A FINGERPRINTING SYSTEM MOBILE MODEL FOR VIDEO COPY PROTECTION

Curtis Kelsey

University of Missouri [email protected]

MOTIVATION

• • Create Application/Database ecosystems free of copyright infringement Reduce computational cost incurred on the provider.

PROPOSED TECHNIQUE

• Use a modified pairwise boosting on visual Viola-Jones features to learn top-M discriminative filters on a mobile platform for querying.

CHARACTERISTIC ANALYSIS

• • Benefits • As accurate as the time spent training • Allows for poor false positive rate Weaknesses • All classifiers must have a high detection rate

OPENCV HARRTRAINING (IMPLEMENTATION ANALYSIS)

• • • • Training the classifier requires: • Negative samples for training/testing • Positive samples for training/testing Training Time • ~90 minutes w/ 1350+ and 5500- images [5] Classifier Accuracy • > 5000 false detections per 1.3 billion [5] Naotoshi Seo extensively tests OpenCV’s training [6] • As training time increases, accuracy increases in a logarithmic form

FEASIBILITY

• • • Can we use cascading classifiers on a mobile device?

• No Why?

• Video Data is unknown until submission. Classifier training cannot be done in real time What now • Use another fingerprinting technique for the mobile platform

MODIFIED PROPOSED TECHNIQUE

• Use a modified block-based luminance signature generated by a client for submission to a server for copy detection.

METRICS

• • • • Precision • 𝑃 𝑟 % = 𝑁 𝑡𝑝 𝑁 𝑝 ∗ 100 Recall • 𝑅 𝑒 % = 𝑁 𝑡𝑝 𝑁 𝑒𝑝 ∗ 100 False Positive • 𝑅 𝑓𝑝 % = 𝑁 𝑓𝑝 𝑁 𝑒𝑛 ∗ 100 • 𝑅 𝑓𝑛 % = 𝑁 𝑓𝑛 𝑁 𝑒𝑝 ∗ 100 In a system attempting to filter copyrighted intellectual property, the false negative rate can be discarded, giving the benefit of the doubt to the user uploading video into your environment.

FIRST THINGS FIRST

• Eliminate Preprocessing • What was done?

• • • Video size constrained Frame rate constrained Encoding bit rate constrained

TRANSITION INTENSITY

• • • • Calculate frame intensity • FI = 𝑤𝑖𝑑𝑡ℎ,ℎ𝑒𝑖𝑔ℎ𝑡 𝑐 ∈𝐶𝑜𝑚𝑝𝑜𝑛𝑒𝑛𝑡𝑠 𝑐 255 ∗𝑛𝑢𝑚𝑂𝑓𝐶𝑜𝑚𝑝𝑜𝑛𝑒𝑛𝑡𝑠 Calculate transition intensity • 𝑡𝑟𝑎𝑛𝑠𝑖𝑡𝑖𝑜𝑛 𝑡 = 𝐹𝐼 𝑡 − 𝐹𝐼 𝑡−1 Determine the threshold between scenes • t hreshold = 𝑡 𝑡𝑟𝑎𝑛𝑠𝑖𝑡𝑖𝑜𝑛 𝑡 𝑛𝑢𝑚𝑂𝑓𝑇𝑟𝑎𝑛𝑠𝑖𝑡𝑖𝑜𝑛𝑠 Determine the number of scenes 𝑡𝑟𝑎𝑛𝑠𝑖𝑡𝑖𝑜𝑛 𝑡 > 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑, 𝑖𝑛𝑐𝑟𝑒𝑚𝑒𝑛𝑡 𝑠𝑐𝑒𝑛𝑒𝑠

CONVERT RGB TO YUV

• • Y` is a measure of overall luminance Can be used instead of components

SCENE FRAMES

• Meng et al. describes multiple solutions. I use a basic luminance differencing in the temporal domain.

• Threshold needs to be trained

GENERATE FINGERPRINT

• • Use the scene frames to generate block luminance signatures of each frame Base on ordinal ranking • Weak to affine transformations

𝐼𝑚𝑎𝑔𝑒 (𝐼) 𝐵𝑙𝑜𝑐𝑘 𝐶𝑜𝑙𝑢𝑚𝑛𝑠(𝐵𝐶) = ∆𝑥/8 𝐵𝑙𝑜𝑐𝑘 𝑅𝑜𝑤𝑠(𝐵𝑅) = ∆𝑦/8 S 𝑜𝑟𝑡𝑒𝑑 𝐿𝑖𝑠𝑡 𝑆𝐿 𝐴𝑐𝑐𝑢𝑚𝑢𝑙𝑎𝑡𝑜𝑟 (𝐴) 𝐹𝑜𝑟 𝑒𝑎𝑐ℎ 𝑖 ∈ 𝐵𝑅 𝑎𝑛𝑑 𝑗 ∈ 𝐵𝐶 𝐴 = 0 𝐹𝑜𝑟 𝑖 − 1 ∗ 8 𝑡𝑜 𝑖 ∗ 8 //Represented with k 𝐹𝑜𝑟 𝑗 − 1 ∗ 8 𝑡𝑜 𝑗 ∗ 8 //Represented with m 𝐴 += 𝐼 𝑘 [𝑚] 𝐹𝑜𝑟 0 𝑡𝑜 𝑆𝐿. 𝑙𝑒𝑛𝑔𝑡ℎ //Represented with k 𝐼𝑓 𝐴 < 𝑆𝐿 𝑘 𝑆𝐿 𝑘 . 𝑖𝑛𝑠𝑒𝑟𝑡(𝐴) 𝑏𝑟𝑒𝑎𝑘

SUBMITTING THE FINGERPRINT

• • POST fingerprint to php script via internet Use Direct Hashing Algorithm (DHA) previously presented.

• • • Hash fingerprints Insert into a standard hash table if query returns no match Query up to hamming distance of 2

RESULTS

• • • Frames process in approx. 12.5 seconds each • • Core i7 4GB DDR3 Video Size • 1676 x 985 Data Rate • 159kbps

RESULTS

• • • • Like hardware 1280 x 720 15,513 kbps 29 fps

• • • • •

REFERENCES

[1] Lian, H. C., Li, X. Q., & Song, B. (2011). A fingerprinting system for video copy detection. Fuzzy Systems and Knowledge Discovery (FSKD), 2011 Eighth International Conference on (Vol. 4, pp. 2146–2149). IEEE. Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6019957 [2] Viola, P. (2001). Rapid object detection using a boosted cascade of simple features. , 2001. CVPR 2001. Proceedings of the. Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=990517 [3] Zhang, Z., Cao, C., & Zhang, R. (2010). Video copy detection based on speeded up robust features and locality sensitive hashing. Automation and Logistics (, 13-18. Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5585375 [4] Meng, J., Juan, Y., & Chang, S.-fu. (1995). Scene Change Detection in a MPEG Compressed Video Sequence 2 . Previous Approaches 3 . MPEG Compression Standard. Symposium A Quarterly Journal In Modern Foreign Literatures, 2419(February), 1-12. Retrieved from http://csce.uark.edu/~jgauch/library/Video-Segmentation/Meng.1995.pdf

[5] Adolf, Florian. How-to build a cascade of boosted classifiers based on Haar-like features. Retrieved from http://lab.cntl.kyutech.ac.jp/~kobalab/nishida/opencv/OpenCV_ObjectDetection_HowTo.pdf

REFERENCES CONT.…

• • [6] Seo, Naotoshi. Tutorial: OpenCV haartraining (Rapid Object Detection With A Cascade of Boosted Classifiers Based on Haar-like Features). Retrieved from http://note.sonots.com/SciSoftware/haartraining.html

[7] Mohan, R. (1998). Video sequence matching. Acoustics, Speech and Signal Processing, 1998., 3697-3700. Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=679686

QUESTIONS