Robust video fingerprinting system Daniel Luis

Confidential 22 Robust video fingerprinting system Summary Purpose of the system What is video fingerprinting Practical problems to solve Proposed solution Results analysis

Confidential 33 Robust video fingerprinting system Purpose of the system SkillUpJapan distributes digital contents FujiTV, TV Tokyo, SkyPerfecTV, Warner Brothers Japan, … Our platform, Uliza, is an extensible digital content management system Piracy and DRM are of importance to digital contents rights holders

Confidential 44 Robust video fingerprinting system Video fingerprinting A way to effectively tie a video, or a segment of it, to a unique hash value Information needs to be stored and searched efficiently Avoid to store original contents provided by clients Contents should not be recreated from said fingerprint

Confidential 55 Robust video fingerprinting system Key technical aspects about video Measured characteristics Luma and chroma (brightness and color components) Edge detection, gradient orientation Time variance A movie is, after all, a sequence of images that change over time at a defined rate Amount of data per frame

Confidential 66 Robust video fingerprinting system Efficiency metrics Uniqueness Accurately find videos we search; not return videos that are not what we search Database Efficiently index the results in a database Solution must be fast Find the clip among many other videos in fastest time

Confidential 77 Robust video fingerprinting system Some practical problems to solve Current solutions have relatively accurate algorithms, however the process is computationally expensive Partitioning of frames, complex algorithm Database storage is not taken seriously It is unaffordable to store information for every frame, or large arrays of information for each fingerprint Slow search times when database grows

Confidential 88 Robust video fingerprinting system Proposed solution Lowers the needed resources and process time, while improving upon results (Luma and time based indexes) Addresses algorithmic complexity by using simple methods (Euclidean distance of vectors and Tanimoto correlation) Stores information in an efficient way, allowing for quick retrievals, with use of Look-up Tables

Confidential 99 Robust video fingerprinting system Proposed solution (video properties): Average the Luma value of each frame Luma values show prolonged, relatively constant, values that can be indexed to an interval of time Luma Time Threshold Luma Time

Confidential 10 Robust video fingerprinting system Proposed solution (video properties): Average Luma calculated according to Luma Duration [start-time, end- time[ (secs) Average Luma calculated in T seconds [2, 8[102 [8,13[107 [13,14[198 [14,19[85 [590,600]48 … Time

Confidential 11 Robust video fingerprinting system Proposed solution (database): Using those indexes to store only segments we can save lots of space Each segment of several seconds has a value of 2 bytes Luma values range from 0 to 255 Look-up table for segments 2.0 Luma Time Luma time

Confidential 12 Robust video fingerprinting system Proposed solution (database): 4 Luma …… Time 2 Luma …… Time 2 Luma …… Time Comparisons Fingerprints on databaseFingerprint A … Fingerprint 1 Fingerprint 3 Time

Confidential 13 Robust video fingerprinting system Proposed solution (algorithm): Tanimoto Tanimoto makes a correlation between C and the remaining elements outside C Euclidean vector distance A B C

Confidential 14 Robust video fingerprinting system Proposed solution (algorithm): Hierarchical approach 1.Look-up Table of segments 2.Compares the time indexes 3.& 4. Tanimoto Correlation and Vector Distance of Luma Look-up Tables discard perceptually different movies efficiently Comparison of time indexes also behaves efficiently The number of movies that are ultimately analyzed with Tanimoto Correlation and Euclidean Vector Distance is very low

Confidential 15 Robust video fingerprinting system Evaluation of algorithm: 220 movies were analyzed with each other Quality varies from FullHD to SD Duration ranges from 15 second commercials to full length movies Frame-rate of movies varies from 15fps to 30fps Comparison against C.G.O. (Centroids of Gradient Orientation) [1] Tests were conducted by searching scenes of 10 seconds Evaluation compares algorithm, database size and robustness of solutions [1] Sunil Lee and Chang D. Yoo, “Robust Video Fingerprinting for Content-Based Video Identification”, IEEE Trans. Circuits and Systems for Video Technology, vol. 18, no. 7, pp , July, 2008

Confidential 16 Robust video fingerprinting system Obtained results (database size): SUJCGO DB ThresholdSize (Kbytes) 1 4, , , , , ,

Confidential 17 Robust video fingerprinting system Obtained results (run-time): SUJCGO DB ThresholdTime (Seconds)

Confidential 18 Robust video fingerprinting system Obtained results (robustness): SUJCGO DB threshold False Positives False Negatives False Positives False Negatives

Confidential 19 Robust video fingerprinting system Obtained results (distortion robustness) CIF = 352x288 pixels QCIF = 176x144 pixels SUJ P FN CGO P FN Resize to CIF Resize to QCIF Lossy compression Frame-rate change to 15 fps Frame-rate change to 5 fps Rotation of 1 degree Rotation of 2 degrees Rotation of 3 degrees

Confidential 20 Robust video fingerprinting system Summary State of the art solutions need to better address practical issues The proposed algorithm can improve upon state of the art algorithms on storage and speed of analysis Evaluation shows that the proposed solution also provides higher robustness

Confidential 21 Robust video fingerprinting system Questions? Daniel Luis

Confidential 22 Robust video fingerprinting system Daniel Luis