LYU 0102 : XML for Interoperable Digital Video Library Recent years, rapid increase in the usage of multimedia information, Recent years, rapid increase in the usage of multimedia information, New approach: DIGITAL VIDEO LIBRARY New approach: DIGITAL VIDEO LIBRARY Automated video and audio indexing Automated video and audio indexing Navigation, visualization Navigation, visualization Search and retrieval Search and retrieval Video segmentation and summarization

Video Information Integration of speech, language, and image processing Integration of speech, language, and image processing Text processing Text processing Audio processing Audio processing Image processing Image processing Video processing Video processing

Digital Video Library System Overview

Techniques to segment data

Techniques we may apply VOCD VOCD Scene changes Scene changes Text processing Text processing Face detection Face detection Storage as XML Storage as XML

Techniques to be discussed VOCR VOCR Scene changes Scene changes Storage and editing with XML Storage and editing with XML

Video OCR for Digital News

Detection of Text Region Video news program comprises huge numbers of frames Video news program comprises huge numbers of frames Roughly detect text region Roughly detect text region Increase processing speed Increase processing speed Reduce processing cost Reduce processing cost

Detection of Text Region Typical text region can be characterized as a horizontal rectangular structure Typical text region can be characterized as a horizontal rectangular structure With clustered sharp edges With clustered sharp edges Regions of high contrast against the background Regions of high contrast against the background

Image Enhancement Sub-pixel Interpolation: Sub-pixel Interpolation: –To magnify the text area –To increase the resolution of caption Multi-frame Integration: Multi-frame Integration: –Video motion of non-caption areas, caption relatively stable –To reduce the variability on background

Character Segmentation Vertical project profile Vertical project profile Character segmentation Character segmentation

Character Recognition Binarize the character image with threshold Binarize the character image with threshold Filter the binary image with morphological filter Filter the binary image with morphological filter Filter the character image with connected component filter Filter the character image with connected component filter

Post-Processing Further improve the recognition rate Further improve the recognition rate 1.Using the words of dictionary to refine the character 2.Integrate the recognition result of multiple frames

Scene change detection technique detection technique effective method for segmenting a video sequence into significant components effective method for segmenting a video sequence into significant components

Existing Method Image difference method Histogram Difference Method Histogram Difference Method using DC Coefficient Image Our Method Our Method  histogram difference method with a dynamic threshold

Scene change grasp scene from the video for every 0.05 second grasp scene from the video for every 0.05 second grasped scenes are 24-bit image, 8 bits for each color (red R, green G, blue B) grasped scenes are 24-bit image, 8 bits for each color (red R, green G, blue B) check each pixel with the most 2 significant bits check each pixel with the most 2 significant bits classify them into 64 different classes classify them into 64 different classes build a color histogram build a color histogram

Scene change Compared the histogram with the pervious scene Compared the histogram with the pervious scene For each column of the histogram, calculate the difference For each column of the histogram, calculate the difference Sum all the difference Sum all the difference If (total difference) > threshold If (total difference) > threshold => scene change Use the first frame as key frame Use the first frame as key frame

XML Extensible Markup Language Extensible Markup Language Create its own mark-up language for describing the contents Create its own mark-up language for describing the contents Look like a big database Look like a big database

Advantages of using XML Platform and system independent Platform and system independent Create your own tag Create your own tag Adopt Unicode Adopt Unicode Universal format Universal format Easy to search Easy to search

Design schema Starts with choosing a vocabulary Words and phrases that are able to describe extracted video information content and therefore can be used as tag name Show relationship between vocabulary entries

XML Parser A parser is a interface between an XML document and the application program A parser is a interface between an XML document and the application program Document Object Model (DOM) Document Object Model (DOM)

How to present XML Tree model becomes very similar to an XML schema Tree model becomes very similar to an XML schema Represented as nodes that show element/attribute names or the text content and their relative places within the XML Represented as nodes that show element/attribute names or the text content and their relative places within the XML

OUR TOOL

COMING EXTRACT SECONDARY INFORMATION EXTRACT SECONDARY INFORMATION

THE END