An Automatic Video Information Processing and Conversion System for Multimedia Messaging Service and Mobile Internet Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith Video over InternEt and Wireless Technologies Lab., CUHK 6 th IT Excellence Awards (Post Secondary) 2004/06/09
VIEWLab, CSE, CUHK Video over InternEt and Wireless Technologies Laboratory VIEWLab is a research and application development laboratory Computer Science and Engineering, CUHK Prof. Michael R. Lyu Focus on Video Information Processing Technologies Mobile and Wireless Application Development
VIEWLab, CSE, CUHK Presentation Outline Motivation Functionality & Design Demonstration Innovation Prospect for future commercialization Q&A
VIEWLab, CSE, CUHK Motivation Needs of mobile video content is increasing Video information processing technologies developed A prototype system is created to automate the video authoring process Enabling faster and cheaper mobile video entertainment
VIEWLab, CSE, CUHK Functionality & Design An automatic video information processing and conversion system for MMS and Mobile Internet Targets Fully automation Mechanical processing with pre-defined knowledge Various video content type Short processing time Configurable for different needs
VIEWLab, CSE, CUHK Overview of System Advertisement Removal Program Identification Scene Analysis MMS (Pictures + Sound) / Mobile Video Video Streaming / Progressive Download MMS Component Creation Mobile Internet Video Clip Conversion TV Broadcast
VIEWLab, CSE, CUHK Video Information Processing These are the technologies to extract information from video using visual and audio channels By extracting/identifying the information, “effective” condensation/summary techniques can be applied through information fusion or recognition steps We define a type of information a “modality” Transcript/Speech On-screen characters Face Visual features
VIEWLab, CSE, CUHK Every Bit Counts MMS size is limited to within hundred KB In order to delivery “useful” information to viewer Condensation/summary techniques are required to locate/prioritize the most important bits Bandwidth cost is relatively higher than other channels the “value” of the content transmitted is curial to successfully MMS business Utilize the “value” of the bandwidth use, only transmitting the important bits
VIEWLab, CSE, CUHK Main Features of Video2MMS Advertisement removal TV program identification Video condensation/summary MMS creation and adaptation for different mobile handsets
VIEWLab, CSE, CUHK Advertisement Removal Based on the logo detection techniques developed, TV program and non TV program segments can be separated automatically Program TV Broadcast Non Program
VIEWLab, CSE, CUHK TV Program Identification Based on the visual features and audio characteristics, the desire TV program can be identified
VIEWLab, CSE, CUHK Video Condensation / Summary Condense the long video into short slide show presentation Segment the video into different scene Two pass minimum difference color histogram Choose the most representative shots Video editing style Longest scene Video Summary
VIEWLab, CSE, CUHK MMS Creation and Adaptation Different handsets have different MMS size and screen size Nokia 7650, KB, 176 x 136 SonyEricsson T68i, T300 – 30KB, 101 x 80 Panasonic GD88 – 45KB, 176 x 136 Convert the pictures to correct screen size Utilize the MMS size for most information (number of pictures and duration) Configurable for new handsets
VIEWLab, CSE, CUHK MMS Matrix MMS Size = audio size + (picture size * n) Audio size = length of audio * 0.8k, (4.75 bit-rate) Picture size = f(screen size, number of colors) HandsetMMS Size DurationScreen Size Number of Color Average Size Number of Pictures Nokia KB60sec174 x K4 ~ 5 SonyEricsson T68i 30KB30sec101 x K3 ~ 4 Panasonic GD88 45KB45sec176 x K4 ~ 5
VIEWLab, CSE, CUHK Mobile Internet Video Clips Conversion Control third party modules to cut and convert video into short video clips Different handsets have different MMS size and screen size 3GPP format RealVideo format Microsoft Windows Media Format Clips size configurable for different usage 100KB for MMS delivery >100KB for video streaming
VIEWLab, CSE, CUHK Demonstration Video (1) (click to play) ATV News Headline AMR + Animated Gif Nokia KB 20 seconds 174 x colors Animated Gif (6 pic.) AMR Audio
VIEWLab, CSE, CUHK Demonstration Video (2) (click to play) ATV News Headline 3GP for Mobile Internet Nokia KB 20 seconds 174 x colors
VIEWLab, CSE, CUHK Demonstration Video (3) (click to play) ATV Weather Report AMR + Animated Gif Nokia KB 60 seconds 174 x colors Animated Gif (5 pic.) AMR Audio
VIEWLab, CSE, CUHK Demonstration Video (4) (click to play) ATV Hourly News AMR + Animated Gif Nokia KB 40 seconds 174 x colors Animated Gif (6 pic.) AMR Audio
VIEWLab, CSE, CUHK Demonstration Video (5) (click to play) Horse Racing 3GP in MMS Nokia KB 20 seconds 174 x 136
VIEWLab, CSE, CUHK Demonstration Video (6) (click to play) Horse Racing RealVideo in MMS Nokia KB 20 seconds 120 x 96
VIEWLab, CSE, CUHK Demonstration Video (7) (click to play) Soccer Goal Shot RealVideo in MMS Nokia KB 5 seconds 120 x 96
VIEWLab, CSE, CUHK System Demonstration Modules Editor Main Panel Task Scheduler Task Profile
VIEWLab, CSE, CUHK Innovation We believed this is the first application applied Video Information Processing technologies on automating the content preparation processes for mobile handset in Hong Kong
VIEWLab, CSE, CUHK Prospect for future commercialization Base on this prototype, our laboratory technology commercialization team developed a production system called Video2MMS™ Video2MMS™ was licensed to HKCSL in spring 2003 to prepare daily news and weather video clips and MMS Press release 7&type1=1&rhtID1=3&lid=2
VIEWLab, CSE, CUHK Tested Handsets MMS Nokia 7560, 3650 SonyEricssion T68i, T300 Panasonic GD88 VideoClips (Download) MS PocketPC, XDA
VIEWLab, CSE, CUHK Recommended System Requirements Intel P4 1.0G Hz 512MB RAM Microsoft Windows 2000 or above DirectX 8.1 TV Capture Card (MPEG-1) Optional third party software Real Encoder Nokia 3GP Encoder Sony Ericsson AMR Convertor Microsoft Windows Media Encoder
VIEWLab, CSE, CUHK Q&A