1 Samson Cheung EE 639, Fall 2004 Lecture 1: Applications & Trends Multimedia Information Systems advent: open communicator browser, screen cam, hari’s directory, and Di’s interface advent: open communicator browser, screen cam, hari’s directory, and Di’s interface

2 Definition A computer hardware/software system used for Acquiring and Storing Indexing and Searching Manipulating (editing and quality enhancement) Distributing Protecting large amount of visual information Images, video, animations, and associated meta-data Examples: Web Media Search Engines Mobile Multimedia Portals Digital Libraries for Large Organizations

3 Why it’s important? New Content Creation Multimedia authoring software Diagnostic multimedia in scientific applications High-Speed Access Networks New Services and Devices Multimedia Messaging Multimedia Enabled Intelligent Archive Media portals Standards become available Digital TV, DVD, MPEG-4, MPEG-7, and IPMP Shih-Fu Chang: 0.56% of web pages has streaming media Shih-Fu Chang: 0.56% of web pages has streaming media

4 Multimedia Web Search Google Image Search, Ditto, Altavista, 3Dup, Singingfish, AllTheWeb, Kazaa Media Desktop, Lycos Pictures and Sounds, MIDI Explorer, PicsearchAllTheWeb Issues Search keywords, display interfaces, more advanced search tools

5 Video Cluster Search Cluster near- duplicate web video clips Fast summarization of video using low- level features Issues Search keywords, no semantics

6 Hermitage Museum Mixed media search tools: keyword and visual Virtual tour (using panoramic and 3D view) Zoom view gallery: multi-resolution Watermark protection (invisible) Balanced use of aesthetic, informational, and technical components Issues: Acquisition, representation, user interfaces, search tools, copyright protection Link

7 Blob World Apply image segmentation to partition images into blobs Weigh importance based on color, texture, location, shape/size Issue: image segmentation is hard for general picture Link

8 Specific Domains Geological Images Human settlement, forest, … (link)link Oil exploration Medical Imageries Compare simulation vs. experiments Porn filtering (link)link Issues: Need domain experts, difficult to get data Potentially very fruitful Put human in the loop (relevance feedback)

9 Any-Time TV (TiVo) Time-shift, local storage: 60 hours Skipping mode Instant record, live pause, simultaneous record/playback Highlight/trail viewing Search/retrieval, bookmarking, multi-source comparison/summarization Personal profile, multi-user profile Target services, ads, consumer usage data, Pay per choice e-Commerce

10 Any-Place TV: Universal Multimedia Access Technical issues Scalable coding, skimming, bandwidth-restricted adaptive streaming

11 Broadcast monitoring Issue: Watermarking is expensive and the watermark may not survive the production process. Legacy content?

12 Virage News Video Indexing Key frames Linking transcripts to video Keyword search (Virage & ABC News)

13 Game 1 Game 2 Commercial Serve 1, base-line play, 2 strokes Serve 2 Key Frame Columbia’s Sports Video Navigator and Streaming

14 3-D Search Hot topic Availability of 3D models High-dimensional unstructured features Issues: representation, efficiency, imperfection in modeling, IBR? Link

15 HP SpeechBot Get sound clips from radio website Create time-aligned transcript based on speech recognition Index audio segments based on words Over 14K hours Issue: High speech quality with no music or noise, Poor recognition results Link