Image and Video Retrieval INST 734 Doug Oard Module 13.

Slides:



Advertisements
Similar presentations
1 Multimodal Technology Integration for News-on-Demand SRI International News-on-Demand Compare & Contrast DARPA September 30, 1998.
Advertisements

What is ENG? It is the process of reporting events and activities that occur outside of the television studio. It is getting the story and presenting.
Information Extraction from Spoken Language Dr Pierre Dumouchel Scientific Vice-President, CRIM Full Professor, ÉTS.
Multimedia Retrieval. Outline Audio Retrieval Spoken information Music Document Image Analysis and Retrieval Video Retrieval.
Broadcast News Parsing Using Visual Cues: A Robust Face Detection Approach Yannis Avrithis, Nicolas Tsapatsoulis and Stefanos Kollias Image, Video & Multimedia.
Chapter 11 Beyond Bag of Words. Question Answering n Providing answers instead of ranked lists of documents n Older QA systems generated answers n Current.
1 CS 430: Information Discovery Lecture 22 Non-Textual Materials 2.
Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,
Multimedia IR LBSC 796/INFM 718R Session 11 November 9, 2011.
MUSCLE movie data base is a multimodal movie corpus collected to develop content- based multimedia processing like: - speaker clustering - speaker turn.
LYU 0102 : XML for Interoperable Digital Video Library Recent years, rapid increase in the usage of multimedia information, Recent years, rapid increase.
Image and Video Retrieval LBSC 796/INFM 718R Session 13, May 4, 2011 Douglas W. Oard.
AdvAIR Supervised by Prof. Michael R. Lyu Prepared by Alex Fok, Shirley Ng 2002 Fall An Advanced Audio Information Retrieval System.
                      Digital Video 1.
Beyond Text INFM 718X/LBSC 708X Session 10 Douglas W. Oard.
Motion graphics. Film title sequence Film title sequence is a way of presenting a film title or television programs which includes the cast members, producer,
Batch VIP — A backend system of video processing VIEW Technologies The Chinese University of Hong Kong.
Digital Storytelling Tell me a fact and I’ll learn
1 Image Video & Multimedia Systems Laboratory Multimedia Knowledge Laboratory Informatics and Telematics Institute Exploitation of knowledge in video recordings.
Stop Motion By Skylar Allison and Maddie Rund. What We Need To Do Research and watch tutorials on how to make a stop motion movie Create stop motion movies.
1 Lessons Learned From Building a Terabyte Digital Video Library Presented by Jia Yao Multimedia Communications and Visualization Laboratory Department.
Skills: none Concepts: introduction to and history of speech (with and without text) and music processing, audio file formats, the audio processing workflow,
MOVIES MOVIES Lidia Parra 3ºA MY FAVOURITE INVENTION…..
Instructional Technology Postproduction techniques for adding captions, titles, and transitions to video and graphics. ED 6336 Dr. N. Hadley MJ Kiefer.
WP5.4 - Introduction  Knowledge Extraction from Complementary Sources  This activity is concerned with augmenting the semantic multimedia metadata basis.
Lecture #32 WWW Search. Review: Data Organization Kinds of things to organize –Menu items –Text –Images –Sound –Videos –Records (I.e. a person ’ s name,
Module 3: The media  Unit 9 Lesson 1-2 Unit 9 Lesson 1-2 Uses of Cameras.
Image and Video Retrieval INST 734 Doug Oard Module 13.
RIAO video retrieval systems. The Físchlár-News-Stories System: Personalised Access to an Archive of TV News Alan F. Smeaton, Cathal Gurrin, Howon.
Contactforum: Digitale bibliotheken voor muziek. 3/6/2005 Real music libraries in the virtual future: for an integrated view of music and music information.
Digital Video 101.
Chapter 10-Basic Software Tools. Overview Text-based editing tools. Graphical tools. Sound editing tools. Animation, video, and digital movie tools. Video.
A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008.
1 CS 430 / INFO 430 Information Retrieval Lecture 23 Non-Textual Materials 2.
Documentary Film Note Taking For Use With: Vietnam: A Television History Frontline Films.
M ULTIMEDIA IN S PORTS Martin Sebera
1 CS 430: Information Discovery Lecture 22 Non-Textual Materials: Informedia.
M ULTIMEDIA IN S PORTS Martin Sebera
Beyond the PC Kiosks & Handhelds Albert Huang Larry Rudolph Oxygen Research Group MIT CSAIL.
Speech and Music Retrieval INST 734 Doug Oard Module 12.
WRITING FictionNon-fiction. FICTION  From the mind of the writer  Dramas Soaps  Commercials  Features  Situation comedies  Interactive games.
 Digital Storytelling  Module #3 TIE585AC Integrating Web 2.0 Applications in the Classroom Module #3 Image source:
Ulead Video Studio is an easy to use video editing software that allows even the novice of movie makers to produce a professional project complete with.
MMHS Technology and Media Festival Guidelines FEBRUARY 22, 2008.
Image and Video Retrieval INST 734 Doug Oard Module 13.
Digital Video Library Network Supervisor: Prof. Michael Lyu Student: Ma Chak Kei, Jacky.
Detection of Illicit Content in Video Streams Niall Rea & Rozenn Dahyot
Scanned Documents INST 734 Module 10 Doug Oard. Agenda Document image retrieval  Representation Retrieval Thanks for David Doermann for most of these.
1 CS 430 / INFO 430 Information Retrieval Lecture 17 Metadata 4.
Film as Literature: A New Section April 21. Agenda Review Introduce New Section Begin Film.
Some research areas:  Medicine: ◦ analysis of bio-signals, ◦ medical imaging ◦…◦… 1.
Introduction to Digital Video. Digital Video Digital vs. Analog Analog video uses a continuous electrical signal to capture footage on a magnetic tape.
What is Windows Movie Maker? Windows Movie Maker is an easy to use video editing software that allows you to make home movies, automated photo albums,
1.Read the story “Off and Running” in Journeys. 2.Work with the other students at your table to identify story elements. Be prepared to come forward to.
Scanned Documents INST 734 Module 10 Doug Oard. Agenda Document image retrieval Representation  Retrieval Thanks for David Doermann for most of these.
Tips and Protocols. Why the presentation? Why Caption? Conventions/Guidelines for Captioning Options for Captioning Resources.
What is a “Book Trailer?” A Book Trailer is a video advertisement for a book..a way to hook readers like movie trailers hook viewers.
Digital Video 101.
Digital Video Library - Jacky Ma.
Visual Information Retrieval
ECE 417 Lecture 1: Multimedia Signal Processing
CS 430: Information Discovery
ROBUST FACE NAME GRAPH MATCHING FOR MOVIE CHARACTER IDENTIFICATION
Talking Titles.
Presenter: Ibrahim A. Zedan
Basic Multimedia Broadcast Terms
Discussion Class 9 Informedia.
Speaker name Title Title
Speaker name Title Title
Presentation transcript:

Image and Video Retrieval INST 734 Doug Oard Module 13

Agenda Image retrieval Video retrieval  Multimedia retrieval

Multimedia A set of time-synchronized modalities –Video Images, object motion, camera motion, scenes –Audio Speech, music, other sounds –Text Closed captioning, on-screen captions, signs, …

Multimedia Genres Television programs –News, sports, documentary, talk show, … Movies –Drama, comedy, mystery, … Meeting records –Conference, video teleconference, … Others –Lifelogging, surveillance cameras, …

Story Segmentation Video often lacks easily detected boundaries –Between programs, news stories, etc. Accurate segmentation improves utility –Too large hurts effectiveness, to small is unnatural Multiple segmentation cues are available –Genre shift in shot-to-shot structure –Vocabulary shift in closed captions –Intrusive on-screen text –Musical segues

Closed Captions Designed for hearing-impaired viewers –Pre-processed, live stenography, automatic –Speech content, speaker id, non-speech audio Complementary to speech recognition –Each contains different types of errors –Each provides unique information Weakly synchronized with the video

On-Screen Captions Useful –Speaker names, event names, program titles, … Challenging –Low resolution, variable background Possible –Absolutely stable over multiple frames –Standard locations and orientations

Identity-Based Retrieval Face recognition and speaker identification –Both exploit information that is usually present On-screen captions provide useful cues –Confounded by OCR errors and varied spelling Closed captions and speech retrieval help too –Announcers often introduce speakers before cuts

Summary Multimedia retrieval builds on all we know –Controlled vocabulary retrieval & social filtering –Text, image, speech and music retrieval New sources of evidence are added –Video structure, closed & on-screen captions Cross-modal alignment adds new possibilities –One modality can make another more informative –One modality can make another more precise