1. WBI/WCI - SKM 14 July 2003 1 Analysis and Knowledge Extraction from Video & Audio Rick Parent Jim Davis Raghu Machiraju Deleon Wang Department of Computer and Information Science Ohio State University

2. WBI/WCI - SKM 14 July 2003 2 Overview Human operators & large data sets Extract important events Focus on human behavior Use multimodal approach Security (real-time processing) Annotating recorded video Processing archival material Streaming data from video & audio Problem Solution Motivation Applications

3. WBI/WCI - SKM 14 July 2003 3 Objectives Detect and track people to extract audio-visual events Present graphical summaries to human operator via secure web-based interface 3 level system Person/action detection Sequential long-term tracking Multi-modal identification Incrementally constructs event model to focus attention and resources to track and recognize people across sequences Build prototype system

4. WBI/WCI - SKM 14 July 2003 4 Person Detection and Activity Recognition (Jim Davis) Thermal-based image analysis and person detection Framework for recognizing basic human activities

5. WBI/WCI - SKM 14 July 2003 5 Sequential-frame tracking (Raghu Machiraju, Rick Parent) Monitor across sequences Characterize motions Capture appearance Tack human figure poses

6. WBI/WCI - SKM 14 July 2003 6 Robust Speaker Recognition (Deleon Wang) Usable speech extraction from multiple speaker audio =+ By tracking pitch and extracting voiced segments

7. WBI/WCI - SKM 14 July 2003 7 Deliverables Demonstration subsystems Person detection Long-term tracking Speech recognition 6 mos: review of basic work 12 months: demo of capabilities, summary report

8. WBI/WCI - SKM 14 July 2003 8 Expenditures 6 Student-quarters of support over 12 months 2 Qtrs: Person detection (Davis) 3 Qtrs: Tracking (Machiraju & Parent) 1 Qtr: Speech (Wang)