WBI/WCI - SKM 14 July Analysis and Knowledge Extraction from Video & Audio Rick Parent Jim Davis Raghu Machiraju Deleon Wang Department of Computer and Information Science Ohio State University

WBI/WCI - SKM 14 July Overview Human operators & large data sets Extract important events Focus on human behavior Use multimodal approach Security (real-time processing) Annotating recorded video Processing archival material Streaming data from video & audio Problem Solution Motivation Applications

WBI/WCI - SKM 14 July Objectives Detect and track people to extract audio-visual events Present graphical summaries to human operator via secure web-based interface 3 level system Person/action detection Sequential long-term tracking Multi-modal identification Incrementally constructs event model to focus attention and resources to track and recognize people across sequences Build prototype system

WBI/WCI - SKM 14 July Person Detection and Activity Recognition (Jim Davis) Thermal-based image analysis and person detection Framework for recognizing basic human activities

WBI/WCI - SKM 14 July Sequential-frame tracking (Raghu Machiraju, Rick Parent) Monitor across sequences Characterize motions Capture appearance Tack human figure poses

WBI/WCI - SKM 14 July Robust Speaker Recognition (Deleon Wang) Usable speech extraction from multiple speaker audio =+ By tracking pitch and extracting voiced segments

WBI/WCI - SKM 14 July Deliverables Demonstration subsystems Person detection Long-term tracking Speech recognition 6 mos: review of basic work 12 months: demo of capabilities, summary report

WBI/WCI - SKM 14 July Expenditures 6 Student-quarters of support over 12 months 2 Qtrs: Person detection (Davis) 3 Qtrs: Tracking (Machiraju & Parent) 1 Qtr: Speech (Wang)