Download presentation
Presentation is loading. Please wait.
1
SAMI: Situational Awareness from Multi-modal Input Naveen Ashish
2
Talk Organization Why are we at RESCUE interested ? Situational Awareness (SA) –Introduction System architecture Research challenges Expected outcomes and artifacts Extraction system demonstration
3
Team Naveen Ashish Sharad Mehrotra Nalini Venkatasubramanian Utz Westermann Dmitry Kalashnikov Stella Chen Vibhav Gogate Priya Govindarajan Ram Hariharan John Hutchinson Yiming Ma Dawit Seid Jay Lickfett Chris Davision Quent Cassen Bhaskar Rao Mohan Trivedi Rajesh Hegde Sangho Park Shankar Shivappa Ron Eguchi Mike Mio Jacob Green
4
Information from Various Sources People/Victims at disaster Emergency responders News, video, audio footage GIS, satellite imagery, maps Pushing “Human-as-sensor”
5
More Data ≠ More Information SA Where are the fire personnel ? Have all medical supplies reached ? What areas should we start evacuating first ?
6
Situational Awareness Wide variety of fields –Beginning in mid-80s, accelerating thru 90s –Fighter aircraft, ATM, Power plants, Manufacturing Definitions –"the perception of elements in the environment along with a comprehension of their meaning and along with a projection of their status in the near future" –"the combining of new information with existing knowledge in working memory and the development of a composite picture of the situation along with projections of future status and subsequent decisions as to appropriate courses of action to take" Situational awareness and decision making Areas –Cognitive science –Information processing –Human factors Knowing what is going on
7
Abstraction of Information Multimodal Input: Text, Audio, Video Events Awareness
8
First-cut Architecture EVENT BASE Querying and Analysis Graph View VISUALIZATION and USER INTERFACES Spatial Indexing PDF Histogram KNOWLEDGE: ONTOLOGIES Text Audio Video Internet RAW DATA EVENT EXTRACTION REFINEMENT Disambiguation Location Centered around EVENTS as fundamental abstractions
9
Research Areas Event Modeling Event Extraction Disambiguation Location Uncertainty Graph Analysis GIS Querying
10
Event Modeling What is an event ? Event Representation RELIABILITY PEOPLE EVACUATION LOCATION TIME REPORT TYPE NAME LOCATION AGENCY FROM TO OPERATION NUMBER
11
Domain Knowledge Captured as Ontologies ROAD EVACUATION AIR EVACUATION IS-A THAILANDSOUTHERN REGION …….PHUKETPHUKET, CHANGWAT
12
Event Extraction Long history of information extraction –IR (MUC efforts) –Web data extraction DARPA ACE –Entities, Relations, Events –Events in 2004 Event extraction accuracy is still low SA Domain –Stream of information –Duplicated, ambiguous –Reliability –Conversations Modalities –Text
13
Semantics Driven Approach Semantics Driven Challenges –Framework –Ontologies What semantics required for event extraction ? Application With NLP, ML techniques Performance –SA specific Duplicates, reconciliation, temporal, conversations …..
14
Disambiguation
16
–point-location in terms of landmarks uncertain, not (x,y) –reasoning on such data support all types of queries Uncertainty is a Challenge Report 1: “... a massive accident involving a hazmat truck on I5-N between Sand Canyon and Alton Pkwy...” Report 2: “... a strange chemical smell on Rt133 between I405 and Irvine Blvd...” Report 1 Report 2
17
Implications of Uncertainty in Text How to model uncertainty? –probabilistic model –P(location | report) e.g. report says “near building A” Queries –cannot be answered exactly... use probabilistic queries all events: P(location R | report) > 0 –SA requirements triaging capabilities fast response –top- k –threshold: P(location R | report) > – -RQ, k-RQ, k -RQ How to map text to probabilities? –use spatial ontologies A B R
18
Graph Analysis GAAL Inherent spatio-temporal properties Graphs are powerful for querying and analysis
19
Current FGDC Search GIS Search
20
Progressive Refinement of Data GIS Search
21
Deliverables, Outcomes, Artifacts “Vertical” thrusts –Event extraction system (TEXT) –Disambiguation system –GIS search system Overall system demonstration ? “By-products” –Ontologies Computer science research areas Databases Semantic-Web Information Retrieval Intelligent Agents (AI)
22
Thank you ! http://sami.ics.uci.edu
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.