1. SAFIRE: Situational Awareness for Firefighters SITUATIONAL AWARENESS FOR FIRE FIGHTERS (SAFIRE) Goal: Improve the safety of firefighters by providing decision makers with greatly improved situational awareness during response activities SAFIRE Project DHS Update – September 15, 2009

2. 2 Agenda for the Day Project Overview SAFIRE Concept / Technology SAFIRE Streams Testing/Validation/Outreach Future Plans: Project Outcomes, Continuity FEMA participation in upcoming DHS workshop on “Incident Management, Resource Management, and Supply Chain Management” Nov. 5 and 6 th at CERT, UCI, Irvine.

3. SAFIRE: Situational Awareness for Firefighters SAFIRE Concept Overview

4. SAFIRE: Situational Awareness for Firefighters SAFIRE System FICB Visualization External Data Sources Acoustic data SAFIRE Core Technology Areas GIS hazmat occupancy Multimodal Sensing Robust Network Infrastructure Visualization and User Interfaces (FICB) Sensor stream processing Integration of external data sources (Ebox) Speech Video data Environmental sensors Sensor database FF physio. & location.

5. SAFIRE: Situational Awareness for Firefighters Progress: Core Technology Areas Speech  Speech for situational awareness Networking & Sensing  Incorporation of new sensors (Co, SpCO, motes)  New antenna array for increased coverage, multi-network & store- and-forward architecture Stream management  ability to incorporate variety of sensors, multimodal sensor archival and retrieval functionality FICB  New functionalities in FICB – simplified UI, annotations, ebox integration, etc. Ebox  Prototype development, ontologies for resource selection, integration of static and dynamic data such as sensing infrastructure of buildings 5 May 29, 2009 July 15, 2009 Today demo/video July 15, 2009 July 15, 2009

6. 6 SAFIRE – Project Focusing Following our May teleconference, development effort was refocused on two key infrastructure components:  Fire Incident Command Board (FICB)  SAFIRE Streams Other modules still very important for SAFIRE system but sufficiently developed to complete current effort.  Networking and Sensing  Acoustic Sensing / Speech  EBox  Localization Framework Final integration work of these being completed this summer. Projects will be well positioned to pursue future funding.

7. 7 SAFIRE Streams: A Semantic Middleware for Multi Sensor Applications

8. 8 SATDeployer SATQL Sensor and computing infrastructure Heterogeneous sensors and processing nodes Distributed Mobile- agent based runtime Deployment of operators Convert Query -> VS -> opGraph FICB / SAFIRE Server SATRuntime SAFIRE Streams Architecture SATScheduler SATMonitor Schedule to meet QoS Query results Semantic context Query (entity, attribute, value) VS context1 Query i Infrastructure DB SATRepository Operator DB Policy DB Semantic DB (entities, Relationships, VS) Semantic knowledge...

9. 9 SAFIRE Stream Middleware Writing sensor applications is hard: -Continuous data -Sensor heterogeneity -Diversity of platforms -Tolerance to failures Powerful programming abstractions to ease application development Hide heterogeneity, failures, concurrency Core Services alerting, triggering, data & stream management, queries. Mediation application needs with resource constraints of devices & networks Sensor FICB FiltersAlerts Analysis Networks SA Applications Middleware – glue between H/w, networks, OS and applications Networks Stream Middleware Goals

10. 10 Key Concepts Driving SAFIRE Streams Semantic Level: Entities -- people, appliances, and buildings, rooms; Relationships – interactions. Infrastructure Level: sensing devices, computing devices, network devices. Virtual Sensors: maps data captured by sensors into events in the semantic world. Event Logs: evolution of physical world as observed by the sentient system 10 SAFIRE Streams models sensor embedded spaces at two levels sentient Applications Virtual Sensor High level stream language like CQL

11. 11 Key Concept: Virtual Sensors Provide the “bridge” between sensors & the semantic “real” world concepts. L, Room12, t> Filter [L=Room1] AP Readings Listener AP Readings to location Translate Location to Lon./Lat. Finger print DB Location Virtual Sensor WiFi fingerprints, t>

12. 12 Virtual Sensors: Multi-Sensor Fusion to improve quality AP Readings Listener AP Readings to location Finger print DB Signal strength Listener Signal strength triagulation AP locations Merge <Person, L, Room12, t> Location Virtual Sensor Using fingerprints Location Virtual Sensor Using signal Strength triangulation

13. 13 Virtual Sensors: Speech illustrating how semantics can help improve quality speech DB Acoustic analysis Location Virtual Sensor Using speech recognition Location Virtual Sensor Using acoustic analysis Audio listeners Audio stream Speech recognizer Data Cleaning using semantics Merge

14. 14 Building Applications using Semantic Model Virtual Sensors “hide” complexity of sensor programming from application developers  Convert heterogeneous sensor streams into semantic event streams  Hide sensor failures / imprecision through Noise reduction (e.g., averaging over multiple samples) multi-sensor fusion (e.g., multiple location sensing technologies provide more accurate location assessment) Semantics (e.g., speech sensors exploit word correlation to improve on ASR) Applications can view the system as consisting of high level concepts such as entities, events, artifacts, spaces, etc. SAFIRE Streams supports high level query languages for implementing queries & triggers:  SQL style stream language (at design stage – not yet implemented)  Event graph based language

15. 15 Demo 5/27/09

16. 16 Multi-sensor localization in SAFIRE Streams

17. 17 Event Graphs in SAFIRE Streams Triggers/continuous queries are converted into an event graph network. SATWARE Deployer submits the resulting event graph into an executable pipeline based on available resources, machines and networks.  Mediates with resources to guarantee application needs are met  Multiple optimizations possible in executing such networks. Loc operator [FF1] Join [t] Filter [L=first floor] Loc operator [FF2] { } Near [5 Rooms] Detect when Fire Fighter 1 is on the 1 st floor Detect when FF1 & FF2 are near each other

18. 18 Multi-sensor store / query / visualize in SAFIRE Streams

19. 19 SAFIRE Streams Summary Middleware to ease multi-sensor applications  provides a powerful semantic interface for complex multi-sensor applications this feature used extensively in building SAFIRE SA Applications  Supports core services Alerts, triggers, storage, archival, & replay capabilities.  Mediation between application needs & system resources E.g., sensor stream scheduling based on application quality requirement 5/27/09

20. 20 Safire Project: Outreach Presentation

21. 21 Possible Future Directions Rework FirePack CO / physiological monitoring data capure platform proposal  Chris working on asummary slide EBox Localization from Speech UWB Incident Site Networking

22. 22 SAFIRE: Future Directions Completion of Project  Creation of Robust technology Testing  Test system with Firefighters Outcomes Continuity

23. 23 SAFIRE: Future Directions Outcomes Safire System Product  FICB Research Products  Ebox  Speech  Networking / Sensing

24. 24 SAFIRE: Future Directions Outcomes Thoughts / Expectations on Technology Transfer?  SAFIRE Advisory Board Newport Beach FD LA County Fire City of Ontario OCFA EH&S  Deltin Corporation

25. 25 SAFIRE: Future Directions Continuity Future Project Directions  Ebox  Speech  Networking / Sensing  SpCO data management [FireTrack]

26. 26 FireTrack System Goals of FireTrack 1.A prototype framework for collecting, communicating, storing, and analyzing exposure data entitled FireTrack 2.A detailed evaluation of the FireTrack system in a pilot study including specific recommendations on how to expand the system to a large scale deployment. 3.Data collected during the pilot study including exposure information at both the level of individual firefighters as well as at the environment under different conditions. 4.Database design to represent data captured about respiratory environments during fires including taxonomies to appropriately classify and analyze such data. 5.Specific recommendations on interventions techniques that can be realized through exposure monitoring that minimize avoidable exposure to toxins during firefighting. 6.Identification of long term partners who will be willing to (a) maintain such a system for capturing and managing exposure information, and (b) work with research groups to launch further (more comprehensive) data collection and analysis studies the FireTrack system will enable in the future.

27. 27 FireTrack System