Managing Situational Awareness using Web Feeds and Social Media During Emergency Events CSIRO Robert Power, Ryan Fraser | 7 November 2013

Projects: Emergency Response Intelligence Capability Emergency Situation Awareness Highlighting: What has been achieved to date What is possible Challenges to overcome Lessons learned Outline ERIC ESA

Emergency Response Intelligence Capability ERIC ESA

ERIC – Drivers

Supports strategic and operational decision making during an emergency by: automated gathering of information from a range of sources integrating the information into a single place presenting the information online using a map generating Situation Reports customized for different types of emergency events at specific locations maintaining a collection of Situation Reports notifying when new relevant information becomes available ERIC – Description

ERIC – Data DHS (zones, sites, demographics) ABS (ASGS boundaries, 2011 Census) BOM (weather alerts, cyclone warnings) GA (hotspots, gazetteer) State based RSS feeds (RFS, ESA, CFA, FESA, TFS, QFRS)

ERIC – Overview

Emergency Situation Awareness ERIC ESA Context: Situation information reported in real-time on social networks, but not easily consumable by crisis coordinators

Problem: Crisis coordinators need tools to manage issues arising from the Tweet deluge  capture and search tweets  detecting and alerting on events of interest  condensing and summarising content to reduce watch officer load  classify and review high-value messages enabling officers to quickly understand impact of real-world events on people and infrastructure  focus and track issues as they arise, evolve and decay over time  perform forensic analysis of incidents Solution: ESA is a service to collect, detect, assess, simplify and report situation information in near-real-time from Twitter text streams Problem/Solution

ESA – Capture Regions Number of Tweets in 5 minute window Red circles show capture regions Markers show geo-coded locations of Tweets

ESA – Detecting and Alerting When you don't know what to look for, such as with unexpected incidents, the ESA Alert Monitor can provide some clues as to what is going on in Twitter Australia-wide. Alerts generated by our system are shown as a tag cloud. Alert colour and size indicates deviation from expected. Moe earthquake as reported by our Melbourne Alert Monitor (Friday 20 th July :13:08)

Detecting And Alerting

Tweet Display Near real time search Replay historical events Classification Tweet content model Burst detection Alert monitor Location mapping Data backups ESA Toolkit

Anomaly detection and alerting (2012) Streaming classifiers (2013) Streaming incremental clustering (2013) Forensic awareness (2014) Situation awareness (2015) All-hazards Natural Disasters Earthqu akes Tsunam i Cyclone s Storms Bushfire s Floods Man-made Protests Public events Socio- political Acciden ts Chem- bio Hazmat Pandem ic Detect Australian Earthquakes faster than USGS Classify infrastructure impact in real- time Track issues in real-time Classify event types in real-time Strategic Roadmap

Government organisations (federal, state and local) are invited to trial CSIRO's advanced social media tools and work together with other leading organisations to make the most from social media. Our Early Adopters Reference Group is a community of government innovators with access to prototypes of our newest social media monitoring and analysis tools. The group started in August Feedback from the group is used to enhance the CSIRO tools during the trial. The Early Adopters Reference Group meet regularly to discuss CSIRO tools and broader issues in social media monitoring. The group is led by a steering committee with representatives from participating organisations. Early Adopters Reference Group

Emergency Situation Awareness Engagement Ecosystem

From ERIC to ESA Differences ERICESA Small number of usersMany users Support a few specific use casesSupport many general use cases Current situationNear real time (NOW) User does the analysisData analysis software Modest data requirementsLarge data volume/velocity (~1700 tweets per/min - 1 Gb per day) Includes static dataOnly dynamic data (and archive) Authoritative informationUser driven content Includes user (DHS) specific dataCrowd sourced data ERIC ESA

Dynamic data Data archived Browser operated Web server Data warehouse approach Demonstration prototype From ERIC to ESA Similarities ERIC ESA

Summary ERIC authoritative current data modest data user analysis ESA crowd sourced near real time big data tool analysis dynamic data data archive alerts

Future work ERIC ESA …increase the overlaps…

Obstacles Overcome – data ERICESA Inconsistent Web feed formatsVolume Information gapsVelocity Access to DHS data (right resolution to preserve privacy) Near real time processing: classification

Obstacles Overcome – users ERICESA Close engagementInitial CCC engagement Early demonstrationsThen others … Regular meetingsEarly Adopters Group Observation exercises… next steps

Know the users Who are they – specifically – how do they work now What can’t they do now – what do they want to do Observe Know the data Access (open data – service access?) Custodian (is the data fit for purpose?) Format (PDF/ word documents) Syntax, semantics, relationships Role of the system administrator (restricted/open) Lessons Learned

Documentation Concept of Operations - non technical description: who, why, where, when. Requirements - complete and unambiguous description: what. Communication Speak the same language Regular contact (meetings and demos) Development Methodology Iterative development (mockups/demonstrators) Adopt, adapt, build Use proven techniques/technologies Lessons Learned

Thank you CSIRO Government and Commercial Service Ryan Fraser Research Manager t wwww.csiro.au