Ó 2008, Aptima, Inc. 1 Boston ▪ DC ▪ Dayton © 2008, Aptima, Inc. Improving the Display of River and Flash Flood Predictions Presentation to DOH Workshop, Silver Spring Lawrence Wolpert, Tom Aten, Bob McCormack July 17, 2008

Ó 2008, Aptima, Inc. 2 Agenda  Introductions  Short Aptima Overview  Flood Prediction –Task 1: Knowledge Acquisition –Task 2: Uncertainty Modeling –Task 3: Visual Display Design  Future Work

Ó 2008, Aptima, Inc. 3  2008, Aptima, Inc. 3 Aptima, Inc.  Founded in 1995; consistent annual growth (43% CAGR)  Serving government and commercial clients  300+ contracts with the DoD  Offices –Woburn, MA, (HQ) –Washington, DC –Dayton, OH –Ft Walton Beach, FL Interdisciplinary Small Business  100+ staff (80% graduate degrees)

Ó 2008, Aptima, Inc. 4  2008, Aptima, Inc. 4 Human–Centered Engineering Technology Capabilities Social & Organizational Structures Mission, Tasks & Work Processes Human Agents Congruence or... Disruption

Ó 2008, Aptima, Inc. 5  2008, Aptima, Inc. 5 Optimizing Performance in Complex Environments

Ó 2008, Aptima, Inc. 6  2008, Aptima, Inc. 6 Flood Prediction: The Challenge Weather and flood predictions are filled with uncertainty, which can be difficult to convey –Misconceptions about probabilistic events “They say we had a 500 year flood last spring. At least that will never happen again in my lifetime.”

Ó 2008, Aptima, Inc. 7 The Challenge - 2 Weather and flood predictions are filled with uncertainty, which can be difficult to convey –Uncertainty make decision-making more difficult What would you do if:  You lived along the NC coast?  You had an important business meeting in New Orleans?  You were planning an outdoor party?

Ó 2008, Aptima, Inc. 8  2008, Aptima, Inc. 8 Flood Prediction Grant - Summary  Contract #: NA07NWS  Customer: NWS  Client Team: –Pedro Restrepo –Mary Mullusky –Donna Page  Contract Duration: –10/05/07 – 05/31/08  Contract Value: –$60,748 Knowledge ElicitationExamine Current Capabilities Visualizations Uncertainty Models

Ó 2008, Aptima, Inc. 9 Task 1: Knowledge Acquisition through Interviews and Group Discussions  Participants –2 Hydrologists –6 Emergency Managers –2 Red Cross Disaster Coordinators –Flood Evacuation Shelter Operator –Dam Operator –2 Fly Fishermen –3 Kayakers –TV Weather Reporter –6 NWS Employees Interviews conducted either in person or by phone

Ó 2008, Aptima, Inc. 10 Task 2: Uncertainty Model  Purpose –Identify the sources/types of uncertainty related to predicting, communicating, and acting upon flood events. –Identify ways to factor uncertainty estimates into the decision-making processes of each community of interest to improve overall quality and efficiency of outcomes. –Determine what each community of interest needs to know about the uncertainty that exists to assess potential risks and choose a course of action.  Method –Analytical investigation and integration of uncertainty research from multiple disciplines.  Probability theory  Categorization theory  Perception theory  Deliverables –Description of the model of uncertainty and how it can be used to enhance decision-making.

Ó 2008, Aptima, Inc. 11 Theoretical Models of Uncertainty  Probabilistic model of uncertainty –signal processing, information theory –“model predictive uncertainty”  Categorization model of uncertainty –set theory, fuzzy logic –“meaning, implication, knowledge organization uncertainty”  Perception model of uncertainty –human psychology, complexity theory, AI –“semantic uncertainty”, perceptual filters, information presentation vs. expectation mismatch

Ó 2008, Aptima, Inc. 12  2008, Aptima, Inc. 12 Likelihood vs. Confidence

Ó 2008, Aptima, Inc. 13  2008, Aptima, Inc. 13  2008, Aptima, Inc. 13 Task 3: Visual Display Design  Support EMs’ decision making regarding courses of action. –Visualizations of river forecasts with uncertainty –Visualizations of “impact” of various flooding scenarios  Visualization of uncertainty model inputs for clear understanding of “why” predictions are what they are.  Allow for local knowledge to be incorporated into predictions  Fusion / organization of information for clear understanding of relationship to one another and enhanced orientation.

Ó 2008, Aptima, Inc. 14  2008, Aptima, Inc. 14  2008, Aptima, Inc. 14 Landmark Map  Ever present geographic depiction of current state of affairs  Enhanced ability to select desired information for presentation on the map  Increased saliency/access to relevant related information needed for accurate SA and decision making

Ó 2008, Aptima, Inc. 15  2008, Aptima, Inc. 15  2008, Aptima, Inc. 15 River Height/Flow Prediction with Uncertainty  Visual depiction of “river forecast” in terms of height / flow –Clear depiction of uncertainty with cone of uncertainty  Visualization of model parameters with ability of the EM to apply local knowledge.

Ó 2008, Aptima, Inc. 16  2008, Aptima, Inc. 16  2008, Aptima, Inc. 16 River Extent Prediction Visualizations  Visualization of river extent for given point selected in the river prediction of flow/height  Stage, Likelihood and Confidence of a selected point  Impact of given scenario shown (items affected)

Ó 2008, Aptima, Inc. 17  2008, Aptima, Inc. 17  2008, Aptima, Inc. 17 Interplay of Visualizations  Enhanced Fusion / organization of information for enhanced decision making –Clear understanding of relationship –Easy access to needed information for decision making

Ó 2008, Aptima, Inc. 18  2008, Aptima, Inc. 18  2008, Aptima, Inc. 18 Further Detail Regarding Flood’s Impact  Provide further understanding of implications of flood at a: –Global level –Individual variable level  Current and Future river impact predictions shown  Additional understanding provided through references to similar previous floods

Ó 2008, Aptima, Inc. 19 Summary  Elicited knowledge from multiple stakeholders –job roles, responsibilities, and information needs  Formulated a comprehensive theory of uncertainty –Environmental/weather –NWS models –Users  Designed visualization techniques –to maximize the comprehension and utility of river and flash flood data for Emergency Managers  Developed prototype displays –demonstrated these techniques in screen shots

Ó 2008, Aptima, Inc. 20 Contact information  Lawrence Wolpert (PM)  Bob McCormack (PI)  Tom Aten

Ó 2008, Aptima, Inc. 21  2008, Aptima, Inc. 21