1. NOAA CLIMAS NASA EOS NSF SAHRA NOAA GAPP Considering Equity in Climate Prediction Applications: Implications for Development and Evaluation of Decision Support Tools 1 Department of Hydrology and Water Resources, University of Arizona 2 Department of Civil and Environmental Engineering, University of California-Irvine Holly Hartmann 1 and Soroosh Sorooshian 2 NASA HyDIS Raytheon Synergy

2. Calls for Societally Relevant Research and Products Information is appropriate to the knowledge and concerns of the recipient. Ensure that.. modeling improvements and data products are useful to the water resources management community. Develop a strategy for… how these could be made more useful for [user] purposes. Need studies of the benefits and costs of [hydroclimatic] information services. Increase the value of weather and related … information to society. Bring scientific outputs and users’ needs together. Make climate forecasts more socially useful. Stronger sense of responsibility for delivering timely and relevant tools. Accelerate activities to integrate science with the needs of decision makers. Integrate user needs… and ensure that research results are provided in a form useful for users. Sources: Various USGCRP and NRC reports, 1997-2001

3. Issue: So Many Stakeholders! Continental Scale: Focus of modelers Watershed/Local Scale: Where impacts happen Where stakeholders exist Different Scales (time & space) Different Issues Different Stakeholders

4. Poor interactions with users affects: opportunities for future work credibility of agencies, institutions and products “What are your motives?” (agenda) “How long is your project really going to last?” (failed promises of past projects) “What did you do with the last survey?” (checking your responsiveness) Lessons from Stakeholders Building Expectations and Trust Building trust requires repetition & responsiveness Concerns: agendas, science will be used to hurt them Effective stakeholder integration  generate support for science funding & programs

5. Changed decisions & decision processes Enabling system-wide change (transferability, scalability) Public support for climate research Concerns for Climate Science Enterprise Project Objectives Affect… Metrics Structure of stakeholder interactions Research products Perceptions of climate science enterprise Research funding Evaluating Societally Relevant Research and Products

6. Objective: Economic Efficiency Metrics: Cost/benefits. Return on Investment. Stakeholder Interaction Structure: Consultant-client relationship with high-value clients (e.g., hydropower). Research Products: Customized Decision Support Systems. System optimization rules. Perceptions: Science serving special interests. Increasing competitive imbalances. Research Funding: By clients through private sector.

7. Objective: Agency Impact Metrics: Policy and regulatory impact. Stakeholder Interaction Structure: Work with agencies. Important role for policy analysts/scientists. Research Products: Traditional products. Refereed methodology and results. Hold up in court. Perceptions: Science serving special interests, agendas. Increasing regulatory burden. Research Funding: By managed sector, perhaps public.

8. Objective: Societal Equity Metrics: Breadth/diversity of applicability, accessibility, usability. Sectoral ‘market’ penetration. Stakeholder Interaction Structure: Engagement with diversity of stakeholders. Important role for social scientists. Potentially huge demand on researchers’ time. Research Products: Diverse. Non-traditional, but not “dumbed down”. Note: data << information << knowledge << wisdom Perceptions: Science providing useable information and practical tools. Increasing capacity to adapt to climate variability. Research Funding: Public.

9. Evaluating Success of Products and Process ??? PRODUCTS: Forecasts - traditional publications - MS/PhD degrees - newsletter outreach - database of forecasts - forecast evaluation tool - “Climate in a Nutshell” (450+) - presentations to stakeholder groups (25+) - workshops (research/forecast/stakeholder) (8+) Frequent interaction, from the outset Interaction… not outreach! Getting and giving Starting where the stakeholders are Moving dialogue & action forward

10. Common across all groups Uninformed, mistaken about forecast interpretation Understand implications of “normal” vs. “unknown” forecasts Use of forecasts limited by lack of demonstrated forecast skill Stakeholder Use of Hydroclimatic Forecasts Common across many, but not all, stakeholders Have difficulty distinguishing between “good” & “bad” products Have difficulty placing forecasts in historical context

11. Common across all groups Uninformed, mistaken about forecast interpretation Understand implications of “normal” vs. “unknown” forecasts Use of forecasts limited by lack of demonstrated forecast skill Stakeholder Use of Hydroclimatic Forecasts Unique among stakeholders Relevant forecast variables, regions (location & scale), seasons, lead times, performance characteristics Role of of forecasts in decision making Technical sophistication: base probabilities, distributions, math Common across many, but not all, stakeholders Have difficulty distinguishing between “good” & “bad” products Have difficulty placing forecasts in historical context

12. Efficiency Work with hydropower agencies & other high-value clients Develop customized evaluation tools Transfer to agencies Equity Also work with stakeholders affected by changing supplies & policies Develop tools for knowledge development and diverse decision processes Requires on-going support of research products and tools Impact Work with regulatory & policy agencies Inform water supply policy via peer-reviewed science & policy analysis Forecast Assessment: CLIMAS Alternatives

13. http://hydis6.hwr.arizona.edu/ForecastEvaluationTool/ Initially for NWS CPC climate forecasts Six elements in our webtool: Exploring Forecast Progression Forecast Interpretation - Tutorials Forecast Performance Historical Context Use in Decision Making Details: Forecast Techniques, Research

14. 1. Forecast Progression

15. Unknown EC Sometimes forecasters don’t know what the chances are… EC - EQUAL CHANCES THE PROBABILITY OF THE MOST LIKELY CATEGORY CANNOT BE DETERMINED = Unknown Chances!! 63% 33% 3% “+30% Chance of Warm” Each colored contour indicates a shift in the normal chances. 33% Climatology Climatology is only a reference (1971- 2000), not a substitute forecast 2. Forecast Tutorial

16. 3. Forecast Performance Evaluation Sub-setting: Seasons, Leadtimes, Regions Criteria: Simple/Intuitive to Complex/Informative Transparency: Data behind analysis

17. 3. Forecast Performance Evaluation Sub-setting: Seasons, Leadtimes, Regions Criteria: Simple/Intuitive to Complex/Informative Transparency: Data behind analysis

18. 4. Historical Context for Forecasts Recent History | Possible Futures Requested by Fire managers… Applicable to any climate variable 20032002 Neutral Non-ENSO sequences 2004 La Nina

19. Wet Near-Dry Normal 10 years had more than 3.7 inches 10 years had less than 1.9 inches 10 years were in the middle Willcox Jan-March Total Precipitation 1930-2001 Year Precipitation (Inches) 1971-2000 Willcox Jan-March Total Precip. 1971-2000 Dry Norm Wet 0” 1.9” 3.7” 8+” Exceedance Probability 4. Historical Context for Forecasts

20. La NinaEl Nino El Nino La Nina 50% Wet 0% 30% Norm 25% 20% Dry 75% Willcox, AZ: Precipitation, JFM 4. Historical Context for Forecasts

21. Custom real-time data access, analysis, and information Value-added interpretation Multiple entry points along continuum of sophistication Opportunities and tools for increasing sophistication Knowledge development emphasis vs. decision support Data << Information << Knowledge << Wisdom Are these concepts & tools transferable and scalable? Test with new products, inter-RISA opportunities Lessons of FET for Climate Services Other issues: Accessibility, ease of use (information management, updating)

22. Ease of Use  Profile and Projects: save a history of your work on each "project", so you can return to your work any time, easily repeat past analyses using updated data. Facilitating Information Intermediaries Accessibility  Report Generation create PDF reports of your analyses for non-Internet users automatically includes legends, data sourcing, contact information, caveats, explanations sections for user-customized comments Future: Automated Updating & Additional Products: water supply forecasts, experimental climate forecasts, drought monitoring

23. Lessons Learned: Knowledge Development Tools Stakeholders Information needs, understanding, access Social Science Effective communication Natural Science Forecast skill, interpretation Transferable, scalable tools are possible! Focus on knowledge development, not just data & information.

24. Lessons Learned: Knowledge Development Tools Stakeholders Information needs, understanding, access Social Science Effective communication Natural Science Forecast skill, interpretation Computer Science Web programming Transferable, scalable tools are possible! Focus on knowledge development, not just data & information. Interactive webtools require major commitment and resources. Prototypes insufficient! Stakeholders need reliable tools, which require solid software foundation, organized development, sustainability for maintenance and expansion.

25. Climate Research Impacts Society… Unexpectedly? Have you heard comments about role of science? Which objectives are supported by your research and products? What are your success metrics? Are there synergies in webtool development?