# 1 Environmental Management: Risk Assessment, Multi-Criteria Decision Analysis, and Adaptive Management Techniques for Addressing Model Uncertainty and Reliability Igor Linkov & Pat Deliman US Army Engineer Research and Development Center Environmental Laboratory Phone: Phone:

# 2 Adaptive Risk-Based Planning

# 3 The Government Performance and Results Act (GPRA) –“provide for the establishment of strategic planning and performance measurement in the Federal Government” (OMB, 1993). –embodied a push for better planning, greater accountability, and straightforward performance evaluation in government. OMB Program Assessment Rating Tool (PART) –rates the performance of a program through a series of yes/no questions –Scores on four primary areas: program purpose & design, strategic planning, management, and results & accountability. –performance metrics used by the program are essential to PART. Agencies need to relate response to the mission goals and track the progress and performance Risk-based Planning: Top-down Drivers

# 4 For stakeholders, the root issue is: fear of becoming a victim to (uncompensated) loss –Layperson: Risk = Hazard x Perception –Expert: Risk = Hazard x Exposure x Consequence Core concerns tend to be: trust, control, process, information and timing Local communities need to understand actions by the Agencies and like to see their values accounted for Risk-based Planning: Bottom-up Drivers

# 5 What are the flood and storm threats to coastal Louisiana/Mississippi? What do we have to lose and how vulnerable are we? What should be our planning timeframe? What can be done to reduce risks for our planning timeframe? How is the public, agencies, and others involved? For taking action at varying scales, what is the cost and risk reduction? For taking action at varying scales, what are the adverse impacts to significant resources? How do you decide what actions to take? How much information is necessary to make decisions? Coastal Louisiana Restoration Planning: What Questions are We Trying to Answer? After E. Russo Risk and/or Uncertainty elements are present almost in every question

# 6 Information and Planning/Decision Cycles Information gathering and decision-making are two separate cycles in environmental management Modeling/Software/GIS… Technology-based Fix in Information Age Integration – Need for Revolutionary Changes After Roman, 1996

# 7 Main Points Risks and benefits associated with alternative management strategies are difficult to quantify. Model, Parameters and Scenario uncertainty and variability associated with predicting efficiency of management options as well as stakeholder value judgment are important to consider Challenges of risk assessment and planning for situations with a limited knowledge base and high uncertainty and variability require coupling traditional risk assessment and planning with multi-criteria decision analysis (MCDA) to support regulatory decision making

# 8 Risk-based Planning: Top-down and Bottom-up Drivers Risk and Uncertainty –Traditional Way of Dealing with Uncertainty –Need for Formal Decision Analysis MCDA -Summary Example: –MCDA Use to Select Performance Metrics for Oil Spill Response Planning –RA/MCDA Application for Sediment Management Conclusion References Presentation - Overview

# 9 AD HOC Process Quantitative?Qualitative? Current Decision-Making Processes Decision-Maker(s) Include/Exclude? Detailed/Vague? Certain/Uncertain? Consensus/Fragmented? Iterative? Rigid/unstructured? Risk Analysis Modeling / Monitoring Stakeholders’ Opinion Cost or Benefits Tools Challenge: Multiple & Uncertain Criteria

# 10 Challenges to Complex Decision-making “Humans are quite bad at making complex, unaided decisions” (Slovic et al., 1977). Individuals respond to complex challenges by using intuition and/or personal experience to find the easiest solution. At best, groups can do about as well as a well-informed individuals if the group has some natural systems thinkers within it. Groups can devolve into entrenched positions resistant to compromise “There is a temptation to think that honesty and common sense will suffice” (IWR-Drought Study p.vi)

# 11 Model Uncertainty –Differences in model structure resulting from:  model objectives  computational capabilities  data availability  knowledge and technical expertise of the group –Can be addressed by  considering alternative model structures  weighting and combining models  Eliciting expert judgment Problem: Model Uncertainty Mechanistic models for environmental risk assessment are very uncertain and expert judgment is required

# 12 Parameter Uncertainty –Uncertainty and variability in model parameters resulting from  data availability  expert judgment  empirical distributions –Can be addressed by  Probabilistic Simulations (Monte- Carlo)  Analytical techniques (uncertainty propagation)  Expert estimates Problem: Parameter Uncertainty Many parameters and factors important for risk assessment are not well known, reported ranges are large and often unquantifiable

# 13 Subjective Interpretation of the Problem at Hand What is the relative influence of modeler perception on model predictions? Problem: “Modeler/Scenario Uncertainty”

# 14 Subjective Interpretation of the Problem at Hand What is the relative influence of modeler perception on model predictions? Problem: “Modeler/Scenario Uncertainty”

# 15 Multi-Criteria Decision Analysis and Tools Multi-Criteria Decision Analysis (MCDA) methods: –Evolved as a response to the observed inability of people to effectively analyze multiple streams of dissimilar information –Many different MCDA approaches based on different theoretical foundations (or combinations) MCDA methods provide a means of integrating various inputs with stakeholder/technical expert values MCDA methods provide a means of communicating model/monitoring outputs for regulation, planning and stakeholder understanding Risk-based MCDA offers an approach for organizing and integrating varied types of information to perform rankings and to better inform decisions

# 16 Risk Analysis Modeling / Monitoring Stakeholders’ Opinion Cost Decision Analytical Frameworks Agency-relevant/Stakeholder-selected Currently available software Variety of structuring techniques Iteration/reflection encouraged Identify areas for discussion/compromise Decision-Maker(s) Sharing Data,Concepts and Opinions Evolving Decision-Making Processes Tool Integration Decision Integration

# 17 Simplified Decision Matrix PlanCostEco Health Human Health A B C15010 D

# 18 Criteria 1Criteria 2Criteria 3Criteria 4 Alt. 1 Monitoring Results Stakeholder Preference Economic CostNon-monetary benefit Alt. 2 Monitoring Results Stakeholder Preference Economic CostNon-monetary benefit Alt. 3 Monitoring Results Stakeholder Preference Economic CostNon-monetary benefit Alt. 4 Monitoring Results Stakeholder Preference Economic CostNon-monetary benefit How to interpret these results? How to combine these criteria? How to compare these alternatives? Example Decision Matrix

# 19 Decision Analysis Methods and Tools

# 20 Problems Alternatives Criteria Weights Synthesis Decision Decision Matrix Evaluation RA MCDA Feeds RA MCDA RA Feeds MCDA Adaptive Management Linking RA, AM and MCDA

# 21 Risk Informed Decision Framework: Restoration Planning for Coastal LA and MS

# 22 Example 1: Performance Metrics for Oil Spills Response Planning* Framework for selecting metrics –Multiple stakeholders  Agencies (federal, state, local)  Responsible parties  Local residents  NGOs (business, environmental, etc.) –Integrate deliberation and science to link goals, objectives, metrics, and measures –Compatible with existing planning, decision- making, and assessment processes –Completed as part of preparedness planning *based on Linkov, Seager, Figueira, Tkachuk, Levchenko, Trevonnen (2007), funding provided by NOAA through CRRC, UNH.

# 23 Examples (oil spills response) Endpoint –Miles of shoreline impacted or cleaned vs. areas protected (e.g., by redirecting or containing oil). –Number of fish, birds or other wildlife killed or injured (per unit search area). –Number of “appropriate” (not exotics) animals rehabilitated and released. –Degree of change to beaches and sandbars from clean-up actions. –Types of animals and vegetation present after spill cleanup. Process –Did getting required permits delay response action? –Rate of bird handling at rehabilitation center. –Time to deploy booming and double-booming in sensitive areas. Resource –Amount of oil containment boom deployed. –Number of volunteers deployed. –Number sandbags deployed.

# 24 Challenges Challenges to defining “good” metrics –What is most relevant may be very difficult to measure. –Metrics may be indirect measures of what people really care about. –What is easy to measure may not be relevant to what people care about. –There can be disagreements about thresholds to differentiate “good” versus “bad.” –Accuracy and reliability of data recording is a challenge. –Paucity of baseline data. –Timing of measurement can affect assessment of performance. –Difficult to communicate to the public – or at least that is managers’ perception. –Weightings and aggregation.

# 25 Characteristics of Good Measures –scientifically verifiable –cost-effective –easy to communicate to a wide audience –relevant to what people care about –decision or action relevant –credible –scalable over an appropriate time period and geographic region –sensitive to change

# 26 Oil Spill Response Metrics Taxonomy by Type of Information Measured EconomicThermodynamicEnvironmentalEcologicalHuman HealthSocio-Political Clean up costs. Property & eco- system damage. Ecosystem damages or lost services. Lost marginal profits. Volunteer opportunity costs. Volume of oil spilled, recovered, des- troyed, or contained. Slick area and thickness. Mass of clean up wastes generated. Volume cleaning agent deployed. Chemical concen- tration & toxicity. Habitat suitability, e.g., acres shellfish bed. Length of oiled shoreline. Degradation rates. Residual Risk Wildlife deaths or populations, fecundity and recovery rates. Biodiversity. Catch sizes. Plantings, seedings. Habitat Suitability Threatened pop- ulation Quality-adjusted-life- years (QALYS) Disability-adjusted- life-years (DALYS) Life-expectancy Injuries Newspaper column inches, minutes TV coverage, web hits. Volunteerism. Public meeting attendance. Critical sites pro- tected Historic sites pro- tected

# 27 Assessment Criteria

# 28 Metric Assessment by Criteria

# 29 Criteria Weight

# 30 Rank Acceptability Analysis

# 31 Pairwise Metrics Domination

# 32 Sensitivity Analysis COST Most Important ENVIRONMENTAL Relevance Most Important

# 33 Example 2: NY/NJ Harbor

# 34 Issues Harbor among most polluted in U.S. >10 6 yd 3 fail regional criteria for ocean disposal Existing disposal site closed 1 Sep. 97 Proposed deepening Example: NY/NJ Harbor

# 35 Example: Decision Methodology Proof of Concept Study Objectives –Integrate comparative risk assessment results with cost and stakeholder decision criteria –Use decision criteria/performance measures from published data and proposed costs –Test decision tools, methodology and results Set contaminated sediment management options Set decision criteria/performance measures Software - Criterium DecisionPlus Stakeholder Values / Expert Surveys –USACE/EPA dredged material managers meetings (New Orleans 2004) –SRA/USACE/Contaminated Sediments Meeting (Palm Beach 2004)

# 36 Conceptual Illustration of Disposal Alternatives Landfill Upland CDF Nearshore CDF CAD Pit No-Action Island CDF Water Line In-place Sediment Dredged Material Effluent Manufactured Liner Dike Wall Cap Standard Landfill Waste KEY: In-place Soil Kane Driscoll, S.B., W.T. Wickwire, J.J. Cura, D.J. Vorhees, C.L. Butler, D.W. Moore, T.S. Bridges A comparative screening- level ecological and human health risk assessment for dredged material management alternatives in New York/New Jersey Harbor. International Journal of Human and Ecological Risk Assessment 8: Manufactured Soil Cement Lock

# 37 $ / Cubic Yard Contaminated Sediment Management Decision Impacted Area / Capacity Cost Ecological Health Human Health Public Acceptance # of complete ecological exposure pathways Largest Ecological Hazard Quotient (HQ) calculated for any one pathway # of complete human exposure pathways Largest Cancer Risk calculated for any one pathway Estimated Fish COC Concentration / Hazard Level Decision Criteria: NY/NJ Harbor Source: Kane Driscoll et al. (2002). Source: NY/NJ Dredged Material Management Plan and Expert Opinion

# 38 NY/NJ Harbor in Criterium DecisionPlus GoalCriteriaSub-Criteria

# 39 NY/NJ Harbor in Criterium DecisionPlus GoalCriteriaAlternatives Hierarchy Rating Technique: Weights Alternatives Rating Technique: SMART with linear value functions Sub-Criteria

# 40 Criteria Levels for Each DM Alternative CostPublic Acceptability Ecological Risk Human Health Risk DM Alternatives ($/CY) Impacted Area/Capacity (acres / MCY) Ecological Exposure Pathways Magnitude of Ecological HQ Human Exposure Pathways Magnitude of Maximum Cancer Risk Estimated Fish COC / Risk Level CAD E -528 Island CDF E -592 Near-shore CDF E -538 Upland CDF E -538 Landfill E –40 No Action E –4220 Cement-Lock E -50 Manufactured Soil E –30 Blue Text: Most Acceptable Value Red Text: Least Acceptable Value

# 41 USACE/EPA DM Managers Meeting: NY/NJ Harbor Weighting Form Attribute Swung from Worst to best Consequence to compareRank (1-9) Rate (0- 100) Benchmark: Worst case on everything Impacted Area/Capacity of Facility = 6500 (acres/ 10 6 cubic yards) Magnitude of Ecological Hazard Quotient – Maximum Exposure = 5200 Number of Complete Ecological Exposure Pathways = 41 Number of Complete Human Exposure Pathways = 25 Magnitude of Maximum Cancer Probability (Non-barge worker) = 1* Ratio of Estimated Concentration of COCs in Fish to Risk-Based Concentrations = 220 Cost = $/CY 90 Impacted Area/Capacity of Facility Change from 6500 (acres/ 10 6 cubic yards) to 0 (acres/ 10 6 cubic yards) Magnitude of Ecological Hazard Quotient –Maximum Exposure Change from 5200 to 0 Number of Complete Ecological Exposure Pathways Change from 41 to 0 Number of Complete Human Exposure Pathways Change from 25 to 12 Magnitude of Maximum Cancer Probability (Non- barge worker) Change from 1* to * Ratio of Estimated Concentration of COCs in Fish to Risk-Based Concentrations Change from 220 to 0 CostChange from (54-75 $/CY) to (0-5 $/CY)

# 42 USACE/EPA Survey Results: Criteria Weights (%) EPAUSACESRA Public Acceptability Ecological Health Human Health Cost

# 43 Criteria Contributions to Decision Score

# 44 Solution vs. MCDA Method: Does it Matter? Software & Method Alternatives (NY Case Study) CADIsland CDF Near-Sh CDF Upland CDF LandflCemnt Lock Manuf Soil NoAct ExpertChoice, AHP DecisionLab, PROMETHEE CritDecPlus, SMART

# 45 Risk Assessment, Adaptive Management and MCDA Implementation Framework

# 46 Key Take-Away Points Risk-Based MCDA offers planners: Reproducible and defensible management of complex multiple criteria A means to define and gauge what is important Balancing of expert opinion and stakeholder values Better responses, better reporting with opportunities to more clearly get it “out on the table”

# 47 MCDA workshop sites with posted lectures–    Papers –Yatsalo, B., Kiker, G., Kim, J., Bridges, T., Seager, T., Gardner, K., Satterstrom, K., Linkov, I Application of Multi-Criteria Decision Analysis Tools for management of contaminated Sediments. Integrated Environmental Assessment and Management. –Seager, T., Satterstrom, K., Linkov, I., Tuler, S., Kay, R Typological Review of Environmental Performance Metrics (with Illustrative Examples for Oil Spill Response). Integrated Environmental Assessment and Management. –Linkov, I., Satterstrom, K., Kiker, G., Bridges, T., Benjamin, S., Belluck, D. (2006). From Optimization to Adaptation: Shifting Paradigms in Environmental Management and Their Application to Remedial Decisions. Integrated Environmental Assessment & Management 2: –Linkov, I., Satterstrom, K., Seager, T.P., Kiker, G., Bridges, T., D. Belluck, A. Meyer (2006). "Multi-Criteria Decision Analysis: Comprehensive Decision Analysis Tool for Risk Management of Contaminated Sediments". Risk Analysis 26: –Linkov, I., Satterstrom, K., Kiker, Batchelor, C., G., Bridges, T.(2006). From Comparative Risk Assessment to Multi-Criteria Decision Analysis and Adaptive Management: Recent Developments and Applications. Environment International 32: References