MRC-MDBC STRATEGIC LIAISON PROGRAM BASIN DEVELOPMENT PLANNING TRAINING MODULE 3 SCENARIO-BASED PLANNING for the MEKONG BASIN Napakuang, Lao PDR 8-11 December 2003 Formulating Indicators for Assessing Scenarios

A process for this part of basin planning DEVELOPING INDICATORS A process for this part of basin planning

Summary of steps needed to prepare for assessment of scenarios DEVELOPING INDICATORS Summary of steps needed to prepare for assessment of scenarios Derive BDP Development Objectives (derived from 95 Agreement) Select Assessment Criteria (Tests of how well scenarios will meet objectives) Select Indicators (Measurements to be used in assessment tests)

Step 1 - Defining development objectives DEVELOPING INDICATORS Step 1 - Defining development objectives First and foremost: Objectives must be derived from the Basin Agreement. This is the mandate of the Mekong River Commission. But the objectives in the Agreement are broad and not easy to quantify. An agreed set of more precise and/or detailed BDP development objectives needs to be developed through consultative processes. These derived development objectives should cover all aspects of each of the objectives (whether explicit or implicit) of the Agreement.

Example of BDP Development Objective DEVELOPING INDICATORS Example of BDP Development Objective Start point: Chap III of Agreement - Objectives and Principles of Cooperation. For example: Article 3. Protection of the Environment and Ecological Balance Suggested BDP Development Objectives derived from Article 3 could include: “To achieve and maintain acceptable water quality in the Mekong River” “To maintain fisheries production.” and so on...

Step 2 - Select Assessment Criteria DEVELOPING INDICATORS Step 2 - Select Assessment Criteria Assessment criteria are the tests (for achievement of objectives) to be applied to the anticipated outcome of development scenarios. At least one such criterion (or test) is needed per BDP Development Objective. The tests (criteria) should preferably be capable of being measured in a quantifiable manner and linked to water flow regime. Selection of the criteria should be done in a consultative way - especially with relevant experts (MRC programs + expert line agencies)

Example of Assessment Criteria DEVELOPING INDICATORS Example of Assessment Criteria For the objective “To maintain fisheries production” some criteria could be: Changes in dry season flow regime Changes in wet season flood flows Changes in selected flooded areas Changes in connectivity for fish migration These are examples of factors that support or affect fisheries production. Note that they have all been related back to flow. Expert advice is needed for this.

Step 3 - Selection of Indicators DEVELOPING INDICATORS Step 3 - Selection of Indicators Indicators are the specific measurements to be used as part of the assessment tests. At least one indicator is needed per criterion. To the maximum extent possible the indicators should be capable of being generated by the DSF - either as direct numerical output or as spatial data for interpretation. Again, the indicators should be selected in a consultative way - especially with relevant experts (MRC programs + expert line agencies) - and with modellers.

Examples of Indicators DEVELOPING INDICATORS Examples of Indicators For the fisheries maintenance assessment criterion “Changes in wet season flood flows”, possible indicators are: Annual flood frequency (of selected flood sizes) Rate of rise of wet season flood Rate of fall of wet season flood Note these are all numerical indicators and can be computed through DSF simulations.

Assessment - It is done by TEAMWORK DEVELOPING INDICATORS Assessment - It is done by TEAMWORK Derive BDP Development Objectives (derived from 95 Agreement) Select Assessment Criteria (Tests of how well scenarios will meet objectives) Select Indicators (Measurements to be used in assessment tests) BDP Team (MRCS + NMCs) BDP Team + Experts BDP Team + Experts + Modellers

Some ideas for shortcuts DEVELOPING INDICATORS Some ideas for shortcuts MRC has done a lot of work already. Capitalise on it! For example: Transboundary issues are thoroughly identified and listed. Good start point to derive indicators. WUP has already prepared a comprehensive list of suggested indicators. See DFR 640 Technical Reference Report: Impact Analysis Tools. Start with these. Vital to take a structured approach, with all steps documented.

Building on work already done DEVELOPING INDICATORS Building on work already done 1995 Agreement Transb’d’y Issues WUP suggested indicators Anything else you can find Agreed Indicators

A Structured Format DEVELOPING INDICATORS A Structured Format Any format will do. Important thing is to have one.

Finding out if the indicators work DEVELOPING INDICATORS Finding out if the indicators work The only method is to actually apply the indicators to some scenario outcomes generated by the DSF, or otherwise. Revise them and repeat. An adaptive, iterative process. Ask the questions: Do we have the minimum set of indicators? Are any redundant? Are any not sensitive enough to show variations? - even when we ‘know’ there must be change. Have we missed any vital indicators? (Go back to experts with results and check that they are satisfied their issues are being assessed.)

MRC-MDBC STRATEGIC LIAISON PROGRAM BASIN DEVELOPMENT PLANNING TRAINING MODULE 3 SCENARIO-BASED PLANNING for the MEKONG BASIN Napakuang, Lao PDR 8-11 December 2003 Multi-criteria Analysis for Ranking Development Scenarios

Scenario-Based Planning - How do we do this bit? MULTI-CRITERIA ANALYSIS Scenario-Based Planning - How do we do this bit?

The need for MCA MULTI-CRITERIA ANALYSIS The need for MCA The ideal situation is to be able to measure the outcomes of alternative developments in the same units such as dollars. Comparison is then a simple numerical exercise. And in such a situation it doe not matter that for any particular scenario it may meet some criteria extremely well, and others extremely poorly. The alternative developments can simply be optimised to maximise net dollar outcomes. However, work already done on outcome indicators shows that this ideal will rarely occur in practice.

Example of simulation results MULTI-CRITERIA ANALYSIS Example of simulation results Which scenario is best?

Some questions MULTI-CRITERIA ANALYSIS Some questions Are all criteria of equal importance? Who says so? Who are the decision makers? Should the evaluation be passed over to the community or kept for ‘professionals’ to do? Some indicators do not show much change between scenarios - should they be dropped from the analysis? Is there some other indicator(s) that might be of more help in the scenario evaluation? etc

Some answers MULTI-CRITERIA ANALYSIS Some answers Experience shows that stakeholders need to be involved if the evaluation is to have credibility. (At basin level this can be high level and broad) A structured process is the only successful approach to dealing with complexity. At the same time, every effort should be made to have the absolute minimum complexity. There are a number of recognised methodologies and proprietary MCA software available. However, it is important to remember that the decisions will be socio-political, not technical.

Some Structured Approaches MULTI-CRITERIA ANALYSIS Some Structured Approaches Criteria weighting. This requires joint discussions by stakeholders and ‘experts’ to agree on weighting. Numerical techniques can then be used to score the scenarios. DELPHI - a way of combining expert/informed opinions (not especially useful in scenario evaluation, but can help develop weightings) AHP (Analytical Hierarchy Process) - one of many pair-wise preference processes - plenty of software available. etc

Which is the best approach? MULTI-CRITERIA ANALYSIS Which is the best approach? In basin planning, MDBC experience is that the outcomes of any unclear or partially invisible process will frequently be rejected by the stakeholders (and community). MDBC and its member States tend not to use MCA software - both for the above reason, and because none seems to work very well for the complex systems of basin natural resources. (Research into MCA continues however.) Instead, the common approach is to use stakeholder reference panels and an independent facilitator to work towards consensus.

Evaluating Development Choices Scenarios, Indicators, MCA and all that. Thank you GOOD LUCK! Questions?