1 Choosing from alternative approaches for assessing management effectiveness Prepared by Graeme Worboys *1, Dan Salzer *2, Marc Hockings *3 and Richard Margoluis *4 *1: Griffith University, Gold Coast, Queensland Australia: IUCN WCPA *2: The Nature Conservancy, Oregon, USA: Conservation Measures Partnership *3:The University of Queensland, Gatton, Queensland Australia: IUCN WCPA *4: Foundations of Success, Maryland, USA: Conservation Measures Partnership

2 This paper….. Discusses management effectiveness evaluation needs and methods Discusses evaluation needs of protected area stakeholders and some methods available Describes some attributes of methods Describes opportunities for further research Introduces the work of the Conservation Measures Partnership to this topic

3 Benefits of management effectiveness evaluations Effectiveness evaluation can: Improve planning Demonstrate accountability Provide a pivotal role in adaptive management But: in undertaking evaluation, it is critical that: It provides the right information to the right people at the right time The evaluation methodology is the right one for the intended purpose

4 What are some protected area management effectiveness evaluation needs? Design Needs Evaluating whether the design of the protected area site or system is appropriate to the values it seeks to maintain. Management Process Needs Evaluating whether the management systems and processes are adequate and appropriate for the protected area site or the protected area system Conservation Outcome Needs Is the site or system effective in maintaining biodiversity, abating threats and achieving other management objectives?

5 What triggers an effectiveness evaluation? Political requirements Legal requirements Managerial requirements Environmental requirements Social considerations Economic/financial requirements

6 Current status of management effectiveness approaches Many different approaches used by different agencies/organizations Little collaboration on what approaches to use Lack of knowledge on what works, what doesn’t No consensus on framework on which to compare and contrast different approaches

7 Harmonization of approaches can help protected area management Enhances the comparability of data across systems Improves communication through a “common” language Reinforces a core group of attributes to use when evaluating management effectiveness

8 What are some attributes for PA management effectiveness evaluation approaches? Principal evaluation subject: Protected Area/System Design Management processes Conservation condition/status of natural and cultural heritage Mixed subjects Internal or External (to a management organization) Low cost or High cost

9 What are some attributes for PA management effectiveness evaluation approaches? Duration; frequency; time frame for evaluations (e.g., short duration, single evaluation event) Focus on specific element(s) of an identified Evaluation Framework (e.g., WCPA framework) ContextPlanningInputProcessOutputOutcome

10 What are some attributes for PA management effectiveness evaluation approaches? Type of Data analysis/compilation Scorecards based on perception-based monitoring Analysis derived from protected area monitoring program data A diversity of subject areas evaluated within elements Descriptions of how the evaluation works

11 If we look at some evaluation approaches, what are their broad characteristics? A preliminary appraisal of 30 evaluation approaches illustrated that: There were a large number of attributes (more than 300) External organizations played a key role (16 were external) The majority of evaluations were short in duration (22) The majority used questionnaires (17) Outcomes (23), inputs (18) and processes (16) were important WCPA elements

12 If we look at the 30 evaluation approaches, what are the most common attributes that have been used? Boundaries (5) Natural heritage condition (6) Supportive Legislation (10) Plan of Management existence (14) Key threats (issue identification) (4) Sufficient staff (12), $ (8), equipment resources (4) Education (4)

13 If we just look at evaluations conducted by the Protected Area agencies(9) within the 30 approaches reviewed, we observe the following characteristics: Many were: Site based (5) Long term (7) Outcome focused (9) Included Qualitative and Quantitative data (7)

14 Important questions to answer before selecting the effectiveness evaluation approach: What is needed? Who needs it? Why is it needed? Who does it? When is it needed by? At what cost?

15 We suggest you obtain guidance from existing evaluation approaches that have similar attributes to your evaluation needs: Short term; low cost; questionnaire based; mixed elements; local area; external evaluator Long term; high cost; internal; output/outcome focused; on site survey work; local area. Eg: Some generic attributes that you may need: Example of an existing evaluation approach with similar attributes (for your background research) Tasmania World Heritage Area Monitoring and Evaluation Framework ( Jones 2000) WWF/World Bank Forest Alliance: Reporting Progress at Protected Area Sites (Stolton et al 2003)

16 Choosing between alternative evaluation approaches……. Needs a clear understanding of what is required Needs an understanding of the alternative approaches available Needs an awareness that there are pre- existing methods that can help Needs further research to facilitate the process between needs and approaches

17 Opportunities for research and action….. Develop an approach to clearly identifying PA management Effectiveness Evaluation needs Determine generic attributes of Effectiveness Evaluation methods Clarify conditions under which Effectiveness Evaluation methods work to address specific needs. Develop a decision tree

18 Acknowledgements: Appreciation is extended to Dr Jamison (Jamie) Ervin for the opportunity to use unpublished material for this presentation.

19 Harmonizing Management Effectiveness Approaches The Conservation Measures Partnership Dan Salzer, TNC World Parks Congress Durban, South Africa September 12, 2003

20 Core Members African Wildlife Foundation Conservation International The Nature Conservancy Wildlife Conservation Society World Wildlife Fund/World Wide Fund for Nature Collaborating Members Enterprise Works Worldwide Foundations of Success (CMP – facilitator) World Commission on Protected Areas Cambridge Conservation Forum Who is the Conservation Measures Partnership (CMP)?

21 Why did CMP form? Capitalize on collective experience Reduce duplication of effort Promote rapid diffusion of successful tools and strategies Adopt best-practices from ongoing projects Make conservation more efficient

22 CMP’s Vision Transform the practice of conservation by developing, testing and sharing tools to credibly assess and improve the effectiveness of conservation investments

23 Focus of CMP’s current work Consensus set of standards for the practice of conservation Conservation “Rosetta Stone” Peer-review conservation audit standards Activity-based cost accounting Global status indicators Management effectiveness indicators

24 CMP’s role in comparing alternative management effectiveness approaches Members contribute alternative approaches to include in comparisons Develop a CMP conservation audit process derived from our agreed upon conservation practice standards Help with analysis of alternatives & guidance for selecting alternative approaches

25 For more information ….