Methodology Overview: Developing Marine Ecological Scorecards Commission for Environmental Cooperation

Overview  Identify key reporting themes:  Water  Habitats  Living Resources  Use key questions to report on each theme  12 questions in total  Questions across a range of ecological factors  Consider state and trend  Answers are “predetermined”  Gradations help to ensure consistency and remove discretion

12 Questions Water  To what extent do human activities influence water quality and inputs and how are they changing?  To what extents are altered nutrient loads affecting ecosystem health and how are they changing?  To what extent do water conditions pose risks to human health and how are they changing?

12 questions Habitats  To what extent do human activities influence habitat extent and quality, and how are they changing?  To what extent do contaminants in habitats affect living resources or water quality, and how are they changing?  To what extent does habitat alteration, including the extent and distribution of major habitat types, affect ecosystem health, and how is it changing?

12 questions Living Resources  To what extent do human activities influence living resource quality and how are they changing?  What is the status of biodiversity & how is it changing?  What is the status of extracted species & how is it changing?  What is the status and condition of key species & how is it changing?  What is the status and condition of species at risk and how is it changing?  What is the status of alien species & how is it changing?

Overview continued  Draw on scientific evidence, monitoring information, community and traditional knowledge (best available knowledge) to address questions  Questions answered through grid  Evidence required  Gaps to be documented  Grid converted to scores where answers possible  Apply a consistent baseline: the most pristine condition for a site that can be identified and supported by scientific or other evidence  Focus on the last five years  Draw on historical data to project future

Capture knowledge  Monitoring data is key, but understanding what it means is also important  Need a systematic way to capture knowledge and interpret best available information  Can’t make an assessment where knowledge is insufficient and confidence is low

Key Challenges  Overcoming the “perfect knowledge” syndrome  The exercise is evidence-based  Those who know the evidence are the only ones able to make statements on the basis of best available information  Overcoming the “perfect data” syndrome  If there are clear gaps, and confidence in making statements is low, this should be the result  Data is not perfect, but is there enough to inform an answer to the question?

What exactly is the method?  Go through all 12 questions  Introduce and understand the question  Share knowledge if you have any  Determine state, with a description of findings  Determine trend, with a description of findings  Document evidence  Convert result to scores

What is a state? What is state?

State  State is the relative condition of a variable in time  With respect to the scorecard, it is selected based on graded, predetermined values for each question/variable

Example  Theme: Water/Nutrient health  Question: To what extents are altered nutrient loads affecting ecosystem health and how are they changing?

Factors to consider  What is the evidence?  What does the evidence suggest?

What is a trend?

Trend  Trend is the direction and speed with which a variable is changing in time  With respect to the scorecard, it is selected based on graded, predetermined values for each question/variable

Example  Theme: Environmental Quality  Question: To what extents are altered nutrient loads affecting ecosystem health and how are they changing?

Rapidly improving Conditions appear to be changing at a rate that will lead to an improved state within 5 years ImprovingConditions are improving Stable Within the bounds of normal variation, no consistent changes are anticipated from either anthropogenic or other sources DiminishingConditions are diminishing Rapidly diminishing Conditions appear to be changing at a rate that will lead to a diminished state within 5 years ? Undetermined There is insufficient information to establish a basis for the trend or data highly variable and trends can’t be distinguished

Factors to consider  Needs to be consistently scoped  Baseline 5 years, next trend assessment  If possible, distinguish between anthropogenic and natural sources of change in description –Anecdotes –Describe links and causes –Promote “self-directed inquiry” - as clear as needed to ask questions

How should results/evidence be tracked?

Results?  Documentation  Supporting data, reports  Supporting monitoring programs/protocols  Other?  Scoring results  Scorecard  Opinions  Description of findings

How is this done best?  Need input and reactions from all participants  Sharing evidence and methods is important to credibility of initiative and to developing an inventory of monitoring protocols and approaches

NAMPAN portal  Used to collect and share evidence  Used to collect and share scorecards  Visit or for more information

Other tools  Notes at the meeting  Collect and manage hardcopy  Reference/document management system

How to capture expert opinion?

Implement in workshop setting  Face-to-face is key  Questions are shared beforehand  Clear communication of approach  Preparation of evidence in advance of workshop

Aim for Consensus  Consensus is reached when everyone can live with a single collective answer to a question (state and trend)  Where consensus is not possible, it will be best to apply a process that ensures any dissent is tracked and a final score is reached  One solution is to use the Delphi method

Use multiple voting  Apply to controversial questions (consensus not possible)  Be sure question is clear  Share evidence  Allow voting on  State  Trend  Discuss results, revisit possibility of consensus

Questions?