1. 1 Collaborative Systemwide Monitoring and Evaluation Project (CSMEP) Presentation to PNAMP Steering Committee August 21, 2007

2. 2 Presentation Outline 1. CSMEP Overview 2. Examples of CSMEP Work Products 2.1 Status and Trend M&E 2.2 Habitat M&E 2.3 Hydro M&E 2.4 Hatchery M&E 2.5 Harvest M&E 2.6 Integration of M&E 3. Summary & Future Plans

3. 3 CSMEP is an M&E Coordination Forum for Fish and Wildlife Agencies and Tribes A forum for Federal, State Agencies and Tribes to collaboratively develop a regional M&E program, consistent with: –2000 FCRPS BiOp –Fish and Wildlife Program –Subbasin planning –Recovery Planning Vision: improve the quality, consistency, and focus of fish population and habitat data to answer key M&E questions relevant to major decisions in the Columbia Basin

4. 4 Participants and Approach CBFWA ESSA Technologies Ltd. State Agencies IDFG ODFW WDFW Federal Agencies NOAA USFWS EPA DFO Tribal Agencies CRITFC Nez Perce Tribe Colville Tribes Yakama Nation Umatilla Tribes Consultants Eco Logical Research Quantitative Consultants PER WEST

5. 5 2.1 Status and Trends M&E Purpose: Assess viability of the population and overall management scheme. 4H ImpactsActionsMonitoring Harvest√ Hydro√ Habitat√ Hatchery√ Status and Trends will tell you what the population is doing but not why.

6. 6 CSMEP Pilot for Snake Basin; learn from this and extend to other regions

7. 7 Status and Trends: a Management Example Priority Question: Are Snake River spring/summer Chinook salmon viable using IC-TRT guidelines? Related Decision: Has there been sufficient improvement in the population status of Snake River spring/summer Chinook salmon to justify delisting and allow removal of ESA restrictions?

8. 8 How likely is the correct decision with different levels of monitoring intensity? Data Needs: Abundance Productivity Spatial structure Diversity We used a model to test the ability of monitoring programs to correctly assess spring/summer Chinook salmon population viability in the Snake River ESU using a simulated abundance dataset. Technical Recovery Team viability criteria Viability Status: Not Viable Maintained Viable Highly Viable

9. 9

10. 10

11. 11

12. 12

13. 13 Status quo 55% correct A single run of the simulation Medium 87% correct High 94% correct

14. 14 Objectives by Alternatives Matrix: Status & Trends Designs Status and Trends Subgroup

15. 15 Results and Conclusions for Status & Trends M&E TRT rule set is conservative, so high uncertainty results in conservative viability assessments. Weakness of status quo monitoring design for Snake Pilot: – lack of spatial structure information. – Lack of an abundance estimate in the non-index areas for populations without weirs. – One MPG lacks a weir. The medium design cost less than the status quo, yet performs better (for this question). Measurement error may be less than year to year variability, causing small difference between medium and high designs’ reliability

16. 16 2.2 Habitat Effectiveness Monitoring Lemhi River example The task: Apply CSMEP’s adapted DQO process to a habitat effectiveness monitoring example The Lemhi Habitat Conservation Plan (HCP) Lemhi Basin: heavily impacted with agriculture diversions and consequent passage issues 10-17 tributary reconnection projects planned under Lemhi HCP Phased in over the next 35 years

17. 17 From Questions to Designs To what degree have these actions affected fish populations in the Lemhi Basin? Developed a general “Question Clarification Process” to make M&E designs rigorous Monitored effects: salmon distribution, survival, productivity What are the mechanistic connections between recovery actions, key habitat attributes and fish responses?

18. 18 Information collected with Designs DesignPerformance Measures Status QuoOK for current status and trends data, very limited for habitat effectiveness LowFish abundance, juvenile survival, some fish distribution and habitat MediumLow design + lots of habitat and fish distribution information HighMedium design + juvenile/adult movement, more precise adult returns

19. 19 Performance Measures Considered DataPerformance MeasureLowMidHigh Redd CountsAdult Abundance  Juvenile Counts @ trapsJuvenile Emigrant Abundance  Parr & Smolt Tag detectionsParr-Smolt Survival  Snorkel counts (targeted)Juvenile Distribution  Habitat survey (presence)Effect of Actions on Habitat  Snorkel counts (extensive)Parr Density & Distribution  Habitat surveyCovariates for fish population response  Adult detection (& tagging) @ Weirs Adult Returns & Distribution  PIT detections at treatment/control sites Fish distribution Juvenile movement  Carcass surveysPrespawn mortality & Adult distribution 

20. 20 DesignTop-DownBottom-Up Status Quo $125,000 Low$323,000$354,000 Mid$377,000$493,400 High$580,000$643,600 Top-Down = based on per project costs and contracting history Bottom-up = based on cost per unit time per person multiplied by the sample sizes identified in the plans. Costs of CSMEP Designs

21. 21 Habitat Effectiveness M&E Results & Conclusions Identifying mechanistic effects requires more intense M&E Effectiveness monitoring may fulfill requirements of status and trends monitoring A one-size-fits-all template for habitat effectiveness monitoring is not workable Aspects of the design template for the Lemhi River habitat effectiveness monitoring project may be transferable to other systems.

22. 22 2.3 Hydro Action Effectiveness Hydro decisions occur at multiple scales: – Operations at individual projects (spill, bypass, RSW) – Overall operations (within season transport, performance measure compliance) – Longer term hydro decisions (flow, transportation, system configuration) – Adequacy of hydrosystem operations for population recovery

23. 23 CSMEP Hydro M&E Analyses of Survival Rates (SARs, TIRs, Passage survival) Spatial Scales Examined Population (Tributary) Major Population Group Snake River Aggregate Time Scales Examined 5-10 year Average XXX Annual Estimate X X Seasonal Estimate X

24. 24 Alternative Monitoring Designs for Snake River Spring/Summer Chinook

25. 25 What is our ability to estimate in-river survival from LGR to BON using different M&E designs? Design199819992000200120022003200420052006 L SQ, M, H 4 * SQ Survival exceeds standard Survival below standard Uncertain, confidence intervals straddle standard ? ? ? ? ? ? ? ? ? ? ? X ? X X X X X X X X X        

26. 26 Objectives by Alternatives Matrix: Hydro Example – In-River Survival

27. 27 Hydro M&E Conclusions Optimal design & confidence level in answers depends on: – spatial and temporal scales – decision criteria, and – true value of parameter being estimated relative to target. Increasing number of tags/year: – can improve precision of annual estimates; but – doesn’t help multiple-year averages (# years more important) Ability to integrate tags from multiple sources, for multiple questions: – depends on how questions are framed, but – great potential for using PIT-tags, and multi-year averages for many scales & questions

28. 28 Uncertainty regarding the effects of hatchery fish limit management evaluations of : 1.Impacts and benefits of hatcheries. 2.De-listing – how do hatchery fish “count” in productivity estimates? 3.Habitat Action Effectiveness – do hatchery fish reduce Relative Reproductive Success (RRS) and thus mask productivity improvements? 4.Effectiveness of hatcheries for target populations. 2.4 Hatchery Effectiveness

29. 29 Hatchery Evaluations 1.Existing hatchery M&E in the basin is primarily focused investigations at the project scale. 2.Assessing more generic hatchery impacts are likely to require larger spatial scale designs. 3.Designs were developed to investigate the effects of hatchery Straying and Relative Reproductive Success.

30. 30 What is our ability to assess hatchery straying? Status Quo Stray rates/ratios currently calculated using CWT recoveries in non-random locations. CSMEP Alternative Representative distribution of tag recovery effort, increasing sampling intensity & improved reporting. No need to sample every population, allows statistical extrapolation

31. 31 What is our ability to estimate RRS? Relative Reproductive Success (RRS): Status Quo Current studies opportunistically selected, unlikely to represent the range of hatchery management practices. # Programs Harvest Augmentation Supplementation

32. 32 What is our ability to estimate RRS? CSMEP Alternative Distribute RRS studies representatively so that results can be applied across all programs. # ProgramsSupplementation Harvest Augmentation

33. 33 Objectives by Alternatives Matrix: Straying

34. 34 Conclusions Problems with Status quo – Current straying estimates and RRS studies are not representative, are likely biased, and cannot be used in a predictive fashion for un-sampled programs or populations. CSEMP Designs: – Do not address all hatchery uncertainties. – Focus on PRIMARY uncertainties that limit management decisions. – Useful for aggregate and individual benefit/risk evaluations.

35. 35 2.5 Harvest Effectiveness M&E Key Management Decision: Open or close fisheries based on allowable Impacts on listed stocks? Priority Questions: What are the in-season, post season estimates of run size & escapement for each management group (target & non-target)? How do they compare to preseason estimates? What is the target and non-target harvest and when is it projected to reach allowable levels?.

36. 36 Weakness of Current Harvest M&E Fisheries are managed to keep listed-stock mortality (“Impacts”) less than permitted rates, but Status quo harvest monitoring generally does not provide precision estimates. Improved monitoring needed for more accurate and precise information for models (or run reconstructions) used in population viability, status, and trend assessment

37. 37 Assess Precision and Bias of Impact Estimates Consequences of poor (biased or imprecise) estimates Underestimate Impact – over harvest of listed stocks Overestimate Impact – lost opportunities for user groups Poor impact estimates reduce the utility of harvest mortality estimates in run reconstructions, and in assessments of status, trends, and viability

38. 38 Identify Areas of Uncertainty and Bias Columbia River Fisheries Mainstem Sport LCR Commercial Zone 6 Tribal Tributary Key Fisheries Metrics Run-Size Estimates Stock Composition Harvest Number Release Rates Post-Release Mortality Rates

39. 39 Status Quo Visual stock ID + Dam Counts + CWT based run reconstruction LowStatus quo + PIT-tag sampling of kept catch under current tagging programs MediumRepresentative PIT-tagging of wild populations CWT-indicator stocks for ESU-level resolution. Genetic Stock Identification for ESU-level resolution. HighCWT-indicator stocks for MPG-level resolution GSI for MPG-level stock composition. Monitoring Design Alternatives

40. 40 Harvest M&E Conclusions 1.Run-Size Estimates – improved at modest cost using available data and methods 2.Stock Composition – could be improved with PIT tags, GSI sampling, or index stock 3.Harvest Number – best in commercial fisheries; good in tribal and sport fisheries 4.Release Rates – good in commercial fisheries; potential biases in sport fisheries 5.Post-Release Mortality Rates – are based on limited studies; difficult to estimate Harvest Subgroup

41. 41 2.6 Integration of M&E What are the features of an ‘integrated’ monitoring program? Scale integration: data can be used at the scale of interest for decisions (e.g. ESUs for viability analysis, population level for local management). Integrates across separate monitoring programs: information gathered serves multiple functions (e.g. same PIT-tagged fish used for multiple evaluations); coordinated costing. Integrates policy and technical domains: precision of data fits time frames and acceptable risks for decisions. Integrates across life history: evaluate survival and habitat requirements throughout the life cycle. Species integration: collect data for multiple species in an efficient manner.

42. 42 Multiple species

43. 43 Status and Trends Abundance, Productivity, Distribution HarvestHatchery Straying and Relative Reproductive Success studies Stock composition Age structure Hydrosystem Survival studies Habitat Abundance, productivity, spatial structure, spatial diversity changes from habitat actions PIT Tags Integration example using PIT-tags

44. 44 3. Summary of M&E Designs and Tools Hybrid designs for abundance monitoring (census + Index) Database to estimate costs of M&E designs Using PIT-tags to integrate S/T, action effectiveness M&E Regional scale hatchery effectiveness M&E designs IMW and regional scale habitat effectiveness designs MPG and ESU scale hydro effectiveness designs SOTR Data Quality Guide – strengths and weakness of reporting metrics, gaps Evolving towards a reliable, regional M&E framework

45. 45 CSMEP Benefits to Agencies & Tribes WDFW Improved monitoring designs for winter steelhead and Chinook IDFG Retooling natural production monitoring (integrate across fish species, assess spatial distribution, improve habitat effectiveness evaluation) ODFW Developing a cost-effective, statistically robust steelhead monitoring program for Mid-Columbia Steelhead viability assessments

46. 46 CSMEP Benefits to Agencies & Tribes cont’d Technical Recovery Teams (TRTs) CSMEP models assess effects of different M&E strategies on accuracy of viability assessments using TRT rules US vs. Oregon Technical Advisory Committee (TAC) CSMEP analyses being used to review plans for monitoring mortality of listed species caught in various fisheries Ad Hoc Supplementation Workgroup CSMEP’s hatchery group is providing leadership and technical expertise to the ISAB Ad Hoc supplementation Work group

47. 47 The Future of Collaborative Monitoring Collaborative monitoring will become increasingly important as recovery plans are implemented Coordination among federal, state and tribal agencies remains critical for developing logical cost-effective M&E (especially for fish populations that cross state and tribal boundaries) CSMEP provides a forum for federal, state and tribal fish managers to develop cost-effective, and coordinated regional monitoring for status and trends and the 4-“H”s.

48. 48 Summary of proposed CSMEP tasks for FY08 - if half funding Complete design analyses for Hydro, Harvest, & Hatcheries Complete analytical tools (e.g., Sp Ch viability model, Cost Database) Facilitate regional hatchery M&E framework Contribute to SOTR data quality summaries Present CSMEP’s Snake River Pilot results to ISAB for review Consolidate CSMEP M&E recommendations for Basin decisions

49. 49 Summary of proposed CSMEP tasks for FY08 – if full funding (see handout) S&W assessments for additional Chinook & steelhead populations (collaborating with StreamNet) Extend CSMEP’s viability simulation model to steelhead Work with local recovery teams to use CSMEP’s analytical tools for improving M&E designs Extend CSMEP hydro analyses to other areas, new BiOP standards Assist development of recommended standards for monitored Performance Measures

50. 50 Develop (with StreamNet) protocols to maintain TRT stock viability databases Assess current viability analyses and data inputs; recommend adaptive changes in M&E Develop alternative designs for steelhead harvest monitoring Work with habitat restoration managers to improve habitat action effectiveness monitoring designs/plans Summary of proposed CSMEP tasks for FY08 – if full funding cont’d

51. 51 Questions?