1. Pest Assessment and Prioritization Process
The Cooperative Agricultural Pest Survey Presents
A CAPS Webinar
Pest Assessment and Prioritization Process 
To join the meeting:
To connect audio:
To join the audio portion of the conference:
Select “Dial-out”
Enter the phone number you will be using during the webinar
Select “Join”
The conference will automatically call you.
Lobby slide

2. For Best Results
This webinar will be recorded. To eliminate background noise:
Mute your phone
Close other programs running on your desktop
DO NOT place your phone on hold!
If you need to answer a second call, disconnect from this call first and reconnect when you are ready
Patrick housekeeping slide

3. The Cooperative Agricultural Pest Survey Program: Pest Assessment and Prioritization Process
CAPS Leads
OPEP Leads
Dan Mackesy
Leslie Newton
Heather Moylett
Alison Neeley
John Bowers to provide introduction to the webinar
Hi. My name is John Bowers and I am the National Policy Manager for Pest Detection and CAPS in Riverdale, MD.
I would like to welcome you to our presentation today, where Dan, Leslie, and Heather will be presenting and providing information on the processes we use to assess and prioritize pests and pathogens for the CAPS Priority Pest List.

4. Cooperative Agricultural Pest Survey (CAPS) program
Intro to Pest Prioritization and Assessment Process
CAPS Pre-assessment
OPEP model (Leslie Newton)
Results of OPEP model: Outcome for CAPS
CAPS Post-assessment
General timeline for addition of a pest to CAPS list
Presented by Dan Mackesy
CAPS is the primary PPQ program devoted to surveying for exotic plant pests in the United States.
In this presentation, we will cover the following topics:
Explain the three step CAPS pest prioritization and assessment process, including:
Pre-assessment process
OPEP Model and outcomes for CAPS
Post-assessment process
General timeline for addition of pests to CAPS list.

5. Pest Assessment and Prioritization Process
This process ensures that pests included in the CAPS Priority Pest List are appropriate survey targets with effective and practical survey and identification methods.
Pre-assessment
Post-assessment
OPEP Model
CAPS Priority Pest List
Presented by Dan Mackesy
And to give you an overview of the pest prioritization process, when we get pest suggestions:
First, we conduct a pre-assessment to determine if the pest is a good candidate for CAPS.
If it passes the pre-assessment, the potential impact of the pest is evaluated using the Objective Prioritization of Exotic Pests (OPEP) model.
The pests that rank as high impact and some pests that rank as moderate impact are then run through a post-assessment.
The post-assessment is used to determine if there are adequate survey and identification or diagnostic methods available.
Those that pass the post-assessment are added to the CAPS prioritized pest list and are considered ready for the field.

6. Pests are suggested by CAPS community and other sources.
Completed pre-assessment form is archived. Pest may be re-submitted in the future if more information becomes available.
Pests that fail the Pre-Assessment are not candidates for the Prioritized Pest List
Pests are run through Pre-assessment questionnaire.
Moderate impact pests evaluated on a pest by pest basis. If recommended for CAPS the pest will be run through the Post-assessment questionnaire .
Pest is a candidate for the Objective Prioritization of Exotic Pests (OPEP) model. Pest is run through model.
Low impact pests are not added to the final CAPS Pest List unless there is a significant reason.
High impact pests are run through the Post-assessment questionnaire (evaluate Survey and ID methods/ capacity).
Presented by Dan Mackesy
This is the Pest Assessment and Prioritization Process in greater detail.
A pest is suggested to the CAPS program by the CAPS community or other sources, like the New Pest Advisory Group (CPHST PERAL).
The pest is run through the CAPS Pre-Assessment questionnaire.
If the pest fails the CAPS Pre-Assessment, it is archived and may be resubmitted to CAPS in the future as new information becomes available.
If the pest passes CAPS Pre-Assessment, it is passed to the Objective Prioritization of Exotic Pests (OPEP) team for evaluation.
Pest is evaluated using OPEP model
Pests found to be low impact are archived and do not advance to the CAPS Post-Assessment unless significant reason exists (political/trade implications).
Moderate impact pests are evaluated on a pest by pest basis
High impact and some moderate impact pests advance to the CAPS Post-Assessment
Pest is run through the CAPS Post-Assessment questionnaire to evaluate detection and identification methods available.
If high impact pest fails the CAPS Post-Assessment, it is placed on a research list for survey and ID methods development/improvement
Moderate impact pests that fail post-assessment may or may not be listed as priorities for research.
Pests that pass the CAPS Post-Assessment are added to the CAPS Prioritized Pest List and approved methods for survey and identification are posted to the CAPS Resource and Collaboration Site:
Pests that fail the Post-assessment are put on a research list for survey or ID methods development/ improvement.
Pests that make it through the Post-assessment will be on the CAPS Pest List.

7. Pest Assessment and Prioritization Process
Pre-assessment Questionnaire
The Pre-assessment questionnaire is used to assess potential new CAPS pests before the pests are run through OPEP model. The Pre-assessment determines if:
The pest causes measurable damage.
The pest is established in the United States.
The pest is reportable by PPQ.
A pathway of introduction exists for the pest.
Presented by Dan Mackesy
As you remember, the pre-assessment first determines if the species is
Is it a plant pest as defined by the IPPC?
Does the pest cause measurable damage on any plant of value or interfere with trade?
Is citrus the only important host for this pest? (If yes, refer to CPHST citrus team.)
Is the pest exotic to the United States?
Is it listed in the AQAS database as non-reportable at the species level? Yes = Fail
Is there a demonstrated pathway of introduction, not including smuggling ?
Is the deliberate smuggling of this pest or any host of this pest likely to occur?

8. Pre-assessment questionnaire
Pest is not a candidate for Prioritized Pest List.
Completed pre-assessment form is archived. Pest may be re-submitted in the future if more information becomes available.
Pests are run through Pre-assessment Form.
FAIL
PASS
Pest is a candidate for the Objective Prioritization of Exotic Pests (OPEP) model. Pest is run through model.
Presented by Dan Mackesy
Pests which pass the Pre-assessment are run through the OPEP model.
Pests which fail the pre-assessment are archived and may be re-evaluated if new information becomes available.

9. Objective Prioritization of Exotic Pests
(OPEP Model)
Presented by Leslie Newton

10. Why have a model for prioritizing exotic plant pests?
Presented by Leslie Newton
Arthropods: 36 (current list) + 19 (new suggestions) = 55 total
Plant pathogens/nematodes: 19/20 (current list) + 20 (new suggestions) = 39/40 (depending if we run Xanthomonas oryzae at the species or pathovar level).

11. Why a new model? Presented by Leslie Newton Criteria: Weightings:
Shortcomings of AHP (as applied to this purpose):
Subjective
Weightings of criteria determined by experts rather than data & evidence
Requires independent criteria
Nearly impossible to validate
Arthropod-centric
Biological impacts were not separated from policy considerations
Presented by Leslie Newton
Criteria:
Entry potential
Establishment potential
Post-establishment proliferation and spread
Economic impact
Non-economic impact
Weightings:
Derived through pair-wise comparisons of criteria and sub-criteria made by decision-makers
Some questions highly subjective: e.g., The pest’s reproductive potential is:
Extremely high
High
Moderate
Low
Very low
This made the model somewhat arthropod-centric

12. What we wanted in a new model
Objective – evidence-driven, not opinion-driven
Valid & comparable across pest types & systems
Transparent – separates analysis based on scientific information from that based on policy
Separate uncertainty from risk score
Flexible – can be used to look at risk by region and host
Defendable
Uses methods that can be tested & statistically validated
Based on other proven PPQ risk assessment methods
Presented by Leslie Newton
Be evidence-driven
Use scientifically-valid methods that can be tested and validated
Evaluate pests across pest types using a common metric
Make it clear what part of the ranking is based on science and what part is based on policy

13. How should pests be prioritized?
Ideally, we want to spend our limited resources on those pests that pose the greatest risk
Risk = the likelihood of an adverse event and the magnitude of the associated consequences
Analysts must separate evidence from uncertainty to ensure that risk is not artificially elevated – in the past, many pests were ‘guilty by association’
Presented by Leslie Newton

14. Presented by Leslie Newton
“When everyone’s super, no one will be”
By focusing on those organisms that have a high likelihood of being high impact pests, we free up our resources for those things that will surprise us.

15. Risk analysis, uncertainty, and decision-making:
Risk assessment is not a crystal ball: We can’t predict everything
We often don’t always know what we don’t know
We have to evaluate pests and make predictions and decisions with incomplete information (we “do the best we can with what we’ve got”)
There is always a chance that our prediction may be wrong
Our estimates of risk may change as we get more information
There will always be pests that surprise us (that we didn’t even know we should worry about or interact w/ the environment in ways we couldn’t predict)
* But, there are pests and pest situations that we can be fairly certain about.
Presented by Leslie Newton
We have to make decisions and prioritize resources under uncertainty
We must often adopt an attitude of “do-the-best-you-can-with-what-you’ve-got”

16. How should pests be prioritized?
Likelihood of introduction
How likely is the pest to enter the United States, establish a viable population & spread?
Consequences of introduction
Is the pest likely to cause serious impacts upon introduction & spread
Feasibility and Cost Effectiveness
Is it possible to survey for the pest?
Do the expected impacts of the pest justify the cost of a survey program?
Political considerations
Pest Risk
Presented by Leslie Newton

17. By focusing on those organisms that we determine have a high probability of causing serious impacts, we can free up resources that can then be spent on those pests that will inevitably surprise us.
Presented by Leslie Newton
Uncertainty forces decision-makers to judge how probable it is that risks will be overestimated or underestimated for every member of the exposed population, whereas variability forces them to cope with the certainty that different individuals will be subjected to risks both above and below any reference point one chooses.”
See Thompson, (
National Research Council, 1996

18. Objective Prioritization of Exotic Pests
(OPEP)
Phase III:
To be completed FY 2019
Phase I:
Completed (finishing up validation)
Phase II:
To be completed FY 2018
Impact Potential
Likelihood of Introduction
Survey Feasibility & Cost Effectiveness
Add to survey program?
Presented by Leslie Newton
Policy considerations*
(Decision Lens/AHP)

19. PPQ’s Weed Risk Assessment Model
Very successful tool for evaluating the “invasive” potential of plants
Widely evaluated, tested, and validated
Adopted by other stakeholders
WRA
Guidelines
Presented by Leslie Newton

20. Model Development
We developed a set of yes/no and multiple choice questions (criteria) we thought might be predictive of impact
Previous CAPS models
Other risk analysis models
Scientific literature
Expert opinion
Currently we have developed 2 separate models: arthropods and pathogens (mollusk model to follow)
Presented by Leslie Newton

21. Factors for evaluating U.S. impacts
Severity of unmitigated damage (e.g., yield loss, mortality)
Frequency of severe outbreaks
Impact on production practices
Environmental & social impacts
Level of management & cost of control
Amount of research into methods of control (including host resistance/ biocontrol)
Ease of control
Presented by Leslie Newton

22. Model Development
We analyzed each arthropod (100) and pathogen (72) as if it were not present in the United States using the potential questions
Statistician compared results to observed impacts & tested the predictability of each question; removed non-predictive questions
For predictive model, we selected ordinal logistic regression
Weighted each question by its predictive power
Presented by Leslie Newton
Recall that with the AHP model, weightings were derived though pairwise comparisons of criteria and sub-criteria made by decision-makers, here the questions are weighted by its predictive power as determined through statistical analysis.

23. Consideration of U.S. conditions
Are there already organisms in the U.S. that fill the same ecological niche?
Are there tools in the U.S. that have already been developed and are in use that would be effective at controlling the pest?
Would current production practices or conditions in the United States be effective at mitigating the pest?
Presented by Leslie Newton

24. Results
Results are provided as probabilities for a pest resulting in High, Moderate, or Low impact
Presented by Leslie Newton

25. Uncertainty analysis
We consider uncertainty through a Monte Carlo simulation (5000 iterations) where alternate answers are applied based on analyst’s uncertainty rating
Presented by Leslie Newton

26. Impact assessment interface
The current model is housed in Excel
Background questions
Impact potential (predictive model)
Endangered area
References
Results
Presented by Leslie Newton

27. Presentation to stakeholders
A summary document encapsulates the assessment with background information, results from the predictive model, endangered area, references, and an appendix with predictive questions & answers
Presented by Leslie Newton

28. Validation
Validation of the arthropod impact assessment model has been completed
We assessed 75 additional pests & retested some questions that were not predictive
Vector status
Parthenogenic reproduction (specifically, the ability of one female to begin a new population)
Host expansion (new species within a genus or new genera or families)
Pathogen model validation underway (65+ additional pests)
Retesting questions on spread
Presented by Leslie Newton

29. Current uses for the OPEP IA
Cooperative Agricultural Pest Survey (CAPS)
Conducts national & state surveys targeted at exotic plant pests
For 2017 CAPS survey season, the OPEP IA was used to analyze 91 pests
43 ranked as high or moderate impact and were added to the national survey list for 2017
New Pest Advisory Group (NPAG)
NPAG uses the OPEP IA as a ‘first look’ at new pests to assess potential impacts
Non-routine requests for economic analysis of exotic pests
Presented by Leslie Newton

30. Likelihood of Introduction
Phase II
Further refine prioritized list by the likelihood of introduction (FY 2017)
We will develop a likelihood of introduction model (building on Port Environs & State and County Risk Mapping work)
* Highest priority pests would be those that have a high likelihood of causing serious impacts AND have a high likelihood of being introduced
Presented by Leslie Newton

31. Survey Feasibility & Cost Effectiveness
Phase III
Develop an assessment that looks at feasibility of survey and cost effectiveness
Develop a method for comparing the cost of the survey to the expected losses
Determine which pest surveys will give us the greatest value for our dollar
Eliminate surveys where the cost of survey is greater than the potential damage
Presented by Leslie Newton

32. Objective Prioritization of Exotic Pests (OPEP)
Moderate impact pests (Category 2) evaluated on a pest by pest basis. If recommended for CAPS the pest will go through the Post-assessment questionnaire
Pest is a candidate for the Objective Prioritization of Exotic Pests (OPEP) model. Pest is run through model
Low impact pests (Category 3) are not added to the final CAPS Pest List unless there is a significant reason
Presented by Heather Moylett
High impact pests (Category 1) are run through the Post-assessment questionnaire (evaluate Survey and ID methods/ capacity)

33. OPEP Results: Outcome for CAPS
Category 1 (High Impact Pest)
Significant likelihood to have a high impact in the U.S.
Greater than 20% probability of being a high impact pest.
Post-assessment candidate.
Category 2 (Moderate Impact Pest)
Most likely have a medium impact in the U.S.
10 to 20% probability of being a high impact pest.
Evaluated on a case by case basis.
Category 3 (Low Impact/Undetermined)
Have a low impact in the U.S or not enough info to evaluate.
Less than 10% probability of being a high impact pest.
Not evaluated further unless significant reason.
Presented by Heather Moylett
Category 1 (High and some Moderate pests; Pests in Risk Groups A, B, C, and D)
• Pests have a significant likelihood to have a high impact in the United States. Pests have a greater than 20% probability of being a high impact pest.
• If the pests pass the Post-assessment (which evaluates the survey and identification/ diagnostic method), they will be listed on the Pests of Economic and Environmental Importance List. Pests will also be added to relevant commodity manuals.
• Pests on the Pests of Economic and Environmental Importance List are Priority Pests.
• If pests fail the Post-assessment, they will be placed on a priority list for research.
Category 2 (Other Moderate pests; Pests in Risk Groups E, F, and G)
• Pests are most likely to have a medium impact in the United States. Pests have a 10 to 20% probability of being a high impact pest.
• These pests will not be part of the Pests of Economic and Environmental Importance List.
• Pests will be evaluated on a case-by-case basis to determine if surveys are warranted. Pests that are recommended for survey and pass the Post-assessment (which evaluates the survey and identification/ diagnostic method) will be added to commodity manuals or posted as free-standing datasheets.
• If the pest is added to a commodity manual, it will be considered a Priority Pest. If the pest does not fit into an existing commodity manual, it will be added to the Additional Pests of Concern List and will not be considered a Priority Pest.
• If pests are recommended for survey but fail the Post-assessment they may or may not be listed as priorities for research.
Category 3 (Low impact or Undetermined pests; Pests in Risk Groups H, I, and J)
• Pests have a less than 10% probability of being a high impact pest.
• Low impact pests: pests are most likely to have a low impact in the United States. Undetermined pests: there is not enough information available to evaluate likely impacts.
• These pests will not be included on the Pests of Economic and Environmental Importance List.
• New pests assessed will not be included in a commodity manual or posted as free-standing datasheets unless there is a significant reason (political/ trade implications, human health impacts, etc.) for including them.
• Existing pests present in a commodity manual will likely be removed from manuals over time. These pests will still be available for bundling into other surveys.

34. Pest Assessment and Prioritization Process
Post-assessment Questionnaire
Evaluates:
Survey method
Identification/diagnostic method
Capacity/expertise to perform ID/diagnostics
Exclude pests that don’t have effective methods.
These pests would not be listed on the final prioritized pest list.
Pests would go to a research and development list.
Presented by Heather Moylett
The post-assessment evaluates the:
Ease of detection for this pest: (specific pheromone, generic sticky trap, visual)
Ease of identification:
Are there validated- diagnostic methods?
Does it require extensive sample preparation?
Is the target easily confused with many native/endemic pests?
and if there is sufficient capacity and available expertise to identify the pest should a large scale survey be conducted

35. Post-assessment questionnaire
High impact pests, and some moderate, are run through the Post-assessment Questionnaire. Evaluate Survey and ID methods/ capacity
Pests that fail the Post-assessment are put on a research list for survey or ID methods development/ improvement.
FAIL
PASS
Presented by Heather Moylett
Pests that pass the post-assessment are added to the CAPS priority pest list.
Pests that fail the post-assessment are added to a research list for methods development.
Pests that make it through the Post-assessment will be on the CAPS Pest List.

36. Economic and Environmental Importance
CAPS Pest List
Survey Manuals
Economic and Environmental Importance
Additional Pests of Concern
Presented by Heather Moylett
The CAPS Priority Pest List includes the target pests in the CAPS Program. As of 2017, there are 153 pests on the CAPS list. This number is subject to change each year as CAPS adds new pests or removes existing pests. Pests are removed based on changes in US distribution, regulatory status, or changes to impact.
The pests listed are included in one or more of the following categories: Additional Pests of Concern, Survey Manuals, and Economic and Environmental Impact.

37. Economic and Environmental Importance
Provide guidelines for early detection surveys.
Organized by Taxonomy or Commodity.
Priority Pests: High impact pests and moderate impact pests when appropriate.
Survey Manuals
List of pests ranked as high impact by OPEP.
These pests are also included in relevant survey manuals.
Priority Pests
Economic and Environmental Importance
Moderate impact pests that do not fit into an existing survey manual. Datasheets are freestanding.
Low impact pests with significant reason.
Not Priority Pests
Additional Pests of Concern
Presented by Heather Moylett
Survey Manuals:
All CAPS surveys use a bundled survey approach, and survey manuals provide guidelines for conducting these surveys. In bundled surveys, groups of exotic pests are surveyed for concurrently. The purpose of this commodity-based (pests with the same host plant), taxon-based (similar pest taxa; e.g., cyst nematodes, bark beetles), and pathway-based (pests that follow the same pathway; e.g., Asian defoliators) bundling is to increase efficiency and cost-effectiveness of surveys.
Economic and Environmental Impact:
Pests on the Economic and Environmental Impact list (currently ~55 pests) are predicted to have significant economic or environmental impact if they become established in the U.S.
Additional Pests of Concern:
Pests on the Additional Pests of Concern list are moderate pests recommended to CAPS that do not fit into a survey manual or low impact pests that are retained for significant reasons (political, trade, etc.). Pests on the APC List will have a free-standing datasheet accessible from the Approved Methods page and will not be considered a Priority Pest.
Ex. Mamestra brassicae

38. Pests are suggested by CAPS community and other sources.
Completed pre-assessment form is archived. Pest may be re-submitted in the future if more information becomes available.
Pests that fail the Pre-Assessment are not candidates for the Prioritized Pest List
Pests are run through Pre-assessment questionnaire.
Moderate impact pests evaluated on a pest by pest basis. If recommended for CAPS the pest will be run through the Post-assessment questionnaire .
Pest is a candidate for the Objective Prioritization of Exotic Pests (OPEP) model. Pest is run through model.
Low impact pests are not added to the final CAPS Pest List unless there is a significant reason.
High impact pests are run through the Post-assessment questionnaire (evaluate Survey and ID methods/ capacity).
Presented by Heather Moylett
Revisit the entire Pest Assessment and Prioritization Process.
A pest is suggested to the CAPS program by the CAPS community or other sources, like the New Pest Advisory Group (CPHST PERAL).
The pest is run through the CAPS Pre-Assessment questionnaire.
If the pest fails the CAPS Pre-Assessment, it is archived and may be resubmitted to CAPS in the future as new information becomes available.
If the pest passes CAPS Pre-Assessment, it is passed to the Objective Prioritization of Exotic Pests (OPEP) team for evaluation.
Pest is evaluated using OPEP model
Low impact pests are archived and do not advance to the CAPS Post-Assessment unless significant reason exists (eg., political/trade implications).
Moderate impact pests are evaluated on a pest by pest basis
High impact and some moderate impact pests advance to the CAPS Post-Assessment
Pest is run through the CAPS Post-Assessment questionnaire to evaluate detection and identification methods available.
If high impact pest fails the CAPS Post-Assessment, it is placed on a research list for survey and ID methods development/improvement
Moderate impact pests that fail post-assessment may or may not be listed as priorities for research.
Pests that pass the CAPS Post-Assessment are added to the CAPS Prioritized Pest List and approved methods for survey and identification are posted to the CAPS Resource and Collaboration Site:
Pests that fail the Post-assessment are put on a research list for survey or ID methods development/ improvement.
Pests that make it through the Post-assessment will be on the CAPS Pest List.

39. CAPS Timeline General Timeline for Prioritized Pest List Development
Year 1: Prioritization Process
Pre-assessments
Model
Post-assessments
Year 2: Develop support products for new pests
Pest datasheets
Approved survey methods
Approved identification/diagnostic methods
Trap and lure procurement
Identification and diagnostic capacity
Screening aids (if appropriate)
Host and Risk maps
Presented by Heather Moylett
A new pest list is developed every two years. The process involves both the prioritization process to create a pest list and also development of the support tools that make the pest list “field-ready.” In this way, when the pest list is made available to the CAPS community, the datasheets, approved methods, and necessary infrastructure (traps and lures, diagnostics, screening aids, etc.) are in place so that surveys can take place. This slide represents our two-year timeline.

40. CAPS Timeline In 2017, the CPHST CAPS Support Team will:
provide support to the field for the 2017 survey season,
work to finalize datasheets/manuals for the 2018 survey season,
evaluate survey methods for new pests for the 2018 survey season,
coordinate procurement of new traps and lures for 2018,
determine which pests to add to/remove from the CAPS Priority Pest List for the 2019 Guidelines
Presented by Heather Moylett
While the prioritization and assessment process takes two years, the CPHST CAPS Support team is working on products for the 2018 and 2019 survey seasons while providing support for the 2017 survey season.
In 2017, the team will:
……..
Each year, the CAPS Guidelines are released in April on Earth Day.

41. Contact CAPS Reach out to Dan Mackesy or Heather Moylett:
if you have questions about the CAPS Survey Guidelines, Approved Survey Methods, etc;
to suggest a pest for evaluation for possible inclusion on the CAPS Priority Pest List;
if you would like to receive the minutes from the monthly National CAPS Committee call.
Presented by Heather Moylett
You can always reach out to Dan or I with questions.
You can suggest pests to us to evaluate as potential CAPS targets.
You can ask to receive the monthly minutes to the National CAPS Committee calls. This is a good way to keep current on what is going on in CAPS. We currently forward the notes to Lab Directors and any interested CPHST scientists.

42. Questions? Questions facilitated by John Bowers
I would like to thank you all for coming today. We would like to continue to provide these opportunities, and in order to make them worthwhile for you, we need to know what topics and issues you would like more information on or what discussions you would like to have and participate in (webinars and/or virtual town halls). Just let me, Lisa, and/or your NCC representative know what topic interests you, and we will see what we can do.
Again, thanks for coming. We appreciate your participation in the CAPS Program.
Questions?