1. Food SafetyResearch Consortium A MULTI-DISCIPLINARY COLLABORATION TO IMPROVE PUBLIC HEALTH Ranking Pathogens in Foods for Broad Priority Setting The Foodborne Illness Risk Ranking Model Michael Batz Research Associate, Resources for the Future mbatz@rff.orgmbatz@rff.org, (202) 328-5020 RAC Workshop on Food and Waterborne Pathogen Risk Ranking Models: From Policy to Practice College Park, Maryland 18 August 2005

2. 2 Food SafetyResearch Consortium  Multi-disciplinary collaboration to improve public health, focused on creating tools and analysis to foster a science- and risk-based food safety system in the United States  Member institutions / steering committee U. California at Davis (Jerry Gillespie) U. Georgia (Mike Doyle) Iowa State (Cathie Woteki) U. Maryland (Glenn Morris) U. Massachusetts (Julie Caswell) Michigan State (Ewen Todd) Resources for the Future (Mike Taylor)

3. 3 Food SafetyResearch Consortium Additional FIRRM Researchers  Glenn Morris (U of Maryland School of Medicine)  Mike Taylor (Resources for the Future)  Alan Krupnick (Resources for the Future)  Sandy Hoffmann (Resources for the Future)  Holly Gaff (U of Maryland School of Medicine)  David Hartley (U of Maryland School of Medicine)  Marisa Caipo (U of Maryland School of Medicine)  Jody Tick (Resources for the Future)  Diane Sherman (Resources for the Future)

4. 4 Food SafetyResearch Consortium What does FIRRM do?  Ranks food-pathogen combinations by public health impact 28 pathogens 13 food categories, 48 subcategories 5 measures of public health impact  Illnesses  Hospitalizations  Deaths  Dollars  QALY loss  Choice of assumptions and data sources

5. 5 Food SafetyResearch Consortium Some Characteristics  Not a predictive model –not a risk assessment  Created in Analytica Graphical user interface: point-and-click, drop-down menus, follow the arrows Changeable assumptions and choices of data Uncertainty (Monte Carlo) Relatively user friendly: takes some time to learn, but no command-line prompts  Open and free to download/use/change

6. 6 Food SafetyResearch Consortium Some more characteristics  Transparency Built-in documentation No secrets – the math is right there, though it might take some work to follow the dots Decisions make explicit uncertainties that are usually hidden or glossed over  Adaptable to new data  Vetted via workshops, policy input

7. 7 Food SafetyResearch Consortium Why was it created?  First step in priority setting  Complex food system: many pathogens, many foods, many points of contamination  Use data driven approach  Compare food-pathogen vectors, not just pathogens  Determine the economic impacts of illnesses  The Question: Which pathogen-food vectors have the most significant impacts on public health?

8. 8 Food SafetyResearch Consortium Phase II Development  Phase I: 2002- 2003 Thanks to Robert Wood Johnson Foundation Resulted in FIRRM as presented today Significant data gaps Not ready for prime time  Phase II: 2004 - 2006 Thanks to CSREES Fill many data gaps Incorporate more uncertainty information Create web-interface Ready to inform policy

9. 9 Food SafetyResearch Consortium Where does FIRRM fit in?  FIRRM is part of a larger conceptual framework of priority setting tools and models  CSREES Project: Prioritizing Opportunities to Reduce Foodborne Disease 3 Regional Workshops 1 National Conference: National Conference for Stakeholders and Experts September 14, 2005 RFF Conference Center, Washington, DC http://www.card.iastate.edu/food_safety/national_conference/

10. 10 Food SafetyResearch Consortium Conceptual Framework Risk Ranking Priority Setting Decision - Purpose I: Resource allocation, research, data, etc Conceptual Framework for Prioritizing Food Safety Interventions

11. 11 Food SafetyResearch Consortium Conceptual Framework Intervention Assessment -Cost of Interventions -Effectiveness (in terms of contamination indicators) -Cost-Effectiveness (indicator) Risk Ranking Priority Setting Decision - Purpose I: Resource allocation, research, data, etc - Purpose II: Risk management, private intervention, etc Conceptual Framework for Prioritizing Food Safety Interventions

12. 12 Food SafetyResearch Consortium Conceptual Framework Intervention Assessment -Cost of Interventions -Effectiveness (indicators) -Cost-Effectiveness (indicator) Risk Ranking Health Benefit Assessment -Health Outcomes -Health Valuation Combined Assessment -Cost-Benefit -Cost-Effectiveness Priority Setting Decision - Purpose I: Resource allocation, research, data, etc - Purpose II: Reg. action, private intervention, etc Conceptual Framework for Prioritizing Food Safety Interventions

13. 13 Food SafetyResearch Consortium Conceptual Framework Intervention Assessment -Cost of Interventions -Effectiveness (indicators) -Cost-Effectiveness (indicator) Risk Ranking Health Benefit Assessment -Health Outcomes -Health Valuation Combined Assessment -Cost-Benefit -Cost-Effectiveness Priority Setting Decision - Purpose I: Resource allocation, research, data, etc - Purpose II: Reg. action, private intervention, etc Post Hoc Evaluation Data Collection Conceptual Framework for Prioritizing Food Safety Interventions

14. 14 Food SafetyResearch Consortium How is the model structured?  Incidence Estimates National Maryland  Health Valuation Economic QALY  Food Attribution Based on outbreak data Based on expert judgment Based on risk assessments and other data  Rankings

15. 15 Food SafetyResearch Consortium Module 1: Incidence  National estimates based on Mead et al. (1999) Reported illnesses multiplied by underreporting factors Similar underreporting factors for hosps, deaths Foodborne is percent of Total illness (for each path)  FIRRM adaptations: Uncertainty as probability distributions Alternate multipliers, hospitalization & fatality rates Estimates for Maryland based on FoodNet laboratory data (two years only of stripped, summarized data) Mead, P. S., L. Slutsker, V. Dietz, et al., Food-Related Illness and Death in the United States, Emerging Infectious Diseases (1999), 5, 607-625.

16. 16 Food SafetyResearch Consortium Incidence in Phase II  Year-by-year data  Add more years through 2003  New underreporting factors Now: based on Mead (1999), single factor per pathogen Soon: based on Voetsch et al. (2004), a three-tiered approach (at least for FoodNet paths) Voetsch, A. C., T. J. V. Gilder, F. J. Angulo, et al., FoodNet Estimate of the Burden of Illness Caused by Nontyphoidal Salmonella Infections in the United States, Clinical Infectious Diseases (2004), 38, S127-134.

17. 17 Food SafetyResearch Consortium Module 2: Valuation  Aggregate measure  Economic impact of disease  Useful for later cost-benefit  Create outcome trees for each pathogen to capture symptoms, severities, treatments  Compute dollars & QALYs for each health state

18. 18 Food SafetyResearch Consortium Health Outcome Tree (example) 65% are mild cases and recover fully 1,300 cases 35% are severe cases 700 cases 50% do not visit a physician and recover fully 5,000 cases 30% visit a physician and recover fully 3,000 cases 20% are hospitalized 2,000 cases 55% recover fully 385 cases 25% chronic sequelae 175 cases 20% die in first year 140 cases Total cases of Pathogen A 10,000 cases

19. 19 Food SafetyResearch Consortium Module 2: Valuation (cont’d)  Economic valuation Cost of Illness (COI) – morbidity Willingness to Pay (WTP) – mortality (VSL) COI values drawn primarily from ERS studies WTP values drawn from literature  Quality Adjusted Life Years (QALYs) Quantify based on scale of 0 to 1 Values drawn from surveys Subtract from baseline & multiply by duration Numerous health indices available (QWB, HUI, EQ5D) FIRRM currently uses Quality of Well Being (QWB) index

20. 20 Food SafetyResearch Consortium Chronic sequelae in Phase I FIRRM  Campylobacter Guillain-Barre Syndrome (GBS)  Hospitalized with and without ventilation  Eventual recovery (return to work)  Permanent disability (never return to work)  E. coli O157:H7 Hemolytic Uremic Syndrome (HUS)  Dialysis and transplants  Kidney transplants  Premature death  Listeria monocytogenes Stillbirths and newborn deaths Mild, moderate, and severe retardation  Nontyphoidal Salmonella No chronic sequelae

21. 21 Food SafetyResearch Consortium Valuation in Phase II  8 additional pathogens: Cyclospora Cryptosporidium Shigella Vibrio vulnificus Vibrio parahaemolyticus & other marine Vibrios Yersinia enterolotica Norovirus Toxoplasma gondii  Additional chronic sequelae Reactive arthritis Irritable Bowel Syndrome  New QALY index Probably will use EQ-5D (EuroQoL)

22. 22 Food SafetyResearch Consortium Impact of VSL on Valuation

23. 23 Food SafetyResearch Consortium Module 3: Food Attribution  For each pathogen, apply percent of total due to each food category  No ideal data source  Primary data options: Outbreak data Expert elicitation FDA/USDA Listeria risk assessments “Shorthand” risk assessment approach (consumption/contamination)  Two-tier food categorization (eg. seafood/ finfish)

24. 24 Food SafetyResearch Consortium Module 3: Food Attribution  Outbreak data (1990-2000) CSPI compilation: mostly CDC data (88%) Approx 2000 outbreaks & 80,000 cases Percents based on cases summed across years  Expert Elicitation Mail survey, peer-reviewed set of respondents 101 contacted, 45 completed 11 pathogens Best estimates, also low/high estimates Self-assessed expertise, confidence in answers

25. 25 Food SafetyResearch Consortium Food Attribution for Campy.

26. 26 Food SafetyResearch Consortium Food Attribution in Phase II  Update food categories  Update outbreak data Add years: 2001 - 2003 Focus on CDC line listings Allow user to choose which years to use  Expand incorporation of expert elicitation  Incorporate FoodNet case-control studies  Further develop “shorthand” risk assessment approach based on food consumption and contamination data

27. 27 Food SafetyResearch Consortium Interface 1  Insert screen grab: main model screen  Say: open the model and interact by double-clicking… double-click on ‘model interface’ to run some scenarios

28. 28 Food SafetyResearch Consortium Interface 2  Insert screen grab: main interface screen

29. 29 Food SafetyResearch Consortium Ranking by Dollars These rankings are provided as an example. They are based on midpoint values and were computed in 2003 using default model settings, including a VSL of $2.2M and attribution based on outbreak data, among other assumptions. Only four pathogens are currently valued in dollar or QALY terms.

30. 30 Food SafetyResearch Consortium Ranking by Deaths These rankings are provided as an example. They are based on midpoint values and were computed in 2003 using default model settings. Note that Toxoplasma and E coli STEC do not have enough outbreaks in the attribution dataset to estimate food-pathogen combinations.

31. 31 Food SafetyResearch Consortium Phase II Tasks  Already mentioned Update & improve incidence estimates More pathogens valued Different QALY index Update & improve food attribution  Treatment of uncertainty Incorporate more variance information Uncertainty and sensitivity analysis Importance assessment  Web-based model Simplified interface Ability to save changes, compare different runs Contract with Enrich Consulting

32. 32 Food SafetyResearch Consortium Conclusions  Lots of uncertainty Underreporting multipliers in incidence estimates Mortality valuation Food attribution estimates  Largest data gaps in food attribution  Valuation changes ranking  Norovirus and Toxoplasma are important  Preliminary results are useful for priority setting For more information about the Foodborne Illness Risk Ranking Model, or to download a draft version: http://www.rff.org/fsrc/firrm.htm

33. 33 Food SafetyResearch Consortium Appendices

34. 34 Food SafetyResearch Consortium Analytica  Modeling environment developed primarily for risk and decision analysis  Visual modeling framework Hierarchical influence diagrams Point and click interaction  Embedded uncertainty Inputs as probability distributions Monte Carlo simulation to propagate uncertainties

35. 35 Food SafetyResearch Consortium Why Analytica  Transparency Data and math is explicitly visible; documentation of sources and assumptions  Flexibility and adaptability Visual programming means fast development; modular; collaborative tool; easy to expand and/or change;  Accessibility modest software costs; distribution; web interface;  Ease of use Drop down menus allow to easily change assumptions; don’t have to be an expert or programmer to use it.

36. 36 Food SafetyResearch Consortium Key Activities  Map current information landscape for food safety  Convene public and private stakeholders to establish guiding principles  Identify key obstacles to sharing existing data and strategies for resolution  Develop cost effective strategies and priorities for collecting new data  Establish mechanisms for data housing and dissemination  Maintain links to decision processes at all levels

37. 37 Food SafetyResearch Consortium Food Categories

38. 38 Food SafetyResearch Consortium Food Attribution: E. coli O157:H7

39. 39 Food SafetyResearch Consortium Food Attribution: Salmonella

40. 40 Food SafetyResearch Consortium Food Attribution: Toxoplasma

41. 41 Food SafetyResearch Consortium Method: Expert Elicitation Survey  Peer-reviewed list of potential respondents  Mail survey with phone follow-up  Survey Content Developed in collaboration with national expert on elicitation (Paul Fischbeck, CMU) Respondent background information Respondent self-evaluation of level of expertise Quantitative (%) attribution with best judgment and upper and lower bounds Reporting on information used in response

42. 42 Food SafetyResearch Consortium Example of Expert Elicitation Survey Part 1

43. 43 Food SafetyResearch Consortium Example of Expert Elicitation Survey Part 2 Questions: 1. What information did you principally rely on to fill out this table? (list all that apply) general knowledge your own research or clinical experience specific journal articles, data sets, or other specific professional publications. Please list: 3. Is there a factor other than the type of food that determines Whether a food is associated with illnesses caused by this pathogen? For example, for listeria, it may be that illnesses are associated with refrigerated foods regardless of whether the food is dairy or poultry. 2. Please give your best estimate of the total number of foodborne cases of illness caused by this pathogen in a typical year. Provide a brief description of your reasoning.

44. 44 Food SafetyResearch Consortium Rankings Sorted by Cases

45. 45 Food SafetyResearch Consortium Ranked by Dollars VSL Comparison

46. 46 Food SafetyResearch Consortium Data Challenges  Utility of models requires multiple types of data: Attribution of illnesses to foods Effectiveness and cost of interventions Link between interventions and health outcome  Data is already collected but spread out Federal and state agencies Food industry Academic researchers  Focused effort needed to access and use existing data and fill critical gaps

47. 47 Food SafetyResearch Consortium The Data Opportunity  Key agencies embrace the systems approach to food safety and the need to set priorities  Data needs have been highlighted by the NAS, GAO, and FSRC  Technical tools now exist to collect and manage the needed data  Fragmentation of the data “system” is a recognized problem

48. 48 Food SafetyResearch Consortium A Food Safety Information Infrastructure is Needed To...  Build buy-in on system goals, data needs, and priorities  Make certain that the right questions are being asked  Assure technical compatibility of data systems  Assure data access and sharing  Provide information at all levels to decision makers