1. Lessons Learned in Initiating and Conducting Risk Assessments within a Risk Analysis Framework: A FDA/CFSAN Approach Robert Buchanan DHHS Food and Drug Administration Center for Food Safety and Applied Nutrition

2. Perspective Risk Assessor vs. Risk Manager Risk Manager vs. Risk Assessor

3. The Paradox Codex Alimentarius “Principles and Guidelines for the Conduct of Microbial Risk Assessments” Principle #2: There should be a functional separation between risk assessment and risk management Principle #2: There should be a functional separation between risk assessment and risk management Important principle that doesn’t work if you achieve it

4. Setting the Stage Experience: Experience: conducted two large quantitative microbial risk assessments Vibrio parahaemolyticus Listeria monocytogenes Learning: Learning: address the “bumps” encountered to discuss lessons learned to develop a risk analysis framework for future major risk assessments

5. The Bumps Resource intensive Setting time frames and establishing deadlines Establishing why risk assessment is being done Establishing expectations What a risk assessment can do? Establishing boundaries Who is in charge? Who gets to decide? Who settles conflicts?

6. The Bumps Communicating with risk managers (Why can’t risk assessors speak English?) Modeling Explaining uncertainty vs. variability Lack of understanding of risk assessment techniques (Why are risk managers so dense?) Getting risk managers to pay attention early

7. The Bumps Presenting the results Interpretation Transparency Plain language (Who rights the report?) Dealing with unexpected results Review of risk assessments Impact on risk management programs Protecting the risk assessors against outside pressure

8. Risk Analysis Working Group Senior managers Risk managers Risk assessors Scientists “Modelers” Project leaders Risk Communicators Facilitator

9. Risk Analysis Working Group Overall Goal: Overall Goal: To improve the quality and consistency of “major” risk assessments conducted by the Center for Food Safety and Applied Nutrition

10. What Are ‘Major’ Risk Assessments? Non-routine, complex, and/or quantitative Involve multiple program offices and/or cross-cutting in nature Require a commitment of significant resources to complete

11. Risk Analysis Working Group Specific Tasks: 1) Identify boundaries and responsibilities of key participants in the risk analysis process 2) Develop a process for identifying and selecting risk assessments conducted by CFSAN 3) Establish procedures for the conduct of risk assessment within a risk analysis framework

12. Risk Analysis – The Theory Traditional definition: risk analysis is composed of : Risk Management Risk Management Risk Assessment Risk Assessment Risk Communication Risk Communication Risk Management Risk Communication Risk Assessment

13. Risk Analysis – The Reality Risk Analysis – The Reality In CFSAN risk analysis is a “stew” of : Center Management Risk Management Risk Assessment Risk Communication Research Etc Risk Mgmt Risk Communi- cation Risk Assess- ment Research CFSAN Mgmt FDA Mgnt FDA GC

14. Procedures to Initiate and Conduct Risk Assessments INITIATE: INITIATE: A decision-based approach to identify and select all ‘major’ risk assessments Four phase process: concept generation, problem identification, data feasibility evaluation, disposition CONDUCT: CONDUCT: A systematic and iterative approach to conducting risk assessments Four step process: Plan, Perform, Review, Publish

15. The Identification and Selection Process

16. Selection of Major Risk Assessments Select major risk assessments on basis of Center’s risk management needs Center’s risk management needs Center’s resources Center’s resources Data feasibility Data feasibility

17. Personal Observation If science is complex, there is no consensus among stakeholders, and there are multiples ways of managing the risk – consider a quantitative risk assessment If science is straight forward, there is general consensus among stakeholders, and there is only one risk management option – do not conduct a quantitative risk assessment

18. Where do risk assessment ideas come from? Stakeholders (public) Follow up to completed risk assessment Regulatory staff Management Researchers Other agencies

19. A Four Phase Selection Process Phase 1: Concept Generation Phase 2: Problem Formulation Phase 3: Feasibility Determination Phase 4: Disposition

20. Selection Process Concept Generation Problem Formulation Feasibility Determination Disposition (selection) Identify and maintain list of potential RM questions for which a RA would assist with policy decisions Identify hazards/commodities State the RM question(s) State the RA question(s)

21. Selection Process Concept Generation Problem Formulation Feasibility Determination Disposition (selection) Management reviews list and approves specific RAs for further evaluation (feasibility determination)

22. Selection Process Concept Generation Problem Formulation Feasibility Determination Disposition (selection) Review literature to determine availability of data needed to conduct RA to answer RM questions Recommend action: More research needed Modify RM question Qualitative RA vs. Quantitative RA

23. Selection Process Concept Generation Problem Formulation Feasibility Determination Disposition (selection) Using the results of the feasibility determination as an aid, management selects RAs to be conducted based on technical merit, resource availability, and other factors and get commitment of resources

24. Conducting the Risk Assessment

25. A four step process: Step 1: Plan Step 2: Perform Step 3: Review Step 4: Publish

26. Conducting the Risk Assessment Step 1: Planning Step 1: Planning Define scope Identify resource needs Assign teams Develop timelines

27. Conducting the Risk Assessment Step 2: Perform Step 2: Perform Answer risk management questions

28. Receive Charge from Risk Managers Receive Assumptions from RM Refine Assumptions for modeling Assemble data/model inputs Verify data/model inputs Develop model Audit model Run model/iterations Review results New data Draft Report Conduct sensitivity analysis

29. Conducting the Risk Assessment Step 3: Step 3: Review – ongoing process Advisory Committees Peer Review SGE Panels Public Comment Approve Clear

30. Technical and Scientific Reviews of the Draft Risk Assessment Documents

31. Conducting the Risk Assessment Step 4: Publish Step 4: Publish Develop “roll out” strategy Public release of documents Handling comments Public comment

32. Organization of Risk Analysis Risk analysis should be conducted by teams: Risk Management Team Risk Management Team Risk Assessment Team Risk Assessment Team Risk Communication Team Risk Communication Team Meet regularly to discussion process and surface issues

33. Organization of Risk Analysis Established three unique positions to help with consistency, coordination, and making decisions Established three unique positions to help with consistency, coordination, and making decisions Science Advisor for Risk Analysis Risk Analysis Coordinator Risk Assessment Project Manager

34. Summary of Recommendations Implemented Decision-based approach to identify and select all ‘major’ risk assessments Procedure for the conduct of risk assessment within risk analysis framework Established new organizational structure

35. Summary of Recommendations Implemented Criteria to evaluate data quality Procedures for the approval and release of risk assessments Formal peer review process Enhance capabilities to conduct risk assessments and interaction between chemical and microbiological risk assessments

36. Report Available “Initiation and Conduct of All ‘Major’ Risk Assessments within a Risk Analysis Framework” Available on FDA website: http//:www.cfsan.fda.gov/~dms/raf w-toc.html http//:www.cfsan.fda.gov/~dms/raf w-toc.html

37. New Twists OMB Guidelines on Data Quality Court decisions of status risk assessment when used as part of a regulatory activity Security issues when risk assessment used for food security activities