Toward More Rational Risk Analysis: Improving the ranking process in #152 Tony Cox Cox Associates FDA Anti-Infective Drugs Advisory Committeee Meeting

© Cox Associates, (303) What is necessary for rational decision-making? Identify risk management alternatives. –E.g., continue, ban, restrict, wait-and-see, etc Assess the probable human health consequences of each alternative decision option. –Quantify probabilities of health consequences Choose the option giving the most desirable probable consequences. –Choose options giving larger probabilities of preferred consequences

© Cox Associates, (303) CVM’s proposed qualitative approach does not support rational decisions Crucial quantitative information needed for rational decision-making is omitted. –Quantitative extent of exposure is ignored, but is essential for determining probable human health outcomes Requires subjective probability rating of ambiguous events and undefined concepts –Release: Pr(resistant bacteria present in animal “as a result of” the drug use) is not defined. (Usually zero when the drug just selects for resistant bacteria that are already there!) –Consequence: Pr(exposure “results in” adverse consequence) is not defined, especially when exposure is not quantified.

© Cox Associates, (303) Other key ambiguities Hazard: “illness caused by a specified resistant bacteria”, “attributable to a specified animal-derived food commodity” – italicized terms are undefined. Hazardous agent: “resistant bacteria attributable to a specified commodity” – “attributable to” is not defined. Risk = Pr(human illness caused by a specified resistant bacteria attributable to a specified commodity, treated with drug of interest) –For fluoroquinolones, CVM interpreted “attributable to” to mean “not attributed by us to anything else”. No objective definition or data-based criteria used – purely subjective judgments. –This definition of risk does not address probable consequences of risk management alternatives  No support for rational decisions!

© Cox Associates, (303) Performance problems The key aspect of rational decision-making is that it recommends the actions with the most preferred probable net benefits. –Example: “Worst first” problem ranking But CVM’s proposed approach recommends actions without regard for their net benefits (which depend on quantitative exposures) –Ignores effects on non-resistant microbial load Ignores human health benefits of continued use Ignores probable extent of adverse effects

© Cox Associates, (303) A key disconnect Potential human health consequences of exposure can not be estimated from human medical importance of a drug. –Example: Drug is very important but exposure does not reduce (or perhaps enhances) its performance. For decision-making purposes, it is not the health consequences of exposures, but health consequences of risk management interventions that must be assessed.

© Cox Associates, (303) CVM’s three factors do not predict risk The three factors of “Release”, “Exposure”, and “Consequence” (as interpreted by CVM) cannot logically or rationally be used to estimate or rank human health risks. The key questions – how much harm does exposure cause (if any) and how would this change if a ban or intervention were implemented – are not addressed by these factors. No expected correlation between “qualitative” risk ranking and “rational” (or quantitative) ranking

© Cox Associates, (303) Example: Virginiamycin and Synercid CDER/CVM Criteria for ranking drugs based on “importance”: Factors related to drug efficacy 1) sole therapy/limited available therapies for treating humans 2) therapy of choice for human infection(s) 3) spectrum of activity of particular importance 4) importance as an oral (rather than parenteral) therapy 5) importance in treating food-borne infections 6) unique mechanism of antimicrobial action Factors related to development of antimicrobial resistance 7) cross-resistance within drug class 8) cross-resistance across drug classes 9) ease of transmissibility of resistance determinants 10) cross resistance between animal and human drugs

© Cox Associates, (303) Qualitative vs. Quantitative Based on the proposed criteria, it appears that the qualitative risk assessment procedure could assign a risk rating of “high” to virginiamycin (VM) – even if VM does not cause human harm. A quantitative risk assessment for VM and Synercid suggests that banning VM in 1Q-02 would have saved not more than 0.18 statistical lives in the whole US population over the next five years. Such a perspective is (a) Useful for decision-making; and (b) Missing from CVM’s proposed qualitative approach.

© Cox Associates, (303) Quantitative vs. Qualitative: What factors determining risk are considered ? Quantitative FactorMeanKey Data VRE cases x % that are vanA VREF 9371 x 0.61 Eliopoulos et al., 1998 Clark et al., 1993 Jones et al, 1995, Rice 2001 % not nosocomial0.17 Bischoff et al., 1999; Austen et al., 1999; Thal et al., 1998 % chicken-compatible0 to Willems et al., 2000, 2001 % prescribed QD0.92 t AMR, 2001; declining over time, Zyvox increasing % QD-resistant0 to 0.01 Eliopoulos et al., 1998 Jones et al, 1999 % Effective treatments0.72 Moellering et al., 1999; Linden, 2002

© Cox Associates, (303) Quantitative vs. Qualitative A rough bounding quantitative assessment can be… (a)Quicker, easier, and less costly than a qualitative assessment (b)Produce clearer, more objective results (c)Provide a more constructive framework for surfacing concerns and resolving conflicts (d)Better able to support rational decision-making

© Cox Associates, (303) Key Steps in Health Risk Assessment 0. Bound/scope the analysis to support better decisions –What decisions and consequences matter? 1.Hazard identification: What are the source and effect? 2.Exposure assessment: Who and how much? –Frequency/magnitude of individual exposures in the population. Who is exposed to how much how often? 3.Individual dose-response model: How do exposure and other factors affect the frequency/severity of health effects? 4.Risk characterization: How will decisions change frequency of adverse health effects in population? 5.Uncertainty analysis: How confident are we in the results? What new information could change the recommendations?

© Cox Associates, (303) Conclusions Risk is quantitative, not qualitative (at least as used in rational risk-management decision-making) Human health impacts of regulations are transmitted by causal chains that carry amounts of physical quantities (e.g., colony-forming units of susceptible and resistant bacteria) from animals to humans. Accurate risk assessment must estimate effects of regulation on these quantities (exposure assessment) and on their probable adverse effects (dose-response modeling). Not doing so can randomize resource allocations and fail to protect human health. Doing so supports effective regulatory decisions.

© Cox Associates, (303) Recommendations Use quantitative risk assessment methods (e.g., upper-bounding analyses) to quantify or bound the probable human health consequences of proposed decisions. Clearly define key terms. Provide objective interpretations. Use simulation to quantify/validate performance of the final recommended guidance

© Cox Associates, (303) Appendix: Details of VM Example

© Cox Associates, (303) Quantifying Risk Human treatment failures due to VM = Product of: 1.Total vanA E. faecium (VREF) cases 2.Proportion not of nosocomial origin 3.Proportion with genogroup found in chickens 4.Proportion that are prescribed QD 5.Proportion that are QD-resistant 6.Proportion that would have treatment success Each parameter can be estimated from available medical and genetic typing data.

© Cox Associates, (303) Proportion of total VRE isolates that are vanA VREF Focus is on vanA VREF because vanB VREF are susceptible to teicoplanin (Eliopoulos, 1998); QD is usually not prescribed to vanB VREF patients (Murray, 2000); QD is not active against Enterococcus faecalis, and “Only vanA E. faecium can be linked to food animals.” (EAGAR, 2002) Parameter estimates based on data: –USA: E. faecium isolates submitted to the CDC from 1988 to 1992 were 83% vanA (Clark et al., 1993). In a 1992 survey of 97 US laboratories, 79% of VREF isolates were vanA (Jones et al, 1995).

© Cox Associates, (303) Proportion of vanA VREFs Attributed to Chickens Willems et al (2000) found four major AFLP genogroups among 255 vanA VREF strains. Among hospitalized patients: 4 had genogroup A strains (occurs in pigs, most community patients) 10 had genogroup B strains (found in 30 of 31 chickens, veal calves, community patients); 73 had genogroup C strains (found in hospitalized patients, veal calves, isolates from cats and dogs) Attributing all Group B strains (10 of 87 hospitalized patients) to chicken (conservative) gives a chicken- attributable fraction of Beta(11, 78), mean = 0.124

© Cox Associates, (303) QD-Resistance Levels in Humans Eliopoulos et al., 1998 found that QD failed to inhibit of the first isolates submitted by any single patient at  2  g/ml. Among 875 isolates from hospitals across the US (including multiple submissions) 29 (3.3%) had MIC was  4  g/ml The NCCLS has established the following QD resistance breakpoints for enterococci:  1 mg/L susceptible, 2 mg/L intermediate, and  4 mg/L resistant.

© Cox Associates, (303) VM-Attributable Risk Calculation FactorMeanKey Data VRE x % vanA VREF9371 x 0.61 Eliopoulos et al., 1998 Clark et al., 1993 Jones et al, 1995, Rice 2001 % not nosocomial0.17 Bischoff et al., 1999; Austen et al., 1999; Thal et al., 1998 % chicken-compatible0 to Willems et al., 2000, 2001 % prescribed QD0.92 t AMR, 2001; declining over time, Zyvox increasing % QD-resistant0 to 0.01 Eliopoulos et al., 1998 Jones et al, 1999 % Effective treatments0.72 Moellering et al., 1999; Linden, 2002

© Cox Associates, (303) Effects Model “Conservative Bayesian” (uniform prior, beta posterior) uncertainty models + Monte-Carlo uncertainty analysis. (Bias means toward.5) Health consequences per treatment failure:  Mortalities = x treatment failures (Linden et al., 1997) (ignoring new treatment options such as Linezolid)  Life-years = 22 (if same as healthy people)  Quality-Adjusted Life Years = 0.04 (if treatment-days are as bad as life days lost)

© Cox Associates, (303) Main Results for US Banning VM now (1Q-02) could save up to: cases of treatment failure statistical lives life-years for whole U.S. population over next 5 years. If ban is delayed until 2Q-04, expected impact declines to statistical lives saved.

© Cox Associates, (303) Timing of Impacts

© Cox Associates, (303) What drives the numbers? Risk = product of small numbers. Key values come from available data. Risk = 0 if gene transfer from chickens to humans does not occur Upper-bounds risk estimates assume maximum attribution of human resistance to chickens.