Benchmark Dose Modeling Jay Zhao, M.P.H., Ph.D., DABT Office of Research and Development U.S. Environmental Protection Agency (EPA) 513-569-7373 Zhao.jay@epa.gov Subject of this training model is … Much of the materials make use of models within BMDS (check website for latest release) Other modeling programs are available and encouraged However, BMDS was designed with Agency assessment needs and guidance in mind Other tools may need to be supplemented to include recommended statistical considerations

Disclaimer The views in this presentation are those of the authors and may not represent their Agency or organization policy.

Objective Introduce Benchmark Dose (BMD) Modeling method and its application in dose-response analysis.

Outline Introduction to benchmark dose method Traditional method (NOAEL) Concept of BMD method Pros and cons for BMD method EPA’s BMD software (BMDS) and its available models BMD modeling procedure Evaluation of BMD modeling results

Glossary BMD: An estimate of the dose or concentration that produces a predetermined change in response rate BMDL: 95% lower-bound confidence limit on the BMD BMR: A predetermined response level based on which a BMD or BMDL is calculated POD: a point of departure used in estimate risk values when divided by an uncertainty factor UF: Factors used in risk assessment to account for uncertainty in the data or extrapolations to human no-effect levels Confidence interval: an interval estimate of a population parameter at a certain confidence level AIC: Akaike’s Information Criterion used to assist in model evaluation based on overall data fitting and number of parameters used in the model

Reference Dose/Reference Concentration NOAEL or LOAEL RfD or RfC = UF This shows how a RfD or risk value is calculated in the traditional NOAEL approach. We need to identify the NOAEL or LOAEL, and UF. NOAEL or LOAEL: No or Low Observed Adverse Effect Level UF: Uncertainty Factor

Study Conducted with 50 Animals per Dose 0.2 0.4 0.6 0.8 1 50 100 150 200 Fraction Affected Dose (mg/kg-day) Gamma Multi-Hit * * * NOAEL Once a sensitive endpoint is identified, a dose response analysis should be conducted to identify the point of departure. In this case, the NOAEL. NOAEL is usually the highest dose in an experiment that does not cause a significant adverse effect, and it is usually judged by a statistical analysis. However, there might be a problem with the NOAEL approach. As we know, a statistical analysis highly depends on statistical power in the data. For example, if we have a sample size of 50 in each dose group in this animal study, the highest dose that does not cause significant effect is the 50 mg/kg. NOAEL=50 *:Statistically significant

Study Conducted with 10 Animals per Dose 0.2 0.4 0.6 0.8 1 50 100 150 200 Fraction Affected dose Gamma Multi-Hit * * NOAEL In contrast, if we have a sample size of 10 in each dose group shown on this slide, the highest dose that does not cause significant effect is the 100 mg/kg. Please note that NOAEL is not saying it does not cause any effect. It might cause effect, but this effect is just not statistically significant compared to the control group. While both studies show the same percentage response, due to a high statistical power from a large sample size, the 50 sample size study shows the response at 100 mg/kg statistically different from the control group, but the small sample size study doesn’t. Therefore, 100 mg/kg that was LOAEL in the previous study now becomes the NOAEL in this small sample size experiment, while the next higher dose of 150 mg/kg becomes the LOAEL. Thus, in the NOAEL approach, smaller sample size will result in the use of a higher NOAEL as the point of departure. This will result in a high (less health protective) risk value. Theoretically, because a smaller sample size will provide less confidence in the data, we should use a more conservative (lower) value as the point of departure to estimate the risk value instead of a higher or less conservative point of departure which will result in higher risk value (a less stringent value). Until the benchmark dose approach, this type of uncertainty could only be handled in a rather non-quantitative fashion via the use of uncertainty factors. NOAEL=100

Limitations of Using a NOAEL or LOAEL Limited to the doses tested Response levels not comparable Does not represent 0% response Does not consider dose-response slope (“wastes” data) NOAEL is not always available Highly dependent on sample size This slide summarizes the limitations of the NOAEL approach.

Study Conducted with 50 Animals per Dose 0.2 0.4 0.6 0.8 1 50 100 150 200 Fraction Affected dose BMDL BMD Best estimate BMD Lower Bound A new approach to address these limitations of NOAEL approach is to use benchmark dose as the point of departure. In BMD approach: Model the data with a curve fitting; Identify the dose that cause the predetermined benchmark response; Estimate the lower 95% confidence limit on the estimated the dose, i.e., BMDL, which broadens with decreasing sample size, resulting in a lower (more health protective) risk value. BMR=0.1 BMDL=65.5 BMD=78

Study Conducted with 10 Animals per Dose 0.2 0.4 0.6 0.8 1 50 100 150 200 Fraction Affected dose BMDL BMD Best estimate BMD Lower Bound A new approach to address these limitations of NOAEL approach is to use benchmark dose as the point of departure. In BMD approach: Model the data with a curve fitting; Identify the dose that cause the predetermined benchmark response; Estimate the lower 95% confidence limit on the estimated the dose, i.e., BMDL, which broadens with decreasing sample size, resulting in a lower (more health protective) risk value. BMR=0.1 BMDL=48.5 BMD=78

Benchmark Dose Definitions BMD: An estimate of the dose or concentration that produces a predetermined change in response rate of an adverse effect (called the benchmark response or BMR) compared to background. For example, an estimate of the dose that causes a 10% increase in the number of animals developing fatty liver compared with untreated animals. BMDL: 95% Lower-Bound Confidence Limit on the BMD.

Benchmark Dose Goal is to estimate a point of departure (POD) that is relatively independent of study design.

Deriving an RfD/RfC using a BMD Equation for an RfD or RfC becomes: BMDL or BMCL UF No UF for LOAEL to NOAEL extrapolation RfD or RfC = Once we identify the BMDL, we can use it as the POD to calculate RfD. It is important to understand that the BMDL is not a NOAEL The BMDL represents an estimate of the dose (with 95% confidence) associated with the lowest response that can be comfortably modeled by the data Since BMD is not a NOAEL, no LOAEL to NOAEL UF is needed when a BMDL from such data is used as the point of departure.

Advantages of BMD Approach Not limited to doses tested experimentally Less dependent on dose spacing Takes into account the shape of the dose-response curve Flexibility in determining biologically significant rates Comparable results across chemicals and endpoints Incentive to conduct better (larger) studies (less uncertainty) The software has been peer reviewed, freely available, and widely accepted by regulatory agencies.

Challenges in the Use of BMD Ability to estimate a BMD may be limited by the format of the data presented Generally more complicated and time consuming e.g., means and SD or individual animal response data are required for continuous measurements However, NCEA is working on improvements to the BMDS interface and batch processing capabilities that should help to facilitate the analysis of multiple endpoints and models.

Are the Data Worth Modeling? Evaluate database as for NOAEL approach good quality studies appropriate duration and route of exposure measured endpoints of concern Several procedures we need to follow in conducting BMD modeling. The first is study evaluation. This is same as we need to do in NOAEL approach.

Are the Data Worth Modeling? Significant dose-related trend Two doses with responses in excess of the control Responses that define the low end of the dose-response region are preferred Data requirement for BMD modeling.

Are the Data Worth Modeling? Model all biologically, statistically significant responses, if feasible Model all the endpoints with LOAEL < 10-fold above the lowest LOAEL of the database Consider dropping high dose group(s) that negatively impact low dose fit To model a high dose “plateau” consider using a Hill or other models that contain an asymptote term

Benchmark Dose Software Benchmark Dose Software is also called BMDS software. It is developed by US EPA and it is available free from website: http://www.epa.gov/ncea/bmds/

Types of Models Dichotomous Model: for dichotomous or quantal data Continuous Model: for continuous data Nested Model: for nested dichotomous data

Model Selection - Dichotomous Data Dichotomous models are used to evaluate quantal data, where an effect for an individual may be classified by one of two possible outcomes. For example: dead or alive, tissue pathology (present/absent), and cancer incidence (yes/no) What is dichotomous data.

BMDS Models for Dichotomous Data Gamma Logistic Dose Log dose Probit Multi-stage Weibull Quantal-Linear (power = 1) Available dichotomous models in the BMDS software.

Model Selection - Continuous Data Effects measured on a continuum For example: body weight, organ weight, enzyme levels What is continuous data

BMDS Models for Continuous Data Polynomial (all-purpose model) Linear (simplest model) Non linear Power (L-shaped dose-responses) Linear Hill (dose-responses that plateau) Show the available continuous models in the BMDS software.

Model Selection - Nested Dichotomous Data Developmental Toxicity Study Dose 25 50 100 Dams …. …. …. …. Litters/Pups Show what nested data is. …. …. …. …. Endpoints – Fetal weight, malformations, etc.

Nested Dichotomous Data Malformation in neonates Sternebral defect Vertebral arch defect Ossification changes in neonates What is continuous data

BMDS Models for Nested Dichotomous Data Logistic Nested Model (NLogistic) NCTR Rai & Van Ryzin Model Show the available nested models in the BMDS software.

Model Selection – Other Considerations? Most BMD models are not biologically based, but all model fits must be biologically tenable To be biologically tenable, model parameters may need to be restricted Consider using model with asymptote term for saturable responses Explain what is nonlinear models.

Consider combining BMDLs START 1. Choose BMR(s) to Evaluate 2. Select a model, set parameters and run the model No 3. Does the model fit the data? Yes No 4. Have all models/model options been considered? Yes No 5. Evaluate BMDLs. Are they in 3-fold range? Use lowest BMDL This EPA’s decision making flow chart in BMD analysis: 1. Select a BMR 2. Select a model/modeling approach 3. Evaluate fit (p>=0.1; low-dose fit) 4. Consider all models that adequately fit and describe the relevant low-dose portion of the dose-response. 5. If the BMDL estimates from the adequate fitting models are within a factor of 3, then they are considered to show no appreciable model dependence and the “best fitting” model should be used. If a single “best fitting model can not be determined, consider combining BMDLs somehow. 6. If the BMDL estimates from the remaining models are not within a factor of 3, some model dependence of the estimate is assumed. Since there is no clear remaining biological or statistical basis on which to choose among them, the lowest BMDL is selected as a reasonable conservative estimate. If the lowest BMDL from the available models appears to be an outlier, compared to the other results (e.g., there are several other results, all within a factor of 3), then additional analysis and discussion would be appropriate. Additional analysis might include the use of additional models, the examination of the parameter values for the BMDLs. In any case, after steps 5 or 6, document as outlined…of the EPA 2000 technical guidance document Yes No 6. Does one model fit best? Consider combining BMDLs Yes Use BMDL from the model that provides the best fit Document the BMD analysis as outlined in reporting requirements.

Select A Benchmark Response BMR should be near the low end of the range of increased risks that can be detected by a bioassay. Low BMRs can impart high model dependence, i.e., different models will provide different BMDL estimates.

Benchmark Dose BMR 0.2 0.4 0.6 0.8 1 50 100 150 200 Fraction Affected 0.2 0.4 0.6 0.8 1 50 100 150 200 Fraction Affected dose Gamma Multi-Hit Model with 0.95 Confidence Level BMDL BMD Gamma Multi-Hit BMD Lower Bound NOAEL A new approach to address these limitations of NOAEL approach is to use benchmark dose as the point of departure. In BMD approach: Model the data with a curve fitting; Identify the dose that cause the predetermined benchmark response; Estimate the lower 95% confidence limit on the estimated the dose, i.e., BMDL, which broadens with decreasing sample size, resulting in a lower (more health protective) risk value. BMR

BMR Selection: Choose BMR(s) to Evaluate Dichotomous Data Extra risk of 10% is a BMR for comparison purpose by US EPA, since the 10% response is at or near the limit of sensitivity in most cancer bioassays and in some non-cancer bioassays. If a study has greater than usual sensitivity, a lower BMR can be used. US EPA recommends that BMD10 and BMDL10 should always be presented for comparison purpose. Various papers have proposed a BMR for quantal responses in the range of 1% to 10%. Comparison of the BMD with the NOAEL for a large number of developmental toxicity data sets indicated a BMR in the range of 5-10% resulted in a BMD that was on average similar to the NOAEL.

BMR Selection: Choose BMR(s) to Evaluate (Continuous Data) If there is an accepted level of change in the endpoint that is considered to be biologically significant, then that amount of change is the BMR. In the absence of any other idea of what level of response to be considered biologically significant, a change in the mean equal to one control standard deviation (1.0 SD) from the control mean can be used. 1. A change in average adult body weight of 10%, or the doubling of average level for some liver enzyme is usually considered biologically significant. 2. If lower 1% (or upper 99%) of control group is considered suitable marker of adversity, a BMR of a mean change of 1.1 SD from the control mean will result in a 10% increase in adversely effected individuals. Therefore, one SD as the BMR should always be used, at least for a reference purpose.

Restricting Parameters BMD Guidance suggests restricting models initially. Unrestricted, some models take on unrealistic forms. Number of parameters in a model cannot exceed the number of dose groups. Discussion of parameter restriction.

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 20 40 60 80 100 120 140 Fraction Affected dose Gamma Multi-Hit Model with 0.95 Confidence Level BMDL BMD Gamma Multi-Hit BMD Lower Bound

Does the Model Fit the Data? Global measurement: goodness-of-fit p value (p > 0.1) Local measurement: Scaled residuals (absolute value < 2.0) Visual inspection of model fitting Note: Consider how well the model predicts both responses and response variance (in the case of continuous data). Evaluation of a model fit to the data usually can be grouped into three parts. BMD output and curve fitting figure provide this information.

Have All Options Been Considered? Goal of BMD modeling - fit a model to dose-response data that best describes the data, especially at the lower end of the dose-response range. This may require the application of several models and model options, or just a few. We need keep this in mind when we evaluate the BMD results.

Summary of BMD Results Model P value AIC Residual at 0 at 5% BMD BMDL Gamma 0.9032 295.6 0.230 64.3 54.5 Logistic 0.3317 298.5 -0.996 -0.246 70.5 61.8 Log-Logistic 0.8851 0.287 65.1 55.3 Multistage 0.2520 297.9 -1.545 51.3 45.7 Probit 0.6543 296.6 -0.628 -0.077 67.8 58.9 Log-Probit 0.6513 296.2 0.443 63.7 54.9 Quantal-Linear 324.4 -3.69 24.1 20.5 Weibull 0.9912 295.4 -0.083 53.7

BMDL Estimates Within 3-fold Range? Often, more than one model will result in an acceptable fit to the data. Consider using the lowest BMDL if BMDL estimates from acceptable models are widely divergent (e.g., outside of a 3-fold range). Consider relative model fit if model results in similar BMDL estimates (e.g., within a 3-fold range). NOTES: Using a value of a=0.1 to determine a critical value. ________________________________________________ Further reject models that apparently do no t adequately describe

Is There One Model That Fits The Data Best? Global measurement: goodness-of-fit p value (p > 0.1) Local measurement: Scaled residuals near the BMR Visual comparison of model fits (e.g., to detect systemic or high dose bias) Comparison of Akaike’s Information Criterion (AIC) (smaller is better)

Akaike’s Information Criterion (AIC) AIC = -2 x LL + 2 x P LL = log-likelihood at the maximum likelihood estimates for parameters p = number of model parameters estimated Within a family of models, fit will improve as parameters are added. For a similar degree of fit, AIC rewards the less complex model (with less parameters). Akaike’s Information Criterion (AIC) can be used to compare models from different families using a similar fitting method (for example, least squares or a binomial maximum likelihood) (smaller number is best)

Summary of BMD Results Model P value AIC Residual at 0 at 5% BMD BMDL Gamma 0.9032 295.6 0.230 64.3 54.5 Logistic 0.3317 298.5 -0.996 -0.246 70.5 61.8 Log-Logistic 0.8851 0.287 65.1 55.3 Multistage 0.2520 297.9 -1.545 51.3 45.7 Probit 0.6543 296.6 -0.628 -0.077 67.8 58.9 Log-Probit 0.6513 296.2 0.443 63.7 54.9 Quantal-Linear 324.4 -3.69 24.1 20.5 Weibull 0.9912 295.4 -0.083 53.7

Deriving an RfD/RFC from a BMDL BMDL or BMCL UF RfD or RfC = Apply the BMDL in the RfD or risk value calculation.

Conclusions BMD method uses more dose-response information. It provides a better way for comparing different endpoints. This method gives incentive to conduct better studies (with less uncertainty). BMD modeling requires more information on the data and it is more time consuming.

References Collins, J. F., Alexeeff, G. V., Lewis, D. C., Dodge, D. E., Marty, M. A., Parker, T. R., Budroe, J. D., Lam, R. H., Lipsett, M. J., Fowles, J. R., & Das, R. (2004). Development of acute inhalation reference exposure levels (RELs) to protect the public from predictable excursions of airborne toxicants. J Appl Toxicol, 24(2), 155-166. Crump, K. (2002). Critical Issues in Benchmark Calculations from Continuous Data. Critical Reviews in Toxicology, 32(3), 133-153. Crump, K. S. (1984). A new method for determining allowable daily intakes. Fundam Appl Toxicol, 4(5), 854-871. Filipsson, A. F., Sand, S., Nilsson, J., & Victorin, K. (2003). The benchmark dose method--review of available models, and recommendations for application in health risk assessment. Crit Rev Toxicol, 33(5), 505-542. U.S. Environmental Protection Agency. (1995). The Use of the Benchmark Dose Approach in Health Risk Assessment (EPA/630/R-94/007): Office of Research and Development. (http://cfpub.epa.gov/ncea/raf/recordisplay.cfm?deid=42601) U.S. Environmental Protection Agency. (2000). Benchmark Dose Technical Guidance Document. External Review Draft (EPA/630/R-00/001). Washington, DC: Risk Assessment Forum. (http://www.epa.gov/ncea/pdfs/bmds/BMD-External_10_13_2000.pdf) U.S. Environmental Protection Agency. (2007). BenchMark Dose Software (Version 1.4.1c): National Center for Environmental Assessment. (http://www.epa.gov/ncea/bmds/)