1. Benchmark Dose Modeling
Jay Zhao, M.P.H., Ph.D., DABT
Office of Research and Development
U.S. Environmental Protection Agency (EPA)
Subject of this training model is …
Much of the materials make use of models within BMDS (check website for latest release)
Other modeling programs are available and encouraged
However, BMDS was designed with Agency assessment needs and guidance in mind
Other tools may need to be supplemented to include recommended statistical considerations

2. Disclaimer
The views in this presentation are those of the authors and may not represent their Agency or organization policy.

3. Objective
Introduce Benchmark Dose (BMD) Modeling method and its application in dose-response analysis.

4. Outline Introduction to benchmark dose method
Traditional method (NOAEL)
Concept of BMD method
Pros and cons for BMD method
EPA’s BMD software (BMDS) and its available models
BMD modeling procedure
Evaluation of BMD modeling results

5. Glossary
BMD: An estimate of the dose or concentration that produces a predetermined change in response rate
BMDL: 95% lower-bound confidence limit on the BMD
BMR: A predetermined response level based on which a BMD or BMDL is calculated
POD: a point of departure used in estimate risk values when divided by an uncertainty factor
UF: Factors used in risk assessment to account for uncertainty in the data or extrapolations to human no-effect levels
Confidence interval: an interval estimate of a population parameter at a certain confidence level
AIC: Akaike’s Information Criterion used to assist in model evaluation based on overall data fitting and number of parameters used in the model

6. Reference Dose/Reference Concentration
NOAEL or LOAEL
RfD or RfC = UF
This shows how a RfD or risk value is calculated in the traditional NOAEL approach.
We need to identify the NOAEL or LOAEL, and UF.
NOAEL or LOAEL: No or Low Observed Adverse Effect Level
UF: Uncertainty Factor

7. Study Conducted with 50 Animals per Dose
0.2
0.4
0.6
0.8 1 50
100
150
200
Fraction Affected
Dose (mg/kg-day)
Gamma Multi-Hit * * *
NOAEL
Once a sensitive endpoint is identified, a dose response analysis should be conducted to identify the point of departure. In this case, the NOAEL.
NOAEL is usually the highest dose in an experiment that does not cause a significant adverse effect, and it is usually judged by a statistical analysis. However, there might be a problem with the NOAEL approach. As we know, a statistical analysis highly depends on statistical power in the data. For example, if we have a sample size of 50 in each dose group in this animal study, the highest dose that does not cause significant effect is the 50 mg/kg.
NOAEL=50
*:Statistically significant

8. Study Conducted with 10 Animals per Dose
0.2
0.4
0.6
0.8 1 50
100
150
200
Fraction Affected
dose
Gamma Multi-Hit * *
NOAEL
In contrast, if we have a sample size of 10 in each dose group shown on this slide, the highest dose that does not cause significant effect is the 100 mg/kg. Please note that NOAEL is not saying it does not cause any effect. It might cause effect, but this effect is just not statistically significant compared to the control group.
While both studies show the same percentage response, due to a high statistical power from a large sample size, the 50 sample size study shows the response at 100 mg/kg statistically different from the control group, but the small sample size study doesn’t. Therefore, 100 mg/kg that was LOAEL in the previous study now becomes the NOAEL in this small sample size experiment, while the next higher dose of 150 mg/kg becomes the LOAEL.
Thus, in the NOAEL approach, smaller sample size will result in the use of a higher NOAEL as the point of departure. This will result in a high (less health protective) risk value. Theoretically, because a smaller sample size will provide less confidence in the data, we should use a more conservative (lower) value as the point of departure to estimate the risk value instead of a higher or less conservative point of departure which will result in higher risk value (a less stringent value). Until the benchmark dose approach, this type of uncertainty could only be handled in a rather non-quantitative fashion via the use of uncertainty factors.
NOAEL=100

9. Limitations of Using a NOAEL or LOAEL
Limited to the doses tested
Response levels not comparable
Does not represent 0% response
Does not consider dose-response slope (“wastes” data)
NOAEL is not always available
Highly dependent on sample size
This slide summarizes the limitations of the NOAEL approach.

10. Study Conducted with 50 Animals per Dose
0.2
0.4
0.6
0.8 1 50
100
150
200
Fraction Affected
dose
BMDL
BMD
Best estimate
BMD Lower Bound
A new approach to address these limitations of NOAEL approach is to use benchmark dose as the point of departure.
In BMD approach:
Model the data with a curve fitting;
Identify the dose that cause the predetermined benchmark response;
Estimate the lower 95% confidence limit on the estimated the dose, i.e., BMDL, which broadens with decreasing sample size, resulting in a lower (more health protective) risk value.
BMR=0.1
BMDL=65.5
BMD=78

11. Study Conducted with 10 Animals per Dose
0.2
0.4
0.6
0.8 1 50
100
150
200
Fraction Affected
dose
BMDL
BMD
Best estimate
BMD Lower Bound
A new approach to address these limitations of NOAEL approach is to use benchmark dose as the point of departure.
In BMD approach:
Model the data with a curve fitting;
Identify the dose that cause the predetermined benchmark response;
Estimate the lower 95% confidence limit on the estimated the dose, i.e., BMDL, which broadens with decreasing sample size, resulting in a lower (more health protective) risk value.
BMR=0.1
BMDL=48.5
BMD=78

12. Benchmark Dose Definitions
BMD: An estimate of the dose or concentration that produces a predetermined change in response rate of an adverse effect (called the benchmark response or BMR) compared to background.
For example, an estimate of the dose that causes a 10% increase in the number of animals developing fatty liver compared with untreated animals.
BMDL: 95% Lower-Bound Confidence Limit on the BMD.

13. Benchmark Dose
Goal is to estimate a point of departure (POD) that is relatively independent of study design.

14. Deriving an RfD/RfC using a BMD
Equation for an RfD or RfC becomes:
BMDL or BMCL UF
No UF for LOAEL to NOAEL extrapolation
RfD or RfC =
Once we identify the BMDL, we can use it as the POD to calculate RfD.
It is important to understand that the BMDL is not a NOAEL
The BMDL represents an estimate of the dose (with 95% confidence) associated with the lowest response that can be comfortably modeled by the data
Since BMD is not a NOAEL, no LOAEL to NOAEL UF is needed when a BMDL from such data is used as the point of departure.

15. Advantages of BMD Approach
Not limited to doses tested experimentally
Less dependent on dose spacing
Takes into account the shape of the dose-response curve
Flexibility in determining biologically significant rates
Comparable results across chemicals and endpoints
Incentive to conduct better (larger) studies (less uncertainty)
The software has been peer reviewed, freely available, and widely accepted by regulatory agencies.

16. Challenges in the Use of BMD
Ability to estimate a BMD may be limited by the format of the data presented
Generally more complicated and time consuming
e.g., means and SD or individual animal response data are required for continuous measurements
However, NCEA is working on improvements to the BMDS interface and batch processing capabilities that should help to facilitate the analysis of multiple endpoints and models.

17. Are the Data Worth Modeling?
Evaluate database as for NOAEL approach
good quality studies
appropriate duration and route of exposure
measured endpoints of concern
Several procedures we need to follow in conducting BMD modeling.
The first is study evaluation. This is same as we need to do in NOAEL approach.

18. Are the Data Worth Modeling?
Significant dose-related trend
Two doses with responses in excess of the control
Responses that define the low end of the dose-response region are preferred
Data requirement for BMD modeling.

19. Are the Data Worth Modeling?
Model all biologically, statistically significant responses, if feasible
Model all the endpoints with LOAEL < 10-fold above the lowest LOAEL of the database
Consider dropping high dose group(s) that negatively impact low dose fit
To model a high dose “plateau” consider using a Hill or other models that contain an asymptote term

20. Benchmark Dose Software
Benchmark Dose Software is also called BMDS software.
It is developed by US EPA and it is available free from website:

21. Types of Models Dichotomous Model: for dichotomous or quantal data
Continuous Model: for continuous data
Nested Model: for nested dichotomous data

22. Model Selection - Dichotomous Data
Dichotomous models are used to evaluate quantal data, where an effect for an individual may be classified by one of two possible outcomes.
For example: dead or alive, tissue pathology (present/absent), and cancer incidence (yes/no)
What is dichotomous data.

23. BMDS Models for Dichotomous Data
Gamma
Logistic
Dose
Log dose
Probit
Multi-stage
Weibull
Quantal-Linear (power = 1)
Available dichotomous models in the BMDS software.

24. Model Selection - Continuous Data
Effects measured on a continuum
For example: body weight, organ weight, enzyme levels
What is continuous data

25. BMDS Models for Continuous Data
Polynomial (all-purpose model)
Linear (simplest model)
Non linear
Power (L-shaped dose-responses)
Linear
Hill (dose-responses that plateau)
Show the available continuous models in the BMDS software.

26. Model Selection - Nested Dichotomous Data
Developmental Toxicity Study
Dose 25 50
100
Dams …. …. …. ….
Litters/Pups
Show what nested data is. …. …. …. ….
Endpoints – Fetal weight, malformations, etc.

27. Nested Dichotomous Data
Malformation in neonates
Sternebral defect
Vertebral arch defect
Ossification changes in neonates
What is continuous data

28. BMDS Models for Nested Dichotomous Data
Logistic Nested Model (NLogistic)
NCTR
Rai & Van Ryzin Model
Show the available nested models in the BMDS software.

29. Model Selection – Other Considerations?
Most BMD models are not biologically based, but all model fits must be biologically tenable
To be biologically tenable, model parameters may need to be restricted
Consider using model with asymptote term for saturable responses
Explain what is nonlinear models.

30. Consider combining BMDLs
START
1. Choose BMR(s) to Evaluate
2. Select a model, set parameters and run the model No
3. Does the model fit the data?
Yes No
4. Have all models/model options been considered?
Yes No
5. Evaluate BMDLs. Are they in 3-fold range?
Use lowest BMDL
This EPA’s decision making flow chart in BMD analysis:
1. Select a BMR
2. Select a model/modeling approach
3. Evaluate fit (p>=0.1; low-dose fit)
4. Consider all models that adequately fit and describe the relevant low-dose portion of the dose-response.
5. If the BMDL estimates from the adequate fitting models are within a factor of 3, then they are considered to show no appreciable model dependence and the “best fitting” model should be used. If a single “best fitting model can not be determined, consider combining BMDLs somehow.
6. If the BMDL estimates from the remaining models are not within a factor of 3, some model dependence of the estimate is assumed. Since there is no clear remaining biological or statistical basis on which to choose among them, the lowest BMDL is selected as a reasonable conservative estimate. If the lowest BMDL from the available models appears to be an outlier, compared to the other results (e.g., there are several other results, all within a factor of 3), then additional analysis and discussion would be appropriate. Additional analysis might include the use of additional models, the examination of the parameter values for the BMDLs.
In any case, after steps 5 or 6, document as outlined…of the EPA 2000 technical guidance document
Yes No
6. Does one model fit best?
Consider combining BMDLs
Yes
Use BMDL from the model that provides the best fit
Document the BMD analysis as outlined in reporting requirements.

31. Select A Benchmark Response
BMR should be near the low end of the range of increased risks that can be detected by a bioassay.
Low BMRs can impart high model dependence, i.e., different models will provide different BMDL estimates.

32. Benchmark Dose BMR 0.2 0.4 0.6 0.8 1 50 100 150 200 Fraction Affected
0.2
0.4
0.6
0.8 1 50
100
150
200
Fraction Affected
dose
Gamma Multi-Hit Model with 0.95 Confidence Level
BMDL
BMD
Gamma Multi-Hit
BMD Lower Bound
NOAEL
A new approach to address these limitations of NOAEL approach is to use benchmark dose as the point of departure.
In BMD approach:
Model the data with a curve fitting;
Identify the dose that cause the predetermined benchmark response;
Estimate the lower 95% confidence limit on the estimated the dose, i.e., BMDL, which broadens with decreasing sample size, resulting in a lower (more health protective) risk value.
BMR

33. BMR Selection: Choose BMR(s) to Evaluate Dichotomous Data
Extra risk of 10% is a BMR for comparison purpose by US EPA, since the 10% response is at or near the limit of sensitivity in most cancer bioassays and in some non-cancer bioassays.
If a study has greater than usual sensitivity, a lower BMR can be used.
US EPA recommends that BMD10 and BMDL10 should always be presented for comparison purpose.
Various papers have proposed a BMR for quantal responses in the range of 1% to 10%.
Comparison of the BMD with the NOAEL for a large number of developmental toxicity data sets indicated a BMR in the range of 5-10% resulted in a BMD that was on average similar to the NOAEL.

34. BMR Selection: Choose BMR(s) to Evaluate (Continuous Data)
If there is an accepted level of change in the endpoint that is considered to be biologically significant, then that amount of change is the BMR.
In the absence of any other idea of what level of response to be considered biologically significant, a change in the mean equal to one control standard deviation (1.0 SD) from the control mean can be used.
1. A change in average adult body weight of 10%, or the doubling of average level for some liver enzyme is usually considered biologically significant.
2. If lower 1% (or upper 99%) of control group is considered suitable marker of adversity, a BMR of a mean change of 1.1 SD from the control mean will result in a 10% increase in adversely effected individuals. Therefore, one SD as the BMR should always be used, at least for a reference purpose.

35. Restricting Parameters
BMD Guidance suggests restricting models initially.
Unrestricted, some models take on unrealistic forms.
Number of parameters in a model cannot exceed the number of dose groups.
Discussion of parameter restriction.

38. 0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8 20 40 60 80
100
120
140
Fraction Affected
dose
Gamma Multi-Hit Model with 0.95 Confidence Level
BMDL
BMD
Gamma Multi-Hit
BMD Lower Bound

40. Does the Model Fit the Data?
Global measurement: goodness-of-fit p value (p > 0.1)
Local measurement: Scaled residuals (absolute value < 2.0)
Visual inspection of model fitting
Note: Consider how well the model predicts both responses and response variance (in the case of continuous data).
Evaluation of a model fit to the data usually can be grouped into three parts. BMD output and curve fitting figure provide this information.

41. Have All Options Been Considered?
Goal of BMD modeling - fit a model to dose-response data that best describes the data, especially at the lower end of the dose-response range.
This may require the application of several models and model options, or just a few.
We need keep this in mind when we evaluate the BMD results.

42. Summary of BMD Results Model P value AIC Residual at 0 at 5% BMD BMDL
Gamma
0.9032
295.6
0.230
64.3
54.5
Logistic
0.3317
298.5
-0.996
-0.246
70.5
61.8
Log-Logistic
0.8851
0.287
65.1
55.3
Multistage
0.2520
297.9
-1.545
51.3
45.7
Probit
0.6543
296.6
-0.628
-0.077
67.8
58.9
Log-Probit
0.6513
296.2
0.443
63.7
54.9
Quantal-Linear
324.4
-3.69
24.1
20.5
Weibull
0.9912
295.4
-0.083
53.7

43. BMDL Estimates Within 3-fold Range?
Often, more than one model will result in an acceptable fit to the data.
Consider using the lowest BMDL if BMDL estimates from acceptable models are widely divergent (e.g., outside of a 3-fold range).
Consider relative model fit if model results in similar BMDL estimates (e.g., within a 3-fold range).
NOTES:
Using a value of a=0.1 to determine a critical value.
________________________________________________
Further reject models that apparently do no t adequately
describe

44. Is There One Model That Fits The Data Best?
Global measurement: goodness-of-fit p value (p > 0.1)
Local measurement: Scaled residuals near the BMR
Visual comparison of model fits (e.g., to detect systemic or high dose bias)
Comparison of Akaike’s Information Criterion (AIC) (smaller is better)

45. Akaike’s Information Criterion (AIC)
AIC = -2 x LL + 2 x P
LL = log-likelihood at the maximum likelihood estimates for parameters
p = number of model parameters estimated
Within a family of models, fit will improve as parameters are added.
For a similar degree of fit, AIC rewards the less complex model (with less parameters).
Akaike’s Information Criterion (AIC) can be used to compare models from different families using a similar fitting method (for example, least squares or a binomial maximum likelihood) (smaller number is best)

46. Summary of BMD Results Model P value AIC Residual at 0 at 5% BMD BMDL
Gamma
0.9032
295.6
0.230
64.3
54.5
Logistic
0.3317
298.5
-0.996
-0.246
70.5
61.8
Log-Logistic
0.8851
0.287
65.1
55.3
Multistage
0.2520
297.9
-1.545
51.3
45.7
Probit
0.6543
296.6
-0.628
-0.077
67.8
58.9
Log-Probit
0.6513
296.2
0.443
63.7
54.9
Quantal-Linear
324.4
-3.69
24.1
20.5
Weibull
0.9912
295.4
-0.083
53.7

47. Deriving an RfD/RFC from a BMDL
BMDL or BMCL UF
RfD or RfC =
Apply the BMDL in the RfD or risk value calculation.

48. Conclusions BMD method uses more dose-response information.
It provides a better way for comparing different endpoints.
This method gives incentive to conduct better studies (with less uncertainty).
BMD modeling requires more information on the data and it is more time consuming.

49. References
Collins, J. F., Alexeeff, G. V., Lewis, D. C., Dodge, D. E., Marty, M. A., Parker, T. R., Budroe, J. D., Lam, R. H., Lipsett, M. J., Fowles, J. R., & Das, R. (2004). Development of acute inhalation reference exposure levels (RELs) to protect the public from predictable excursions of airborne toxicants. J Appl Toxicol, 24(2),
Crump, K. (2002). Critical Issues in Benchmark Calculations from Continuous Data. Critical Reviews in Toxicology, 32(3),
Crump, K. S. (1984). A new method for determining allowable daily intakes. Fundam Appl Toxicol, 4(5),
Filipsson, A. F., Sand, S., Nilsson, J., & Victorin, K. (2003). The benchmark dose method--review of available models, and recommendations for application in health risk assessment. Crit Rev Toxicol, 33(5),
U.S. Environmental Protection Agency. (1995). The Use of the Benchmark Dose Approach in Health Risk Assessment (EPA/630/R-94/007): Office of Research and Development. (
U.S. Environmental Protection Agency. (2000). Benchmark Dose Technical Guidance Document. External Review Draft (EPA/630/R-00/001). Washington, DC: Risk Assessment Forum. (
U.S. Environmental Protection Agency. (2007). BenchMark Dose Software (Version 1.4.1c): National Center for Environmental Assessment. (