Christoffer W Tornoe Oct Population PK Model Building 1 Q & A on Session 1 What is naïve pooled analysis? –Definition –One advantage/disadvantage What is naïve averaged analysis? –Definition –One advantage/disadvantage What is a Two stage method? –Definition –One advantage/disadvantage What is a One stage method? –Definition –One advantage/disadvantage

Christoffer W Tornoe Oct Population PK Model Building 2 Q & A on Session 1: Mixed-effects concept 0 ++ -- (Individual-Pop Mean CL,V) Between Subject Variability 0 ++ -- Pred-Obs Conc Residual Variability Between-occasion variability = zero ??? Pop Avg i th patient

Christoffer W Tornoe Oct Population PK Model Building 3 Q & A on Session 1: Bayes theorem 0 ++ -- Prior 0 ++ -- Current 0 ++ -- Posterior 0 ++ -- Prior 0 ++ -- Current 0 ++ -- Posterior

Christoffer W Tornoe Oct Population PK Model Building 4 Q & A on Session 1: Residual variability models True CV -Variability (SD) is same at low and high true values -Called “additive” model True SD -Variability (SD) increases with true values -Called “proportional” or “constant CV” model True SD

Christoffer W Tornoe Oct Population PK Model Building 5 Q & A Homework Assignment 3 Why is it S2=V/1000 for homework 3 and S1=V/1000 for homework 2? In homework 3, which of the following would not work? And if it works, what changes will have to made to the code? 1.VC=THETA(2)*EXP(ETA(2)) 2.V=THETA(1)*EXP(ETA(1)) and CL=THETA(2)*EXP(ETA(2)) 3.CL=TVCL*EXP(ETCL) If the drug in homework 3 followed a two compartment model, what changes will you make to the code? Is it necessary to include $COVARIANCE block in every run? What do you specify in $OMEGA block?

Christoffer W Tornoe Oct Population PK Model Building 6 Q & A Homework Assignment 3 Where do the initial estimates of (theta), omega and sigma come from? Is there a difference between the omega and sigma estimates in the *.smr and *.lst output files? What is F and Y in $ERROR? What does the NOAPPEND do?

Oct Population PK Model Building 7 Christoffer W Tornoe Pharmacometrics Office of Clinical Pharmacology Food and Drug Administration

Christoffer W Tornoe Oct Population PK Model Building 8 Agenda Population PK Model Building –Model based inference Hypothesis testing Likelihood ratio test –Base model selection (not the focus of this session) –Covariate model building Continuous covariates Discrete covariates Covariate search methods Model Qualification and Assumption Checking –Likelihood profiling –Introduction and application of bootstrap to derive confidence intervals Parametric and non-parametric –Posterior predictive check and predictive check –Internal and external validation –Sensitivity analysis

Christoffer W Tornoe Oct Population PK Model Building 9 Hypothesis testing Wikipedia definition - A method of making statistical decisions using experimental data In population PK modeling building, hypothesis testing is used to choose between competing models Null-hypothesis –Assuming the null hypothesis is true (H 0 :  =  0 ), what is the probability of observing a value (c) for the test statistic (  that is at least as extreme as the value that was actually observed? –Critical region of a hypothesis test is when the null hypothesis is rejected (  ≥ c, reject H 0 ) and the alternative hypothesis (H A :  =  A ) is accepted (  < c accept (don’t reject) H 0 )

Christoffer W Tornoe Oct Population PK Model Building 10 Likelihood Ratio Test Likelihood Ratio Test (LRT) is used to compare goodness-of-fit for nested models –Nested models: One model is a subset of the other, e.g. base model (without covariates) is a subset of the full model (with covariates) CL = CL pop + slope * WT ? First-order elimination [CL*C] vs. Michaelis-Menten [Vmax*C/(C+Km)] ? One-, two-, three-compartment model ? Combined residual error model Y = IPRED*(1+EPS(1)) + EPS(2) ? The ratio of likelihoods (L 1 /L 2 ) can be used to test for significance –Objective Function Value (OFV) = - 2 log-likelihood, i.e. sum of squared deviations between predictions and observations Distribution of -2 log(L 1 /L 2 ) follows a  2 distribution –-2 log(L 1 /L 2 ) = -2 (log L 1 – log L 2 ) = 2 (LL 2 – LL 1 ) –Difference in log likelihoods follows  2 distribution

Christoffer W Tornoe Oct Population PK Model Building 11 Likelihood Ratio Test With a probability of 0.05, and 1 degree of freedom, the value of the  2 distribution is 3.84  Parameters  -2LL p

Christoffer W Tornoe Oct Population PK Model Building 12 Other Information Criterions Akaike Information Criterion (AIC) is another measure to compare goodness-of-fit between competing models –Lower AIC = better model fit to the data –AIC = - 2LL + 2*k where k = no. of model parameters Bayesian Information Criterion (BIC or Schwarz) –Lower BIC = better model fit to the data –BIC = - 2LL + k*ln(n obs ) where n obs = number of observations Which criterion penalizes the most for the number of parameters?

Christoffer W Tornoe Oct Population PK Model Building 13 Population PK Model Building – Base Model Base Model –Structural Input (IV bolus, first-order absorption, zero-order input) Distribution (one-, two-, three-compartment model) Elimination (linear or non-linear) Single/multiple dose –Between-subject variability Individual PK estimates should be positive (i.e. CL i =CL pop *exp(  i )) –Residual variability Additive (Constant residual error (LLOQ)) Proportional (Increasing variability with increasing concentrations, CCV) Combined

Christoffer W Tornoe Oct Population PK Model Building 14 Methods for Assessing Goodness-of-Fit Hypothesis Testing –Likelihood-ratio test (Compare OFV) –AIC, BIC Precision of parameter estimates –Large standard errors indicate over-parameterization Diagnostic plots –Observed and predicted concentration vs. time –Observed vs. predicted concentration –Residuals vs. time –Residuals vs. predictions

Christoffer W Tornoe Oct Population PK Model Building 15 Covariate Model Building Why build covariate models? –Explain between-subject variability in parameters and response using patient covariates –Improve predictive performance –Understand causes of variability Patient covariates –Demographic (weight, age, height, gender, ethnicity) –Biomarkers (renal/hepatic function) –Concomitant medication (beta-blocker, CYP inhibitors) –Comorbidity (other diseases)

Christoffer W Tornoe Oct Population PK Model Building 16 Different Ways to Implement Covariate Models Continuous covariates –Linear CL = CL pop + slope * WT CL = CL pop + slope * (WT-WT pop ) (Centered around population mean) –Piecewise linear CL = CL pop + (WT<40)*slope 1 * (WT) + (WT≥40)*slope 2 * (WT) –Power CL i = CL pop * WT i exponent (Allometric model: exponent=0.75) CL i = CL pop * (WT i /WT pop ) exponent (Normalized by population mean) –Exponential CL i = CL pop * exp (slope*WT i )

Christoffer W Tornoe Oct Population PK Model Building 17 Different Ways to Implement Covariate Models Categorical covariates –Linear CL = CL pop,female + Male_diff * SEX –Proportional CL = CL pop,female * (1 + Male_diff * SEX) –Power CL = CL pop,female * Male_diff SEX –Exponential CL = CL pop,female * exp(Male_diff * SEX) (SEX = gender, 0 = Female, 1 = Male)

Christoffer W Tornoe Oct Population PK Model Building 18 Covariate Model Building Essentials Visualize the range and distribution of the covariate data Identify strong correlations or co-linearities between covariates Apply prior knowledge about the PK of the drug –Renally cleared drug (e.g. CL~CrCL) –Fix covariate parameters to literature value if they can’t be estimated (CL ~ WT 0.75, V ~ WT 1.0 ) Keep clinical utility in mind when incorporating covariates –Use body weight instead of BSA as covariate for clearance when dosed mg/kg –Limit to clinical important covariates, e.g cause >20% difference Consider study design before ruling out a covariate effect –Too narrow covariate range –Insufficient information to estimate effect (e.g. 95% CI includes 0)

Christoffer W Tornoe Oct Population PK Model Building 19 Example One-compartment model with 1-order absorption –100 subjects –Samples at t=1, 2, 6, 8, 12, 16, and 24 hours postdose –Single dose of 50 mg

Christoffer W Tornoe Oct Population PK Model Building 20 Visualization of Covariate Data Continuous Covariates Categorical Covariates

Christoffer W Tornoe Oct Population PK Model Building 21 Identify Covariate Correlations or Co-Linearities Body weight and age a co-linear Body weight and sex are correlated

Christoffer W Tornoe Oct Population PK Model Building 22 Clearance Model Building Base Model Clearance ( CL i = CL pop * exp(  i ) ) vs Body Weight –OFV: 8277 –Try linear model:CL i = (CL pop + slope * WT i ) * exp(  i )

Christoffer W Tornoe Oct Population PK Model Building 23 Clearance Model Building Covariate Model 1: CL i = (CL pop + slope * WT i ) * exp(  i ) –  OFV = -30 (Base OFV = 8277, Cov1 OFV = 8247) –Correlation between CL pop and slope =

Christoffer W Tornoe Oct Population PK Model Building 24 Clearance Model Building Covariate Model 2: CL i = (CL pop + slope * (WT i -70) * exp(  i ) –Try centering around median body weight –  OFV = 0 (Cov1 OFV = 8247, Cov2 OFV = 8247) –Corr(CL pop, slope) = 0.307

Christoffer W Tornoe Oct Population PK Model Building 25 Clearance Model Building Covariate Model 3: CL i = (CL pop * (WT i /70) exponent * exp(  i ) –Try power model to avoid problems for WT = 0 –  OFV = 0 (Cov2 OFV = 8247, Cov3 OFV = 8247)

Christoffer W Tornoe Oct Population PK Model Building 26 Clearance Model Building Covariate Model 3: CL i = (CL pop * (WT i /70) exponent * exp(  i ) –Look for other potential continuous clearance covariates –Clearance appears correlated with Age due to co-linearity with WT –IIV Clearance does not show a trend with Age

Christoffer W Tornoe Oct Population PK Model Building 27 Clearance Model Building Covariate Model 3: CL i = (CL pop * (WT i /70) exponent * exp(  i ) –Look for other potential categorical clearance covariates –Higher clearance in males compared to females – Why?

Christoffer W Tornoe Oct Population PK Model Building 28 Clearance Model Building Covariate Model 3: CL i = (CL pop * (WT i /70) exponent * exp(  i ) –Females have lower body weight compared to males –No trend in ETA CL

Christoffer W Tornoe Oct Population PK Model Building 29 Covariate Search Methods Generalized Additive Modeling (GAM) –Multiple linear regression to quickly screen for linear and non-linear covariate-parameters relationships –Based on empirical Bayes parameter estimates from NONMEM –Does not account for correlation between model parameters Stepwise Covariate Modeling (SCM) –Forward addition –Backward elimination –Forward/backward stepwise

Christoffer W Tornoe Oct Population PK Model Building 30 Generalized Additive Modeling (GAM) Implemented in Xpose4 in R –Clearance covariate model (Revisited) xpose.gam(xp0, parnam="CL", covnams = xvardef("covariates", xp0)) –Initial Model: CL ~ 1 –Final Model: CL ~ BW –Call: gam(formula = CL ~ BW, data = gamdata, trace = FALSE) Deviance Residuals: – Min 1Q Median 3Q Max – (Dispersion Parameter for gaussian family taken to be 0.293) –Null Deviance: on 99 degrees of freedom –Residual Deviance: on 98 degrees of freedom –AIC: Coefficients –(Intercept) BW –

Christoffer W Tornoe Oct Population PK Model Building 31 Generalized Additive Modeling (GAM)

Christoffer W Tornoe Oct Population PK Model Building 32 Stepwise Covariate Modeling (SCM) Implemented in Perl-Speaks-NONMEM –Forward Inclusion Step Includes covariates one step at a time using LRT (typically p<0.05) Univariate analysis of all specified covariate-parameter relationships Adds best covariate and repeats univariate analysis with remaining covariates Continue until no more significant covariates are left –Backward Elimination Step Starts with final model in forward inclusion step and removes covariates one at a time in a stepwise manner using LRT (typically p<0.01 or p<0.001) Remove covariate that has the smallest increase in OFV when fixed to 0 Continues until all remaining covariates are significant

Christoffer W Tornoe Oct Population PK Model Building 33 Stepwise Covariate Modeling (SCM) –Forward inclusion (p<0.05), Backward eliminition (p<0.001) –Continuous (Age, BW, Exponential=4) and Categorical (Sex, Linear=2) ModelTest Base OFV New OFV Test ValueGoalSignificant? CLAGE-4OFV >3.84YES! CLBW-4OFV >3.84YES! CLSEX-2OFV >3.84YES! VAGE-4OFV >3.84YES! VBW-4OFV >3.84YES! VSEX-2OFV >3.84YES! Parameter-covariate relation chosen in this forward step: CL-BW 1. Forward Step

Christoffer W Tornoe Oct Population PK Model Building 34 Stepwise Covariate Modeling (SCM) –Forward inclusion (p<0.05), Backward eliminition (p<0.001) –Continuous (Age, BW, Exponential=4) and Categorical (Sex, Linear=2) ModelTest Base OFV New OFV Test ValueGoalSignificant? CLAGE-4OFV >3.84 CLSEX-2OFV >3.84 VAGE-4OFV >3.84YES! VBW-4OFV >3.84YES! VSEX-2OFV >3.84YES! Parameter-covariate relation chosen in this forward step: V-BW 2. Forward Step

Christoffer W Tornoe Oct Population PK Model Building 35 Stepwise Covariate Modeling (SCM) –Forward inclusion (p<0.05), Backward eliminition (p<0.001) –Continuous (Age, BW, Exponential=4) and Categorical (Sex, Linear=2) 3. Forward Step ModelTest Base OFV New OFV Test ValueGoalSignificant? CLAGE-4OFV >3.84 CLSEX-2OFV >3.84 VAGE-4OFV >3.84 VSEX-2OFV >3.84 Parameter-covariate relation chosen in this forward step: -

Christoffer W Tornoe Oct Population PK Model Building 36 Stepwise Covariate Modeling (SCM) –Forward inclusion (p<0.05), Backward eliminition (p<0.001) –Continuous (Age, BW, Exponential=4) and Categorical (Sex, Linear=2) 1. Backward Step ModelTest Base OFV New OFV Test ValueGoalSignificant? CLBW-1OFV > VBW-1OFV > Parameter-covariate relation chosen in this backward step: -

Christoffer W Tornoe Oct Population PK Model Building 37 Summary of Covariate Model Building Why build covariate models? –Explain between-subject variability in parameters and response using patient covariates –Improve predictive performance –Understand causes of variability Before building covariate models –Apply prior knowledge about the PK of the drug when deciding on which covariates to test –Keep clinical utility in mind when incorporating covariates –Consider whether the available data and design is adequate to detect covariate effect Covariate search methods –Generalized additive modeling –Stepwise covariate modeling