Two bootstrapping routines for obtaining uncertainty measurement around the nonparametric distribution obtained in NONMEM VI Paul G. Baverel 1, Radojka M. Savic 2 and Mats O. Karlsson 1 1 Division of Pharmacokinetics & Drug Therapy, Uppsala University, Sweden 2 INSERM U73 8, University Paris Diderot Paris 7, Paris, France  Two novel bootstrapping routines intended for nonparametric estimation methods are proposed. Their evaluation with a simple PK model in the case of informative sampling design was performed when applying FOCE- NONP in NONMEM VI but it is easily transposable to other nonparametric applications.  These tools can be used for diagnostic purpose to help detecting misspecifications with respect to the distribution of random effects.  From the sampling distribution obtained, standard uncertainty metrics, such as standard errors and correlation matrix can be derived in case reporting uncertainty is intended. References: [1]. Perl-speaks-NONMEM (PsN software): L. Lindbom, M. Karlsson, N. Jonsson. [2]. Savic RM, Baverel PG, Karlsson MO. A novel bootstrap method for obtaining uncertainty around the nonparametric distribution. PAGE 18 (2008) Abstract [3]. Efron B. Bootstrap methods: another look at the jackknife. Ann Stat 1979; 7:1-26.  Overall, the trend and the magnitude of the 95% CI derived with the full and the simplified nonparametric bootstrapping methods (N=100 and N=500) matched the true 95% CI in all distributional cases and regardless of individual numbers in original data.  The simplified version induced slightly less bias in quantifying uncertainty (prediction errors of 95% CI width) than the full version. This is expected as the former methodology derives uncertainty from the original data. To provide quantitative measures of uncertainty around nonparametric distribution (NPD) and nonparametric population parameter estimates To assess the appropriateness of parameters distributional shape by means of two resampling-based diagnostic methods aiming at quantifying uncertainty around NPD  Quantifying uncertainty in parameter estimates is essential to support decision making throughout model building process.   Despite providing enhanced estimates of parameter distribution, nonparametric algorithms do not yet supply uncertainty metrics. Two different permutation methods automated in PsN [1] were developed to quantify uncertainty around NPD (95% confidence interval) and nonparametric estimates (SEs and variance-covariance matrix): o The full method [2] relies on N bootstraps of the original data and a re-analysis of both the preceding parametric as well as the nonparametric step o The simplified method relies on N bootstrap samples of the vectors of individual probabilities associated with each unique support point of the NPD  Six informative datasets of 50 or 200 individuals were simulated from an IV bolus PK model in which CL and V conformed to various underlying distributional shapes (log-normal, bimodal, heavy-tailed). Residual variability was set to 10% CV.  Re-estimation was conducted assuming normality under FOCE, and NPDs were estimated by applying FOCE-NONP method in NONMEM VI. Figure 1: Sequential steps of the operating procedure of both the full and simplified nonparametric bootstrapping methods intended for nonparametric estimation methods. Figure 2. On the left : 95% confidence intervals obtained based on the full and simplified nonparametric bootstrapping methodologies in case of various underlying distributions of CL. The true 95% CI around the true parameter distribution is also represented for comparison, as well as the parametric cumulative density function. On the right: Prediction errors of the 95% CI width are displayed for each quartile of parameter distribution, the true uncertainty being taken as reference. UNDERLYING BIMODAL CLEARANCE DISTRIBUTION UNDERLYING HEAVY-TAILED CLEARANCE DISTRIBUTION (200 IDs) Simplified nonparametric bootstrapping method: 5-step procedure (ca 2 mn.) Full nonparametric bootstrapping method: 7-step procedure (CPU time: ca 4 hr.) BOOTSTRAP* N times original data D J  Bootstrapped data B 1...B N PARAMETRIC ESTIMATION: B 1... B N  N sets (θ,Ω,σ) each defined at <J support points NONPARAMETRIC ESTIMATION: D J given (θ,Ω,σ)  N sets NPD N defined at J support points for <J individuals in B 1...B N PARTITIONING NPD N into J individual probability densities IPD N BOOTSTRAP* IPD N according to sample scheme B 1...B N  NxN matrices M boot N of bootstrapped IPD <J RE-ASSEMBLING bootstrapped IPD <J  From NPDnew N construct nonparametric 95% CI around NPD  Derive SEs and correlation matrix of nonparametric estimates PARTITIONING NPD into J individual probability densities IPD J NONPARAMETRIC ESTIMATION: Original data D J  Matrice M (JxJ) of individual probabilities: Row entries J individuals Column entries J support points  NPD defined at J support points BOOTSTRAP IPD J N times RE-ASSEMBLING bootstrapped IPD J  N matrices M boot N of bootstrapped IPD J IDs 200 IDs  The true uncertainty was derived by standard nonparametric bootstrapping (N=1000) [3] of the true individual parameters and used as reference for qualitative and quantitative assessment of the uncertainty measurements derived from both techniques.  N sets NPDnew N defined at J support points of NPD N  N matrices M N (JxJ) of individual probabilities: Row entries J individuals Column entries J support points  A single set of NPDnew N defined at J support points of NPD Prediction errors N=100Prediction errorsN=500 Table 1. SEs of parameter estimates obtained from 100 stochastic simulations followed by estimations given the true model under FOCE, FOCE-NONP and the analytical solution under FOCE ($COVARIANCE) in NONMEM VI. The simplified methodology was applied to each simulated dataset; SEs were computed and average SEs were reported for comparison with SSE. SE 100 Stochastic Simulations followed by Estimations (SSE) Simplified nonparametric bootstrap version True empirical FOCE Asymptotic ( $ COV) FOCE True empirical FOCE-NONP (N= 100 ) FOCE-NONP Θ CL Θ V Ω CL Ω CL,V Ω V  SEs obtained with the simplified methodology matched the ones obtained by SSE.