Nonparametric Estimation with Recurrent Event Data Edsel A. Pena Department of Statistics University of South Carolina Research supported by NIH and NSF Grants Based on joint works with R. Strawderman (Cornell) and M. Hollander (Florida State) Talk at UNCC, 10/12/01

2 A Real Recurrent Event Data (Source: Aalen and Husebye (‘91), Statistics in Medicine) Variable: Migrating motor complex (MMC) periods, in minutes, for 19 individuals in a study concerning small bowel motility during fasting state.

3 Pictorial Representation of Data for a Unit or Subject Consider unit/subject #3. K = 3 Gap Times, T j : 284, 343, 529 Censored Time,  -S K : 4 Calendar Times, S j : 284, 343, 529 Limit of Obs. Period:   =533 S 1 =284 S 2 =343 S 3 =529 0 T1T1 T2T2 T3T3 T4T4 Calendar Scale

4 Features of Data Set Random observation period per subject (administrative constraints). Length of period:  Event of interest is recurrent. A subject may have more than one event during observation period. # of events (K) informative about MMC period distribution (F). Last MMC period right-censored by a variable informative about F. Calendar times: S 1, S 2, …, S K. Right-censoring variable:  -S K.

5 Assumptions and Problem Aalen and Husebye: “Consecutive MMC periods for each individual appear (to be) approximate renewal processes.” Translation: The inter-event times T ij ’s are assumed stochastically independent. Problem: Under this IID assumption, and taking into account the informativeness of K and the right- censoring mechanism, to estimate the inter-event distribution, F.

6 General Form of Data Accrual Calendar Times of Event Occurrences S i0 =0 and S ij = T i1 + T i2 + … + T ij Number of Events in Observation Period K i = max{j: S ij <  i } Upper limit of observation periods,  ’s, could be fixed, or assumed to be IID with unknown distribution G.

7 Observables Main Theoretical Problem Based on this data, to obtain an estimator of the unknown gaptime or inter-event time distribution, F; and determine its properties.

8 Relevance and Applicability Recurrent phenomena occur in a variety of settings. –Nuclear power plant stoppages. –Outbreak of a disease. –Terrorist attacks. –Labor strikes. –Hospitalization of a patient. –Tumor occurrence. –Epileptic seizures. –Non-life insurance claims. –When stock index (e.g., Dow Jones) decreases by at least 6% in one day.

9 Limitations of Existing Estimation Methods Consider only the first, possibly right-censored, observation per unit and use the product-limit estimator (PLE). –Loss of information –Inefficient Ignore the right-censored last observation, and use empirical distribution function (EDF). –Leads to bias. –Estimator actually inconsistent.

10 Review: Prior Results T 1, T 2, …, T n IID F(t) = P(T < t) Empirical Survivor Function (EDF) Asymptotics of EDF where W 1 is a zero-mean Gaussian process with covariance function Complete Data Setting

11 In Hazards View Hazard rate function Cumulative hazard function Equivalences Another representation of the variance

12 Right-Censored Data Failure times: T 1, T 2, …, T n IID F Censoring times: C 1, C 2, …, C n IID G Right-censored data (Z 1,  1 ), (Z 2,  2 ), …, (Z n,  n ) Z i = min(T i, C i )  i = I{T i < C i } with Product-limit or Kaplan-Meier Estimator

13 PLE Properties Asymptotics of PLE where W 2 is a zero-mean Gaussian process with covariance function If G(w) = 0 for all w, so no censoring,

14 Relevant Stochastic Processes for Recurrent Event Setting Calendar-Time Processes for ith unit is a vector of square-integrable zero-mean martingales. Then,

15 is the length since last event at calendar time v Needed: Calendar-Gaptime Space 0 Difficulty: arises because interest is on (.) or  (.), but these appear in the compensator process in form GapTime Calendar Time s t For Unit 3 in MMC Data

16 Processes in Calendar-Gaptime Space N i (s,t) = # of events in calendar time [0,s] for ith unit whose gaptimes are at most t Y i (s,t) = number of events in [0,s] for ith unit whose gaptimes are at least t: “at-risk” process

17 “Change-of-Variable” Formulas

18 Estimators of  and F for the Recurrent Event Setting By “change-of-variable” formula, RHS is a sq-int. zero-mean martingale, so Estimator of  (t)

19 Estimator of F Since by substitution principle, a generalized product-limit estimator (GPLE). GPLE extends the EDF for complete data, and the PLE or KME for single- event right-censored data.

20 Computational Forms If T ij ’s are distinct, If s is large, combine data for all n units taking into account right-censoring of each unit’s last observation. GPLE could be computed like the product-limit estimator. Statistical packages such as SAS, Splus, etc., could be utilized.

21 GPLE Finite-Sample Properties GPLE is positively biased, but with bias disappearing at an exponential rate as sample size increases. Finite-sample variance function estimate: Special Case R(t) = renewal function of F

22 Asymptotic Properties of GPLE Special Case: If F = EXP(  ) and G = EXP(  )

23 Weak Convergence Theorem: ; The proof of this result relied on a weak convergence theorem for recurrent and renewal settings developed in Pena, Strawderman and Hollander (2000), which utilized some ideas in Sellke (1988) and Gill (1980).

24 Comparison of Limiting Variance Functions PLE: GPLE (recurrent event): For large s, For large t or if in stationary state, R(t) = t/  F, so approximately, with  G (w) being the mean residual life of  given  > w. EDF:

25 Wang-Chang Estimator (JASA, ‘99) Beware! Wang and Chang developed this estimator to be able to handle correlated inter- event times, so comparison with GPLE is not completely fair to their estimator!

26 Frailty-Induced Correlated Model Correlation induced according to a frailty model: U 1, U 2, …, U n are IID unobserved Gamma( ,  ) random variables, called frailties. Given U i = u, (T i1, T i2, T i3, …) are independent inter-event times with Marginal survivor function of T ij:

27 Frailty-Model Estimator Frailty parameter, , determines dependence among inter-event times. Small (Large)  : Strong (Weak) dependence. EM algorithm is needed to obtain the estimator, where the unobserved frailties are viewed as missing values. EM implementation parallels that of Nielsen, Gill, Andersen, and Sorensen (1992).

28 Resulting Estimator: FRMLE Able to prove that the GPLE is inconsistent in the presence of dependence: that is, when the frailty parameter  is finite. E-STEP: Given, estimate Z’s via, M-STEP: Given  is estimated from profile likelihood. EM Algorithm

29 Monte Carlo Studies Under gamma frailty model. F = EXP(  ):  = 6 G = EXP(  ):  = 1 n = 50 # of Replications = 1000 Frailty parameter  took values in {Infty (IID), 6, 2} Computer programs: combinations of S-Plus and Fortran routines. Black = GPLE; Blue = WCPLE; Red = FRMLE

30 IID Simulated Comparison of the Three Estimators for Varying Frailty Parameter

31 Effect of the Frailty Parameter for Each of the Three Estimators

32 The Three Estimates of Inter- Event Survivor Function for the MMC Data Set IID assumption seems acceptable. Estimate of  is 10.2.