1. Workshop Semiparametric frailty models
coxph and coxme

2. The data set: Time to first insemination
Database of regional Dairy Herd Improvement Association (DHIA)
Milk recording service
Artificial insemination
Select sample
Subset of 2567 cows from 49 dairy farms

3. Fixed covariates data set insemfix.dat

4. Time-varying covariates data set insemtvc.dat

5. Fitting semiparametric models with fixed covariates
setwd("c://docs//onderwijs//survival//Flames//")
insemfix<-read.table("insemfix.dat",header=T)
coxph(Surv(timeto,stat)~heifer+frailty(herd))
HR=exp(-0.24)=0.79
Call:
coxph(formula = Surv(timeto, stat) ~ heifer + frailty(herd))
coef se(coef) se2 Chisq DF p
heifer e-08
frailty(herd) e+00
Iterations: 10 outer, 23 Newton-Raphson
Variance of random effect= I-likelihood =
Degrees of freedom for terms=
Likelihood ratio test=281 on 41.9 df, p=0 n= 2579

6. Exercise Can we maximise the penalised partial likelihood at once?
Depict the penalised partial likelihood as a function of theta
The PPL is the second item of the named list loglik, e.g., for our example
insemfix$loglik[2]

7. Penalised partial log likelihood
detloglik<-function(x)
{coxph(Surv(timeto,stat)~heifer+frailty(herd,theta=x),data=insemfix)$loglik[2]}
thetlist<-seq(0.05,0.15,0.005)
logliklist<-sapply(thetlist,detloglik)
plot(thetlist,logliklist,type='l',xlab='theta',ylab='penalised partial loglikelihood')

8. Exercise Do we obtain the maximum profile likelihood?
Depict the profile likelihood as a function of theta
Theta can be fixed at x by frailty(herd,theta=x)
The marginal likelihood can be obtained as insemfix$history[[1]]$c.loglik

9. Profile log likelihood
detloglik<-function(x)
{coxph(Surv(timeto,stat)~heifer+frailty(herd,theta=x),data=insemfix)$history[[1]]$c.loglik}
thetlist<-seq(0.05,0.15,0.005)
logliklist<-sapply(thetlist,detloglik)
plot(thetlist,logliklist,type='l',xlab='theta',ylab='profile loglikelihood')

10. Adjusting outer number of iterations
coxfit<-coxph(Surv(timeto,stat)~heifer+frailty(herd,eps= ),data=insemfix,outer.max=100)
coxfit
Call:
coxph(formula = Surv(timeto, stat) ~ heifer + frailty(herd, eps = 1e-08),
outer.max = 100)
coef se(coef) se2 Chisq DF p
heifer e-08
frailty(herd, eps = 1e e+00
Iterations: 21 outer, 39 Newton-Raphson
Variance of random effect= I-likelihood =
Degrees of freedom for terms=
Likelihood ratio test=277 on 39.5 df, p=0 n= 2579

11. Time-varying covariates data
Contribution to the denominator:
tij=10 : 2.29
tij=55 : 2.61

12. Fitting Cox models with time-varying covariates (1)
#Read data
setwd("c://docs//onderwijs//survival//Flames//")
insemtvc<-read.table("insemtvc.dat",header=T)
coxph(Surv(begin,end,stat)~ureum+frailty(herd,eps= ), outer.max=100,data=insemtvc)

13. Fitting Cox models with time-varying covariates (2)
> coxph(Surv(begin,end,stat)~ureum+frailty(herd,eps= ), outer.max=100,data=insemtvc)
Call:
coxph(formula = Surv(begin, end, stat) ~ ureum + frailty(herd,
eps = 1e-07), data = insemtvc, outer.max = 100)
coef se(coef) se2 Chisq DF p
ureum
frailty(herd, eps = 1e
Iterations: 35 outer, 82 Newton-Raphson
Variance of random effect= I-likelihood =
Degrees of freedom for terms=
Likelihood ratio test=2066 on 166 df, p=0 n= 93608

14. Alternative semiparametric models
Unadjusted and marginal semiparametric model
Stratified semiparametric model
Semiparametric frailty model

15. Unadjusted semiparametric model
ureumtv.unadjust <-coxph(Surv(begin,end,stat)~ureum,data=insemtvc)
summary(ureumtv.unadjust)
Call:
coxph(formula = Surv(begin, end, stat) ~ ureum, data = insemtvc)
n= 93608
coef exp(coef) se(coef) z p
ureum
exp(coef) exp(-coef) lower .95 upper .95
ureum
Rsquare= 0 (max possible= )
Likelihood ratio test= on 1 df, p=0.0926
Wald test = on 1 df, p=0.0929
Score (logrank) test = on 1 df, p=0.0929

16. Marginal semiparametric model
ureumtv.marg<-coxph(Surv(begin,end,stat)~ureum+cluster(herd),data=insemtvc)
summary(ureumtv.marg)
Call:
coxph(formula=Surv(begin,end,stat)~ureum+cluster(herd),data=insemtvc)
n= 93608
coef exp(coef) se(coef) robust se z p
ureum
exp(coef) exp(-coef) lower .95 upper .95
ureum
Rsquare= 0 (max possible= )
Likelihood ratio test= on 1 df, p=0.0926
Wald test = on 1 df, p=0.338
Score (logrank) test = on 1 df, p=0.0929, Robust = p=0.339

17. Stratified semiparametric model
ureumtv.strat<-coxph(Surv(begin,end,stat)~ureum+strata(herd),data=insemtvc)
summary(ureumtv.strat)
Call:
coxph(formula = Surv(begin, end, stat) ~ ureum + strata(herd),
data = insemtvc)
n= 93608
coef exp(coef) se(coef) z p
ureum
exp(coef) exp(-coef) lower .95 upper .95
ureum
Rsquare= 0 (max possible= )
Likelihood ratio test= on 1 df, p=
Wald test = on 1 df, p=
Score (logrank) test = on 1 df, p=

18. Semiparametric frailty model
ureumtv.frail<-coxph(Surv(begin,end,stat)~ureum+frailty(herd,eps= ),outer.max=100,data=insemtvc)
summary(ureumtv.frail)
Call:
coxph(formula = Surv(begin, end, stat) ~ ureum + frailty(herd,
eps = 1e-07), data = insemtvc, outer.max = 100)
n= 93608
coef se(coef) se2 Chisq DF p exp(coef) lower.95 upper.95
ureum
frailty(herd, eps = 1e
Iterations: 35 outer, 60 Newton-Raphson
Variance of random effect= I-likelihood =
Degrees of freedom for terms=
Rsquare= (max possible= )
Likelihood ratio test= on 166 df, p=0
Wald test = on 166 df, p=1

19. Conclusions
With adjustment, either by stratification or frailties, significant ureum effect
Without adjustment, either with or without variance adjustment, no significant ureum effect
Why?
Plot herd specific ureum effect as a function of mean herd ureum concentration

20. Exercise
Plot percentage events as a function of mean herd ureum concentration
Using following scheme
Derive average ureum conc. for each cow
avureum.cow<-tapply(insemtvc$ureum,list(insemtvc$cowid),mean)
Derive average ureum conc. for each herd
Derive percentage events
Plot

21. Mean ureum and %insemination
setwd("c://docs//onderwijs//survival//Flames//")
insemtvc<-read.table("insemtvc.dat",header=T)
herd<-insemtvc$herdnr;timeto<-(insemtvc$end*12/365.25)
stat<-insemtvc$stat;heifer<-insemtvc$par2
avureum.cow<-tapply(insemtvc$ureum,list(insemtvc$cowid),mean)
herd.cow<-tapply(insemtvc$herd,list(insemtvc$cowid),mean)
avureum.herd<-tapply(avureum.cow,list(herd.cow),mean)
numinsem.herd<-tapply(insemtvc$stat,insemtvc$herd,sum)
totanim.herd<-tapply(insemtvc$stat,insemtvc$herd,length)
percinsem<- 100*numinsem.herd/totanim.herd
plot(avureum.herd,percinsem,xlab="Ureum",ylab="Insemination (%)")

22. Exercise
Plot frailty term as a function of mean herd ureum concentration
Using following scheme
Save frailties from Cox frailty model
Derive average ureum conc. for each herd
Plot

23. Mean ureum and frailties
ureumtv.frail<-coxph(Surv(begin,end,stat)~ureum+
frailty(herd,eps= ),outer.max=100,data=insemtvc)
avureum.cow<-tapply(insemtvc$ureum,list(insemtvc$cowid),mean)
avureum.herd<-tapply(avureum.cow,list(herd.cow),mean)
herd.cow<-tapply(insemtvc$herd,list(insemtvc$cowid),mean)
stat.herd<-tapply(insemtvc$stat,list(insemtvc$herd),sum)
results<-data.frame(frailty=ureumtv.frail$frail,ureum=avureum.herd,stat=stat.herd)
results0<-results[results$stat==0,]
plot(results$frailty,results$ureum,xlab="Frailty",ylab="Mean ureum concentration")
points(results0$frailty,results0$ureum,pch=3)

24. Mean ureum and frailties plot

25. Exercise Derive herd specific ureum effects
Fit the model for each herd separately

26. Herd specific treatment effect
avureum.cow<-tapply(insemtvc$ureum,list(insemtvc$cowid),mean)
herd.cow<-tapply(insemtvc$herd,list(insemtvc$cowid),mean)
avureum.herd<-tapply(avureum.cow,list(herd.cow),mean)
fitmod<-function(herdnr)
{coxph(Surv(begin,end,stat)~ureum,subset=(insemtvc$herd==herdnr),data=insemtvc)$coef}
ureumeffect.herd<-sapply(herd.cow,fitmod)
Error in fitter(X, Y, strats, offset, init, control, weights = weights, :
Can't fit a Cox model with zero failures

27. Exercise
Refit the frailty model and the marginal model leaving out the herds without inseminations

28. Results deleting farms without inseminations
insemtvcr<-insemtvc[insemtvc$herd!= & insemtvc$herd!= & insemtvc$herd!= & insemtvc$herd!= & insemtvc$herd!= & insemtvc$herd!= ,]
ureumtvcr.unadjust<-coxph(Surv(begin,end,stat)~ureum,data=insemtvcr)
summary(ureumtvcr.unadjust)
Call:
coxph(formula = Surv(begin, end, stat) ~ ureum, data = insemtvcr)
n= 90232
coef exp(coef) se(coef) z p lower .95 upper .95
ureum
Rsquare= 0 (max possible= )
Likelihood ratio test= on 1 df, p=
Wald test = on 1 df, p=
Score (logrank) test = on 1 df, p=

29. Results deleting farms without inseminations - alternative
insemtvcr<-insemtvc[insemtvc$herd!= & insemtvc$herd!= & insemtvc$herd!= & insemtvc$herd!= & insemtvc$herd!= & insemtvc$herd!= ,]
ureumtvcr.unadjust<-coxph(Surv(begin,end,stat)~ureum,data=insemtvcr)
summary(ureumtvcr.unadjust)
Call:
coxph(formula = Surv(begin, end, stat) ~ ureum, data = insemtvcr)
n= 90232
coef exp(coef) se(coef) z p lower .95 upper .95
ureum
Rsquare= 0 (max possible= )
Likelihood ratio test= on 1 df, p=
Wald test = on 1 df, p=
Score (logrank) test = on 1 df, p=

30. Deleting farms without inseminations
#alternative
herdstat<-data.frame(herd=names(stat.herd),numinsem=stat.herd)
insemtvcr2<-merge(insemtvc,herdstat,by="herd")

31. Exercise
Fit the frailty model assuming the lognormal distribution for the frailties

32. Fitting the shared normal frailty model
coxph(Surv(begin,end,stat)~ureum+frailty(herd,dist="gaussian",eps= ),outer.max=100,data=insemtvc)
Call:
coxph(formula = Surv(begin, end, stat) ~ ureum + frailty(herd,
dist = "gaussian", eps = 1e-05), data = insemtvc, outer.max = 100)
coef se(coef) se2 Chisq DF p
ureum
frailty(herd, dist = "gau
Iterations: 6 outer, 21 Newton-Raphson
Variance of random effect= 0.411
Degrees of freedom for terms=
Likelihood ratio test=2030 on 166 df, p=0 n= 93608

33. Lognormal frailties versus normal random effects
Assume U has lognormal distribution, with U=exp(W) and W~N(m,s2)
The gamma density with mean 1, variance q is given by
For q = 0.1, 0.5, 1, 1.5, plot the gamma density function and lognormal density chosing values for m and s2 so that
Var(exp(W))= q and m = 0
Var(exp(W))= q and E(U)=1

34. Lognormal frailties versus normal random effects (2)
With m equal to 0, we have
With E(U)=1, we have

35. Plotting densities theta<-c(0.1,0.5,1,1.5)
s1<-log((1+sqrt(1+4*theta))/2)
s2<-log(theta+1)
m2<- -0.5*s2
u<-seq(0,3,0.01)
for (i in 1:length(u)){
dlnorm.theta0.1.EWeq0<-dlnorm(u,0,sqrt(s1[1]))
dlnorm.theta0.5.EWeq0<-dlnorm(u,0,sqrt(s1[2]))
dlnorm.theta1.EWeq0<-dlnorm(u,0,sqrt(s1[3]))
dlnorm.theta1.5.EWeq0<-dlnorm(u,0,sqrt(s1[4]))
dlnorm.theta0.1.EUeq1<-dlnorm(u,m2[1],sqrt(s2[1]))
dlnorm.theta0.5.EUeq1<-dlnorm(u, m2[2],sqrt(s2[2]))
dlnorm.theta1.EUeq1<-dlnorm(u, m2[3],sqrt(s2[3]))
dlnorm.theta1.5.EUeq1<-dlnorm(u, m2[4],sqrt(s2[4]))
dgam.theta0.1.EUeq1<-dgamma(u,1/theta[1], 1/theta[1])
dgam.theta0.5.EUeq1<- dgamma(u,1/theta[2], 1/theta[2])
dgam.theta1.EUeq1<- dgamma(u,1/theta[3], 1/theta[3])
dgam.theta1.5.EUeq1<- dgamma(u,1/theta[4], 1/theta[4]) }