1. Event History Models: Cox & Discrete Time Models
Sociology 229: Advanced Regression
Class 6
Copyright © 2010 by Evan Schofer
Do not copy or distribute without permission

2. Announcements Assignment 4 Handed out Today’s agenda
More complex EHA assignment
Today’s agenda
Cox models
Parametric Models
Reading Discussion

3. Cox Models Where h(t) is the hazard rate
The basic Cox model:
Where h(t) is the hazard rate
h0(t) is some baseline hazard function (to be inferred from the data)
This obviates the need for building a specific functional form into the model
Also written as:

4. Cox Model: Example Mostly similar to exponential model…
Cox regression -- Breslow method for ties
No. of subjects = Number of obs =
No. of failures =
Time at risk =
Wald chi2(6) =
Log pseudolikelihood = Prob > chi2 =
(Std. Err. adjusted for 92 clusters in newid3)
| Robust
_t | Coef. Std. Err z P>|z| [95% Conf. Interval]
gdp |
degradation |
education |
democracy |
ngo |
ingo |
Most effects = similar… though education effect loses significance…

5. Cox Model: Baseline Hazard
Cox models involve a “baseline hazard”
Note: baseline = when all covariates are zero
Question: What does the baseline hazard look like?
Or baseline survivor & integrated hazard?
Stata can estimate the baseline survivor, hazard, integrated hazard. Two steps:
1. You must ask stata to save the info when you run the Cox model
Ex: stcox gdp degradation education democracy ngo ingo, robust nohr basehc(h0)
2. Use “stcurve” command to plot the baseline curves
Ex: stcurve, hazard OR stcurve, survival

6. Cox Model: Baseline Hazard
Baseline rate: Adoption of environmental law

7. Cox Model: Baseline Hazard
Note: It may not always make sense to plot the baseline hazard
Baseline shows hazard when X variables are zero
Sometimes zero values aren’t very useful/interesting
Example: Does it make sense to plot hazard of countries adopting laws, if X vars = zero?
Hazard rate might be quite low
In some cases, you’ll just get a flat zero curve
Or extremely high values
Solutions:
1. Rescale indep vars before running cox model
2. Use stcurve to choose relevant values of vars.

8. Cox Model: Estimated Hazards
You can also use stcurve to plot estimated hazard rates based on values of indep vars
Ex: What is hazard curve if democracy = 1, 5, 10?
Strategy: use “at” subcommand:
stcurve , hazard at(democ=1) at2(democ=10)
NOTE: All other variables are pegged at the mean…

9. Cox: Estimated Hazard Rate
Hazard rate for adoption of environmental law

10. Cox Model Diagnostics Issues that you must deal with:
1. How to estimate results with “ties” in your data
Ties = cases that fail at the exact same time
2. How to identify violations of the proportional hazard assumption
3. Dealing with outliers/influential cases
4. Assessing model fit
Most of this applies to parametric models
Ties are not a concern
But, additional issues come up: choosing the right functional form (shape) to model the hazard.

11. Cox Model Issues: Ties How to handle ties in data
It is mathematically complex to estimate models when there are tied failures
That is: two cases that have events at the exact same time
Several mathematical approaches:
Breslow approximation – simplest approach
Stata default, but not the best choice!
Efron approximation – generally better
More computationally intensive, but given the power of modern computers it is not an issue
stcox var1 var2 var3, efron

12. Cox Model Issues: Ties
Exact marginal – “continuous time approximation”
Box-Steffensmeier & Jones: “Averaged Likelihood”
Assumes ties didn’t happen EXACTLY at the same time… and considers all possible orderings
Exact partial – “discrete”
Box-Steffensmeier & Jones: “exact discrete method”
Assumes ties happened EXACTLY at the same time
Advice:
Use Efron at a minimum
Exact methods are often more accurate
Exact marginal often makes most sense… events rarely occur at the EXACT same time… unless you have discrete data
But, exact methods can take a LONG time.
For big datasets with many ties, Efron is OK.

13. Proportional Hazard Assumption
Key assumption: Proportional hazards
Estimated Hazard ratios are proportional over time
i.e., Estimates of a hazard ratio do NOT vary over time
Example: Effect of “abstinence” program on sexual behavior
Issue: Do abstinence programs lower the rate in a consistent manner across time?
Or, perhaps the rate is lower initially… but then the rate jumps back up (maybe even exceeds the control group).
Groups are assumed to have “parallel” hazards
Rather than rates that diverge, converge (or cross).

14. Proportional Hazard Assumption
Strategies:
1. Visually examine raw hazard plots for sub-groups in your data
Watch for non-parallel trends
A crude method… not the best approach… but often identifies big violations

15. Proportional Hazard Assumption
Visual examination of raw hazard rate
You want them to change proportionally
If one doubles, so does the other…

16. Proportional Hazard Assumption
2. Plot –ln(-ln(survival plot)) versus ln(time) across values of X variables
What stata calls “stphplot”
Parallel lines indicate proportional hazards
Again, convergence and divergence (or crossing) indicates violation
A less-common approach: compare observed survivor plot to predicted values (for different values of X)
What stata calls “stcoxkm”
If observed are similar to predicted, assumption is not likely to be violated.

17. Proportional Hazard Assumption
-ln(-ln(survivor)) vs. ln(time) – “stphplot”
Parallel=good
Convergence suggests violation of proportional hazard assumption
(But, I’ve seen worse!)

18. Proportional Hazard Assumption
Cox estimate vs. observed KM – “stcoxkm”
Predicted differs from observed for countries in West

19. Proportional Hazard Assumption
3. Piecewise Models
Piecewise = break model up into pieces (by time)
Ex: Split analysis in to “early” vs “late” time
If coefficients vary in different time periods, hazards are not proportional
Example:
stcox var1 var2 var3 if _t < 10
stcox var1 var2 var3 if _t >= 10
Look for large changes in coefficients!

20. Proportional Hazard Assumption
In a piecewise model, coefficients would differ in non-proportional models
Non-Proportional
Proportional
Early
Late
Here, the effect is the same in both time periods
Here, the effect is negative in the early period and positive in the late period

21. Piecewise Models Look at coefficients at 2 (or more) spans of time
EARLY
. stcox gdp degradation education democracy ngo ingo if year < 1985, robust nohr
          _t |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
         gdp |           1.05   0.294       
 degradation |          -1.80   0.072       
   education |         -0.59   0.552       
   democracy |           0.87   0.382        
         ngo |           2.18   0.029        
        ingo |           1.39   0.165       
Note: Effect of ngo is larger in early period
LATE
. stcox gdp degradation education democracy ngo ingo if year >= 1985, robust nohr
          _t |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
         gdp |            1.39   0.165        
 degradation |         -2.38   0.017      
   education |           0.99   0.323       
   democracy |           2.59   0.010        
         ngo |           1.20   0.229       
        ingo |         -0.54   0.590       

22. Proportional Hazard Assumption
4. Tests based on re-estimating model
Try including time interactions in your model
Recall: Interactions – effect of A on C varies with B
If effect of variable X on hazard rate (or ratio) varies with time, then hazards aren’t proportional
Recall example: Abstinence programs
Perhaps abstinence programs have a big effect initially, but the effect diminishes (or reverses) later on

23. Proportional Hazard Assumption
Red = Abstinence group; green = control
Positive time
interaction
No time interaction
In non-proportional case, the effect of abstinence programs varies across time

24. Proportional Hazard Assumption
Strategy: Create variables that reflect the interaction of X variables with time
Significant effects of time interactions indicate non-proportional hazard
Fortunately, inclusion of the interaction term in the model corrects the problem.
Issue: X variables can interact with time in multiple ways…
Linearly
With “log time” or time squared
With time dummies
You may have to try a range of things…

25. Proportional Hazard Assumption
Red = Abstinence group; green = control
Linear time interaction
Effect grows consistently over time
Try “Abstinence*time”
Interaction with time-period… Effect differs early vs. late Try “Abstinence*DLate”

26. Proportional Hazard Assumption
5. Grambsch & Therneau test
Ex: Stata “estat phtest”
Test for non-zero slope of Schoenfeld residuals vs time
Implies log hazard ratio function = proportional
Can be applied to general model, or for each variable
stcox gdp degradation education democracy ngo ingo, robust nohr scaledsch(sca*) schoenfeld(sch*)
. estat phtest
Test of proportional hazards assumption
Time: Time
| chi df Prob>chi2
global test |
Significant chi-square indicates violation of proportional hazard assumption

27. Proportional Hazard Assumption
Variable-by-variable test “estat phtest”:
Note: Certain variables are especially problematic…
. estat phtest, detail
Test of proportional hazards assumption
Time: Time
| rho chi df Prob>chi2
gdp |
degradation |
education |
democracy |
ngo |
ingo |
global test |

28. Proportional Hazard Assumption
Notes on estat phtest :
1. STATA 9/10: Requires that you calculate “schoenfeld residuals” when you run the original cox model
And, if you want a test for each variable, you must also request scaled schoenfeld residuals
2. Test is based on identifying non-zero time trend… but how should we characterize time?
Options: normal/linear time, log time, time dummies, etc
Results may differ depending on your choice
Ex: estat phtest, log – specifies “log time”
Plot of smoothed Schoenfeld residuals can indicate best way to characterize time
Linear trend (not a curve) indicates that time is characterized OK
Ex: estat phtest, plot(ngo) OR estat phtest, log plot(ngo)

29. Proportional Hazard Assumption
What if the assumption is violated?
1. Improve model specification
Add time interactions to address nonproportionality
Ex: If high democracies are not proportional to low democracies, try adding “highdemoc*time”
Variables can be interacted with linear time, log time, time dummies, etc., to address the issue
2. Model groups separately
Split sample along variables that are non-proportional.

30. Proportional Hazard Assumption
What if the assumption is violated?
3. Use a stratified Cox model
Allows a different baseline hazard for each group
But, you can’t estimate effect of stratifying variable!
Ex: stcox var1 var2 var3, strata(Dhighdemoc)
4. Use a piecewise model
Split time into chunks… in which PH assumption is met
Requires sufficient sample size in all time periods!

31. Proportional Hazard Assumption
What if the assumption is violated?
5. Live with it (but temper your conclusions)
Violation of proportional hazard assumption tends to:
Overestimate the effect of variables whose hazard ratios are increasing over time
And, underestimate those whose hazard ratios are decreasing
However, Allison points out: Cox model is reasonably robust
Other issues (e.g., model misspecification) are bigger issues

32. Discrete Time EHA Models
Distinction: Continuous vs. Discrete EHA
“Discrete time”: time divided into integer chunks
Years, decades, months
Spell start & end times are essentially “rounded off”
Continuous time: time conceptualized as an unbroken continuum
Times need not be rounded off
High levels of precision are possible
Not just integers, but decimals.

33. Discrete Time EHA Models
Issue: Discrete vs. continuous time gives rise to different EHA models
Example: The hazard rate is defined for continuous time:
The hazard rate over discrete (identical-sized) chunks of time is (ti):

34. Discrete Time EHA Models
Issue: If the hazard rate in discrete time is a probability, maybe we can model it as such…
Standard options for modeling probabilities:
Logistic regression (logit) model
Probit model
Complementary log/log model (cloglog)
An asymmetric function
Starts slowly from p=0, but accelerates more rapidly toward p=1 at the end
Often used when predicted probabilities are very low or high.

35. Discrete Time EHA Models
Example: Discrete time logit model
Where p is the probability of an event (Y=1) for a discrete chunk of time
Complementary log log model looks like this:

36. Discrete Time EHA Models
Basic logit/probit/cloglog models are like constant-rate/exponential models
They assume a constant baseline hazard, represented by constant in the model
Discrete EHA models are are proportional hazard models
Logit output reports coefficients and odds ratios…
But, it is appropriate to refer to them as hazard ratios
Coefficient interpretation is the same
Raw coeficientss require exponentiation to interpret…

37. Discrete Time EHA: Data
Discrete time models require split-spell data where each spell has constant length
Example: every record in your data represents 1 year
Number of cases represents total time at risk
Ex: If caseid 1 has 10 records, it was at risk for 10 years…
This differs from continuous models, where records can represent variable amounts of time
E.g., by providing specific start and end times…

38. Discrete Time EHA Data
Discrete time data looks like other examples of split spell data
But, each record MUST be the same length
Example: Country data over time:
Logit/probit/cloglog simply models outcome of 1
newname2 newid3 year law eventnum start end ss es pop
INDIA
INDIA
INDIA
INDIA
INDIA
INDIA
INDIA
INDIA
INDIA
Event (Y=1)

39. Discrete Time Logit Model
Logit model for discrete time EHA
It is a constant rate model
In fact, results are almost the same as streg…
. logit es gdp degradation education democracy ngo ingo
Logistic regression Number of obs =
LR chi2(6) =
Prob > chi2 =
Log likelihood = Pseudo R =
es | Coef. Std. Err z P>|z| [95% Conf. Interval]
gdp |
degradation |
education |
democracy |
ngo |
ingo |
_cons |

40. Discrete Time and Cox Models
A Cox model can also be estimated in the discrete time context
Indeed, the discrete time example helps illustrate what a Cox model really is (even in continuous time)
Idea: Use a conditional logit model
Conditioned on the cases in the risk set at each point in time
… rather than a traditional logit model

41. Discrete Time and Cox Models
A conditional logit model estimates common coefficients across models for many groups
Looks at within-group factors, net of overall rate within each group… sorta like a fixed-effects model…
Box-Steffensmeier & Jones, p. 80
Thus, effects are modeled net of the “baseline hazard”
Interpretation: A Cox model is like pooling a large set of logit results
In the continuous time context, the group is the current risk set at the time of any failure

42. Discrete Time and Cox Models
A conditional logit model on discrete time EHA yields identical results to a Cox Model;
If you specify the “exact partial” method for handling ties in the continuous time Cox model
We’ll cover this later

43. Discrete Time Cox Model
Conditional logit model – a cox model
Yields identical results to cox when using discrete data
. clogit es gdp degradation education democracy ngo ingo, group(year)
Conditional (fixed-effects) logistic regression Number of obs =
LR chi2(6) =
Prob > chi2 =
Log likelihood = Pseudo R =
es | Coef. Std. Err z P>|z| [95% Conf. Interval]
gdp |
degradation |
education |
democracy |
ngo |
ingo |

44. Discrete vs. Continuous EHA
In practice, we can often use either discrete or continuous methods
Even though time is theoretically continuous, our measures are usually limited to discrete time intervals
Ex: year, month, day…
For yearly spell data (or any other consistent interval) the data sets are pretty much identical
If time resolution is extremely poor, there can be advantages to using discrete time models
Otherwise, continuous time models provide greater flexibility
And more modeling options.

45. EHA Example In-class group activity: Let’s design a study
Outcome of interest: Students dropping a course
What is the risk set?
How would you set up the data?
What are key independent variables?
What kind of model would you use?
Work in groups of 2-4, and be prepared to discuss your thoughts…

46. Reading Discussion
Empirical Example:  Soule, Sarah A and Susan Olzak.  2004.  “When Do Movements Matter? The Politics of Contingency and the Equal Rights Amendment.”  American Sociological Review, Vol. 69, No. 4. (Aug., 2004), pp
Long, J. Scott, Paul D. Allison, and Robert McGinnis.  1993.  “Rank Advancement in Academic Careers:  Sex Differences and the Effects of Productivity.”  American Sociological Review, 58, 5: