1. Modeling Diabetic Hospitalizations for the TennCare Population Application of Predictive Modeling for Care Management Panel AcademyHealth Annual Research Meeting June 28, 2005 Boston Avery Ashby MS Soyal Momin MS, MBA Raymond Phillippi PhD Allen Naidoo PhD Judy Slagle RN, MPA

2. Background

3. BlueCross BlueShield of Tennessee provides care management programs for members with certain chronic illnesses or conditions. Care managers are licensed nurses. Diabetes is a prevalent chronic illness affecting our managed TennCare population. Modeling of diabetic inpatient hospitalizations can help in identifying and directing those members at higher risk to care management. Management Programs

4. Methodology

5. Diabetic members were identified using member level claims data. Data were collected for continuously enrolled diabetic members for the time period of July 1, 2001 through June 30, 2003. Year 1 member specific data were used to model whether a diabetic hospitalization occurred in Year 2. Logistic regression was employed to model the probability of a diabetic hospitalization in Year 2. Study Design Time Period Year 1 July 1, 2001 – June 30, 2002 Member Specific Data Year 2 July 1, 2002 – June 30, 2003 Diabetic Hospitalization?

6. Data Elements

7. Gender Age Zip Code Metropolitan & Rural Region Multiple Regions Eligibility Medicaid subcategories not including dual-eligible members Demographics

8. Diabetic Hospitalizations Emergency Room Encounters Ophthalmologist Encounters Primary Care Physician (PCP) Encounters Endocrinologist Encounters Total Specialist Encounters Utilization

9. Insulin Prescriptions Prescribed or Not Misc. Anti-diabetic Prescriptions Prescribed or Not Sulfonylurea Prescriptions Prescribed or Not Caloric Agents Prescribed or Not Total Prescriptions (Any variety) Pharmacy

10. Cholesterol Screening Received or Not Eye Examination Received or Not Microalbuminuria Screening Received or Not HbA1c Screening Received or Not Evidence Based Guidelines

11. Insulin Dependency Dependent or Not Total Co-morbidities Diagnostic Cost Grouper (DCG) Risk Score Diagnosis and Risk Score

12. Members: 11,002 (313 Year 2 Hospitalizations) Gender: Female 64.7% Age: Mean 47 Median 50 General Data Characteristics

13. Predictive Model

14. Probability of hospitalization = 1/(1+e -z ) Where z = -2.160 + ( 1.164 * Diabetic Hospitalizations) – ( 0.328 * No Insulin prescribed) – ( 0.038 * Age) +( 0.092 * Diagnostic Cost Grouper Risk Score) + ( 0.199 * No Misc. Anti-diabetic prescribed) + ( 0.208 * Ophthalmologist Encounters) – ( 0.015 * Primary Care Physician Encounters) – ( 0.361 * Non-Insulin Dependent) + ( 0.054 * Emergency Room Encounters) – ( 0.031 * Total Specialist Encounters) Model Specifics

15. Sensitivity vs. Specificity

16. Odds Ratio Estimates Model Specifics Covariate Odds Ratio Lower Limit Upper Limit Diabetic Hospitalizations3.2032.5414.039 Insulin PrescribedNo vs. Yes0.5190.2750.982 Age0.9630.9550.971 Diagnostic Cost Grouper Risk Score1.0961.0771.116 Misc. Anti-diabetic PrescribedNo vs. Yes1.4891.1541.922 Ophthalmologist Encounters1.2321.1011.377 Primary Care Physician Encounters0.9850.9700.999 Insulin DependencyNo vs. Yes0.4860.2540.929 Emergency Room Encounters1.0561.0261.086 Total Specialist Encounters0.9690.9510.987

17. Diagnostics CovariateTolerances * Diabetic Hospitalizations0.92 Insulin Prescriptions0.19 Age0.91 Diagnostic Cost Grouper Risk Score0.66 Anti-diabetic Prescriptions0.94 Ophthalmologist Encounters0.94 Primary Care Physician Encounters0.71 Insulin Dependency0.19 Emergency Room Encounters0.54 Total Specialist Encounters0.40 *Tolerance is 1- R 2 x, where R 2 x is the variance in each covariate, X, explained by all of the other covariates.

18. Goodness of Fit

19. Model Performance 97.5%Negative Predictive Value (NPV) 0.223Pseudo-R 2 97.2%Correct Prediction Rate 12.1%Sensitivity 99.7%Specificity 52.8%Positive Predictive Value (PPV) 10,689 10,655 34 No Stay 313 275 38 Actual 11,002Totals 10,930No Stay 72 Prediction TotalsStay

20. Rational Artificial Intelligence

21. An artificial Neural Network (ANN) was trained and validated on the entire data set. Problematic because the ANN tried to maximize the overall correct prediction rate. Similar results to logistic regression models. Initial RAI Results

22. RAI Model Performance 97.5%Negative Predictive Value (NPV) N/APseudo-R 2 97.4%Correct Prediction Rate 10.9%Sensitivity 99.9%Specificity 82.9%Positive Predictive Value (PPV) 10,689 10,682 7 No Stay 313 279 34 Stay Actual 11,002Totals 10,961No Stay 41Stay Prediction Totals

23. Collect equal samples from hospitalized and non-hospitalized members. Build ANN based on this 1:1 (150:150) training data set. Validate ANN on remaining Out-of-Sample members. Repeat process to ensure that the overall pattern is accounted for. Develop credibility intervals for sensitivity, specificity, PPV, and NPV based on this repeated process. Forced Learning Solution

24. Results of repeated forced learning method were collected. 95% credibility intervals were derived from MCMC simulation using WinBUGS 1.4. [4.11%,4.49%] Positive Predictive Value (PPV) [98.36%,98.73%][76.06%,78.13%][66.00%,70.80%] Negative Predictive Value (NPV) SpecificitySensitivity Forced Learning Model Performance

25. Research Implications

26. Begins with the question of allocated resources. Logistic regression model and ANN identified a small percentage of members with an actual Year 2 hospitalization with a “reasonable” PPV. ANN using the Forced Learning Method identified a much larger percentage of members with an actual Year 2 hospitalization with a low PPV. Finding a Balance

27. Coverage No hospitalization hospitalization No hospitalization hospitalization Logistic Regression Model Forced Learning ANN Predicted hospitalization

28. Other covariates like lab values, Health Risk Assessments (HRAs), and psychological indicators. Using a meta-model where clusters of homogenous sub-groups are modeled separately [and possibly] with differing methods. Model probability of co-morbid condition related hospitalizations instead of diabetic hospitalizations. Future Considerations

29. Avery Ashby MS Senior Research Analyst Health Intelligence Group 801 Pine Street – 3E Chattanooga, TN 37402 423.763.7482 p 423.785.8083 f avery_ashby@healthintelgroup.com Contact Information