1. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Assuring an ROI in your Disease Management Program Al Lewis Disease Management Purchasing Consortium April 2003

2. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Agenda Disease Management Purchasing Consortium The ROI Imperative Measuring It –3 Biggest Mistakes Inflating ROI –What ROIs are by disease Why “Built” programs usually have most fallacies 5 Take-Aways

3. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Hey, Butch, Who ARE these guys? 74 payor members (including all large outsourcers and many small ones) 50-million lives of purchasing power Named “most influential organization in DM” by every trade publication, plus British Medical Journal Published 4 articles on ROI in DM

4. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Agenda Disease Management Purchasing Consortium The ROI Definition and Imperative Measuring It –3 Biggest Mistakes Inflating ROI –What ROIs are by disease Why “Built” programs usually have most fallacies 5 Take-Aways

5. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com ROI Definition Gross Savings (before fees)/Fees = ROI Questionable measure in and of itself –No real “investment” if savings accrue in same year as investment and… –Savings are usually guaranteed anyway so nothing was risked Complement or substitute: Net Savings Maximization

6. Examples: 2 Programs #1 Gross 1 st -Year Savings Guarantee: $2-million Fees: $1-million ROI: 2:1 Net Savings: $1- million #2 Gross 1 st Year Savings Guarantee: $5-million Fees: $3-million ROI: 1.6:1 Net savings: $2- million Which is better for health plan’s bank account?

7. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com But for the purposes of this presentation… The measurement issues apply to ROI and net savings maximization

8. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Pop Quiz Program has a 6% guarantee of net savings and fees amount to 4%. What is the ROI?

9. The ROI Imperative Widely Accepted that DM Programs improve health outcomes, satisfy members Still quite contentious whether they save money because ROIs are often invalid –Vendor claims not believed in general because some aren’t valid –Almost every “built” program is measured wrong (present company excepted, which is why I invited the present company) Industry needs to build credibility via validity in ROI

10. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Today’s Goal How to Identify and Avoid the top 3 issues (there are more but they are more subtle) ways in which ROIs are inflated –This will allow you to create or contract for credible numbers

11. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Agenda Disease Management Purchasing Consortium The ROI Definition and Imperative Measuring It –3 Biggest Mistakes Inflating ROI –What ROIs are by disease Why “Built” programs usually have most fallacies 5 Take-Aways

12. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com 3 Biggest Mistakes Inflating ROI Measuring on Moderates and Severes only (creates regression to mean) Measuring only on people who enroll (sample selection bias) Measuring new members after sentinel event

13. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com What is this regression to the mean? The Yankees are 20-4 as of this writing. They are not going to finish the season 135-27. The Tigers are 3-19. They are not going to finish the season 22-140. Same is true of people with chronic disease. If you take all people with chronic disease who had a really bad month like the Tigers—with maybe 2 hospitalizations--they will show ‘improvement’ simply because they aren’t going to have 2 hospitalizations every month.

14. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com More regression to the mean: “ Red Zone” Winning Percentages April 12 of “severely ill” baseball teams Baltimore.333 Cleveland.333 Detroit.000 Atlanta.364 Milwaukee.300 Arizona.200

15. Watch these “red zone” teams regress to the mean by April 26 Baltimore.333.500+.167 Cleveland.333.292-.041* Detroit.000.136+.136 Atlanta.364.600+.236 Milwaukee.300.360+.060 Arizona.200.375+.175 4/104/26 * Not all red-zone patients regress to the mean…we’re talking about populations as a whole

16. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Likewise, “Green Zone” patients regress the wrong way to the mean TeamWinning % April 10 Winning % April 26 Change Yankees.889.833-.056 Kansas city1.000.773-.227 Oakland.700.583-.117 Houston.778.458-.320 San Francisco.909.750-.159

17. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Lessons If you only measure people who are severe or moderate and severe, based on the objective claims data you have, your sample will show improvement no matter what you do If you leave the “green” pts out of your measurement, you will miss a lot of “events” in your “after” calculation

18. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com What you do you to avoid it? MEASURE THE WHOLE POPULATION –The “Whole population” of baseball teams have a.500 record every day –Whole populations cannot regress to the mean; they are the mean A watch-out: One health plan, which understood the fallacy, measured the whole population of asthma patients at its accounts with the most asthma utilization and showed great improvements Watch-out #2: Same plan said those accounts had been high for 3 years

19. source; gregg meyer Asthma studies suffering from regression to the mean (plus ACAAI study just published) Program Study Type 1.Massachusetts Respiratory Hospital/Air Watch A 26Historical Control 2.Valley Home CareP30Historical Control 3.Jacksonville University Medical CenterA25Historical Control 4.University of Tennessee, Memphis A 39Non-randomized Control 5.Access Health — BC/B, Massachusetts A 67Historical Control 6.Lutheran Medical Center Brooklyn, New York P50 — 7.United Health Care of Ohio A/P — Historical Control 8.Lovelace HMO, Albuquerque, N.M.P86 Historical Control 9.Kaiser Permanente, Santa Clare — — — 10.Harvard Community Health Plan (1995)P 53Historical Control 11.Managed Care Services P11Historical Control 12.University of Pennsylvania, Philadelphia A — — 13.National Jewish Medical CenterA/P317Historical Control 14.University of Laval, Quebec A 126Non-randomized Control 15.Henry Ford HospitalA241Randomized Control 16.Harvard Community, Health Plan (1999)P57Randomized Control 17.Hartford Hospital, Connecticut A 23Historical Control 18.Olsten Kimberly Quality Care A/P257 — 19.Blue Cross/Blue Shield of South Carolina — 3,000 — 20.Harvard Pilgrim Health Care, Boston —— — Emergency Room Visits Hospitalization Annualized Savings per Patient R e d u c t i o n s i n :E c o n o m i c s : $5,981 — 4,026407 1,676278—— 3,400 — —— — 45 —— 3,296418— 1,500 — 2,260606— 62885 2,217190 4,137 — 11,172 —— $1.07 million — total savings Patient # in Study Adult/ Ped Annualized Cost per Patient

20. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com 3 Biggest Mistakes Inflating ROI Measuring on Moderates and Severes only (creates regression to mean) Measuring only on people who enroll (sample selection bias) Measuring new members after sentinel event

21. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Example: Participation Bias The vendor says: “ The study group’s claims went down 20% over two periods vs. a control group

22. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Participation Bias The vendor says: “ The study group’s claims went down 20% over two periods vs. a control group

23. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Participation Bias Your antennae go up when vendor says: “ The control group was matched to the study group for age, sex, and claims history, and consisted of people who were not offered the program...”

24. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Participation Bias Your antennae go up when vendor says: “ …and the control group’s claims rose 20% over the same period.” The study group consists of Motivated people only! The popu- ation has everybody

25. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com How to avoid participation bias in measuring ROI (You guessed it) measure on the whole population of people with the disease But you will soon see that’s not as easy as it sounds…

26. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com …Because of sentinel event measurement Measuring on Moderates and Severes only (creates regression to mean) Measuring only on people who enroll (sample selection bias) Measuring new members after sentinel event

27. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Sentinel Events: The key to whole- population measurement The “sentinel event” is the event which tells the health plan that someone has a disease It is often the most expensive claim from that member during the first 12 months with the disease

28. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Let’s do Sentinel Events with more baseball Analogy that the first loss a team has is like the first claim for a disease. You are searching your database for people with a disease, called “lossitis”

29. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Standings after first game TeamWonLostTeamWonLost Yankees10Red Sox01 Tampa10Blue Jays01 Baltimore10White Sox01 Royals10Cleveland01 Seattle10Detroit01 Anaheim10Texas01 Minnesota10Oakland01

30. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Let’s do Sentinel Events with more baseball Analogy that the first loss a team has is like the first claim for a disease. You are searching your database for people with a disease called “lossitis” where each loss costs $1000 After the first game you have identified 7 teams with “claims” (i.e., losses) and they cost you $7000 –So you track those teams for the next claims cycle to reduce lossitis and save money

31. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Standings after second game TeamWonLostTeamWonLost Yankees20Red Sox11 Tampa11Blue Jays11 Baltimore11White Sox02 Royals20Cleveland02 Seattle11Detroit02 Anaheim11Texas11 Minnesota20Oakland11

32. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Let’s do Sentinel Events with more baseball Analogy that the first loss a team has is like the first claim for a disease. You are searching your database for people with a disease called lossitis. After the first game you have identified 7 teams with “claims” (i.e., losses) –So you track those teams for the next claims cycle Now you find that those teams only had 3 “claims” in this cycle so among identified people with lossitis, claims fell by $4000

33. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Just counting previously identified members with lossitis If you don’ count sentinel events This is the “savings” from reducing lossitis

34. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Standings after second game— including new “sentinel events”— patients with lossitis TeamWonLostTeamWonLost Yankees20Red Sox11 Tampa11Blue Jays11 Baltimore11White Sox02 Royals20Cleveland02 Seattle11Detroit02 Anaheim11Texas11 Minnesota20Oakland11

35. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com But when you add in other “sentinel event” claims you learn that didn’t save any money—your overall lossitis rate is still the same

36. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Let’s look at this in disease management Asthma Cardiology

37. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Example of Sentinel Events: Asthmatics People with Asthma claims Last year

38. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com If you exclude asthmatics with sentinel events this year and just look at ALL last year’s asthmatics with claims… People with Asthma claims Last year Those people’s Claims WILL decline This year 2000 2001

39. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Now see what happens if you add this year’s sentinel-event asthmatics People with Asthma claims Last year Those people’s Claims WILL decline This year Newly Dx’ed—No 2000 Claims history New to Plan—No 2000 claims history Not taking Meds since 1999 or No claims since 1999—No 2000 claims 2000 2001

40. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Example: Asthmatics People with Asthma claims Last year Those people’s Claims WILL decline This year Newly Dx’ed New to Plan Not taking Meds If you don’t Count Sentinel Events you Show excellent ROI 2000 2001

41. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com But the REAL ROI is NEGATIVE in this example People with Asthma claims Last year Those people’s Claims WILL decline This year Newly Dx’ed New to Plan Not taking Meds All asthma Cost 2001/ All asthma Cost 2000 Your asthma cost Went up a little 2000 2001

42. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Why you don’t want to WAIT until AFTER the Dx or sentinel event to start counting Example #2: Cardiology

43. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Actual Cardiology Example ($ PMPM)

44. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Actual Cardiology Example: ROI goes from 4:1 to 2:1 (note: always question ROI >2:1) This was the effect of including Sentinel events in the contract Period, after fees

45. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Also, it’s not just measurement…it’s management! You WANT vendors to PREDICT and AVOID first events because 50% of all cardiac events are FIRST events –You should measure on ALL events to create that incentive

46. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Why payors don’t include sentinel events for members added later, even though it inflates ROI Don’t get that they aren’t in (e.g.,”all diagnosed patients”) Get that they aren’t in but don’t get ROI significance of leaving them out Vendors say it’s unfair to them –No: (1) they guarantee lower but “real” ROI and (2) see RFP—you need to get them timely data to count these patients

47. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Just to clarify… Your BASELINE will include people who had their so-called “sentinel diagnosis” DURING the baseline, thus INFLATING it Going forward, if you wait until patients are diagnosed to count them in the program and measure their costs GOING FORWARD, they will cost MUCH LESS than the baseline because you EXCLUDED the sentinel event

48. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Difference between sentinel event (SE) bias and regression to mean They are biases along the same spectrum Differences: –SE includes PEOPLE NOT PREVIOUSLY DIAGNOSED, regression to mean is always among people diagnosed –SE is solved by INCLUDING ALL CLAIMS on some people not otherwise included if you would just be looking at the diagnosed population

49. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Agenda Disease Management Purchasing Consortium The ROI Imperative Measuring It –3 Biggest Mistakes inflating ROI –What ROIs are by disease Why “Built” programs usually have most fallacies 5 Take-Aways

50. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Typical Return on Investment by Disease (first year 100% guarantee as multiple of cost—measured right) Breakeven line

51. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Agenda Disease Management Purchasing Consortium The ROI Imperative Measuring It –3 Biggest Mistakes inflating ROI –What ROIs are by disease Why “Built” programs usually have most fallacies 5 Take-Aways

52. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Why “built” programs ESPECIALLY overstate their ROIs Trivia: –If you look at relative budgets, what are the NUMBER ONE and NUMBER TWO areas in which “built” programs underspend vs. vendors’ own programs and how does that affect reported ROI?

53. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Dollars spent per $20 in claims— typical built vs. bought

54. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Dollars spent per $20 in claims— typical built vs. bought So you measure Only on the 15-25% Who are easiest to Enroll!

55. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com Dollars spent per $20 in claims— typical built vs. bought So you measure Only on the 15-25% Who are easiest to Enroll! And you don’t spend nearly what you need To spend to measure right because there Is no guarantee “on the line”

56. (c) 2003 DMPC LLC. All rights reserved yaddayaddayadda www.DisMgmt.com 5 Things to take Away Three biggest inflators of ROI: Regression to mean, self-selection, failure to include sentinel event If you are building you are almost certainly measuring wrong If you are outsourcing and you’ve never seen this presentation before or your are not a DMPC member, you are probably measuring wrong Even when you measure correctly, there is still a strong ROI