Phaedra Corso, Ph.D. Associate Professor College of Public Health University of Georgia Program Evaluation from an Economic Perspective

Why Care About Economics in the Context of Prevention?  Maximizing outcomes is important.  Minimizing costs is important too.  Resources are limited, so hard (resource allocation) decisions must be made.  Demonstrates the value provided from the resources expended (return on investment).

The PH Model for Prevention – Economics? Problem Identification Program and Policy Evaluation Risk and Protective Factor Identification Implementation and Dissemination Program and Policy Development Economic Evaluation Economic Impact - COI

EE Methods Partial evaluation – costs only Partial evaluation – costs only Cost of illness (COI) analysis Cost of illness (COI) analysis Cost analysis (program costs) Cost analysis (program costs) Full evaluation – costs and outcomes Full evaluation – costs and outcomes Cost-benefit analysis (CBA) Cost-benefit analysis (CBA) Cost-utility analysis (CUA) Cost-utility analysis (CUA) Cost-effectiveness analysis (CEA) Cost-effectiveness analysis (CEA)

COI Analysis Estimates total costs incurred because of a disease or condition Estimates total costs incurred because of a disease or condition (i.e., medical costs, non-medical costs, productivity losses) (i.e., medical costs, non-medical costs, productivity losses) Generally reported as Generally reported as annual total cost annual total cost average per person lifetime cost average per person lifetime cost Used to show potential benefits of prevention efforts Used to show potential benefits of prevention efforts

Costs of Violence in the United States Corso et al., AJPM 2007

Cost Parameters Direct costs Direct costs Medical care Medical care ED visits ED visits Hospitalizations Hospitalizations Ambulance/paramedic Ambulance/paramedic MD visits MD visits Dental Dental Physical Therapy Physical Therapy Prescription Drugs Prescription Drugs Mental health care Mental health care Productivity losses Productivity losses Work losses Work losses Household productivity losses Household productivity losses

Incidence ~2.2. million medically-treated injuries associated with violence occurred in 2000 ~2.2. million medically-treated injuries associated with violence occurred in 2000 ~17,000 homicides, ~30,000 suicides ~17,000 homicides, ~30,000 suicides People aged 15 to 44 years comprise 44 percent of the population, but account for nearly 75 percent of violent injuries People aged 15 to 44 years comprise 44 percent of the population, but account for nearly 75 percent of violent injuries

Costs The total cost associated with nonfatal injuries and deaths due to violence in 2000 was more than $70 billion. The total cost associated with nonfatal injuries and deaths due to violence in 2000 was more than $70 billion. $37 billion for interpersonal violence $37 billion for interpersonal violence $33 billion for self-inflicted violence $33 billion for self-inflicted violence The average cost per homicide was $1.3 million in lost productivity and $4,906 in medical costs. The average cost per homicide was $1.3 million in lost productivity and $4,906 in medical costs. The average cost per case for a non-fatal assault resulting in hospitalization was $57,209 in lost productivity and $24,353 in medical costs. The average cost per case for a non-fatal assault resulting in hospitalization was $57,209 in lost productivity and $24,353 in medical costs. The average cost per case of suicide is $1 million lost productivity and $2,596 in medical costs. The average cost per case of suicide is $1 million lost productivity and $2,596 in medical costs. The average cost for a non-fatal self inflicted injury was $9,726 in lost productivity and $7,234 in medical costs. The average cost for a non-fatal self inflicted injury was $9,726 in lost productivity and $7,234 in medical costs.

So What? The incidence and economic burden of injuries in the US is substantial The incidence and economic burden of injuries in the US is substantial This information can be used to lobby for more prevention resources This information can be used to lobby for more prevention resources Implementation of effective interventions could reduce this burden Implementation of effective interventions could reduce this burden The cost to implement effective interventions less the economic burden prevented – represents the potential returns on investment for prevention. The cost to implement effective interventions less the economic burden prevented – represents the potential returns on investment for prevention. This is what economic evaluation is all about!! This is what economic evaluation is all about!!

Cost Analysis (CA) Estimates total costs of running a program Estimates total costs of running a program Costs are the value of the resources (people, building, equipment and supplies) used to produce a good or a service Costs are the value of the resources (people, building, equipment and supplies) used to produce a good or a service Important for realizing costs from varying perspectives Important for realizing costs from varying perspectives e.g., incurred by program, incurred by participant e.g., incurred by program, incurred by participant Includes not just financial, but also economic costs. Includes not just financial, but also economic costs. Important for budget justification, decision making, and forecasting. Important for budget justification, decision making, and forecasting. Also called: cost consequence or cost identification analysis Also called: cost consequence or cost identification analysis Provides the first step of a full economic evaluation Provides the first step of a full economic evaluation

Cost Analysis of a national replication of a child maltreatment program Corso et al., CDC, OCAN (in progress)

Define Cost Categories

Preliminary Results at end of Year 1 The average cost per family referral ranged from $2,319 to $8,906. The average cost per family referral ranged from $2,319 to $8,906. The average cost per family receiving services ranged from $4,238 to $33,742. The average cost per family receiving services ranged from $4,238 to $33,742. At the end of the first year of implementation, pre-implementation costs as a percentage of total costs ranged from 23% to 42% of the total costs of the program. At the end of the first year of implementation, pre-implementation costs as a percentage of total costs ranged from 23% to 42% of the total costs of the program.

So What? Provides information for Agency X who might want to implement the program in the future. Provides information for Agency X who might want to implement the program in the future. Provides the cost component of a full EE. Provides the cost component of a full EE. Lessons learned on how to conduct a programmatic CA: Lessons learned on how to conduct a programmatic CA: Prospective data collection Prospective data collection Input from site implementers Input from site implementers Technical assistance throughout data collection Technical assistance throughout data collection Revisions of cost collection templates along the way Revisions of cost collection templates along the way Reference: Applying Cost Analysis to Public Health Programs (at

What is Economic Evaluation (EE)? Applied analytic methods to: Identify, Measure, Value, and Compare the costs and consequences of treatment* and prevention** strategies. * Done a lot ** Done “not so much”

Cost-benefit Analysis - CBA A method used to compare costs and benefits of an intervention A method used to compare costs and benefits of an intervention where all the costs and benefits are standardized or valued in monetary terms. where all the costs and benefits are standardized or valued in monetary terms. Provides a list of all costs and benefits over time: Provides a list of all costs and benefits over time: Can have different time lines Can have different time lines Can have different amounts at different times Can have different amounts at different times Provides a single value: Provides a single value: Net Benefits: NB (Benefits – Costs) Net Benefits: NB (Benefits – Costs)

When is CBA Used? To decide whether to implement specific programs To decide whether to implement specific programs If NB > 0, implement If NB > 0, implement When choosing among competing options When choosing among competing options Implement program with highest NB Implement program with highest NB For setting priorities on options given resource constraints For setting priorities on options given resource constraints

Quantify Benefits - CBA Cost-of-Illness (COI) approach Cost-of-Illness (COI) approach Willingness-to-Pay (WTP) or Contingent-valuation surveys Willingness-to-Pay (WTP) or Contingent-valuation surveys (e.g., how much is society willing to pay to reduce the annual morbidity and mortality risk associated with a disease or injury) (e.g., how much is society willing to pay to reduce the annual morbidity and mortality risk associated with a disease or injury)

Corso Survey (Fall 2007, Georgia) “Based on national data, 2 out of every 100,000 children annually, or an average of 4 children every day are killed as a result of child maltreatment by parents or caretakers.” “Now imagine we had a nationally-sponsored child maltreatment prevention program that was available to your state and this this program was proven to reduce the risk of a child being killed due to child maltreatment by 50%. This means that the number of children killed on average every day in the U.S. by child maltreatment is reduced from 4 per day to 2 per day.” “If this program were available to your state, would you be willing to pay $150 in extra taxes per year to sponsor this program?” YES – “Would you be willing to pay $225?” NO – “Would you be willing to pay $75?”

Cost-utility Analysis - CUA A method used to compare costs and benefits of interventions where benefits are expressed as the number of life years saved adjusted to account for loss of quality. A method used to compare costs and benefits of interventions where benefits are expressed as the number of life years saved adjusted to account for loss of quality. Combines Combines Length of life (survival), and Length of life (survival), and Quality of life Quality of life Compares disparate outcomes in terms of utility Compares disparate outcomes in terms of utility Quality-adjusted life years (QALYs) Quality-adjusted life years (QALYs) Disability-adjusted life years (DALYs) Disability-adjusted life years (DALYs) Derives a ratio of cost per health outcome Derives a ratio of cost per health outcome $/QALY or $/DALY $/QALY or $/DALY

When is CUA Used? When quality of life is the important outcome. When quality of life is the important outcome. When the program affects both morbidity and mortality. When the program affects both morbidity and mortality. When the programs being compared have a wide range of different outcomes. When the programs being compared have a wide range of different outcomes. When the program is being compared with a program that has already been evaluated using CUA. When the program is being compared with a program that has already been evaluated using CUA.

Quantify Benefits - CUA Utilities are: Utilities are: A “preference-based” measure of health, that relies on choice and uncertainty to elicit preferences A “preference-based” measure of health, that relies on choice and uncertainty to elicit preferences Typically based on a 0 (death) to 1 (perfect health) scale Typically based on a 0 (death) to 1 (perfect health) scale

Example of Tool to Elicit Utilities: Time Trade-Off (TTO) Quality of Life Length of Life Long and dull Short and fun Which life do you prefer?

Example: TTO 0 Utility Years Dead U(healthy) = 1.0 U (blind both eyes) = ? 2012 blind both eyes healthy

Combining Quality of Life with Length of Life Utility without preventionwith prevention Years QALYS (with prevention) = 1.0*75 = 75 QALYS (without prevention) = 0.7*30 = 21

Cost-effectiveness Analysis - CEA  Estimates costs and outcomes of interventions  Expresses outcomes in natural units  e.g., cases prevented, lives saved  Compares results with other interventions affecting the same outcome  Summary measure: cost-effectiveness ratio  Cost per some outcome achieved  e.g., cost per case prevented, cost per life saved

When is CEA Used? Used to identify Used to identify most cost-effective strategy from options that produce a common effect most cost-effective strategy from options that produce a common effect practices that are not “worth” their costs practices that are not “worth” their costs Used for empirical support for under-funded Used for empirical support for under-funded programs programs

Quantify Outcomes – CEA of parenting intervention Intermediate outcomes Intermediate outcomes Increased child self-esteem and mental health status Increased child self-esteem and mental health status Increased family cohesiveness/coping skills Increased family cohesiveness/coping skills Decreased depression in parents Decreased depression in parents Final outcomes Final outcomes QOL improvements in parents and children QOL improvements in parents and children Child maltreatment cases prevented Child maltreatment cases prevented Lives or life years saved Lives or life years saved

CEA Caveat  Outcomes cannot be combined, so one or two of the most important effectiveness measures should be considered (separately) for the CEA.  The number of summary measures depends on the number of outcomes chosen.  If 2 outcomes, A and B, are considered the most important for evaluation, then  Cost/outcome A  Cost/outcome B  This makes translation for policy makers difficult!!

Example: CM Prevention Program Average CE Ratios for depressive symptoms * Compared to baseline ProgramBaselinescore Ending score Difference Program cost Avg. CE Ratio* 3-mo $984$246 9-mo $2,304$278

Final Comments Economic evaluation (EE) is valuable to decision making and for setting health policy. Economic evaluation (EE) is valuable to decision making and for setting health policy. For new researchers in PH, this is an important specialization to consider – because the demand for these skills is growing. For new researchers in PH, this is an important specialization to consider – because the demand for these skills is growing.

Coming soon….. Center for Economic Evaluation Institute for Behavioral Research and College of Public Health