Welcome to I-TECH HIV/AIDS Clinical Seminar Series An Introduction to Operations Research or How can I make my health program better? Dr. Mark Micek, MD, MPH December 4, 2008

The Issue It is difficult to… It is difficult to… …effectively deliver scientifically proven health interventions in the “real world” …translate research into health program settings Why? Why? Research generally occurs in controlled settings Research generally occurs in controlled settings Homogenous sample, controlled setting Homogenous sample, controlled setting Health programs exist in a complex setting Health programs exist in a complex setting Heterogeneous clients, multiple settings Heterogeneous clients, multiple settings Dependent on external context of care-delivery system Dependent on external context of care-delivery system i.e. policy, resource availability ($ and personnel), community perceptions i.e. policy, resource availability ($ and personnel), community perceptions

Example of difficulty translating proven treatment into practice HAART reduces mortality among patients with HIV HAART reduces mortality among patients with HIV BUT many eligible HIV-positive people don’t start HAART BUT many eligible HIV-positive people don’t start HAART

Example of difficulty translating proven treatment into practice Short-course ARV therapy can decrease mother-to-child HIV transmission Short-course ARV therapy can decrease mother-to-child HIV transmission BUT few HIV+ women receive treatment BUT few HIV+ women receive treatment Data from: Evaluation of United Nations-Supported Pilot Projects for the Prevention of Mother-to-Child Transmission of HIV: Overview of Findings. UNICEF, New York, 2003.

How can we improve the performance of our programs in an evidence- based way?  Use the principles of Operations Research

What is OR? Generic definition “Use of systematic research techniques for program decision-making to achieve a specified outcome.” “Use of systematic research techniques for program decision-making to achieve a specified outcome.” [Population Council, 2006] [Population Council, 2006] Goal = find a best possible solution to improve performance of the organization Goal = find a best possible solution to improve performance of the organization Early examples in military, business Early examples in military, business Use data, statistics, mathematical modeling Use data, statistics, mathematical modeling Goal (health care) = “to increase the efficiency, effectiveness, and quality of services delivered by providers, and the availability, accessibility, and acceptability of services desired by users” Goal (health care) = “to increase the efficiency, effectiveness, and quality of services delivered by providers, and the availability, accessibility, and acceptability of services desired by users” [Population Council, 2002] [Population Council, 2002]

3 Core Principles of OR 1. Study health programs 2. Actively try to make the program better 3. Use results to improve the program Corollary: requires collaboration between managers and researchers

Core principle of OR #1: Study health programs Health program is key to health care delivery Health program is key to health care delivery OR usually focuses on existing program OR usually focuses on existing program Research problem = program problem Research problem = program problem Research intervention = program solution Research intervention = program solution Feasible within context of entire system Feasible within context of entire system Assures problems and solutions are defined by realities of the health program system Assures problems and solutions are defined by realities of the health program system  Without involvement of a health program, it’s not OR

Health program as system Outputs Raw Materials: Finances Costs Staff Facilities Activities: Trainings Supervision Logistics Reporting Record keeping Patient flow strategies Products of program activities: # condoms distributed # people tested for HIV # enrolling for HIV care Effect on knowledge or behavior: # youth using condoms # women using pMTCT services # starting ART % adhering to ART OutcomesImpact Change in health/well- being HIV/STD incidence # deaths Health Program Under managers’ control InputsProcess Effects of Health Program

Health programs are complex systems Adhere to HAART Test for HIV Enroll in HIV clinic CD4 testing Start HAART (if eligible) Schedule appt with clinician Return for appt with clinician Time and drop-off Pt returns to get blood drawn Clinician orders CD4 Time and drop-off Return for appointment with clinician Time and drop-off Blood drawn Schedule appt for results Time and drop-off Enroll in HIV clinic

Core principle of OR #2: Actively try to make the program better Better “understanding” of situation is not enough Better “understanding” of situation is not enough Better can mean… Better can mean… Improve access to services Improve access to services Improve quality Improve quality Limit costs (find cost-effective strategies) Limit costs (find cost-effective strategies) Improve health Improve health  Without actively trying to improve the program, it’s not OR

Core principle of OR #3: Use results to improve program Using results can mean… Using results can mean… Implement new strategy on local / national scale Implement new strategy on local / national scale Influence national / international policy Influence national / international policy Dissemination of results, develop “best practices” Dissemination of results, develop “best practices”  OR successful only if results used to improve the program  Published papers are NOT a valid indicator of OR success

Corollary: OR requires collaboration between managers and researchers Program managers/policy-makers Program managers/policy-makers Should be involved in ALL ASPECTS of research process Should be involved in ALL ASPECTS of research process Understand health care system Understand health care system Help ensure problem is important, solution is feasible Help ensure problem is important, solution is feasible Help ensure results will be implemented Help ensure results will be implemented Researchers Researchers Understand of research methodology Understand of research methodology Responsible of recommending and implementing appropriate research techniques Responsible of recommending and implementing appropriate research techniques Can be the same person Can be the same person

Broad methodologies of OR Modeling (classic) Modeling (classic) Develop mathematical model to mimic health care system Develop mathematical model to mimic health care system Manipulate to find the best possible “solution” Manipulate to find the best possible “solution” Optimize efficiency Optimize efficiency Maximize Y given constraints X Maximize Y given constraints X Intervention-based (Population Council) Intervention-based (Population Council) Design/test best way to deliver services Design/test best way to deliver services Similar to quality improvement (IHI/WHO) Similar to quality improvement (IHI/WHO)

Intervention-based OR Population Council IHI Collaborative LinearCyclical 1. Identify program problem 1. Plan 2. Generate program solution 2. Do 3. Test program solution 3. Study 4. Use/disseminate results 4. Act ActPlan DoStudy

Steps in intervention-based OR 1. Identify program problem Usually determined in ongoing program Usually determined in ongoing program Routine data (M&E, surveillance) vs. program evaluation Routine data (M&E, surveillance) vs. program evaluation Under control of program manager Under control of program manager 2. Generate program solution Review workflow, talk to staff/clients Review workflow, talk to staff/clients Consider exploratory study if causes/solution unknown Consider exploratory study if causes/solution unknown Must be feasible Must be feasible 3. Test program solution Level of intervention: facility vs. individual Level of intervention: facility vs. individual Data measurement: routine vs. added procedures Data measurement: routine vs. added procedures Allocation: non-randomized (quasi-experimental) vs. randomized (experimental) Allocation: non-randomized (quasi-experimental) vs. randomized (experimental) 4. Use/disseminate results Continue/expand successful interventions Continue/expand successful interventions Influence national/international policy Influence national/international policy OR not typically “generalizable” but can be relevant for similar programs (“best practices”) OR not typically “generalizable” but can be relevant for similar programs (“best practices”)

Common OR study designs (Experimental/quasi-experimental) Pre-post control /non-equivalent control group Simple time-series (some control for time) Step-wedge time-series (better control for time)

Example 1: HIV testing in pMTCT program in Rwanda Program problem: 18% of women tested for HIV in a pMTCT program in Rwanda did not return for results 18% of women tested for HIV in a pMTCT program in Rwanda did not return for results System has 2-3 day delay in getting HIV test results System has 2-3 day delay in getting HIV test results  What is a potential solution?  How could we test this solution? Source: An Approach to Rapid Scale-up: Using HIV/AIDS Treatment and Care as an Example. WHO. Geneva, 2004.

Example 1: HIV testing in pMTCT program in Rwanda Problem: 2-3 day delay in getting HIV test results  18% did not return for results Potential solution: Same-day results Test of solution: Time series, dropout ~0 post-intervention Source: An Approach to Rapid Scale-up: Using HIV/AIDS Treatment and Care as an Example. WHO. Geneva, 2004.

Example 2: Low number of people starting ART per month

What is the program’s problem? What is the program’s problem? Do we need more information to determine what the cause of the problem is? Do we need more information to determine what the cause of the problem is? What is the program’s solution? What is the program’s solution? Do we know enough to develop a solution? Do we know enough to develop a solution? What should our next step be? What should our next step be?

Identify steps required to start ART

Using programmatic data: Where are patients lost? Step 1 Step 2 Step 3 Step 4

Using programmatic data: What are priorities to address?

Why do HAART-eligible patients not start ARVs (step 4)? Poor follow-up also reported as reason for not starting HAART in other studies Poor follow-up also reported as reason for not starting HAART in other studies Giordano TP et al, Factors Associated with the Use of Highly Active Antiretroviral Therapy in Patients Newly Entering Care in an Urban Clinic. JAIDS, 32: Giordano TP et al, Factors Associated with the Use of Highly Active Antiretroviral Therapy in Patients Newly Entering Care in an Urban Clinic. JAIDS, 32:

Improving rates of starting ARVs in HAART-eligible patients Reasons for poor follow-up Reasons for poor follow-up Pre-HAART procedure too cumbersome Pre-HAART procedure too cumbersome Dissatisfaction with services Dissatisfaction with services Trouble paying transportation costs Trouble paying transportation costs Poor understanding of clinic procedures Poor understanding of clinic procedures Stigma of going to HIV clinic Stigma of going to HIV clinic Death Death Potential solutions Change workflow around HAART-eligible patients Improve counseling Improve relationship between patients and health care workers Decentralize ARV services

Number of HIV+ pregnant women enrolled at ART site <30 days after HIV testing On-site ART vs. Off-site ART clinic: OR 7.2 (CI , p<0.001) BeiraChimoioNhamatandaCatandica Off-site ART clinicOn-site ART clinic Total HIV + Enrolled 34% 22%76% 74% 30%75%

ART-eligible starting ART (Total and <90 days), Sofala and Manica, % 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% Beira Nhamatanda Caia Ponta Gea Marromeu Chimoio Catandica Manica Espungabera Gondola 1o de Maio Chitobe ART Total ART <90days Vertical Integrated Total stated on ART: Vertical 50% vs. Integrated 65% (p<0.001) ART <90 days: Vertical 37% vs. Integrated 59% (p<0.001) N=9,193

Other examples of OR Program problemPotential OR studies (Pre-post control, time series, step-wedge) Poor adherence to HAARTStudy strategies to improve adherence (DOT, counseling, community-based treatment partners, food/transport subsidies) Low rates of HIV treatment among TB patients, inpatients Study integration of HIV services into TB programs or hospital services Underdiagnosis of TB among HIV+ patientsStudy strategies to increase TB screening/diagnosis (training/mentorship, standardized care algorithms, screen at VCT) Low rates of HIV treatment among pregnant women Study strategies to improve retention into care (decentralize HIV services, improve counseling, involvement of peer counselors) Poor provider adherence to HIV care protocolsStudy strategies to improve adherence to protocols (training/mentorship, performance reviews, involvement of lower-level HCWs) Low level of prevention behaviors among HIV+ people Study strategies to increase prevention (couples counseling/care, intensify prevention counseling in testing and care centers)

OR resources Designing HIV/AIDS Intervention Studies: An Operations Research Handbook. Andrew A. Fisher and James Foreit. The Population Council, New York, Available at: Designing HIV/AIDS Intervention Studies: An Operations Research Handbook. Andrew A. Fisher and James Foreit. The Population Council, New York, Available at: An Approach to Rapid Scale-up: Using HIV/AIDS Treatment and Care as an Example. World Health Organization, Geneva, Available at: An Approach to Rapid Scale-up: Using HIV/AIDS Treatment and Care as an Example. World Health Organization, Geneva, Available at: The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. Institute for Health Care Improvement. Cambridge, MA, Available at E A06D/0/BreakthroughSeriesWhitePaper2003.pdf The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. Institute for Health Care Improvement. Cambridge, MA, Available at E A06D/0/BreakthroughSeriesWhitePaper2003.pdf E A06D/0/BreakthroughSeriesWhitePaper2003.pdf E A06D/0/BreakthroughSeriesWhitePaper2003.pdf Population Council / Horizons program on HIV/AIDS OR: Population Council / Horizons program on HIV/AIDS OR:

Thank you! Next session: December 18 th, 2008 Listserv:

Welcome to I-TECH HIV/AIDS Clinical Seminar Series Next session: December 18 th, 2008 Robert Harrington Opportunistic Infections