Integrating Evaluation with Business Process Modeling for Increased Efficiency and Faster Results in HIV/AIDS Clinical Trials Research Jonathan M Kagan Division of Clinical Research National Institute of Allergy & Infectious Diseases Bethesda, Maryland, USA William MK Trochim Department of Policy Analysis & Management Cornell University Ithaca, New York, USA

Adults and children estimated to be living with HIV, 2007 The HIV/AIDS Problem Adults and children estimated to be living with HIV, 2007 Total: 33 million (30 – 36 million) Western & Central Europe 730 000 [580 000 – 1.0 million] Middle East & North Africa 380 000 [280 000 – 510 000] Sub-Saharan Africa 22.0 million [20.5 – 23.6 million] Eastern Europe & Central Asia 1.5 million [1.1 – 1.9 million] South & South-East Asia 4.2 million [3.5 – 5.3 million] Oceania 74 000 [66 000 – 93 000] North America 1.2 million [760 000 – 2.0 million] Latin America 1.7 million [1.5 – 2.1 million] East Asia 740 000 [480 000 – 1.1 million] Caribbean 230 000 [210 000 – 270 000] Total: 33 million (30 – 36 million)

Evaluation Project Goals Support the success of the clinical trials programs Provide empirically based evidence about process and outcomes to guide decision making and program improvement Ensure the highest scientific priorities are addressed Promote collaboration and shared learning Increase efficiency and research integration Develop a culture of ongoing evaluation Jonathan 5

Stakeholder-Identified Critical Success Factor Concept Map DAIDS Policies and Procedures Operations and Management Resource Utilization Community Involvement Scientific Agenda - setting Biomedical objectives Relevance to Participants 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91

Operations, Policies and Resources Focus Administrative polices, funder polices and procedures, process efficiency and site capacity Lead Evaluation Question How can the process of protocol development be improved to increase efficiency and shorten the timeline? How long does each phase of the protocol development process take? What are the limiting factors in the protocol development process? How can we shorten the timeline without compromising quality? How can the process be improved? What are reasonable targets for each phase? ü ü ü ü ü 7

DAIDS Adverse Experience Reporting System DAIDS-ES UpdateDAIDS-ES Update DAIDS-ES Update DAIDS-ES UpdateDAIDS-ES Update Monday, August 13, 2007 May 7, 2008, August 13, 2007 Monday, August 13, 2007Monday, August 13, 2007 September 24, 2007Monday, August 13, 2007Monday, August 13, 2007 DAIDS Enterprise Information System Desktops Wireless Web Services Protocol Registration Clinical Site Monitoring DAIDS Adverse Experience Reporting System IND Management Enterprise Foundation Person and Institution Registry DAIDS-ES Update to FHI ACTG & IMPAACT OPS and DMC Meeting ACTG & IMPAACT OPS and DMC MeetingACTG & IMPAACT OPS and DMC Meeting DAIDS-ES Update to MTNACTG & IMPAACT OPS and DMC MeetingACTG & IMPAACT OPS and DMC Meeting 8 8 8 8 8 8

DAIDS Harmonized Protocol Statuses In Development Pending Open to Accrual Enrolling Closed to Closed to Follow Up Withdrawn Participants Off Study & Primary Analysis Completed Concluded Archived Proposed

SRC Review Total Elapsed (Days1) Maximum1 60 Minimum1 1 Median1 27 Target2 35 Difference (Median-Target) 2 Std. Deviation 10.41 # of Reviews 106 Note: The numbers shown above the bar represents the total number of days for SRC Review Process (A+B) A= Days from Protocol Receipt to SRC Review B= Days from SRC Review to Consensus Distribution 1 Calendar days 2 Business days

Protocol Timeline Summary 30 Days 60 90 120 180 210 240 270 300 330 360 390 420 450 480 510 540 570 600 660 720 630 690 150 750 133 days 100 days Receipt to CSRC Review (Multiple) SRC Review Completion to RAB Sign Off Pending to Open to Accrual Receipt to Comments Distribution (single) 125 days 27 days 15 days 23 days 160 days Pending to v1.0 Site Registration (US Sites) 517 days Pending to v1.0 Site Registration (Non-US Sites) Open to Accrual to Enrolling Protocol Timeline Summary Receipt to Review (single) 233 days 358 days 381 days 780 Total 57 protocols (both CSRC and PSRC) 29 CSRC Protocols 28 PSRC Protocols

DAIDS Regulatory Review Process

What Good Are These Time-Based Measures? Time measures provide a coarse-grain look at processes – kind of a composite ‘smell test’ Sometimes it’s the process itself Other times it’s the resources/capacity that’s rate-limiting Time measures can help develop focus on where improvements could be sought Excessive time is a disincentive to clinical research We should evaluate quality/value but we may lack the ability or know-how – time studies can get us started

What Have We Learned and How Can We Put it to Use? Things that appear worth considering from these time-based protocol analyses: Shorten/simplify processes that consume time disproportionate to their ‘value’ Develop means of assuring performance accountability Set a ‘drop dead’ date for protocol initiation

Acknowledgements Evaluation planning and concept mapping Mary Kane, Concept Systems, Inc. Kathleen Quinlan, Concept Systems, Inc. Scott Rosas, Concept Systems, Inc. Protocol event analyses Suresh Varghese and Alex Varghese, Digital Infuzion, Inc. NIAID HIV/AIDS Networks Jeffrey Schouten, Network Coordinating Center, FHCRC Network Leadership, Operations Centers, Data Centers