1. Integrating Evaluation with Business Process Modeling for Increased Efficiency and Faster Results in HIV/AIDS Clinical Trials Research
Jonathan M Kagan
Division of Clinical Research
National Institute of Allergy & Infectious Diseases
Bethesda, Maryland, USA
William MK Trochim
Department of Policy Analysis & Management
Cornell University
Ithaca, New York, USA

2. Adults and children estimated to be living with HIV, 2007
The HIV/AIDS Problem
Adults and children estimated to be living with HIV, 2007
Total: 33 million (30 – 36 million)
Western &
Central Europe
[ – 1.0 million]
Middle East & North Africa
[ – ]
Sub-Saharan Africa
22.0 million
[20.5 – 23.6 million]
Eastern Europe & Central Asia
1.5 million
[1.1 – 1.9 million]
South & South-East Asia
4.2 million
[3.5 – 5.3 million]
Oceania
74 000
[ – ]
North America
1.2 million
[ – 2.0 million]
Latin America
1.7 million
[1.5 – 2.1 million]
East Asia
[ – 1.1 million]
Caribbean
[ – ]
Total: 33 million (30 – 36 million)

5. Evaluation Project Goals
Support the success of the clinical trials programs
Provide empirically based evidence about process and outcomes to guide decision making and program improvement
Ensure the highest scientific priorities are addressed
Promote collaboration and shared learning
Increase efficiency and research integration
Develop a culture of ongoing evaluation
Jonathan 5

6. Stakeholder-Identified Critical Success Factor Concept Map
DAIDS Policies and Procedures
Operations and
Management
Resource
Utilization
Community Involvement
Scientific
Agenda -
setting
Biomedical objectives
Relevance to Participants 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91

7. Operations, Policies and Resources
Focus
Administrative polices, funder polices and procedures, process efficiency and site capacity
Lead Evaluation Question
How can the process of protocol development be improved to increase efficiency and shorten the timeline?
How long does each phase of the protocol development process take?
What are the limiting factors in the protocol development process?
How can we shorten the timeline without compromising quality?
How can the process be improved?
What are reasonable targets for each phase? ü ü ü ü ü 7

8. DAIDS Adverse Experience Reporting System
DAIDS-ES UpdateDAIDS-ES Update
DAIDS-ES Update
DAIDS-ES UpdateDAIDS-ES Update
Monday, August 13, 2007
May 7, 2008, August 13, 2007
Monday, August 13, 2007Monday, August 13, 2007
September 24, 2007Monday, August 13, 2007Monday, August 13, 2007
DAIDS Enterprise Information System
Desktops
Wireless
Web Services
Protocol
Registration
Clinical
Site Monitoring
DAIDS Adverse Experience Reporting System
IND
Management
Enterprise Foundation
Person and Institution Registry
DAIDS-ES Update to FHI
ACTG & IMPAACT OPS and DMC Meeting
ACTG & IMPAACT OPS and DMC MeetingACTG & IMPAACT OPS and DMC Meeting
DAIDS-ES Update to MTNACTG & IMPAACT OPS and DMC MeetingACTG & IMPAACT OPS and DMC Meeting 8 8 8 8 8 8

9. DAIDS Harmonized Protocol StatusesIn
Development
Pending
Open to
Accrual
Enrolling
Closed to
Closed to
Follow Up
Withdrawn
Participants Off Study
& Primary Analysis
Completed
Concluded
Archived
Proposed

10. SRC Review Total Elapsed (Days1)
Maximum1 60
Minimum1 1
Median1 27
Target2 35
Difference (Median-Target) 2
Std. Deviation
10.41
# of Reviews
106
Note: The numbers shown above the bar represents the total number of days for SRC Review Process (A+B)
A= Days from Protocol Receipt to SRC Review
B= Days from SRC Review to Consensus Distribution
1 Calendar days 2 Business days

11. Protocol Timeline Summary30
Days 60 90
120
180
210
240
270
300
330
360
390
420
450
480
510
540
570
600
660
720
630
690
150
750
133 days
100 days
Receipt to CSRC Review (Multiple)
SRC Review Completion to RAB Sign Off
Pending to Open to Accrual
Receipt to Comments Distribution (single)
125 days
27 days
15 days
23 days
160 days
Pending to v1.0 Site Registration (US Sites)
517 days
Pending to v1.0 Site Registration (Non-US Sites)
Open to Accrual to Enrolling
Protocol Timeline Summary
Receipt to Review (single)
233 days
358 days
381 days
780
Total 57 protocols (both CSRC and PSRC)
29 CSRC Protocols
28 PSRC Protocols

12. DAIDS Regulatory Review Process

13. What Good Are These Time-Based Measures?
Time measures provide a coarse-grain look at processes – kind of a composite ‘smell test’
Sometimes it’s the process itself
Other times it’s the resources/capacity that’s rate-limiting
Time measures can help develop focus on where improvements could be sought
Excessive time is a disincentive to clinical research
We should evaluate quality/value but we may lack the ability or know-how – time studies can get us started

14. What Have We Learned and
How Can We Put it to Use?
Things that appear worth considering from these time-based protocol analyses:
Shorten/simplify processes that consume time disproportionate to their ‘value’
Develop means of assuring performance accountability
Set a ‘drop dead’ date for protocol initiation

15. Acknowledgements Evaluation planning and concept mapping
Mary Kane, Concept Systems, Inc.
Kathleen Quinlan, Concept Systems, Inc.
Scott Rosas, Concept Systems, Inc.
Protocol event analyses
Suresh Varghese and Alex Varghese, Digital Infuzion, Inc.
NIAID HIV/AIDS Networks
Jeffrey Schouten, Network Coordinating Center, FHCRC
Network Leadership, Operations Centers, Data Centers