Purpose of Network Evaluation Increase understanding of the relationship between the network design, objectives and functions and the outcomes achieved Provide data to examine trends, identify strengths and weaknesses and assist in leadership decisions

Standing HPTN committees Ongoing, internal peer review Protocol Review Committee Study Monitoring Committee Manuscript Review Committee

Network Evaluation Approach in 2000 Qualitative rather than quantitative Why?  Breadth of science  Initial focus on infrastructure development Comprehensive evaluation  Site and protocol performance  Working group functioning  Central resources: CL, CORE, SDMC  Standing committees

2001 Evaluation Report Survey results  Science working groups  Central resources  Standing committees Quantitative data  Protocol implementation  Site performance

HPTN Evaluation - How did we measure up?

Slow overall progress – complex reasons The large number of contributors (NIAID, FDA, CABS) that are an intrinsic part of a network has a direct relationship to protocol progress. Many steps in review process – most seem to be unavoidable Protocols – Development and Progress

Monitor protocol progress and take corrective action when needed to move slow developers along. Shorten review process whenever possible; clear communication of the need for each review step; combine steps whenever possible. Define realistic expectations for fielding a protocol. Protocols – Development and Progress - Suggested Corrective Measures

CWG – Survey Comments Concern that CWG was not properly integrated into the HPTN research agenda CWG/Network leadership not proactive enough in integrating the community into network activities CWG should develop a clearer mission for the group and a master plan for achieving mission goals Need clear definition of the roles of the CWG and CORE community program staff

Central Laboratory – Survey Comments  CL contributions to protocols were applauded  Site laboratory training – Good quality, more quantity More and earlier involvement More site visits by CL Work on timing of training to coincide more closely with study initiation.

CORE – Survey Comments  Protocol meetings and trainings are well managed  Receptive staff members More involvement by CORE staff More site visits Closer monitoring More frequent and focused trainings More vocal involvement in protocol development

CORE – Survey Comments (cont.) More scientific/technical training for CORE staff More technical information on the web site and in the newsletter Better definition of the roles of the CORE’s community program and the CWG

SDMC – Survey Comments  “Knowledgeable,” “Professional”  Accurate and prompt More flexibility in approach to protocol development Increase number of site and training visits to HPTUs Increase interface with site staff in dealing with data management issues

Standing Committees – Survey Comments SMC  General satisfaction with decisions Earlier involvement in protocols during implementation and site visits PRC  Reviews were timely, thorough and helpful Concern that expertise of some reviewers may not be adequate, suggest use of additional outside reviewers

Standing Committees – Survey Comments (cont.) MRC Improve response time to authors Reviews need to be helpful, provide guidance for improvement Provide written comments with review decision

Prevention Leadership Group – Survey Comments  Conflict resolution is handled well by the PLG  Good decisions on making the best use of available resources Clarify and communicate the role of PLG to the Network Need to focus on tough issues and make “tough” decisions

Summary and Conclusion Key Evaluation Results to Consider Increase communications to entire network  Clarify the review process from concept to study initiation  Clarify leadership roles and activities Increase network partner involvement with HPTUs  More visits by CORE, SDMC and CL to improve HPTU performance Increased focus on ethics and community issues in regard to conducting international research