Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archivedOER Public Archive Home Page

Center for Scientific Review National Institutes of Health Department of Health and Human Services Toni Scarpa Peer Review in a Rapidly Changing Biomedical Environment Peer Review Advisory Committee Meeting

July 1, 2005 My First Day at Work

Applications received for all of NIH and applications referred for CSR review, FY ,000 40,000 60,000 80, Fiscal year Number of applications Applications received for all of NIHApplications assigned for review by CSR

CSR Mission Statement To see that NIH grant applications receive fair, independent, expert, and timely reviews -- free from inappropriate influences -- so NIH can fund the most promising research.

Time Complexity and Impact CSR Operations Current Systems New Systems? Possible Changes in CSR Peer Review Operations

Increase communications between CSR, the ICs, our reviewers and applicants Increase uniformity Possible Changes in CSR Operations

Consistency and Transparency of CSR Operations Current Efforts Surveyed SRAs Standardized Resumes Developed Uniform Policies for-- –Posting critiques in IAR before meetings –Posting summary statements –Streamlining 50 percent of the applications

Increased CSR Communications Current Efforts Internal CSR/NIH Communications Met with IC Directors and program staff Integrated IC program staff into CSR committees Established an open-door policy for staff External Communications Developed NIH Review Connect ListSERV Expanded the CSR Peer Review Notes Newsletter

Increase communications between CSR, the ICs, our reviewers and applicants Increase uniformity Increase efficiency Possible Changes in CSR Operations

Potential of Knowledge Management Tools for Peer Review Collexis Software Knowledge management solutions Fingerprinting and text retrieving Disease coding Benefits for Peer Review Assigning applications to Integrated Review Groups Selecting reviewers (one application, multiple applications) Four pilots are underway to begin to assess these benefits Implementation will give NIH the ability to assess the research content of all unfunded applications

Increase communications between CSR, the ICs, our reviewers and applicants Increase uniformity Increase efficiency Facilitate work of IC program staff Possible Changes in CSR Operations

New Secure Speakerphones to Give NIH Program Officers Greater Access to CSR Review Meetings VTX 1000 Sound Station from Polycom

Possible Changes in Current Systems Shorten the review cycle

NIH Committee to Shorten the Review Cycle: Current Efforts Trans-NIH Staff Olivia Bartlett, Ph.D. National Cancer Institute Bettie J. Graham, Ph.D. National Human Genome Research Institute Marcia Hahn Office of Extramural Research Robert Hammond, Ph.D. Ex National Institute of Diabetes & Digestive & Kidney Diseases Jack Jones, Ph.D. Center for Information Technology John J. McGowan, Ph.D. National Institute of Allergy and Infectious Diseases Belinda Seto, Ph.D. National Institute of Biomedical Imaging and Bioengineering Sophonia Simms, M.S. Office of Extramural Research CSR Staff Eileen Bradley, D.Sc. (Chairperson) Surgical Sciences, Biomedical Imaging and Bioengineering Integrated Review Group Suzanne Fisher, Ph.D. Division of Receipt and Referral Richard Panniers, Ph.D. Genes, Genomes and Genetics Integrated Review Group Donald L. Schneider, Ph.D. Division of Molecular and Cellular Mechanisms Thomas Tatham, Ph.D. Small Business Developmental Disabilities, Communication, and Science Education Panel

This is Not a Ford Assembly Line ReceiptRefer Evaluate Scientific Merit of Applications

Possible Changes in Current Systems Shorten the review cycle Address concern that clinical research is not properly evaluated

NMedian HS HS R01s Reviewed by ICs NMedian HS+1, HS-3, R01s Reviewed by CSR Review Outcomes for HS+ vs. HS- R01 Applications Reviewed by Panels Organized by CSR or Other NIH ICs October 2002 Council Data

Possible Changes in Current Systems Shorten the review cycle Address concern that clinical research is not properly evaluated Improve the assessment of innovative, high- risk/high-reward research

“The judging of grants has become a charade.” The American Society for Cell Biology “The judging of grants has become a charade. To be funded, the experimental plan has become a litany of experiments already accomplished so that everything is feasible. When grants come back with unfundable scores, new investigators may not have sufficient resources to do the experiments that “show feasibility.” Zena Werb President, ASCB Newsletter August 2005

Possible Changes in Current Systems Shorten the review cycle Address concern that clinical research is not properly evaluated Improve the assessment of innovative, high- risk/high-reward research Do more to recruit and retain more high quality reviewers

Distribution of Reviewer Academic Titles in May 1998 and May 2005 Reviewers in Chartered CSR Study Section Meetings 0% 10% 20% 30% 40% 50% 60% 70% ProfessorAssociate ProfessorAssistant Professor Academic titles Percent of reviewers May 1998May 2005

Expanding Peer Review’s Platforms Electronic Reviews Telephone Enhanced Discussions Video Enhanced Discussions Asynchronous Electronic Discussions Study Sections Necessity ● Clinical reviewer (especially inter- ventional medicine) Preference ● Physicists, computational biochemists New Opportunities ● Fogarty, International Reviewers

If we didn’t have any peer-review system and we had to design one from scratch, what would it look like? Possible New Systems

This is CSR

The Scientific Review Administrators Our Unsung Heroes They give to peer review “a special, precious character. Their contributions tend to be overshadowed by the forces and discoveries they sustained. Yet they are the pre-eminent pillars of the civilization that strives to leave a legacy of knowledge through an endless process of discovery and utilization of knowledge.” Donald S. Fredrickson NIH Director,