Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archived OER Public Archive Home Page

Grant Formats and Review Mechanisms Innovative and Transformative Research Keith R. Yamamoto NIH Peer Review Advisory Committee August 28, 2006

The only possible source for adequate support of our medical schools and medical research is the taxing power of the Federal Government. …such a program must assure complete freedom for the institutions and the individual scientists in developing and conducting their research work. Surgeon General Thomas Parran December 1945

The only possible source for adequate support of our medical schools and medical research is the taxing power of the Federal Government. …such a program must assure complete freedom for the institutions and the individual scientists in developing and conducting their research work. Surgeon General Thomas Parran December 1945 Discovery and innovation are to some extent taking place in spite of, rather than because of, the current policies and practices of major biomedical funding agencies. Tom Cech Tom Cech

How NIH review mechanisms may impede discovery and innovation Study sections, panels of experts, carry out the review groups achieve consensus by adopting compromise positions acknowledged experts have a stake in defending the prevailing paradigm “They have the power to choke their challengers by cutting off their supply of funding.” -- Gerald Pollack, Univ. Washington NIH grants focus on research projects, not investigators places emphasis on experimental detail and preliminary data reviewers must predict likelihood of specific experiments to work reviewers must predict likelihood of specific experiments to work reduces consideration of past performance or level of promise of reduces consideration of past performance or level of promise of investigator investigator Numerical score is computed as average of all panelist scores primary and secondary reviewers have dominant influence on scoring primary and secondary reviewers have dominant influence on scoring scores cluster close to perceived payline; invites “blackball” scoring scores cluster close to perceived payline; invites “blackball” scoring highly sensitive to concern or criticism, making system failure-averse

The current NIH review and funding mechanisms thus appear to be normalized (pegged to prevailing paradigms) and encrypted (consensus work is proposed and funded, and innovative/transformative work is performed) The Challenge: Devise an NIH review and funding mechanism that fosters and nurtures the best research— research that achieves rapid and significant progress, and that creates quantum leaps in knowledge and understanding. [Note the scaling problem: 80,000 applications per year, 18,000 reviewers]

Two desirable types of research Innovative new originalinventivepioneering advances/shifts paradigm evolutionary Transformative revolutionary creates new field synthesizes new paradigm Proposal: Two-track system; investigator selects track

NIH Innovative Research Awards Very large program Project-focused, 7 page proposal, new format Explicit focus on innovation; de-emphasize preliminary results Increased focus on investigator Proposals are ranked, not scored (Ken Dill, Deep Innovation Grants for NIH) Proposal Statement of question/issue to be approached (w/ Background): 1 page Specific aims: 1 page Impact of achieving aims: 0.5 page Approaches (w/ progress and preliminary results): 4 pages Innovation of concept or approach: 0.5 page Biosketch Publications, up to: 5 most relevant; 5 most significant; 5 most recent Qualifications of investigator for this proposal: 0.5 page Reviews 5-6 per proposal; address criteria; 1 page maximum

NIH Transformative Research Awards Very small program; outgrowth of Pioneer Program (NDPA) Very small program; outgrowth of Pioneer Program (NDPA) Investigator-focused, 3-5 page essay Investigator-focused, 3-5 page essay Explicit focus on revolutionary concepts and approaches Explicit focus on revolutionary concepts and approaches Evidence that investigator is explorer, discoverer Evidence that investigator is explorer, discoverer Candidates are ranked, independently, by highly selected Candidates are ranked, independently, by highly selected generalists (paradigm standard-bearers are excluded; Gerald Pollack, Challenges to Existing Scientific Orthodoxies) Finalists are interviewed to select awardees Finalists are interviewed to select awardees Highly prestigious; well-supported at appropriate level Highly prestigious; well-supported at appropriate level Ten year awards; few reporting requirements Ten year awards; few reporting requirements NDPA applications: self-limiting to reasonable numbers 2004: 1400 applications, 9 awards 2005: 840 applications, 13 awards, 6 to asst or assoc profs 2006: 470 applications, 7-10 awards expected

Assessing outcomes: Transformative Research Awards Control groups: Control groups: 1. Interviewees not awarded 2. Applicants highly rated but not interviewed Track 10 years after award Track 10 years after award Criteria: Criteria: not limited to conventional standards not limited to essay topic “revolutionary” impact of contributions

Wake up call for innovative and transformative research Discovery and innovation are to some extent taking place in spite of, rather than because of, the current policies and practices of major biomedical funding agencies. Tom Cech HHMI HHMI One does not propose transformative research to NSF or NIH. Such proposals are usually dead on arrival. Erich Jarvis Erich Jarvis Northwestern Univ. Northwestern Univ. We can and should develop mechanisms that actively nurture innovative and transformative research.