1 Demystifying Federal Grant Review Process

2 The Panel Margarita Alegria, PhD Margarita Alegria, PhD Francis Chesley, MD Francis Chesley, MD Willard Manning, PhD Willard Manning, PhD Ming Tai-Seale, PhD Ming Tai-Seale, PhD

3 The Agenda Grants for young investigators Grants for young investigators Review criteria and reviewer expectations Review criteria and reviewer expectations How to write a proposal How to write a proposal Responding to Study Section’s Concerns Responding to Study Section’s Concerns Views from AHRQ Views from AHRQ Questions and Answers Questions and Answers

4 Research Grants for Young Investigators Small grant Small grant R03: up to 2 years R03: up to 2 years total direct cost $50,000/yr total direct cost $50,000/yr Exploratory/developmental grant Exploratory/developmental grant R21: 2 years R21: 2 years total DC $275,000 total DC $275,000 up to $200,000/yr up to $200,000/yr R34: 3 years R34: 3 years total DC $450,000 total DC $450,000 up to $225,000/yr up to $225,000/yr

5 Career Development Grants for Young Investigators R34 dissertation grant R34 dissertation grant F31 minority pre-doc F31 minority pre-doc F32 post-doc F32 post-doc K01 Mentored career development award K01 Mentored career development award K02 Independent scientist career development award K02 Independent scientist career development award K08 Mentored clinical scientist development award K08 Mentored clinical scientist development award …

6

7 Responding to Study Section's Concerns Willard Manning Harris School of Public Policy Studies University of Chicago

8 Responding to Study Section's Concerns Expect a second (or third) submission of your proposal Expect a second (or third) submission of your proposal Very few proposals funded first time unless one- shot announcement Very few proposals funded first time unless one- shot announcement Prepare for a critical review Prepare for a critical review Reviewers doing the review "cold" Reviewers doing the review "cold" Reviewers work under pressure of short deadline while still teaching, seeing patients, etc. Reviewers work under pressure of short deadline while still teaching, seeing patients, etc. Reviewers do not have benefit of lengthy discussions with research team Reviewers do not have benefit of lengthy discussions with research team Reviewers do not have time to read the proposal over and over again Reviewers do not have time to read the proposal over and over again

9 Facts about Review and Reviewers Reviewers may not be from same discipline or specialty Reviewers may not be from same discipline or specialty Check the section roster, then PubMed/Google Check the section roster, then PubMed/Google Many reviewers will have trouble with jargon Many reviewers will have trouble with jargon Your technical terminology is unintelligable jargon to me unless explained Your technical terminology is unintelligable jargon to me unless explained "Collective efficacy" means what? "Collective efficacy" means what? "Diff-n-diff" means what? "Diff-n-diff" means what? HSR is multidisciplinary HSR is multidisciplinary Unless the study section has many from your discipline, you must talk to a wider audience Unless the study section has many from your discipline, you must talk to a wider audience

10 Facts about Review & Reviewers (cont'd) Basic Reviewing Principle Basic Reviewing Principle Burden of proof of idea and approach is on the proposer Burden of proof of idea and approach is on the proposer If they cannot find what they need easily, they often will treat it as missing or inadequately described If they cannot find what they need easily, they often will treat it as missing or inadequately described Reviewer's language is often very frank! Reviewer's language is often very frank! But remember reviewers are: But remember reviewers are: Looking for problems Looking for problems Trying to help, esp. if they see some merit in proposal Trying to help, esp. if they see some merit in proposal

11 Initial Response to “Pink” Sheets Sulking is normal Sulking is normal Do not take it personally Do not take it personally Get advice from agency staff ASAP Get advice from agency staff ASAP GET SENIOR HELP TO INTERPRET COMMENTS GET SENIOR HELP TO INTERPRET COMMENTS Reserve a "cold reviewer" to react to draft resubmission Reserve a "cold reviewer" to react to draft resubmission Preferably with study section or area experience Preferably with study section or area experience Plan to resubmit unless "fatally flawed" Plan to resubmit unless "fatally flawed"

12 Responding to "Pink" Sheets Leave plenty of time to Leave plenty of time to Overhaul in response to study section and agency staff comments Overhaul in response to study section and agency staff comments Solicit reaction to revised submission from cold reviewer, preferably with study section or area experience Solicit reaction to revised submission from cold reviewer, preferably with study section or area experience Respond accordingly Respond accordingly Do not rush to resubmit (July 1st after receiving June 6th) Do not rush to resubmit (July 1st after receiving June 6th)

13 Revising the Proposal Outrage is OK for a personal reaction but never in a response! Outrage is OK for a personal reaction but never in a response! Take the feedback seriously as indicative of Take the feedback seriously as indicative of Gaps in exposition or logic Gaps in exposition or logic Overly terse in key areas Overly terse in key areas Organizational issues Organizational issues Identify common themes across reviewers Identify common themes across reviewers Respond to all concerns in "Response" as well as in text Respond to all concerns in "Response" as well as in text Thank the reviewers for their valuable comments Thank the reviewers for their valuable comments Apologize for inadequacy of … Apologize for inadequacy of …

14 Revising the Proposal (cont'd) Even if you are "right," clean up the exposition to make the logic more transparent Even if you are "right," clean up the exposition to make the logic more transparent Revise the whole proposal if needed Revise the whole proposal if needed Ask research team, senior colleague, cold reviewer for reaction Ask research team, senior colleague, cold reviewer for reaction Revise again Revise again For substance For substance For ease of reviewers to evaluate For ease of reviewers to evaluate

15 Common problems Specific Aims Specific Aims Not specific Not specific Largely rhetorical Largely rhetorical Process aims rather than hypothesis-driven Process aims rather than hypothesis-driven Background Background Has literature but no synthesis Has literature but no synthesis Conceptual framework unclear Conceptual framework unclear Value-added unclear Value-added unclear Preliminary studies section lacks Preliminary studies section lacks Pilot data Pilot data Exploratory analysis Exploratory analysis Experience with these or similar data or methods Experience with these or similar data or methods

16 Common problems (cont'd) Research design and methods lack Research design and methods lack Conceptual framework -- A flow chart is useful but inadequate Conceptual framework -- A flow chart is useful but inadequate Alternative approaches not considered Alternative approaches not considered Weak or no link to hypotheses Weak or no link to hypotheses Research design flaws or weak design Research design flaws or weak design Failure to address reliability and validity of data Failure to address reliability and validity of data Questions of internal or external validity Questions of internal or external validity Wrong sample for research question, esp. if using a convenient sample you happen to have Wrong sample for research question, esp. if using a convenient sample you happen to have

17 Common problems (cont'd) Not feasible as constituted Not feasible as constituted Budget too large or too small Budget too large or too small Budget lacks adequate justification for Budget lacks adequate justification for Length of grant Length of grant Level of effort Level of effort Scope too broad Scope too broad Research team lacks appropriate qualifications Research team lacks appropriate qualifications Substantial data analysis but no statistician/econometrician Substantial data analysis but no statistician/econometrician Economic analysis but no economist Economic analysis but no economist Large clinical component but no clinician Large clinical component but no clinician

18 Common problems (cont'd) PI is too junior, not enough senior involvement PI is too junior, not enough senior involvement Project full of 5 percenters Project full of 5 percenters Lacks sense of own limitations Lacks sense of own limitations Errors or omissions in human subjects, minority, gender areas Errors or omissions in human subjects, minority, gender areas

19

20 Questions & Answers Panel

21 Resources R34 PA: files/PAR html R34 PA: files/PAR htmlhttp://grants.nih.gov/grants/guide/pa- files/PAR htmlhttp://grants.nih.gov/grants/guide/pa- files/PAR html R21 PA: files/PA html R21 PA: files/PA htmlhttp://grants.nih.gov/grants/guide/pa- files/PA htmlhttp://grants.nih.gov/grants/guide/pa- files/PA html R03 PA: R03 PA:

22