1. Peer Review of NIH Research Grant Applications Center for Scientific Review National Institute of Diabetes and Digestive and Kidney Diseases National Institutes of Health

2. U. S. Department of Health and Human Services Administration for Children and Families (ACF) Administration for Children and Families (ACF) Food and Drug Administration (FDA) Food and Drug Administration (FDA) Health Resources and Services Administration (HRSA) Health Resources and Services Administration (HRSA) The Secretary Deputy Secretary The Secretary Deputy Secretary Administration on Aging (AoA) Administration on Aging (AoA) Health Care Financing Administration (HCFA) Health Care Financing Administration (HCFA) Indian Health Services (IHS) Indian Health Services (IHS) National Institutes of Health (NIH) National Institutes of Health (NIH) Centers for Disease Control and Prevention (CDC) Centers for Disease Control and Prevention (CDC) Substance Abuse and Mental Health Services Administration (SAMHSA) Substance Abuse and Mental Health Services Administration (SAMHSA) Program Support Center (PSC) Program Support Center (PSC) Agency for Toxic Substances and Disease Registry (ATSDR) Agency for Toxic Substances and Disease Registry (ATSDR) Agency for Health Care Policy and Research (AHCPR) Agency for Health Care Policy and Research (AHCPR)

3. National Institutes of Health Much of the biomedical research in the united states is supported by the federal government, primarily the National Institutes of Health (NIH)

4. Office of the Director National Institute on Alcohol Abuse and Alcoholism National Institute on Alcohol Abuse and Alcoholism National Institute of Arthritis and Musculoskeletal and Skin Diseases National Institute of Arthritis and Musculoskeletal and Skin Diseases National Cancer Institute National Cancer Institute National Institute of Diabetes and Digestive and Kidney Diseases National Institute of Diabetes and Digestive and Kidney Diseases National Institute of Dental and Craniofacial Research National Institute of Dental and Craniofacial Research National Institute on Drug Abuse National Institute on Drug Abuse National Institute of Environmental Health Sciences National Institute of Environmental Health Sciences National Institute on Aging National Institute on Aging National Institute of Child Health and Human Development National Institute of Child Health and Human Development National Institute on Deafness and Other Communication Disorders National Institute on Deafness and Other Communication Disorders National Eye Institute National Eye Institute National Human Genome Research Institute National Human Genome Research Institute National Heart, Lung, and Blood Institute National Heart, Lung, and Blood Institute National Institute of Mental Health National Institute of Mental Health National Institute of Neurological Disorders and Stroke National Institute of Neurological Disorders and Stroke National Institute of General Medical Sciences National Institute of General Medical Sciences National Institute of Nursing Research National Institute of Nursing Research National Library of Medicine National Library of Medicine Center for Information Technology Center for Information Technology Center for Scientific Review Center for Scientific Review National Center for Complementary and Alternative Medicine National Center for Complementary and Alternative Medicine National Institute of Allergy and Infectious Diseases National Institute of Allergy and Infectious Diseases Fogarty International Center Fogarty International Center National Center for Research Resources National Center for Research Resources National Institutes of Health Clinical Center Clinical Center National Center on Minority Health and Health Disparities National Center on Minority Health and Health Disparities National Institute of Biomedical Imaging and Bioengineering National Institute of Biomedical Imaging and Bioengineering

5. A Typical Institute/center Office of the IC DirectorNationalAdvisoryCouncil Board of ScientificCounselors Extramural ScientificPrograms Grants Contracts Intramural LaboratoryStudiesClinicalStudies

6. NIH Extramural Program Mission Identify scientific opportunities Foster the best science Ensure proper stewardship Promote effective communication Exemplify and promote the highest level of scientific integrity, public accountability, and responsibility in the conduct of science Promote effective ways to communicate scientific information to scientists, health practitioners, and the public Manage a portfolio of investments to improve health through science

7. NIH Extramural Program Grantpatron (Assistance, encouragement) Cooperativepartner Agreement(assistance but substantial Program involvement) Contractpurchaser (Procurement)

8. Overall Peer Review Process

9. Dual Review System for Grant Applications Second Level of Review Council  Assesses Quality of SRG  Review of Grant Applications  Makes Recommendation to  Institute Staff on Funding  Evaluates Program Priorities  and Relevance  Advises on Policy First Level of Review Scientific Review Group (SRG)  Provides Initial Scientific Merit  Review of Grant Applications  Rates Applications and Makes Recommendations for Appropriate Level of Support and Duration of Award

10. Review Process for a Research Grant Research Grant Application School or Other Research Center National Institutes of Health Center for Scientific Review Initiates Research Idea Conducts Research Submits Application Allocates Funds Assigns to IRG/Study Section & IC Evaluates for Scientific Merit Evaluates for Program Relevance Advisory Councils and Boards Institute Director Recommends Action Takes final action for NIH Director Institute Study Section

11. Overall Timeframe From Submission to Award There are three overlapping cycles per year : JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY JUN JUL ReviewCouncil Cycle 1 Receipt Referral Award Review Council Cycle 3 Receipt Referral Award Review Council Cycle 2 Receipt Award Referral

12. Typical Timeline for a New Individual Research Project Grant Application (R01) –Submit in February (June, October) –Review in June (October, February) –Council in September (January, May) –Earliest award in December (April, July) Cycle 1---- Cycle 2---- Cycle 3---- There are three overlapping cycles per year:

13. GROUPS CSR IRGs Study Sections Special Emphasis Panels INSTITUTES INSTITUTES Scientific Review Groups Contract Review Committees APPLICATIONS REVIEWED Research Projects Academic Research Academic Research Enhancement Awards Postdoctoral Fellowships Small Business Innovation Research Shared Instrumentation Program Projects Centers Institutional Training Grants Conference Grants Career Awards Types of Scientific Review Groups Where Are Applications Reviewed? Small Grants RFAs (R01, R21, R03, U01) Contracts

14. Peer Review of NIH Support Mechanisms Research project grant (R01) Program Project grant (P01) Postdoctoral fellowship (F32) Center grant (P30, P50, P60) Senior fellowship (F32) Institutional fellowship (T32) Fogarty international center Academic career award (K07) Fellowship (F05, F06) Mentored clinical scientist Short-term training (T35) development award (K08) Small business grants (R41, R42 Conference grant (R13)* R43, R44) MARC fellowships (F34, F36, T34) Academic Research Enhancement Minority Biomedical Support Award (R15) Grant (S06) Biomedical Research Support Resource grant (P40, P41, R24, Shared Instrumentation R26, R28) Grant (S10) Contract CSR Institutes *Reviewed by CSR upon request

15. Center for Scientific Review (CSR) Focal point for initial review at NIH Central receipt point for PHS applications Referral to Institutes and to IRGs and Study Sections Review of most research and research training applications..for scientific merit

16. Grant Application Receipt and Assignment

17. Applications Submitted to NIH Approximately 46,000 grant applications are submitted to NIH each year, of which 25-30% are funded Competing grant applications are received for three review cycles per year

18. CSR Receipt and Referral: Central Receipt Point for Applications Submitted to the Public Health Service National Institutes of Health Substance Abuse and Mental Health Services Administration CSR Receipt & Referral Centers for Disease Control Food & Drug Agency for Health Care Policy & Research Office of Assistant Secretary for Health

19. Competing Applications Reviewed By: NOTE: Starting in FY93, NIDA, NIAAA, and NIMH are included in NIH totals

20. Receipt Dates Jan, May, Sept 10: institutional training grant applications Jan, May, Sept 25: academic research enhancement award Feb, June, Oct 1: new research applications Mar, Jul, Nov 1: revised, competing continuations, and Depend on the Type of Application Supplemental Applications and Small Business Innovation Research Applications

21. Flow of a Competing Grant Application Through the CSR Referral Section Application (Original & 5 Copies) Mailroom ( Receive, Date Stamp, Preliminary Sort, Add Bar Code) Project Control Unit: Receipt & Record Group (Process, Enter Skeletal Data into Computer, Print Status) Assignment Unit: (Assign to Scientific Review Group and Awarding Organization) (Advance Copy) Scientific Review Group (2 Copies) Scientific Review Group Project Control Unit: Review & Control Group (Add Labels, Send Data to IMPAC, Print Mailers, Distribute) (Original) Print Shop (1 copy) OPERA EIS,DMC (1 copy) Project Control Files (35 copies) SRG (20 copies) Original Awarding Organization (1 copy) Project Control (1 copy) Research Documents (IC only)

22. Applications are Assigned by Referral Officers: Professional scientists, most of whom also serve as scientific review administrators of CSR study sections

23. Applications Are Assigned To: Scientific review groups based on: – Specific review guidelines for each scientific review group Institutes based on: – Overall mission of the Institute – Specific programmatic mandates and interests of the Institute

24. Assignment to Institutes Applications are referred to an institute or center as the potential funding component: –This assignment is based on a match between the research proposed and the overall mission of the institute or center –Where applications are appropriate for more than one institute or center, multiple assignments are made

25. Sample Application Number Individual serial amended Research number Grant 1 R01 DK1234501A1 New national grant Application cancer support Institute year

26. Assignment of Applications Form Approved Through 9-30-97 OMB No. 0925 0001 AA Department of Health and Human Services Public Health Service GRANT APPLICATION Follow instructions carefully. Do not exceed character length restrictions indicated on sample. 1. Title of Project (Do not exceed 56 characters, including spaces and punctuation.) Planning Grant for Diabetes & Sc Ed in Tribal Schools 2. Response to Specific Request for Applications or Program Announcement Number: DK 01-033 Title: Planning Grant for Diabetes and Science Education … X x No Yes (if “Yes” state number and title) 3. Principal Investigator/Program Director 3 a. Name (Last, first, middle) 3 b. Degree(s) 3 c. Social Security No. Smith, Brian E. Ph.D. Leave Blank--for PHS Use Only Type 1 Activity R21 Number A154321-01 Review Group ZDK1 Formerly Council/Board (Month,Year) Date Received 09/01 07-26-01 Review GroupInstitute

27. Assignment Notification Letter  Dear Dr. Smith:  Your grant application entitled “PLANNING GRANT FOR DIABETES AND SCIENCE EDUCATION IN TRIBAL SCHOOLS ” has been received by the National Institutes of Health and assigned to a scientific review group (SRG) for scientific merit evaluation and to an institute/center for funding consideration. Specific information about your assignment is given below. The initial peer review should be completed by AUGUST, 2001, and a funding decision made shortly after the appropriate National Advisory Group meets in SEPTEMBER, 2001. Questions about the assignment should be directed to the scientific review administrator (SRA) or the Division of Receipt and Referral, Center for Scientific Review (formerly Division of Research Grants) at (301) 435-0715. Other questions prior to review should be directed to the scientific review administrator and questions after the review to the program staff in the institute/center.

28. Assignment Notification Letter (continued)  Assignment Number: 1 R01 DK12345 - 01  Dual Assignment: NS  Scientific Review Group:  ZDK1 GRB-5 S1 S  Information about this IRG may be found on the NIDDK Home page (http://www.niddk.nih.gov)  Scientific Review Administrator:  DR. FRANCISCO CALVO, SRA  NATIONAL INSTITUTE OF DIABETES AND  DIGESTIVE AND KIDNEY DISEASES  6707 DEMOCRACY BLVD, ROOM 752  BETHESDA MD 20892-5452  (301) 594-8885

29. Assignment Notification Letter (Continued)  Institute/Center:  NATIONAL INSTITUTE OF DIABETES AND DIGESTIVE AND KIDNEY DISEASES  DIV/EXTRAMURAL ACTIVITIES  6707 DEMOCRACY BLVD, ROOM 752  NATIONAL INSTITUTES OF HEALTH  BETHESDA, MD 20892  (301) 594-8885

30. Initial Review in IC

31. Peer Review CSR and IC Study Sections are managed by a Scientific Review Administrator (SRA) who is a scientist, usually at the Ph.D. level, whose scientific background is close to the expertise of the study section Each CSR/IC standing study section has 12-24 members who are primarily from academia Institute review is also different in that Special Emphasis Panels are constituted specifically to review applications that have been received in response to an RFA or other Institute specific grant mechanisms As many as 60-100 applications are reviewed at each study section meeting, again this depends on the complexity of the review

32. Scientific Review Administrator Performing administrative and technical review of applications to ensure completeness and accuracy Selecting reviewers based on broad input Managing study section meetings Preparing summary statements Providing any requested information about study section recommendations to institutes and national advisory councils/boards Designated Federal official with overall responsibility for the review process, including:

33. Selection of Peer Reviewers Non-Doctoral Scientific Community Non-Research Research Capability Active and Productive Researchers

34. Criteria for Selection of Peer Reviewers Scientific Community Active and Productive Researchers Non-Doctoral Non-Research Active & Productive Researchers Research Capability Doctoral or Equivalent Degree Interest in Serving Expertise in Discipline of Review Group and Specialization Needed Meet Internal Administrative Considerations Geography Institutional Affiliation Non-Federal Employment Gender & Ethnic Status

35. Study Section Meeting

36. Certification of No Conflict of Interest This will certify that in the review of applications and proposals by (study section) on (date), I did not participate in the evaluation of any grant or fellowship applications from (1) any organization, institution or university system in which a financial interest exists to myself, spouse, parent,child, or collaborating investigators; (2) any organization in which I serve as officer, director, trustee, employee or collaborating investigator; or (3) any organization which I am negotiating or have any arrangements concerning prospective employment or other such associations. __________________ __________________ SIGNATURES

37. Confidentiality Review materials and proceedings of review meetings represent privileged information to be used only by consultants and NIH staff. At the conclusion of each meeting, consultants will be asked to destroy or return all review- related material. Consultants should not discuss review proceedings with anyone except the SRA. Questions concerning review proceedings should be referred to the SRA. K185pp.46

38. Review of Research Grants REVIEW CRITERIA: – Significance – Approach – Innovation – Investigator – Environment – Overall Evaluation & Score Reflects Impact on Field

39. Review Criteria (Continued) Significance: Does the study address an important problem? How will scientific knowledge be advanced? Approach: Are design and methods well-developed and appropriate? Are problem areas addressed? Innovation: Are there novel concepts or approaches? Are the aims original and innovative? Investigator: Is the investigator appropriately trained? Environment: Does the scientific environment contribute to the probability of success? Are there unique features of the scientific environment?

40. Specific Review Criteria RFA DK-01-033 Administrative, leadership qualifications and experience of the Program Director. Adequacy and availability of any necessary institutional facilities and resources. Adequacy of the plan for developing a diabetes science education program, including use of existing resources, building partnerships with other institutions, use of consultants, and plans for critical review of the curriculum. Documentation of potential participants for the science education program.

41. Research Involving Human Subjects

42. Education research – Normal educational practices Educational tests, survey or interview procedures, or observation of public behavior – Subjects not identified – Subjects’ privacy rights protected Educational tests, survey or interview procedures, or observation of public behavior not exempt in previous category if: – Subjects are public officials or public office candidates – Federal statute requires confidentiality without exception Areas exempt from human subject review

43. Research Involving Human Subjects ( Continued) Collection or study of existing data, documents, records, pathological specimens – Information publicly available – Subjects not identified Research and demonstration projects regarding certain public benefit or service programs Taste and food quality evaluation and consumer acceptance studies using – Foods without additives – U.S. Government approved food ingredient Areas exempt from human subject review

44. Research Involving Children Children must be considered for inclusion in all human subject research supported by NIH

45. Scientific Review Group or Study Section Actions Scored, Scientific Merit Rating (priority scores and percentiles) Unscored (lower half) Deferral

46. Action Scored -- Scientific Merit Rating 1.0 to approximately 3.0 Based on the relevant review criteria, the application is judged to be in the upper half of applications reviewed by the study section or scientific review group. The recommendation can be for the requested time and amount or for an adjusted time and amount. A priority score is provided, and a summary statement prepared that incorporates the written critiques plus a resume and summary of the discussion.

47. Action Unscored Application is unanimously judged to be in the lower half of applications reviewed by the study section or scientific review group. No priority score is assigned. The summary statement provided to the applicant is a compilation of reviewers’ comments prepared prior to the meeting.

48. Action Deferral The study section cannot make a recommendation without additional information. This information may be obtained by a project site visit or by submission of additional material by the applicant.

49. Post Scientific Review Group Actions Calculations of priority scores and percentile rankings Preparation of summary statements Removal of applications from national advisory council / board consideration

50. Summary Statement Once applications are reviewed, the results are documented by the SRA in a summary statement and forwarded to the Institute (and the PI) where a funding decision is made: The summary statement contains: –Overall Resume and Summary of Review Discussion –Essentially Unedited Critiques –Priority Score and Percentile Ranking –Budget Recommendations –Administrative Notes

51. National Advisory Council or Board Review

52. Council Actions Concurrence with study section action Modification of study section action Deferral for re-review

53. What Determines Which Awards Are Made? Scientific merit Program considerations Availability of funds

54. Mechanisms for Preliminary Studies Small grants (R03) –Feasibility/new technology/innovative high risk ideas –Short term, usually $25,000 - $50,000 –Extremely variable in detail - example: NCRR: 1 year, $35,000, non-renewable NIDDK: 3-6 months, $12,500 - $25,000 NCI: 2 years, $50,000 –Institute review Exploratory/developmental grants (R21) –Feasibility (for those without preliminary data) –$100,000/2 years (for this RFA it is $75,000/1 year) –Increasing use –CSR review (RFA review performed by NIDDK)

55. Preparation of an Application

56. PHS Research Grant Application Kit (form PHS 398) Application Kit (form PHS 398) Mail Completed Forms To: CENTER FOR SCIENTIFIC REVIEW NATIONAL INSTITUTES OF HEALTH ROCKLEDGE II ROOM 1040 MSC-7710 BETHESDA MD 20892-7710

57. When Preparing an Application Read instructions Never assume that reviewers “will know what you mean” Refer to literature thoroughly State rationale of proposed investigation Include well-designed tables and figures Present an organized, lucid write-up Obtain pre-review from faculty at your institution

58. Common Problems in Applications Lack of new or original ideas Absence of an acceptable scientific rationale Lack of experience in the essential methodology Questionable reasoning in experimental approach Uncritical approach Diffuse, superficial, or unfocused research plan Lack of sufficient experimental detail Lack of knowledge of published relevant work Unrealistically large amount of work Uncertainty concerning future directions

59. for Grants NIH GUIDE and Contracts U.S. Department of Health and Human Services Announces NIH Scientific Initiatives Provides NIH Policy and Administrative Information Available on the NIH Web Site : http://www.nih.gov

60. Program Announcement Invites grant applications in a given research area May describe new or expanded interest in a particular extramural program May be a reminder of a continuing interest in a particular extramural program Generally has no funds set aside Applications reviewed in CSR along with unsolicited grant applications

61. Requests for Applications (RFA) Announcement describing an institute initiative in a well-defined scientific area Invitation to the field to submit research grant applications for a one-time competition Set-aside of funds for a certain number of awards Applications generally reviewed within the issuing institute, in this case the NIDDK

62. NIH Information Sources

63. Information on the World Wide Web Selected Sites of Interest National Institutes of Health (http://www.nih.gov) –Office of Extramural Research (http://www.nih.gov/grants/oer.htm) –Grants Policy (http://www.nih.gov/grants/policy/policy.htm) Center for Scientific Review (http://www.csr.nih.gov) –Referral and Review (http://www.csr.nih.gov/refrev.htm) –Overview of Peer Review Process (http://www. csr.nih.gov/review/peerrev.htm) –CSR Study Section Rosters (http://www.csr.nih.gov/committees/rosterindex.asp) –NIH Peer Review Notes (http://www.csr.nih.gov/prnotes/prnotes.htm)

64. Office of Extramural Research Handles requests for grant applications, program guidelines, and general information regarding grant applications Office of Extramural Research National Institutes of Health 6701 Rockledge drive, suite 6095 Bethesda, Maryland 20892-7910 Phone: 301-435-0714 Fax: 301-480-0525 E-mail: grantsinfo@nih.Gov

65. There is no grantsmanship that will turn a bad idea into a good one, but…….. There are many ways to disguise a good one. Dr. William Raub, Former Deputy Director, NIH