1. Bridges to the Future Technical Assistance Workshop Division of Minority Opportunities in Research National Institute of General Medical Sciences National Institutes of Health, USDHHS June 19, 2009 NIGMS

2. Bridges To the Future Program Clifton Poodry, Ph.D. Director, MORE Division National Institute of General Medical Sciences National Institutes of Health, USDHHS June, 19 2009

3. Outline of Presentation Overview of Bridges to Baccalaureate (PAR-07-411) & Bridges to Doctorate (PAR-07-410) programs Guidance for Bridges Program Development

4. Bridges Program Goals An institutional program with a focus on increasing the number of community college/master’s degree students from underrepresented groups and/or health disparities populations (URMs) who transfer and complete the baccalaureate/PhD degree, respectively, in biomedical and behavioral sciences.

5. The Bridges Program Emphasizes : Institutional Focus: impact on Bridges and non-Bridges students alike so more URMs transfer and complete the baccalaureate/PhD degree Partnerships: 2-year community college(s) with four- year institution(s), & master’s degree-granting institution(s) with PhD degree-granting institution(s) Developmental Activities: well-integrated activities that will provide students with the necessary academic preparation and skills to enable their transition and successful completion of the baccalaureate/PhD degree in biomedical/behavioral sciences Clear Expectations

6. Purpose: To facilitate a seamless transition of targeted students from associate to baccalaureate degree-granting institution, and from master’s to the PhD degree-granting institution in biomedical/behavioral sciences Size: Up to 4 institutions for BTB, and 3 for BTD, including the applicant institution, unless strongly justified otherwise An institution may participate in more than one Bridges partnership if strongly justified by the potential to magnify the programs’ and institution’s outcomes Partnership/Consortium

7. Students from groups underrepresented in the biomedical and behavioral research enterprise of the nation and/or populations disproportionately affected by health disparities (targeted groups). Nationally, these groups include, but are not limited to, African Americans, Hispanic Americans, Native Americans (including Alaska Natives), Natives of the U.S. Pacific Islands, and/or rural Appalachians. Must be U.S. citizens or non-citizen nationals or permanent residents. Must be matriculated full-time in associate/master’s degree program in biomedically relevant science fields at the partner community college/master’s degree institution. Bridges Students ( (Students who receive support in the form of salaries/wages)

8. Student Selection Institution’s responsibility to establish student qualifications Describe the criteria for selection and retention of Bridges students into the program Critical Mass: Number of students in the program each year BTB: 15-20, with 4 from each CC BTD: 6-8, with 3 from each master’s degree institution Student Selection & Critical Mass

9.  Baseline Data & Goals to Improve  Effective Partnerships  Strong Pool of Targeted Students  Well-integrated Student Development Activities year-round  Strong Institutional Commitment  Sound Evaluation Plan Key Program Expectations

10. Baseline The baseline is a starting point; it serves as a guide to capacity, and is necessary to gauge the impact of the program. For example, an increase of 100% on a base of 1 is not nearly as impressive as if it were on a base of 10. Usually it’s an average data over a 3-5 year period Measurable Objectives These are brief (and focused) statements of end results – connected to the long-term goal. They can be qualitative and quantitative, but must be measurable. They are not a means to an end, or a checklist of “to do” list. The activities proposed are the means to achieve your goals and objectives. What Are the Baseline Data & Measurable Objectives?

11. Examples of Goals, Baseline Data & Measurable Objectives Goal: To increase the number of CC/master’s degree students who transfer to four-year/PhD degree-granting institutions in biomedically relevant sciences. Baseline: How many and what percent of the students currently transfer to baccalaureate/PhD degree programs? Measurable objective: What is the proposed number and percentage of students who will transfer to baccalaureate/PhD degree programs during the grant period?

12. Examples of Baseline Data & Objectives, cont. Institutional Baseline: Out of a total of 40 students in biomedically relevant fields (40% URM & 60% non-URM), 4 URMs (25%) & 9 non-URMs (37.5%) actually transfer to programs in biomedically relevant disciplines per year (average data from 2005- 2008). Institutional Objective: The URM transfer in biomedically relevant disciplines will increase from the current rate of 25% to 37.5% (from an average of 4 to 6 students per year) by the fourth year of the grant award.

13. Examples of Baseline Data & Objectives, cont. Bridges Baseline on Transfer - An average of 4 out of 8 (50%) Bridges students transferred to programs in biomedically relevant disciplines per year (average data from 2005-2008). Objective - The transfer of Bridges students to programs in biomedically relevant disciplines will increase from the current rate of 50% to 75% (from an average of 4 to 6 students per year) by the fourth year of the grant award. Bridges Baseline on Degree Completion - An average of 2 out of 4 (50%) transferring Bridges students completed the baccalaureate/PhD degree in biomedically relevant disciplines per year (average data from 2005-2008). Objective - Bridges students’ degree completion in biomedically relevant disciplines will increase from the current rate of 50% to 75% (to an average of 4.5 graduates per year) by the fourth year of the grant award.

14. Clearly stated goals, and measurable objectives relative to the baseline and in context of NIH expectations Must improve on the previous record and make substantial progress towards meeting the NIH expectations during the next 5 years Goals to Improve the Track Record

15. Institutional Increase in the institutional transfer of targeted students (Bridges & non-Bridges) to baccalaureate/PhD degree programs in biomedically relevant sciences by 50% in five years Bridges Increase in academic preparation and skills development Increase in the transfer of students to baccalaureate/PhD degree programs (70-75% in five years) Increase in the baccalaureate/PhD degree completion of transferring students (75-80% in five years) NIH Expectations

16. BASELINE DATA AND OBJECTIVES Applicant Institution/ Unified Institutional Program Partner Institutions Institutional Baseline for Transfer of URMs and non-URMs (# of students who transfer/ # of full-time students in biomedically relevant disciplines) Institutional Objectives for Transfer of URMs Bridges Baseline for Transfer (# of Bridges students who transfer/ total # of Bridges students in the program) & Degree Completion (# of Bridges students who complete the baccalaureate/PhD degree/ # of transferring Bridges students) Bridges Objectives for Transfer & Degree Completion Great University Champion College Valley State College Unified Institutional Program

17. To Move the Institution Forward from Baseline to NIH Expectations Bridges to Baccalaureate: Developing community college courses and curricula that are fully transferable to the baccalaureate institution Developing community college courses and curricula that are fully transferable to the baccalaureate institution Faculty from the four-year institution serving as visiting lecturers at the two-year institution, or developing joint team- taught courses Introduction of research concepts into the community college Introduction of research concepts into the community college curriculum and/or offering a research skills course to stimulate students’ interest in science Supplementary instruction in “gate-keeping” courses at the community college Supplementary instruction in “gate-keeping” courses at the community college Mentored research experiences for the Bridges students, Mentored research experiences for the Bridges students, including summer research internships Examples of Developmental Activities

18. Bridges to Baccalaureate, cont. Skills development (e.g., critical thinking, communications skills, study and time management skills) workshops, and research careers seminars Preparing community college students, through college orientation classes, etc., for transfer to the 4-year institution Peer mentoring and tutoring, and research career seminars Research education conferences for Bridges faculty (CC), and advanced or special courses at the partner 4-year institution Research conferences for 4-year faculty mentors, if accompanying students making presentations Others Examples of Developmental Activities, Cont.

19. Bridges to Doctorate: Faculty from the two types of institutions jointly developing courses and curricula, including updating existing or developing new/advanced courses at the master’s degree institution Faculty from the two types of institutions jointly developing courses and curricula, including updating existing or developing new/advanced courses at the master’s degree institution Faculty from the doctorate institution serving as visiting lecturers, offering lectures and/or laboratory courses at the master’s degree institution Faculty from the doctorate institution serving as visiting lecturers, offering lectures and/or laboratory courses at the master’s degree institution Faculty from the doctorate institution providing mentored research experiences to Bridges students, including the summer research internships, and serving on their thesis advisory committees Faculty from the doctorate institution providing mentored research experiences to Bridges students, including the summer research internships, and serving on their thesis advisory committees Fostering research capacity of the master’s degree institution via research collaborations Fostering research capacity of the master’s degree institution via research collaborations Examples of Developmental Activities

20. Bridges to Doctorate, cont. Allowing Bridges students to take some courses, and complete part of their master’s thesis research at the doctoral institution Allowing Bridges students to take some courses, and complete part of their master’s thesis research at the doctoral institution Providing Bridges students access to computer and library facilities, seminars, and workshops, etc., at the doctoral institution Providing Bridges students access to computer and library facilities, seminars, and workshops, etc., at the doctoral institution Establishing a mentoring and academic counseling program for master’s students with faculty at the doctoral institution Establishing a mentoring and academic counseling program for master’s students with faculty at the doctoral institution Advanced or special courses and scientific research conferences for Bridges faculty from the master’s degree institution Advanced or special courses and scientific research conferences for Bridges faculty from the master’s degree institution Examples of Developmental Activities

21. Strong commitment to the goals of the proposed program from all participating institutions, including commitment to: Provide the institutional data on the transfer of URMs and non-URMs in biomedically relevant sciences Provide the data on transfer and baccalaureate/PhD degree completion by Bridges students in biomedically relevant sciences Track Bridges students over a ten-year period as they progress through the pipeline InstitutionalizeInstitutionalize the most effective activities supported by the Bridges program Institutional Commitment

22. Purpose: Provide information useful to the applicant and the partner institutions for improving the program, and for institutionalizing the most effective activities supported by the Bridges program. Provide a sound evaluation plan, with timeline, that’s in-line with measurable goals and objectives Identify a qualified evaluator and include his/her biosketch Evaluation

23. Consortium agreement between the associate and baccalaureate, and between master’s and PhD degree-granting institutions define the participating institutions’ respective roles in administering the Bridges program. The application must include a letter from each collaborating institution signed by the appropriate institutional officials and program director/program coordinator, acknowledging participation in the program. These letters must also include the following: “THE APPROPRIATE PROGRAMMATIC AND ADMINISTRATIVE PERSONNEL OF EACH INSTITUTION INVOLVED IN THIS GRANT APPLICATION ARE AWARE OF THE NIH CONSORTIUM GRANT POLICY AND ARE PREPARED TO ESTABLISH THE NECESSARY INTER-INSTITUTIONAL AGREEMENT(S) CONSISTENT WITH THAT POLICY.” Consortium Agreement

24. The program provides support for student, faculty, and institutional development activities Average awards range from $150,000 to $300,000 (DC) per year. Budget must be reasonable, well documented, and fully justified and commensurate with the scope of the proposed program Awards for up to five years Budget and Years of Support

25. Guidance for Bridges Program Development

26. Develop a Plan Read the FOA and F&Qs Conduct an institutional assessment and gather baseline data: how many transferring and graduating; why not more transferring and graduating? what can change so more will transfer and graduate? What are the institutional needs? What is your long range goal? What are your measurable objectives? What activities will help your institution achieve these objectives How will you evaluate your program outcomes? Follow all of the instructions in writing your application!!

27. Project Summary, & Performance site (s) Project Summary, & Performance site (s) Facilities & other Resources Facilities & other Resources Key Persons and their Biographical Sketches Key Persons and their Biographical Sketches Budget Budget Research Plan: Research Plan: Application Components/Organization

28. Research Plan: ( Introduction) Specific Aims and Measurable Objectives Background and Significance: Institutional & student data; vision and anticipated value Preliminary Studies or Progress Report: Specific Outcome Data & Impact on the Institution Research Design & Methods: Application Components/Org., Cont.

29. Research Design & Methods: PD, PCs, and Program Faculty PD, PCs, and Program Faculty Developmental Activities: Developmental Activities:  The rationale and detailed description  Who will implement and the timeline  Possible pitfalls and alternative approaches Responsible Conduct of Research Responsible Conduct of Research Evaluation Plan Evaluation Plan Consortium Agreements Consortium Agreements Application Components/Org., Cont.

30. Presentation of Data Present data in figures, graphs, tables or text Place figures, tables, and graphs close to where they are referred to in the text Make all figures, tables, and graphs clearly legible Avoid irrelevant information Do not use the Appendix to circumvent the page limitations of the Research Plan

31. Common Reasons for Failure Missing or inadequate baseline data Lack of clear and well-defined measurable objectives Lack of adequate progress Activities poorly related to the objectives Poorly developed or missing evaluation plan Lack of institutional commitment or support Lack of coordination with other institutional programs aimed at accomplishing similar goals “Program-centric” application vs. institutional application

32. Institutional and Bridges baseline data Clear statement of program goals, specific aims, and measurable objectives Detailed progress report, if applicable PD/PI, Coordinators, and program faculty with appropriate training and experience Detailed description of developmental activities Sound evaluation plan (see for example, http://oerl.sri.com/ and http://www.nsf.gov/pubs/2002/nsf02057/start.htm)http://oerl.sri.com/http://www.nsf.gov/pubs/2002/nsf02057/start.htm A plan for training in responsible conduct of research Evidence of strong institutional commitment Consortium Agreement with “specific language” A well justified budget that is commensurate with the scope of the proposed program Summary: Bridges “Essentials”

33. Program Evaluation BRIDGES TO THE DOCTORATE (PAR-07-410) BRIDGES TO THE BACCALAUREATE (PAR-07-411) Clifton Poodry, Ph.D. Division of Minority Opportunities in Research National Institutes of General Medical Sciences National Institutes of Health, USDHHS June 19, 2009 NIGMS

34. Evaluation: What is it? Program evaluations are individual, systematic studies that use objective measurement and analysis to answer specific questions about how well a program is working. - #GAO/GGD-00-204 Program Evaluation Program evaluation and the tracking of students are not the same thing. NIGMS

35. Program Evaluation Answers Questions Like…. Does it work? How well does it work? Does it do what we want it to? Does it work for the reasons we think it does? Is it cost effective? Are the benefits worth it? What are the unintended consequences? NIGMS

36. Why bother?  Supports continuous program improvement  Increases understanding of the program – how are activities and strategies linked to results?  Leads to improved planning and management  Provides shared understanding of program NIGMS

37. G uidelines for Conducting Successful Evaluations Invest heavily in planning early on Use knowledgeable, experienced evaluators (usually social scientists) Integrate evaluation into ongoing activities of the program NIGMS

38. Typical Evaluations Needs Assessment  What is nature & extent of the issues program should address?  Planning phase Process Evaluation  Is program being conducted & producing output as planned?  How can process can be improved? Outcome Evaluation  Extent to which a program’s goals have been met? NIGMS

39. Needs Assessment  What problem is the program attempting to address?  Whom does this program serve; to what extent are their needs met?  What should be the documented goals of the program? Process Evaluation  Is the program being implemented as planned? If not, why?  How could the program’s processes be improved?  Has the program achieved recognized standards of performance? Sample Study Questions NIGMS

40. Outcome Evaluation  To what extent has the program achieved its goals?  Is the current performance different from the past?  Has the program been more successful than a comparable program?  Which characteristics/ activities are most related to success?  What are the intended/ unintended effects of the program? Sample Study Questions NIGMS

41. Why should you care? If you don’t know where you’re going, any road will take you there. - Lewis Carroll, Alice in Wonderland -Illustration by Sir John Tenniel, eBooks@ Adelaide, 2004 NIGMS

42. Key Steps in Evaluation 1. Engage stakeholders 2. Describe the program 3. Focus the evaluation design 4. Gather credible evidence 5. Justify conclusions (present data, analysis used, and findings) 6. Ensure use and share lessons NIGMS

43. 1. Engage Stakeholders Who are the stakeholders? Those involved in program operations, those affected by the program operations, and primary users of evaluation results NIGMS

44. 2.Describe the program  What are the goals and specific aims of the program?  What problem or need is it designed to address?  What are the measurable objectives?  What are the strategies to achieve the objectives?  What are the expected effects?  What are the resources and activities?  How is the program supposed to work? NIGMS

45. Activity ≠ Program You cannot evaluate a program by assessing only an activity Remember: NIGMS

46. Model of a Training Program Resources Activities Impact Research base Workshops & Seminars Mentoring by faculty member Training in scientific methods Short term Knowledge Skills Attitudes Intermediate Behaviors Practices Long term Enter PhD Program Faculty & Staff Money Equipment & Technology What is invested? (Inputs) (Outputs) (Outcomes) What is invested? What is done? What are the changes or benefits? NIGMS

47. 3. Focus the evaluation design  What do you want to know? (key questions)  Who will be involved in or affected by the evaluation or use the findings? (stakeholders)  To focus an evaluation, consider its purpose, uses, questions, methods, roles, budgets, deliverables etc. An evaluation cannot answer all questions for all stakeholders NIGMS

48. 4. Gather credible evidence Evidence must be believable, trustworthy, and relevant Select methodological approach & data collection instruments Determine who is studied and when NIGMS

49. 5. “Justify” conclusions Consider data: Analysis and synthesis - determine findings Interpretation - what do findings mean? Judgments - what is the value of findings based on accepted standards? Recommendations – - what claims can be made? - what are the limitations of your design? NIGMS

50. An evaluation plan should include: Program description with baseline data Purpose & rationale for evaluation Evaluation Design Data Collection & Analyses Products of evaluation & their use Project Management Budget estimate NIGMS

51. If you remember nothing else… Evaluation is a tool to help you make decisions about program management NIGMS

52. June 2009 Application Submission and Review Considerations Mona R. Trempe, Ph.D. Scientific Review Officer NIGMS Office of Scientific Review

53. NIGMS June 2009 Application Process First step - Grants.gov Electronic application via this site now required for all R and most K mechanisms Tentative transition dates for other mechanisms: F transition scheduled for August 2009 T and K12 scheduled for January 2010 P and U transition indefinite at this point

54. NIGMS June 2009 Application Process Second Step - eRA Commons Retrieves the application from Grants.gov and checks it against NIH-specific requirements Allows applicants to electronically track the status of submissions and to receive/transmit application and award information Provides contact information for assigned NIH staff: MORE Division, Review Office, and Grants Management

55. NIGMS June 2009 Online Resources  Overview of Electronic Submission http://era.nih.gov/ElectronicReceipt/ http://era.nih.gov/ElectronicReceipt/  Frequently Asked Questions http://era.nih.gov/ElectronicReceipt/faq.htm http://era.nih.gov/ElectronicReceipt/faq.htm  Avoiding Common Errors http://era.nih.gov/ElectronicReceipt/avoiding_errors.htm http://era.nih.gov/ElectronicReceipt/avoiding_errors.htm  Training Resources, Videos, Quick Reference Materials http://era.nih.gov/ElectronicReceipt/training.htm http://era.nih.gov/ElectronicReceipt/training.htm

56. NIGMS June 2009 Finding Help Grants.gov Contact Center Toll-free: 1-800-518-4726 Email : support@grants.gov Hours : Mon-Fri, 7 a.m. to 9 p.m. EST Support for: Grants.gov registration, Mac issues,support@grants.gov Adobe forms eRA Commons Help Desk Phone: 301-402-7469 Hours : Mon-Fri, 7a.m. to 8 p.m. EST Online Help Ticket : http://ithelpdesk.nih.gov/eRA/http://ithelpdesk.nih.gov/eRA/ Support for: Commons registration, application status, post-submission questions

57. NIGMS June 2009 Review Process: Usual Timeline TimeframeActivity 1 - 2 monthsReferral 2 - 6 monthsReview Panel 6 - 7 monthsSummary Statement Available 7 - 8 monthsAdvisory Council 8 - 9 monthsFunding Decisions 9 - 10 monthsAward Start Date

58. NIGMS June 2009 Review Process: NIH Contacts Who and when? Scientific Review Officer - Prior to summary statement Program Officer - After summary statement Grants Manager - During award activities How to find them? Information always available in eRA Commons If need help, ask your SRO or PO

59. NIGMS June 2009 Review Process: Reviewers Review Panel Organized by a Scientific Review Officer in the NIGMS Office of Scientific Review Special Emphasis Panel or Standing Committee Temporary committee members included on standing committees for particular expertise Reviewer Characteristics Experience with multiple education levels Involvement with research training programs Educators, researchers, and institutional administrators

60. NIGMS June 2009 Enhanced Peer Review: Criteria Core Review Criteria For R mechanisms: Significance, Investigator(s), Innovation, Approach, Environment For T mechanisms: Training Program and Environment, Training Program Director, Preceptors/Mentors, Trainees, Training Record Additional Review Criteria/Considerations Resubmission/Renewal (as applicable) Human Subjects Protection and Inclusion; Vertebrate Animal Use; Biohazards (as applicable) Training in Responsible Conduct of Research

61. NIGMS June 2009 Enhanced Peer Review: Scoring Scale Nine-point whole number scale (1 = exceptional; 9 = poor) Overall Impact and Criterion Scores All discussed applications receive a two digit impact/priority score; this is the average of the score given by each eligible review committee member Scores are given for each of five core review criteria by each of the assigned reviewers “ND” = not discussed (previously unscored) applications will receive criterion scores but no impact/priority score

62. NIGMS June 2009 Enhanced Peer Review: Amendment Limits For original new applications (never submitted) and competing renewal applications only one resubmission (previously amended or revised application) will be accepted If funding is not received after two submissions, the program must be substantially re-designed rather than slightly altered in response to previous reviews in order to be submitted as a new application. There are no more “A2” applications For details, see NIH Guide Notice NOT-OD-09-003NOT-OD-09-003

63. NIGMS June 2009 Details of Enhanced Peer Review NIH Guide NOT-OD-09-024, NOT-OD-09-025, NOT-OD-09-003 http://enhancing-peer-review.nih.gov

64. NIGMS June 2009 Tips about Format Page Limits  Supply all requested materials within page limits  Do not use appendices to get around the limits Appendices  Follow general guidelines (NIH Guide NOT-OD-07-018) and ones specific to the program announcement  Relevant material only, such as large tables, survey instruments, publications that are NOT available online  NO catalogs, lengthy reports, or material that should be in the body of the application

65. NIGMS June 2009 Application Preparation Tips Content Read the program announcement and ensure that your application contains the necessary elements Successful submission through Grants.gov and eRA Commons does not mean appropriate responsiveness to the program announcement Context Present the institutional framework and environment of your program Be realistic in your program’s goals

66. NIGMS June 2009 Application Preparation Tips Comprehensive Address all of the requirements of the program announcement For example: If you don’t have institutional baseline data, explain how you plan to obtain it If you haven’t fully formed your evaluation plan, at least acknowledge that you are working on it Describe how your program “works” For example: How are students recruited and selected? By whom? What does the advisory committee do? How often do they meet? How have you used evaluation information in designing/improving your program?

67. NIGMS June 2009 Application Preparation Tips Clear Don’t bury important information in appendices or expansive prose Don’t expect reviewers to “read between the lines” to figure out what you are proposing Present outcomes data in a straightforward manner: Don’t exaggerate Don’t hide data (reviewers will “do the math”) It is far better to present results as they are and address how the program aims to improve

68. NIGMS June 2009 Application Preparation Tips Current Make sure faculty biosketches are up-to-date Provide data on current and prior students Use the most recent institutional data Consistent Data in tables and text should match Data should be consistent across tables Match justification to budget items Refer to the correct program in text and tables

69. ESSENTIAL “MUSTS” FOR THE BRIDGES PROGRAM APPLICATIONS BRIDGES TO THE DOCTORATE (PAR-07-410) BRIDGES TO THE BACCALAUREATE (PAR-07-411) Jermelina Tupas, Ph.D. Division of Minority Opportunities in Research National Institutes of General Medical Sciences National Institutes of Health, USDHHS June 19, 2009 NIGMS

70. ESSENTIAL “MUSTS” FOR THE BRIDGES PROGRAM APPLICATIONS BRIDGES TO THE DOCTORATE (PAR-07-410) BRIDGES TO THE BACCALAUREATE (PAR-07-411) Jermelina Tupas, Ph.D. Division of Minority Opportunities in Research National Institutes of General Medical Sciences National Institutes of Health, USDHHS June 19, 2009 NIGMS

71. 9 Essential Items for A Bridges Program Application 1.Institutional program 2.Partnership 3.Applicant pool 4.Student Development 5.Specific Aims 6.Budget 7.Responsible Conduct of Research 8.Evaluation Plan 9.Progress Report NIGMS

72. 1. Institutional Program reflect the plans and priorities of the participating institutions and the collective plans and priorities of the partnership; should have the following information baseline data goals and measurable objectives activities or interventions evaluation plan NIGMS

73. Create a partnership program that will focus attention and adequate resources on the institution(s) granting associates or master’s degrees and enhance competitiveness of its (their) science graduates and science programs NIGMS Challenge to the consortium:

74. Significance Are you proposing a program that is important to the mission of NIGMS and NICHD and the science they support? What will be the impact of the proposed program on each participating institutions, i.e. the number of students transferring and graduating (with baccalaureate or PhDs) in the biomedical/behavioral sciences? NIGMS Institutional Program: Review Qs

75. Innovation Is this a duplication of other programs currently supported at the applicant institution or available within the consortium? Environment What are the unique features of the institutions that will contribute to achieving the proposed goals? Are the plans to coordinate the planned activities in multiple sites adequate? Is the institutional commitment to the proposed program appropriate? NIGMS Institutional Program: Review Qs

76. Tracking must be carried out. Official letter of agreement to: a)provide the institutional data on transfer and subsequent graduation of its students in biomedical and behavioral sciences; b)track Bridges students over a ten-year period as they progress through the pipeline; and c)provide the data on transfer, and degree completion (bachelor’s or Ph.D.) for Bridges and non-Bridges students at the partner institutions. NIGMS Institutional Commitment

77. 2. Partnership Consortium-no more than 4 baccalaureate or 3 doctorate institutions Partners-one must be a 2-year or master’s degree institution, another must be a bachelor’s or Ph.D. degree institution Applicant institution (only one)- must name the PD Role of each partner institution- must be well-defined; include partnership/consortium agreement in Appendix NIGMS

78. Requirement for the partners (Bridges to the Baccalaureate) 2-year institution: offers the associate degree as the only undergraduate degree in the biomedical and behavioral sciences within the participating departments Baccalaureate institution: must be a college or university granting the baccalaureate degree in biomedical or behavioral sciences. NIGMS

79. Requirement for the partners (Bridges to the Doctorate) Master’s institution: MS- terminal degree in biomedical or behavioral sciences as the highest degree; must have a high enrollment of targeted MS students Ph.D. institution: must be a research institution; have a significant number of mentors with extramural research support; awards doctoral degrees in biomedical or behavioral sciences.

80. MS institutions with RISE programs that include master’s students must justify the need for the Bridges program and provide evidence that there is a large pool of targeted students A list of faculty research mentors at the doctoral institution must be provided; include in their biosketches past student training record and extramural support NIGMS Requirement for the partners (Bridges to the Doctorate)

81. Partnership: Review Qs Approach Are the roles of the participating institutions well developed, well integrated, and appropriate to the aims of the program? Are the partnership arrangements reasonable and are likely to facilitate the seamless transition of the students? NIGMS

82. Partnership: Review Qs Investigators Is there an adequate pool of research mentors at the PhD institution who are extramurally funded? Do the key personnel selected for program implementation have experience in mentoring students, particularly URMs? NIGMS

83. Partnership: Review Qs Environment Is there appropriate collaboration among participating departments and institutions? Do the consortium agreements and letters from each participating institution provide adequate documentation and assurance that each will contribute to the success of the proposed Bridges to the (Baccalaureate or Doctorate) program? Does the proposed consortium involve appropriate number of institutions NIGMS

84. 3. Applicant Pool Targeted students groups underrepresented in the biomedical & behavioral sciences; those disproportionately affected by health disparities U.S. citizens, nationals, or permanent residents matriculated full-time in (Associate or MS) in targeted areas at the (2-yr or MS) pool at each (2-yr or MS) institution must be clearly described; include total number of targeted students at 2-yr or MS institutions & total number of targeted Bridges participants NIGMS

85. Applicant Pool: Review Qs Approach Are the recruitment, retention, and follow-up activities adequate to ensure a large pool of eligible participants? Environment Is there an adequate pool of students from targeted groups/populations in the participating science department(s) at the (2- year or master’s degree-granting) institutions who are interested in research careers in biomedical/behavioral fields? NIGMS

86. integrated set of student development activities rationale for each developmental activity targeted students’ needs and requirements contribution of each activity to realization of the objectives interventions to increase the number of student transfer and completion of B.S. or Ph.D. degree in biomedical/behavioral sciences overall impact on the capabilities of the master’s degree institution(s) to provide competitive training to their students, must be measured (i.e., measurement of outcomes) NIGMS 4. Student Development

87. Significance Are the proposed developmental activities likely to improve the academic preparation of (associates or master’s degree) students and allow them to be admitted to BS or Ph.D. degree programs in biomedical/behavioral sciences? What will be the impact of the proposed program on the number of students from the targeted groups who transfer (to 4-yr or doctoral institutions) and complete (B.S. or Ph.D.) degrees? NIGMS Student Development: Review Qs

88. Approach: Does the applicant acknowledge potential problem areas and consider alternative approaches? Is there evidence that the program is based on sound research concepts and educational principles? Does the program provide details and rationale for the activities proposed to enhance the academic preparation of targeted students? NIGMS Student Development: Review Qs

89. Innovation Does the project challenge existing paradigms and address the critical barriers that prevent targeted students from pursuing bachelor or doctorate degree programs? Do the proposed academic development activities employ novel concepts, approaches, or methods to attract, retain, and prepare (associate or master’s degree) students for more challenging academic programs at the (B.S. or Ph.D.) degree- granting institution? NIGMS Student Development: Review Qs

90. Environment Does the scientific/educational environment in which the program will be conducted contribute to the probability of success? Does the proposed research education program benefit from unique features of the scientific environment, subject populations, or employ useful collaborative arrangements? NIGMS Student Development: Review Qs

91. 5. Specific Aims must address the overall goals and specific measurable objectives (including anticipated milestones) that the consortium institutions expect to accomplish by the end of the project period objectives must be presented as percent improvement over the current baseline and the baseline must be clearly defined NIGMS

92. Specific Aims: Review Qs Significance How will implementation of the proposed program advance the objectives of this funding opportunity announcement as well as the mission of the NIGMS and NCMHD? Approach: Are the conceptual framework, design, methods, and analyses adequately developed, well integrated, well reasoned, and appropriate to the aims of the project? NIGMS

93. 6. Budget (clearly justified single consolidated budget ) fully itemized; well justified participant cost requested equipment must be critical to improving laboratory instruction at the 2-yr or MS institution ($25k per project period) cost for faculty teaching/developing new courses: reasonable and commensurate with the scope of the proposed program; no salary for mentoring activities cost for evaluator who is an employee of an institution in the consortium must be included in key personnel and listed as person months expenses for items generally available for educational programs at the institutions should not be duplicated NIGMS

94. 7. Progress Report NIGMS Information needed: 1. summary of the overall progress- overall transfer data on Bridges and non- Bridges students from targeted groups number of Bridges students who transferred to ( 4-yr or Ph.D.) institutions number of Bridges students who completed (B.S. or Ph.D.) degrees in biomedical sciences) explicitly identified in renewal applications in place of preliminary studies

95. 7. Progress Report Information needed: 2.student development activities implemented; number of students served; faculty member(s) conducting the activity; progress made in relation to the original goals and objectives 3.list of Bridges students who were supported by the Program during the previous grant period 4.lessons learned from the program evaluation; changes made in the program as a result of the evaluation 5.Bridges activities that are now continuing (or will continue) on institutional funds NIGMS

96. Progress Report: Review Qs Has the research education program successfully achieved its stated objectives, especially in the context of Program’s expectations during the prior project period? What is the track record of the participating institutions on the transfer and graduation rates of students, How does this record compare to the Bridges Program goals (50% increase in the overall institutional transfer rate; and 75% transfer and 80% degree completion rates for Bridges students)? NIGMS

97. Progress Report: Review Qs Has the program been innovative in the past and does it continue to demonstrate innovation? Has the program been adequately evaluated, and is the proposed approach for the next project period responsive to the results of this evaluation? NIGMS

98. 8. Responsible Conduct of Research Include: 1.the subject matter of the instruction, the format of the instruction, the degree of program faculty participation, participant attendance, and the frequency of instruction 2.the rationale for the proposed plan of instruction. NIGMS

99. 9. Evaluation Plan Remember: evaluation must be used as advisory to the PD and the participating institutions specific plans, benchmarks, and procedures must be described to capture, analyze, and report outcome measures that would determine the success of the research education program in achieving its objectives NIGMS required for all applications, lack of evaluation may lead to non-review

100. Evaluation Plan: Review Qs Is the evaluation plan and timeline adequate for assessing the effectiveness (process and outcome) of the program in achieving its goals and objectives? Does the application identify an individual with appropriate credentials to conduct the proposed evaluation? NIGMS

101. BRIDGES TECHNICAL WORKSHOP GRANTS MANAGEMENT PAR-07-411: http://grants1.nih.gov/grants/guide/pa-files/PAR-07-411.html PAR-07-410: http://grants1.nih.gov/grants/guide/pa-files/PAR-07-410.htmlhttp://grants1.nih.gov/grants/guide/pa-files/PAR-07-411.htmlhttp://grants1.nih.gov/grants/guide/pa-files/PAR-07-410.html Lori Burge Grants Management Officer National Institute of General Medical Sciences June 19, 2009

102. New BRIDGES to the Baccalaureate Program and Doctorate Program Current/New PAR Submission style Electronic-Grants.Gov SF424 Project PeriodUp to 5 years Receipt Date 2009January 22 and September 18 Number of Trainees BAC Min 4 from each partner institution DOC Min 3 from each partner institution Length of TrainingUp to 2 years

103. Features (cont’d) Just-in-Time Concepts Used (Other Support, IRB, IACUC submitted to eRA Commons) Budget FormatCategorical Number of Institutions BAC Maximum 4 (includes the applicant institution) DOC Maximum 3 Salary for Students BAC level comparable to that of other students employed in similar activities DOC $12 an hour max/level comparable to other students Total Direct Costs No limit

104. Features – BRIDGES to the Doctorate  Support for students: - salary support for up to 20 hours a week during the academic year while they are fulfilling their course requirements and 40 hours a week during the summer if no courses are being taken. Bridges students in the master’s degree program are also allowed tuition remission as part of a compensation package.  Limited tuition costs of participating faculty (from the master’s institution) to take one advanced course per year at the partner doctoral institution is allowed

105. Features – BRIDGES to the Doctorate  Students must be matriculated full-time.  Salary support: - for the PD to administer the program is limited to a max of 1.8 person months - for a program coordinator at the partnering institutions should not exceed 1.8 person months  Equipment costs are limited to a maximum of $25,000 per project period  Research supplies for Bridges students (not to exceed $2,000/student/year)

106. Features Bridges to the Baccalaureate  Students must be Matriculated full-time.  Salary support: - for the PD to administer the program is limited to a max of 2.4 person months - for a program coordinator at the partnering institutions should not exceed 1.8 person months  Equipment costs are limited to a maximum of $25,000 per project period  Research supplies for Bridges students (not to exceed $1,000/student/year)

107. Allowable Costs for all Bridge Grants  Consultant costs - If evaluator is an employee of an institution within the consortium the cost must be included in the category of key personnel salary and listed in person months  Supplies and equipment  Travel for key persons

108. Allowable Costs for all Bridge Grants  Personnel - Limited administrative and clerical salary costs associated distinctly with the Bridges program that are not normally provided by the institution when justified  Per new guide notice each institution may participate in more than one Bridges partnership if justified  http://grants.nih.gov/grants/guide/notice-files/NOT-GM-07- 116.html http://grants.nih.gov/grants/guide/notice-files/NOT-GM-07- 116.html

109. Participants Allowable Costs  Must be itemized in section B (other Personnel) of Research & Related Budget  Students may be supported with salary/wages (not stipends) STIPENDS

110. All requested costs – All must be justified as specifically required by the proposed research education program and must not duplicate items generally available for same purpose at the applicant institution – Items must be itemized in appropriate sections on 424 forms, section C (Equipment), D (Travel) and F (Other Direct Costs) of Research & Related Budget – PLEASE JUSTIFY ALL COSTS CLEARLY OR REVIEW MAY NOT ALLOW THEM

111. Unallowable Costs for Bridges to the Baccalaureate  Costs for students not matriculated full-time at the partner associate degree-granting institution  Salary support for students who have completed the associate degree  Faculty: if mentoring and other activities with students are considered a regular part of an individuals academic duties these would not be an allowable costs from Bridge grant funds

112. Unallowable Costs for Bridges to the Doctorate  Costs for students not matriculated full-time at the partner master’s degree-granting institution  Salary support for students who have completed their master’s degree and enter a Ph.D program  Faculty: if mentoring and other activities with students are considered a regular part of an individuals academic duties these would not be an allowable costs from Bridge grant funds

113. Participant Terms & Conditions on Notice of Award For compensation the following conditions must be met:  The student must belong to the targeted groups/populations, must be a U.S. citizen or non-citizen national or permanent resident and must be matriculated full-time in degree programs in biomedical or behavioral science fields at the partner institution  The student must be performing necessary work relevant to the proposed program  There is an employer-employee relationship between the student and the institution (NO STIPENDS)  The total compensation is reasonable for the work performed  The institution provides compensation for all students under similar circumstances, regardless of the source of support for the activity

114. Unallowable Costs  Foreign Travel  Alterations and renovations  Housing, food, books, recruitment costs incentives to encourage or motivate students (such as laptops, internet subscriptions)

115. Consortium Agreements  Are a required part of the Bridges Program  Agreements between the eligible parent grantee and the bridging institution define their respective roles in administering the program  Application must include a signed letter from each collaborating institution

116. Consortium Agreements Language  Each letter should include the following statement: “THE APPROPRIATE PROGRAMMATIC AND ADMINISTRATIVE PERSONNEL OF EACH INSTITUTION INVOLVED IN THIS GRANT APPLICATION ARE AWARE OF THE NIH CONSORTIUM GRANT POLICY AND ARE PREPARED TO ESTABLISH THE NECESSARY INTER-INSTITUTIONAL AGREEMENT(S) CONSISTENT WITH THAT POLICY.”

117. EXPANDED AUTHORITIES Beginning this fiscal year all Bridge grants are under Expanded Authorities http://grants1.nih.gov/grants/policy/nihgps_2003/ NIHGPS_Part7.htm#_Toc54600128

118. Expanded Authorities What does it mean?  Carryover of unobligated balances from one budget period to the next is allowed  No cost extension of final budget period can be submitted automatically  Financial Status Reports (FSR) are no longer due after every budget year. FSRs are due 90 days after the final budget year of the project period

119. No Cost Extensions  1 st No Cost Extension (NCE) may be requested through the eRA Commons.  Only students already participating in the current Bridges program may continue in a NCE  No new students may be added to the Bridges program during a NCE

120. Submit Annual Progress Reports by eSNAP  Bridge progress reports are now eligible for Streamlined Non- Competing Award Procedures (SNAP) and may file their annual Progress Reports electronically using the eSNAP feature of the eRA Commons

121. SNAP Questions

122. # 3 SNAP Question  If you answer YES to question # 3 “unobligated balance will be greater than 25% of the current year’s total budget” - Provide a detailed explanation of why there is a large unobligated balance and your specific plans to spend these funds.

123. Restrictions to Expanded Authorities for Bridge Grants  The Bridge grantees may not appoint more than the approved number of Bridges Participants unless approval is first obtained from NIGMS.

124. Reminder  Grantees are reminded that they must continue to exercise proper stewardship over Federal funds and that costs charged to awards are allowable, allocable, reasonable, necessary, and consistently applied regardless of the source of funds.

125. Personnel on 424 Forms  For Bridge grants all students must be listed under personnel in block B  DO NOT add students in block E under Participant/Trainee Support Costs

126. Add Students to B. Other Personnel

127. Inclusion Enrollment Report  If you mark YES to Human Subjects on the BRIDGES application then the 424 format will expect the applicant to complete and provide the Target Inclusion Enrollment Report.  For the Study Title add Bridges to the Baccalaureate or Doctorate and add zeros in all the total fields.  If the application is marked NO to Human Subject involvement then no action is needed on the Target Inclusion Enrollment Report

129. For More Information Contact Your Grants Management Specialist  Robert Altieri altierir@nigms.nih.govaltierir@nigms.nih.gov  Irina Alva-Weinstein alvair@nigms.nih.govalvair@nigms.nih.gov  Michael Mace macem@nigms.nih.govmacem@nigms.nih.gov  Justin Rosenzweig rosenzwj@nigms.nih.gov rosenzwj@nigms.nih.gov  Lori Burge burgel@nigms.nih.gov

130. 6/12/09 WORKSHOP SUMMARY & THE MORE Website Jermelina L.G. Tupas, Ph.D. Division of Minority Opportunities in Research National Institute of General Medical Sciences National Institutes of Health, USDHHS June 19, 2009 NIGMS

131. 6/12/09 NIGMS WORKSHOP SUMMARY  Key features of the new Program Announcements Bridges to the Doctorate:(PAR-07-410) http://grants1.nih.gov/grants/guide/pa-files/PAR-07-410.html http://grants1.nih.gov/grants/guide/pa-files/PAR-07-410.html Bridges to the Baccalaureate: (PAR-07-411) http://grants1.nih.gov/grants/guide/pa-files/PAR-07-411.html - Read the correct Program Announcement - Be responsive to the Program Announcement  Case Studies: Strengths & Weaknesses

132. 6/12/09 NIGMS WORKSHOP SUMMARY  Program Evaluation: How much did we do? How well did we do it? Are the community colleges; baccalaureate degree-granting institution; and students better off ? Are the Master’s granting institution; Ph.D. granting institution; and students better off ?

133. 6/12/09 July 25, 2008 NIGMS WORKSHOP SUMMARY Review Issues Essential Items for Bridges Applications http://www.nigms.nih.gov/Minority/Bridges/EssentialsBridgesApps. htm http://www.nigms.nih.gov/Minority/Bridges/EssentialsBridgesApps. htm also see FAQs: http://www.nigms.nih.gov/Minority/Bridges/FAQs.htm http://www.nigms.nih.gov/Minority/Bridges/FAQs.htm Grants Management: - -allowable/unallowable costs -No caps- “Justify, justify, justify”

134. 6/12/09 SUMMARY real partnership measurable program objectives measurable program outcomes, impact, & implementation for improvement & advice transfer and graduation focusing on institutional impact

135. 6/12/09 July 25, 2008 NIGMS The MORE website: http://www.nigms.nih.gov/Minority/ Other useful website: 1. Frequently asked questions: http://www.nigms.nih.gov/Minority/Bridges/FAQs.htm 2. Participating institutions: http://www.nigms.nih.gov/Minority/Bridges/PartInstBacc.htm http://www.nigms.nih.gov/Minority/Bridges/PartInstDoct.htm 3. Program announcements: http://grants1.nih.gov/grants/guide/pa-files/PAR-07-411.htmlhttp://grants1.nih.gov/grants/guide/pa-files/PAR-07-411.html (Bac) http://grants1.nih.gov/grants/guide/pa-files/PAR-07-410.htmlhttp://grants1.nih.gov/grants/guide/pa-files/PAR-07-410.html (Doc) 4. MORE Staff Contacts http://www.nigms.nih.gov/Minority/more_staffcontacts.htm