The Grant Process at the Institute of education sciences Paul L. Morgan, Ph.D. Eberly Fellow Professor of Education and Demography Department of Education Policy Studies Director, Center for Educational Disparities Research Penn State paulmorgan@psu.edu @PaulMorganPhD

Presentation’s structure Overview of IES, RFA structure, funding mechanisms Elements of weaker and stronger proposals, by section Observations of the submission and review process

Background on IES Established in 2002 with the Education Sciences Reform Act Independent, non-partisan statistics, research, and evaluation office of the U.S. Department of Education For fiscal year 2017, requested a budget of $694,000,000 from Congress Provides scientific evidence designed to inform educational practice and policy

Source: IES Budget Request, Fiscal Year 2017 Source: IES Budget Request, Fiscal Year 2017. Retrieved from https://www2.ed.gov/about/overview/budget/budget17/justifications/v-ies.pdf

Overview of IES Research & Research Training Grant Programs Education Research Programs Special Education Research Programs Statistical & Research Methodology in Education Partnerships & Collaborations Focused on Problems of Practice & Policy Researcher-Practitioner Partnerships in Education Research Continuous Improvement Research in Education Evaluation of State & Local Education Programs & Policies Research Training Grant Programs in the Education Sciences Pre-doctoral Interdisciplinary Research Training Methods Training for Education Researchers Post-doctoral Research training Program in Education Sciences & Special Education Research training Program in Special Education: Early Career Development and Mentoring Training in Education Research Use & Practice Education Research & Development Centers Unsolicited SBIRs

Overview of IES Research & Research Training Grant Programs Education Research Programs Cognition and Student Learning Early Learning Programs and Policies Education Technology Education Leadership Effective Teachers and Effective Teaching English Learners Improving Education Systems STEM Education Postsecondary and Adult Education Reading and Writing Social and Behavioral Context for Academic Learning Special Topics

Overview of IES Research & Research Training Grant Programs Education Research Programs

Overview of IES Research & Research Training Grant Programs Special Education Research Programs Autism Spectrum Disorders Cognition and Student Learning in Special Education Early Intervention and Early Learning in Special Education Families of Children with Disabilities Mathematics and Science Education Professional Development for Teachers and Related Services Providers Reading, Writing, and Language Development Social and Behavioral Outcomes to Support Learning Special Education Policy, Finance, and Systems Technology for Special Education Transition Outcomes for Special Education Secondary Students

Overview of IES Research & Research Training Grant Programs 5 Research Goals Exploration: examine relations between malleable factors and education outcomes; small primary data studies, secondary analyses, or meta-analyses Development and Innovation: develop new education interventions; demonstrate the feasibility of the intervention for implementation in an authentic education delivery setting; collect pilot data on promise of intervention to achieve intended outcomes Efficacy & Replication: evaluate whether fully-developed interventions are effective under-specified conditions and with specific types of students; usually using random assignment to intervention and comparison conditions Scale-up Effectiveness: evaluate the impact & feasibility of interventions at scale; test whether interventions are effective when implemented under typical conditions Measurement: develop and validate assessments or other measurement tools

Research & Research Training Grant Programs

Overview of IES Research & Research Training Grant Programs What is included in an application? Project Summary/Abstract Project Narrative & Appendices Bibliography & References Cited Human Subjects Narrative Biographical Sketch for each senior/key person Lists of Current and Pending Funding Support for each senior/key person Narrative Budget Justification Dissemination Plan

Overview of IES Research & Research Training Grant Programs Four Sections of Project Narrative Significance: Does the applicant provide a compelling rationale for the significance of the project? Research Plan: Does the applicant meet the methodological requirements for the application’s goal? Does the dissemination plan address a range of audiences? Personnel: Do the PI and other key personnel possess appropriate training and experience and will commit sufficient time to competently implement the proposed research? Resources: Are the facilities, equipment, supplies, and other resources sufficient to support the proposed activities? Do the commitments of each partner show sufficient support?

Elements of Stronger and Weaker IES Proposals

Significance section Weaker proposals Stronger proposals Don’t clearly explain how they are responsive to the specifics of the RFA (e.g. lack a theory of change) Poorly organized Assume lots of background knowledge, use lots of jargon Detail their responsiveness to the RFA Provide a rationale and objectives of the proposal can be accurately summarized in 5 min (including by a tired and over-worked reviewer trying to remember what the proposal was about right before the panel deliberates) Fail to survey the existing literature in a manner that provides a compelling justification for the project Leaves a reviewer asking, “so what?” or “don’t we already know this?” Justifies the proposal based on theory and prior empirical work Reads as “cutting edge” by fairly summarizing the boundary between “what we know” and “what we don’t” and how the answers will inform educational practice

Research methods section Weaker proposals Stronger proposals Have not been read by a research methodologist/applied statistician Are under-powered, do not explicitly test directional hypotheses, and fail to consider alternative explanations Are written or at least closely edited by a knowledgeable research methodologist/applied statistician Use the most appropriate methods for the type of goal being applied for Are well powered Evaluate alternative explanations or findings Fail to anticipate likely objections to sampling, design, and measurement questions Anticipate and explicitly address likely objections (e.g., use MLM when the independence assumption is clearly violated) Acknowledges any remaining methodological limitations

Personnel and resources sections Weaker proposals Stronger proposals Fail to include team members with the substantive or methodological expertise Include only on junior team members with no prior grant-funded experience The time commitments of each member are reasonable to their qualifications and responsibilities Include consultants (usually not more than 3) to provide additional oversight and expertise Fail to include letters of support specific to the the project’s resource demands, particularly for school sites Provide letters of support from every personnel member not at the granting institution, including consultants as well as school districts where the project will be conducted. The greater the demands of the project on consultant or LEA, the more detailed these letters should be in the time commitment

Experiences with the submission and review process

Some things to consider for the submissions process Think of likely objections. How can you address these? Give yourself enough time (e.g., 1-3 months of on-going work) to put together the proposal Try to have a complete draft 1 month prior to the deadline Have 2 or 3 other researches provide feedback Collect and report on pilot data or preliminary findings if possible Plan on subcontracts to take 1 month or more

Some more things to consider for the review process Ask the RFA project officer to review the proposal prior submission (give at least 1 month’s time for this) Talk to your project officer following any funding decisions prior to revising and resubmitting Plan on the panel having talked over your project for about 20 minutes Plan on revising the proposal as a result of not being funded IES reviewer will typically provide detailed (1-3 pages) feedback Revisions are required to have a resubmission letter

Some last things to consider for the review process Consider anything as a “win” if you think you can adequately respond to the reviews Be respectful, fair, and exhaustive in your response letter State the criticism fairly, and consider it and your response from the reviewer’s perspective Use the reviews as an opportunity to improve the work Consider other agencies (e.g., NIH) that may have RFAs offering closely aligned, potential funding mechanisms

Thank you! paulmorgan@psu.edu @PaulMorganPhD