ies.ed.gov Connecting Research, Policy and Practice Rebecca Kang McGill-Wilkinson, Ph.D. Institute of Education Sciences Overview of Education Research & Development Centers Program (84.305C)

ies.ed.gov Outline Overview of IES Research & Development Centers – Developmental Education Assessment & Instruction – Knowledge Utilization General requirements Deadlines and notification 2

ies.ed.gov Legislative Mission of IES Describe the condition and progress of education in the United States Identify education practices that improve academic achievement and access to education opportunities Evaluate the effectiveness of Federal and other education programs 3

ies.ed.gov 4 Organizational Structure of IES 4 National Board for Education Sciences Standards & Review Office Office of the Director National Center for Education Evaluation National Center for Education Statistics National Center for Education Research National Center for Special Education Research

ies.ed.gov Missions of the Research Centers NCER – supports rigorous research that addresses the nation’s most pressing education needs, from early childhood to adult education NCSER – sponsors a rigorous and comprehensive program of special education research designed to expand the knowledge and understanding of infants, toddlers, and students with or at risk for disabilities from birth through high school 5

ies.ed.gov IES Grant Programs: Research Objectives Develop or identify education interventions (practices, programs, policies, and approaches) – that enhance academic achievement – that can be widely deployed Identify what does not work and thereby encourage innovation and further research Understand the processes that underlie the effectiveness of education interventions and the variation in their effectiveness 6

ies.ed.gov FY 2014 Research & Research Training Grant Programs Education Research Grants Program (84.305A) Education Research & Development Centers (84.305C) Statistical & Research Methodology in Education (84.305D) – Statistical & Research Methodology Grants – Early Career Statistical & Research Methodology Grants Partnerships & Collaborations Focused on Problems of Practice & Policy (84.305H) – Researcher-Practitioner Partnerships in Education Research – Continuous Improvement Research in Education – Evaluation of State & Local Education Programs & Policies Research Training Grant Programs in the Education Sciences (84.305B) – Predoctoral Interdisciplinary Research Training – Methods Training for Education Research – Training in Education Research Use & Practice 7

ies.ed.gov NCER Investments by Competition ProgramNumber of Awards Investment (in millions) Education Research 548$897.2 Research & Development Centers 20$204.7 Postdoctoral Research Training 35$22 Predoctoral Research Training 26$121.8 Evaluation of State & Local Programs & Policies 13$53.4 Statistical & Research Methodology in Education 33$23.9 Reading for Understanding 6$113.4 Preschool Curriculum Evaluation Research 12$20.2 Social & Character Development Research 7$13.6 Unsolicited 31$44.9 Small Business Innovation Research 58$51 8

ies.ed.gov Research & Development Centers: Purpose Contribute to solving U.S. education problems, improve the education system, and increase student achievement through – Research – Development – Evaluation – National leadership 9

ies.ed.gov Research & Development Centers: Activities Focused research program – Contribute to solution of specific education problem within the Center’s topic – Generate new knowledge and theories Supplemental studies – Research on supplemental questions within the Center’s topic Leadership and outreach – To researchers, practitioners, and policymakers – Advance the field and practice through evidence-based research, practice, and policy 10

ies.ed.gov Center Partnerships Institute encourages partnerships with stakeholder groups to increase – Relevance of work – Accessibility of publications – Usability of findings 11

ies.ed.gov National Research & Development Center on Developmental Education Assessment & Instruction 12

ies.ed.gov Purpose Research – Document current developmental education practices – Identify promising programs – Support innovation – Evaluate programs that serve large numbers of students and/or that can be expanded Assess scalability of programs found to benefit students Leadership and outreach – Bring together policymakers, practitioners, and researchers – Assist states and postsecondary institutions to bring effective models of developmental education to scale 13

ies.ed.gov Background: Research Large numbers of students beginning postsecondary education – especially in community colleges and open-access 4-year institutions – are placed into developmental (remedial) reading, writing, or mathematics courses before they can accumulate credits – Many students are required to take multiple developmental education courses 14

ies.ed.gov Background: Research Research suggests tests and other practices used to place students in these courses may fail to predict who will benefit from developmental education Studies that track progress through coursework show mixed results – Students may not complete coursework – Students assigned to developmental education may be less likely to graduate 15

ies.ed.gov Background: Policy Concerns Decline in support for developmental education – In some corners, seen as more of a hindrance to students than a support – Expenditures on developmental education Target for cost savings More students may be assigned to developmental education than need it Placement and instruction need to be revised or replaced 16

ies.ed.gov Background: Ongoing Work to Improve Developmental Education Change placement procedures – Replacing standardized test with other measures E.g., high school transcript, college-readiness indicators – Change cut scores for tests Identify high school students’ need for developmental education and provide remediation – During high school rather than college – During a “bridge program” before college Providing additional supports during college – E.g., learning communities 17

ies.ed.gov Background: Ongoing Work to Fundamentally Change Developmental Education Acceleration – Compress 1-year’s worth of material into a shorter period Modularization – Students complete a module of study at their own pace – Students who start with more knowledge can finish sooner Mainstreaming – Development education students take college-level courses with additional support Replacement – Replace conventional developmental math education sequence with one on quantitative reasoning and statistics 18

ies.ed.gov The Center’s Focused Program of Research One descriptive study – To document current developmental education practices (assessment and instruction) in use by community colleges and open-access institutions Two evaluation studies – To determine if specific assessments or instructional practices lead to improved student outcomes in reading, writing, math, English language proficiency, or any combination of these 19

ies.ed.gov The Descriptive Study: Research Questions What are the assessment tools and practices used to assess reading, writing, and math skills of incoming students and place them in developmental education (versus college-level) courses? – Why do they use these tools and practices? What are the major strategies used to teach developmental education? Are different assessment and instructional approaches used for nontraditional students? Are the assessment and instructional approaches aligned with the Common Core State Standards? Applicants may expand or add additional research questions (and should provide justification for doing so) 20

ies.ed.gov The Descriptive Study: Research Approach Institutional survey – Stronger if nationally representative of community colleges and open-access institutions – If not national, stronger if focus on State or postsecondary institution in which evaluation studies will take place Qualitative interviews Secondary data analysis Combination of the above 21

ies.ed.gov The Descriptive Study: Methodological Requirements Research questions Population from which study sample will be selected Measures Data analysis plan Timeline 22

ies.ed.gov Evaluation Study 1: Assessment & Placement Possible Research Questions – How can institutions improve on current assessments and placement practices? – What are the effects of alternative assessment and placement strategies on students’ overall academic performance, persistence, and progress? You may propose expanded or modified research questions as long as they meet the purpose and requirements of the evaluation 23

ies.ed.gov Evaluation Study 2: Instruction Possible Research Questions – How can institutions prepare students for college-level courses in the shortest amount of time? – What are the impacts of different instructional approaches on student learning, overall academic performance, persistence, and progress? You may propose expanded or modified research questions as long as they meet the purpose and requirements of the evaluation 24

ies.ed.gov Evaluation Studies: Methodological Requirements Research questions Sample and setting Appropriate research design (meets WWC standards) Power analysis Measures – Proximal and distal outcomes Including student outcomes in reading, writing, math, English language proficiency, or some combination of these – Fidelity of implementation – Comparison group practices 25

ies.ed.gov Evaluation Studies: Methodological Requirements Moderators or mediators Plan to analyze implementation fidelity and comparison group practices Data analysis plan – Overall impacts – Subgroup impacts (e.g., nontraditional students and students taking more vs. fewer developmental ed courses) Issues of scalability – Including cost-benefits of the programs evaluation Timeline 26

ies.ed.gov Supplemental Studies May conduct smaller, supplemental research projects – Usually can be completed within 9 to 12 months – May include exploratory research, piloting innovations, or further development of innovations Center will work cooperatively with the Institute to select and design supplemental studies Must provide two examples of supplemental studies in application – Short rationale and brief description of approach (2 to 3 paragraphs) 27

ies.ed.gov Leadership & Outreach The Center will work with the Institute on its leadership and outreach activities. These may include: – Convening policymakers, practitioners, and researchers – Assisting efforts by States, colleges, and universities to bring effective models to scale The Institute encourages a Center website, webinars, podcasts, and other innovative uses of technology You should describe at least two examples of activities in application 28

ies.ed.gov Significance of Center Significance section at start of Research Narrative Discuss the significance of your proposed Center – Your vision of the Center – What problems it is intended to address – How your set of research and leadership activities will work together to help students in developmental ed – How the research will be relevant to community colleges and open-access 4-year institutions Describe active partnerships with States or postsecondary systems and their role in the work 29

ies.ed.gov Maximum Awards 5 years $10,000,000 Maximum of one award Focused program of research must comprise at least 75% of the total budget 30

ies.ed.gov National Research & Development Center on Knowledge Utilization 31

ies.ed.gov Purpose Research – Develop tools for observing and measuring research use in schools – Identify conditions under which practitioners use research and factors that promote or inhibit research use in schools – Identify or evaluate strategies that researchers can adopt to make their work more meaningful to and impactful on education practice Leadership and outreach – Effective means of knowledge transfer to improve school performance and student outcomes 32

ies.ed.gov Background Accumulation of high-quality research and evaluation on impact of education policies and interventions Concern that this work is not used by practitioners – To adopt interventions with beneficial effects on students – To discard interventions with null or adverse effects 33

ies.ed.gov Background: Potential Causes Misalignment between research questions (e.g., “Does it work?”) and practitioners’ concerns (e.g., “How do I use it with my students?”) Misunderstanding of social context of practitioners – Research use is multi-level phenomenon Need new ways of working with practitioners – Partnerships – Bidirectional model of relationship – Intermediary organizations 34

ies.ed.gov The Center’s Focused Program of Research One measurement study – Develop and validate tools for observing and measuring research use in schools and school districts Two descriptive studies, or one descriptive study + one evaluation/pilot study – Identify the conditions under which research is used and factors that promote research/inhibit research use in schools and districts – Identify or evaluate the skills and strategies researchers can adopt to make their work more meaningful and have a greater impact on practice 35

ies.ed.gov The Measurement Study: Research Questions What are the most reliable and valid indicators of school staff using education research in their decision making? What are the most reliable and valid approaches to capture variation in research use across schools, school districts, and over time? Applicants may expand or add additional research questions (and should provide justification for doing so) 36

ies.ed.gov The Measurement Study: Methodological Requirements Conceptual framework to provide basis for assessment, intended population, and validation Sample and setting Characteristics, size, analytic adequacy of sample Methods for developing and validating Methods for gathering psychometric evidence Appropriateness for end users Need for adaptation for different contexts and purposes Timeline 37

ies.ed.gov Descriptive Study 1: Research Questions What role do State and local education agencies play in making research available to school administrators, faculty, and staff? What role do school leaders play? How do education agencies, school leaders, and teachers obtain and process scientific information? How do social networks within schools and school districts appear to support or hinder research use? What are the major incentives or disincentives for practitioners to better understand and apply education research? What contextual factors seem to affect the use of research in schools and school districts? Applicants may expand or add additional research questions (and should provide justification for doing so) 38

ies.ed.gov Descriptive Study 2: Research Questions Are partnerships between researchers and practitioners associated with greater use? Are intermediaries associated with greater use? What communication strategies or technological innovations are associated with greater use? Applicants may expand or add additional research questions (and should provide justification for doing so) 39

ies.ed.gov Optional Evaluation or Pilot Study In lieu of second descriptive study Examine efficacy of researcher practices on research use OR Rigorous evaluation of impact of particular practice OR Pilot study to show promise of particular practice 40

ies.ed.gov Descriptive Studies: Methodological Requirements Research questions Population from which study sample will be selected – Institute encourages you to select two or more research sites Measures Data analysis plan Timeline 41

ies.ed.gov Optional Evaluation/Pilot Study: Methodological Requirements Research questions Sample and setting Appropriate research design – if doing an evaluation, the design should meet WWC standards Power analysis Measures – Proximal and distal outcomes – Fidelity of implementation – Comparison group practices 42

ies.ed.gov Optional Evaluation/Pilot Study: Methodological Requirements Moderators or mediators Plan to analyze implementation fidelity and comparison group practices Data analysis plan Timeline 43

ies.ed.gov Leadership & Outreach Innovative approaches to fostering dialogue among practitioners and researchers May take advantage of annual conferences Institute encourages websites Institute is open to learning how other applied social sciences encourage research use among practitioners Describe at least two examples of activities in application 44

ies.ed.gov Supplemental Studies May conduct smaller, supplemental research projects – Usually can be completed within 9 to 12 months Center will work cooperatively with the Institute to select and design supplemental studies Must provide two examples of supplemental studies in application – Short rationale and brief description of approach (2 to 3 paragraphs) 45

ies.ed.gov Significance of Center Significance section at start of Research Narrative Discuss the significance of your proposed Center – Your vision of the Center – What problems it is intended to address – How your set of research and leadership activities will work together to promote the use of scientific education research in schools and districts Describe how you will engage researchers and practitioners in the work and communication of findings 46

ies.ed.gov Maximum Awards 5 years $5,000,000 Maximum of one award Focused program of research must comprise at least 75% of the total budget 47

ies.ed.gov General Requirements 48

ies.ed.gov Additional Sections of the Research Narrative for Both Centers In the 35-page Research Narrative, discuss… – Significance – Research Plan for the Focused Program of Research (including supplemental studies) – Other Center Activities (Leadership and Outreach) – Management and Institutional Resources – Personnel We’ve discussed the first three and now let’s move to the last two 49

ies.ed.gov General Requirements: Management & Institutional Resources Describe plans and procedures for overall management of the Center and its activities If plan is to work in schools or other education delivery settings in first year, must include letter(s) of agreement from education organization(s) If developer/distributer of program or intervention is part of Center, describe role and how objectivity will be maintained 50

ies.ed.gov General Requirements: Personnel Expertise in content area(s) Methodological expertise Sufficient experience working with education delivery settings Experience relevant to national leadership – Explain why proposed Center staff is qualified to fulfill leadership role 51

ies.ed.gov Application Deadline Application Package Posted Start Dates Sept 4, :30:00 PM DC Time June 6, 2013July 1, 2014 to Sept 1, 2014 Important Dates & Deadlines 52

ies.ed.gov Notification All applicants will receive notification that the following information is available via the Applicant Notification System (ANS): Status of award Reviewer summary statement If you are not granted an award the first time, consider resubmitting and talking with your program officer

ies.ed.gov Questions? Becky McGill-Wilkinson 54