Orientation for Peer Reviewers: Interdisciplinary Preparation for Personnel Serving Children with Disabilities who have High-Intensity Needs CFDA 84.325K Office of Special Education Programs U.S. Department of Education April 12/13, 2017

Introductions & Logistical Information Welcome Participants are in listen only mode Please put any questions you have into the Chat Pod, and they will be answered during the Question and Answer portion of the event

Logistics Contents of Reviewer Packet Application for New Grants under IDEA (84.325K) Instructions for Standing Panel Reviewers E-Reader Instructions for Reviewers Competition 84.325K PowerPoint Presentation Conflict of Interest Questionnaire Agreement for Grant Application Reviewers OSERS Peer Reviewer Data Form Panel Assignment Sheet Technical Review Form / Panel Summary Form OSERS Review Process Evaluation Survey Honorarium Form W-9 Form The reviewer packet will also include the application package and this ppt.

G5 Minute with Justin Guidance on Using G5 Go to G5 (www.g5.gov) and create/update your profile, check accessibility, and SAVE your username and password Your G5 account will use a Two-Factor Authentication log-in process For specific instructions on how to use the Two-Factor Authentication log-in process, refer to the following link: https://www.g5.gov/G5HELP/G5_Two_Factor_Authentication_ Guidance.pdf

Today’s Agenda Personnel Development Program Purpose 325K Notice Inviting Applications Purpose / Eligible Applicants Selection Criteria Application Requirements Technical Review Process and Scoring Logistics Tips and Reminders Questions

Purpose of CFDA 84.325 The purposes of Personnel Development to Improve Services and Results for Children with Disabilities program are to: help address State-identified needs for personnel preparation in special education, related services, early intervention, and regular education to work with children, including infants and toddlers, with disabilities; and ensure that those personnel have the necessary skills and knowledge, derived from practices that have been determined through scientifically based research and experience, to be successful in serving those children.

CFDA 84.325K Interdisciplinary Preparation in Special Education, Early Intervention, and Related Services for Personnel Serving Children with Disabilities Who Have High-intensity Needs Purpose—To prepare practitioner-level personnel in special education, early intervention, or related services who have the knowledge and skills to use evidence-based practices to improve results of infants, toddlers, children and youth with disabilities, especially those with high-intensity needs. Eligible Applicants— Institutions of Higher Education Private Non-Profit Organizations $9.3M in FY 2017 36 awards expected

CFDA 84.325K—Note the changes in FY 2017 Focus Areas: Early Childhood School Age Interdisciplinary preparation of personnel: “Scholars” from two or more discipline prepared together Pursuing a Masters, Education Specialist, clinical Doctoral degree (except Educational Interpreters which can be a bachelor degree) Special Education, Early Intervention, and Related Services Related Services and Minority Institutions, including HBCUs, funded within each focus area Program Planning Year option available A. Early Childhood 9 awards expected; including 3 MSI (1 HBCU) B. School Age 27 awards expected; including 8 MSI (2 HBCUs)

“Who qualifies as a scholar?” For the purpose of this priority, the term “scholar” means an individual who is pursuing a degree, license, endorsement or certification related to special education, related services, or early intervention and who receives scholarship assistance under section 662 of IDEA (see 34 CFDA 304.3 (g)). [See footnote 1, A-5] Enrolled in a graduate degree program that leads to (State or national) certification/licensure/endorsement needed to serve IDEA-eligible CWD Able to fulfill Service Obligation requirements See https://pdp.ed.gov/OSEP/Home/regulatoryfaqs

“What do you mean by interdisciplinary?” For the purpose of this priority, “interdisciplinary” refers to preparing scholars from two or more disciplines together through shared coursework, group assignments, and coordinated field experiences.” [See footnote 2, page A-4] Preparing “scholars” from two or more disciplines Completing shared coursework, group assignments, and coordinated field experiences Scholars must be able to fulfill service obligation through service to CWD

“Can we partner with general education programs?” Yes, but “general education” does not count as one of the two disciplines that must be included for an interdisciplinary project. Projects may include individuals who are in degree programs that are cooperating with, but NOT funded by, the applicant’s proposed project. [See top of p A-6] Examples: General Education, Law, Nursing, ABA ALLOWABLE: Shared Coursework Group Assignments Field Experiences Featured Speaker series Monthly Seminar CAUTION: Offering opportunities to those not funded by the project, should not diminish the benefit for project-funded scholars by limiting opportunities for scholars to participate in project activities

Consistent with Interdisciplinary Requirements Masters in Special Education / Specialist in School Psychology Masters in Speech-Language Pathology / AuD in Audiology Masters in Special Education from two distinct “disciplines” (i.e., unique programs of study, licenses) Masters in Adapted PE with Masters in Special Education and cooperating with General Education and General Education PE

Not Consistent with Interdisciplinary Requirements Masters in Special Education / Masters in General Education Masters in Speech-Language Pathology will take classes on how to work on interdisciplinary teams in schools or how to work with children with various disabilities Masters in Special Education will take classes taught by faculty in other disciplines

Not Consistent with Interdisciplinary Requirements Masters in Special Education that enrolls school psychologists and speech-language pathologists Masters in Occupational Therapy/Masters in Physical Therapy; but only OT scholars receive funding Masters in Adapted PE cooperating with General Education PE or Special Education

What are Reviewers supposed to do If the Application does not meet the Requirements of interdisciplinary training? Note the shortcoming under “Quality of Project Services” Reduce the points awarded in SCORING Use COMMENTS to justify scoring deduction Report your concerns to your Panel Manager and Competition Manager prior to the panel review

“How much interdisciplinary training is required?” Required: “Shared coursework, group assignments and coordinated clinical experiences” Subject to Review: Components of each degree program Shared coursework, group assignments and clinical experiences Number, nature, combination, & quality of courses, assignment, fieldwork Use “Quality of Project Services” to SCORE and COMMENT [See page A-9 and A-10] Syllabi are in Appendix B

“What do you mean by high intensity needs?” For the purpose of this priority, the term “high intensity needs” refers to a complex array of disabilities (e.g., multiple disabilities, significant cognitive disabilities, significant physical disabilities, significant sensory disabilities, significant autism, significant emotional disabilities, significant learning disabilities; including, dyslexia), or needs of children with these disabilities requiring intensive, individualized intervention(s) that are - Specifically designed to address persistent learning or behavioral difficulties, implemented with greater frequency and for an extended duration than is commonly available in a typical classroom / early leaning setting Requires personnel to have knowledge and skills in identifying and implementing multiple interventions supported by evidence. [See footnote 1, pages A-3 – A-4]

“How will a planning year impact our budget?” Year 1 Budget Request may not exceed $250,000 Up to $100,000 may be used for improving the program and building capacity 65 percent for scholar support will not include that amount Also Note: NIA allows up to 12 months for a “Planning Year.” Scholars do not have to enrolled during that time.

Application Package Dear Colleague Letter Notice Inviting Applications Federal Register Notice Grants.gov System Submission Procedures and Tips for Applicants Priority Description and Selection Criteria General Information

Eligibility Screening Federal Register Notice Inviting Applications Submitted Prior to Deadline (4:30:00 PM DC-time on March 6, 2017) Requirements in this priority Up to $250,000 per year for up to 5 years [A-21] Up to 50 page Narrative Double-spaced, 12-point font Responsive to Priority

See Application Package pages B-44 thru B-46 Selection Criteria FY 2017, CFDA 84.325K Significance of the Project (10 points) Quality of Project Services (45 points) Quality of Project Evaluation (25 points) Quality of Project Personnel, Management Plan, and Resources (20 points) 100 points See Application Package pages B-44 thru B-46

Significance of the Project (0-10 points) In determining the significance of the proposed project, the Secretary considers the following factors: The extent to which the proposed project will prepare personnel for fields in which shortages have been demonstrated; and The importance or magnitude of the results or outcomes likely to be attained by the proposed project, especially improvements in teaching and child achievement. [ See pages B44; also A-6, B-27 - B28]

Quality of Project Services (0-45 points) In determining the quality of project services, the Secretary considers the following factors: The extent to which the project will recruit and retain high-quality scholars; The extent to which the personnel development to be provided by the proposed project reflect up-to-date knowledge from research and effective practice; The extent to which the personnel development to be provided by the proposed project are of sufficient quality, intensity, and duration to lead to improvements in practice among the recipients of those services; The extent to which the personnel development to be provided by the proposed project involve the collaboration of appropriate partners for maximizing the effectiveness of project services; and The extent to which the personnel development to be provided by the proposed project involve the use of efficient strategies, including the use of technology, as appropriate, and the leveraging of non-project resources. [ See pages B45; also A-8, B-28 – B31]

Quality of Project Evaluation (0-25 points) In determining the quality of the evaluation, the Secretary considers the following factors: The extent to which the methods of evaluation are thorough, feasible, and appropriate to the goals, objectives, and outcomes of the proposed project; The extent to which the goals, objectives, and outcomes, including scholar competencies, to be achieved by the proposed project are clearly specified and measurable; The extent to which the methods of evaluation include the use of objective performance measures that are clearly related to the intended outcomes of the project, including scholar competencies, and will produce quantitative and qualitative data to the extent possible; and The extent to which the methods of evaluation will provide performance feedback and permit periodic assessment of progress toward achieving intended outcomes. [ See pages B45 – B46; also A- 11, B-31 – B32]

Quality of Project Personnel, Management Plan, and Resources (0-20 points) In determining the quality of the proposed project personnel, management plan and resources, the Secretary considers the following factors: The qualifications, including relevant training and experience, of key project personnel; The extent to which the time commitments of the project director and principal investigator and other Key project personnel are appropriate and adequate to meet the objectives of the proposed project; The adequacy of the management plan to achieve the objectives of the proposed project on time and within budget, including clearly defined responsibilities, timelines, and milestones for accomplishing project tasks; The adequacy of support, including facilities, equipment, supplies, and other resources, from the applicant organization or the lead applicant organization; and The extent to which the costs are reasonable in relation to the objectives, design, and potential significance of the proposed project. [ See age 46 ]

General & Statutory Requirements As specified in the Application Packet (see pages A-12 thru A-15), projects funded under this priority must: Ensure that they budget for a three-day Project’s Directors’ meeting in Washington, D.C. during each year of the project Make positive efforts to employ and advance in employment qualified individuals with disabilities in project activities (See Section 606 of IDEA) Involve individuals with disabilities or parents of individuals with disabilities ages birth through 26 in planning, implementing, and evaluating the projects (See Section 682(a)(1)(A) of IDEA) Ensure equitable access to, and participation in, its program for students, teachers, and other program beneficiaries with special needs (See Section 427, GEPA) [ Also B-32 thru B-36 ]

Programmatic & Administrative Requirements Budget Limits Maximum $250,000 per year (12 months) Total budget limit of $1,250,000 Page Limits Narrative - Up to 50 pages Appendix A – Required Appendix B - Syllabi Project Period Up to 60 months (5 years)

Statutory & General Requirements See the Statutory Requirements (page A-12) and General Requirements (pages A-12 thru A-15) in the Application Packet Report if the Requirements are met on TRF Type “yes” or “no” and include the e-page number(s); and Enter a score of zero for each Requirement.

Q & A

Purpose of Peer Review To obtain the best professional judgments regarding each application submitted to the program for funding.

Overview of Peer Review Process Webinar titled “Overview of the Peer Review Process for OSEP” at: https://www.osepideasthatwork.org/resources-grantees/oseppeerreview Foundations of Peer Review Independent Review of each Eligible Application based on Selection Criteria Confidentiality Fair and Impartial Peer Review Process General Process for the Technical Review Completing the Individual Technical Review of Applications Participating in the Panel Discussion Reaching Consensus on the Panel Summary Finalizing Technical Review Forms (TRF) and submitting in G5

Peer Review Participants Who’s Involved in the Technical Review of Applications? Competition Manager Panel Manager Reviewers 3 per panel Lead Reviewer Logistics Contractor – Lux Source Solutions

Steps in the Technical Review What Do I Do to Prepare for the Review? Carefully read the Notice Inviting Applications and the Selection Criteria Go to G5 (www.g5.gov) and create/update your profile, check accessibility, and SAVE your username and password Your G5 account will use a Two-Factor Authentication log-in process For specific instructions on how to use the Two Factor Authentication log-in process, refer to the following link: https://www.g5.gov/G5HELP/G5_Two_Factor_Authentication_Guidance.pdf

Steps in the Technical Review How Do I get started? Access applications in G5 after the Orientation As soon as you are able to access the applications, scan each application to check for Conflicts of Interest; If concerned, notify both the Competition Manager and the Panel Manager

Steps in the technical Review And what should I do next? Carefully read each application Review the Narrative, Appendix A, Budget Also consider syllabi (Appendix B) and other materials Complete the Technical Review Form (TRF) for each application: Decide on the appropriate score for each criterion and justify that score by clearly articulating comments on each sub-factor of the criterion Document the presence/absence of each of the Priority Requirements Write in a WORD version of TRF (later copy/paste the contents into e-Reader) Complete a TRF in G5 for each application before the panel convenes for discussion; “Save”

Steps in the Technical Review - continued Complete all reviews (scored with comments) before the panel call TRF for each application is entered in G5 Meet the deadline provided by the Panel Manager Notify your Panel Manager you have finished Actively participate in the panel discussion; be ready to share scores and comments justifying those scores Update scores and comments in G5 based on the panel discussion and panel manager feedback “ Submit” after the panel manager signs off Complete review forms and post-panel assignments following instructions from the Panel Manager and the Logistics Contractor

84.325K Panel calls Focus Area A (FAA) May 2-3, 2017 (T–W) By Teleconference Dates By Focus Area Focus Area A (FAA) May 2-3, 2017 (T–W) Focus Area B (FAB) May 2-3, 2017 (T-W) May 3-4, 2017 (W-Th) May 4-5, 2017 (Th-F) Start / End Times - Determined by your panel manager

Transmission of technical Review forms submitting the TRF in G5 Reviewers must enter and “save” their pre-panel scores and comments on the TRF in G5 (e-reader) by 5:00 PM EST the day before your panel discussion begins After the panel discussions, reviewers go back into G5 to update scores and comments based on the panel discussion and feedback from the Panel Manager Reviewers must finalize and “Submit” Technical Review forms in G5 by 5:00 PM ET the following business day

Technical Review Scoring Procedures Score applications assuming that the rankings will be based on raw scores ranging from 0 to 100. If during the pre-panel review, all three (3) reviewers score an application at 40 or below the application will not be reviewed by the panel. Applications that are eligible for funding are more likely to have average scores of 80 or above. Pre-panel scores will not be provided to applicants. Final scores and comments will be shared with applicants.

Technical review Scoring - continued Individually, each reviewer will read and score an application, and write comments to justify the scores based on the selection criteria. High quality comments provide constructive feedback Address each selection criterion and all sub-factors; Descriptive rather than evaluative or value judgments; Clear, specific and supported by evidence or examples, including e-page numbers; Suggest how to make improvements, if needed; and Respectful and civil Shared with applicants as written

Reminders! A reviewer who comes to the panel discussions without having read, scored, and submitted written comments to justify the scores on each of their assigned applications will be dismissed from the competition and not given an honorarium. During the panel discussion, reviewers share their scores, comments, and the justifications for those scores that each plans to report; scores may change based on panel discussion. AFTER the panel discussions are completed, do not change your scores, unless directed by the panel manager.

Panel Etiquette and Tips Come to the panel meeting on time and prepared to discuss all applications Ensure access to TRFs during the panel discussion and use a land-line phone, if possible, to avoid reception problems Use evidence to support your position, including e-page numbers and examples Hold the dates reserved for panel calls OPEN Function as a team Be respectful of all perspectives shared All day both days!

Last, But Not Least! G5 Questions? Contact the G5 HelpDesk (1-888-336-8930) Let your Panel Manager know the issue Trouble getting back into G5? You may not have logged-out. Wait 10-15 minutes and try again before contacting G5 HelpDesk Develop post-panel amnesia Dispose of grant applications and forms in a secure manner

Q & A

Logistics Contents of Reviewer Packet: Once Again , You should have … Application for New Grants under IDEA (84.325K) Instructions for Standing Panel Reviewers E-Reader Instructions for Reviewers Competition 84.325K PowerPoint Presentation Conflict of Interest Questionnaire Agreement for Grant Application Reviewers OSERS Peer Reviewer Data Form Panel Assignment Sheet Technical Review Form / Panel Summary Form OSERS Review Process Evaluation Survey Honorarium Form W-9 Form The reviewer packet will also include the application package and this ppt.

Logistics Submit Forms to Marlynne Brown After the Review: Submit Forms to Marlynne Brown by COB on the Next Business Day following your Panel Review MBrown@luxsourcesolutions.com Include Technical Review / Panel Summary Forms (completed in e-reader) OSERS Review Process Evaluation Survey (scanned pdf/via email) Honorarium Form (scanned pdf/via email) W-9 Form (scanned pdf/via email)

Logistics Questions? MBrown@luxsourcesolutions.com After the Review; continued - Dispose of Applications Honorarium Processing Maintenance of TRF and Panel Summary Forms Questions? MBrown@luxsourcesolutions.com 301-960-9825, ext. 703

E-Reader @ G-5.gov OSEP uses e-Reader software, available in G5, for Technical Reviews, Location of e-Rader: www.G5.gov; G5 Help Desk: 1-888-336-8930 A webinar for using e-Reader titled “How to Register and Use the G5 E-Reader for Peer Review” is located at: https://www.osepideasthatwork.org/oseppeerreview/

E-Reader Helpful TIPS “Save” frequently but do not “Submit” until after each application has been discussed and your final scores and comments have been entered in G5. Make revisions to your scores and comments supporting the scores after the panel discussion as needed in G5. (Don’t forget to make changes in the Word document as well). For each selection criterion, check the “Answer Complete” button before you hit “Submit.” If you hit “Submit” and then determine you need to make further revisions, the “Save” button will not work. Make your revisions and then “Submit” again to save changes.

Q & A

Lux Source solutions Marlynne Brown Review Manager Contact information Marlynne Brown Review Manager MBrown@luxsourcesolutions.com 301-960-9825, ext. 703

Further Information Competition Managers Technical Support Dawn.Ellis@ed.gov 202-245-6417 Focus Area A—Early Childhood Maryann.McDermott@ed.gov, or Sarah.Allen@ed.gov 202-245-7439 / 7875 Focus Area B—School Age Justin.Hampton@ed.gov 202-245-6111 G5 edcaps.user@ed.gov 1-888-336-8930 G5 HelpDesk Further Information Competition Managers Technical Support

Thank You