1 SPDG Jennifer Coffey 323A State Personnel Development Grants SPDG Webinar on Grant Performance Report for Continuation Funding Jennifer Coffey Office of Special Education Programs US Department of Education Washington, DC February 2014

2 Today’s Agenda Today’s Topic – Grant Performance Report for Continuation Funding   Overview of Performance Reporting   Developing Performance Measures   Completing Section A of the ED 524B   Completing Sections B&C of the ED 524B Program Measures Contractor Review Considerations for your submission

O S E P Annual Grant Performance Report (APR)   An annual report of your activities and performance in meeting the approved objectives of the project and responsible use of federal funds   Required for all active grants, including those in no cost extension (NCE)   OSEP reviews the report to determine if substantial progress has been made in order to receive continued funding or a NCE 3

O S E P Requesting a no-cost extension (NCE) On the Grants Management page ( Requesting No-cost extensions At the time the no-cost extension is requested, about 30 days before the end of the grant, the Project Officer will need a continuation report ed to them. The Project Officer will also need to know: (1) the amount the grantee has remaining in their budget, (2) the activities the grantee wants to continue to conduct that align with approved objectives, (3) how much of the budget will be used for each activity, and (4) why the grantee was not able to spend the entire budget within 5 years. NCE REporting FormNCE REporting Form, This is the form OSEP provides to states to capture information about the no cost extension (NCE). NCE REporting Form 4

O S E P Overview   Recognize strong project objectives that can be associated with high quality performance measures   Develop relevant, measurable, outcome oriented performance measures related to your objectives that maximize the potential for meaningful data reporting and positive outcomes   Complete the ED Grant Performance Report (aka. APR) using form ED 524B. 5

O S E P Why Is This Important? High quality objectives and measures …   Make it easier for you to measure your progress for the purpose of grant management   Allow you to report progress easily and quantitatively   Establish targets (both short-term/annual & long- term)   Allow OSEP staff to gather evidence of program effectiveness Taken from the Center for Evaluation and Education Policy (CEEP) presentation at 6

O S E P Goal – Objectives - Measures 7 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

O S E P Project Objectives What are you trying to accomplish? Objectives should answer this question. Preferred format for objectives: Begin the objective with a verb and define a desired outcome or condition 8

O S E P High Quality Project Objectives   Relevance How relevant is the project objective to the overall goal of the program and/or the goal of your project?   Applicability How applicable is the project objective to the specific activities that are being conducted through your particular project? 9 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

O S E P High Quality Project Objectives   Focus How focused is the project objective?   Measurability Are there concepts in the project objective that lend themselves to measurement? If so, is measurement feasible? Taken from the Center for Evaluation and Education Policy (CEEP) presentation at 10

O S E P Project Objectives -- Examples  Implement a high-quality professional development program to help LEAs implement a multi-tiered system for behavior and academics  Train and mentor new teachers to increase special education teacher retention

O S E P Project Objectives - Examples Provide training that enables personnel to work with and involve families in their child’s education, including families of low income and limited English proficient children with disabilities Provide training that enables personnel to work with and involve families in their child’s education, including families of low income and limited English proficient children with disabilities

O S E P Performance Measures How are you measuring your progress in meeting your objectives? Performance measures should answer this question. 13

O S E P Performance Measures   Measurable indicator used to determine how well objectives are being met.   How will progress be assessed?   How much progress will constitute success?   How will it be known if an objective or part of an objective has been achieved? Taken from the Center for Evaluation and Education Policy (CEEP) presentation at 14

O S E P Performance Measures 15 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

O S E P 2 Types of Performance Measures   Program All grantees funded under the SPDG must report on the PDP program performance measures established by OSEP.   Project Each grantee reports on the approved project performance measures established to meet their project objectives. 16

O S E P Program Performance Measures   Program Measures established by OSEP for the SPDGs. Measures apply to all grants funded under the SPDG. Results on these measures are reported to Congress under the Government Performance and Results Act of Please see the Program Measures Web page for more information and recorded Webinars: 17

O S E P Types of Performance Measures   Project Measures that the grantee establishes to meet their project objectives   Project performance measures can address both the process of working towards an objective and the outcome related to meeting the objective   Ensure a mix of both process and outcome measures, but most will be outcome Taken from the Center for Evaluation and Education Policy (CEEP) presentation at 18

O S E P High Quality Performance Measures High quality performance measures show   What will change   How much change you expect   Who will achieve the change   When the change will take place 19 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

O S E P Project Performance Measure Examples Process measure (e.g.) - SPDG staff (who) will hold 4 (how much) trainings with IHE faculty on how to integrate the transition curriculum into their syllabi (what ) during the first and third years of the grant (when). 20

O S E P Project Performance Measure Examples Outcome measure (e.g.) - By the end of the third year of the grant (when), 80% of SPDG professional development participants (who) will demonstrate 100% reliability (how much) when using the self- assessment rubric established to evaluate implementation of the ---- program (what). 21

O S E P Project Performance Measure Examples Outcome measure (e.g.) - At the end of their third year of training (when), 90% (how much) of partner schools (who) will demonstrate a 15% improvement in the math scores of 4 th grade students who have IEPs (what). 22

O S E P Common Problems   Activities are NOT performance measures   If the best response is “Yes, we did that,” it is likely an activity (not a performance measure)   Activities:   Establish a stakeholder group   Hold an advisory board meeting   Evaluate the project 23 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

O S E P Common Problems   Performance measures need to be measurable   Examples with measurement problems   Will maintain collaborative partnerships with parent organizations   Increase the sustainability of the personnel development program 24 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

O S E P Need additional information on writing performance measures?   All grantees are strongly encouraged to seek training on writing performance measures.   For further information on developing performance measures and logic models, see

O S E P Summary   Projects should have a few clear objectives that explain what the project is doing to support the overall goal(s)   Each objective should have a few, specific performance measures to demonstrate how progress toward meeting the objective will be measured   Both program and project performance measures are included in the ED524B 26

O S E P Completing the ED 524B The ED 524B is a required reporting form with specific instructions.   The form is used by all ED grants and has been approved by the Office of Management and Budget (OMB). Project Directors must follow the directions listed in the Dear Colleague letter and ED 524B Instructions provided by OSEP.   Word or PDF versions of the forms are available at 27

O S E P x 28 X

O S E P Reporting Period: For first year grants, the date is the beginning of the project year to February 28, For grants in years 2-4, it is the date from the end of the previous reporting period to February 28, Reports are due May 9, ANNUAL PERFORMANCE REPORTS

O S E P ANNUAL PERFORMANCE REPORTS Budget Expenditures: Report the expenditures during the “Reporting Period.” Must be data or information from the business or grants office. 30

O S E P Signatory must have authority to sign on behalf of the institution since the grant is from the Department to the institution and not to an individual. The Authorized Representative signs; not the Project Director. Performance Measure Status: This will be checked “No” since OSEP is asking for data for the reporting period covering all years of the grant, not for this budget period. The date entered here will be the due date for your Final Performance Report; which is 90 days after the end of the grant. ANNUAL PERFORMANCE REPORTS 31

O S E P EXECUTIVE SUMMARY SHEET 32 OMB No Exp. 02/28/2011 OMB No Exp. 02/28/2011 *** Provide highlights of the project's activities and the extent to which the expected outcomes and performance measures were achieved during the reporting period. Do NOT include the project abstract. H323A

O S E P PROJECT STATUS CHART 33 H323A

O S E P If you are a 2009, 2010, 2011, or 2012 grantee you will begin with Program Measure 1 as your 1 st project objective. Program Measure 2 will be your 2 nd objective, and so on. After these program measures you will then have your project’s objectives. Please see the program measures presentation for more information: d5-47e a608514d027a d5-47e a608514d027a Also, the Program Measures Web Page: PROJECT STATUS CHART 34 H325T

O S E P Here you identify if the performance measure is a PROGRAM measure, “PRGM,” or a PROJECT measure, “PROJ.” Note: Program measure refers to one of OSEP’s 4 performance measures for the SPDGs. Project measures are unique to your grant. PROJECT STATUS CHART 35 PRGM PROJ

O S E P PROJECT STATUS CHART QUANTITATIVE DATA 36 Depending on your measure, enter either a raw number, or a ratio and percentage. Enter the target number identified in the performance measure and then the actual data for this year. If complete data are not available for the measure, enter “999” (if no baseline) or “NA” in the “Raw Number” or “%” column, as appropriate. Provide an explanation at the bottom of the page under “Explanation of Progress.”

O S E P Information to Include in the Explanation of Progress Section   Describe the data provided (e.g., what data collection methods were used, when were the data collected, how was a sample drawn, are there missing/incomplete data, what was the response rate, was a reliability measure taken). Your Project Officer should be able to understand and interpret the number in the chart from your description in this section.   What changes in the data occurred since last APR (i.e., trend)?   What activities were undertaken to achieve the targets?   If targets were not met, what are possible reasons?   How will activities that failed to meet targets be improved? 37

O S E P Additionally… A “template” is provided for the program measure descriptions in the Program Measure Example Continuation Report ( 5) Program Measure Example Continuation ReportProgram Measure Example Continuation Report 38

O S E P QUALITATIVE DATA If measure requires the collection of qualitative data, report the performance measure and type (PROG or PROJ) and then, enter “N/A” under the Raw Number and Percentage columns. N/A In the “Explanation of Progress” section of the page, referencing the performance measure by number, report applicable qualitative data along with other information about how these data were collected, targets and activities –refer to previous slide for additional content requirements. PROJECT STATUS CHART 39

O S E P Final Page of the Report Section B: Refer to the instructions for Section B in the ED 524B Instructions Section C: Include additional information (recruitment material, syllabi, evaluation instruments, journal articles) 40 H

O S E P Section B – Budget Information This section is never blank! A table can be helpful!   Provide actual expenditures for this reporting period (through 2/29/2012)   Estimate anticipated expenditures for the rest of this budget period and balance remaining, if any.   Explain why you did not expend funds at the expected rate.   Indicate how you plan to use the unexpended funds (carryover) in the next budget period. 41

O S E P Section B – Budget Information   Describe any significant changes to your budget resulting from modifications of project activities.   Describe any changes to your budget that affect your ability to achieve your approved project activities and/or project objectives.   Describe any anticipated changes in your budget for the next budget period that require prior approval from the Department. Any questions … Talk to your Project Officer 42

O S E P Section C – Additional Information   Provide a list of current partners on your grant and indicate if:   Any partners changed during the reporting period. If there were changes, please describe both the changes and any impact that resulted in your ability to achieve approved project objectives and/or project activities.   Any partners are anticipated to change during the next budget period. If so, please describe both the changes and any impact the change might have on your ability to achieve approved project objectives and/or project activities.   Describe any changes that you wish to make in the grant’s activities for the next budget period that are consistent with the scope and objectives of your approved application. 43

O S E P Section C – Additional Information   If requesting changes to the approved Project Director and/or other key personnel, please include the person’s name, title, and contact information. Indicate his/her proposed start date, and percentage of time working on the grant, and attach a resume or curriculum vitae to the annual performance report being submitted.   Do not report on any key personnel changes that were already made during the current or previous budget period(s).   Note: Departmental approval must be requested and received prior to making key personnel changes.   Provide any other information about your project including unanticipated outcomes or benefits 44

O S E P Submitting the ED 524B 45  Submit the ED 524B at  Instructions for using G5 are in the continuation packet.  Signed ED 524B Cover Sheet must be scanned and ed in PDF format to your Project Officer.  Special cases require regular submission of the 524B and signed cover sheet in PDF format to your Project Officer rather than submission through G5 –  Final Performance Reports or APRs for grants in their last performance period, or no-cost extension  The updated APR required for your no-cost extension

O S E P RESULTS OF THE PROGRAM MEASURE REVIEW Considerations for your current submission 46

O S E P What feedback will you receive? Grantee data brief templateGrantee data brief template Conversation with your project officerConversation with your project officer 47

What Feedback will all SPDGs receive? We will plan a future Webinar to review the overall findings. 48

O S E P Considerations for your current submission Overall Results Program Measure 1: 15 initiatives met their target, 18 did notProgram Measure 1: 15 initiatives met their target, 18 did not Program Measure 2: 6 initiatives met their targets and 6 did notProgram Measure 2: 6 initiatives met their targets and 6 did not Program Measure 3: 22 initiatives met their target and 3 did notProgram Measure 3: 22 initiatives met their target and 3 did not 49

O S E P Program Measure 1 Did very well in submitting the necessary data for the reviewers 50

O S E P Problems with Measure 2 Data were reported on 34 initiatives, but most grantees reported insufficient information on this measure.Data were reported on 34 initiatives, but most grantees reported insufficient information on this measure. Data are reported by the contractors for 12 initiatives.Data are reported by the contractors for 12 initiatives. 51

O S E P Data are not included for the remaining initiatives for one of several reasons: Grantees identified a measure but did not provide enough information for DQI to determine if it was a fidelity measure, so DQI was unable to assess progress toward meeting the target since there was uncertainty about whether it was an appropriate measure (n=8); Grantees identified a fidelity measure but did not report data on the measure, often because data had not been collected and/or analyzed at the time the APR was written (n=8); Grantees identified a measure that was not a fidelity measure, so DQI was unable to determine if the target was met since the grantee did not use an appropriate measure of progress toward meeting the goal (n=5) 52

O S E P Measure 3 Make sure you set a target and report on measure 3 53

Contact your OSEP Project Officer with any questions! Thank you for your hard work on behalf of OSEP 54