Office of Special Education Programs U.S. Department of Education GRANT PERFORMANCE REPORT FOR CONTINUATION FUNDING FY 2011

O S E P Annual Grant Performance Report  An annual report of your activities and performance in meeting the approved objectives of the project  Required for all active grants, including those in no cost extension  OSEP reviews the report to determine if substantial progress has been made in order to receive continued funding 2

O S E P Overview  Recognize strong project objectives that can be associated with high quality performance measures  Develop high-quality, measurable performance measures that maximize the potential for meaningful data reporting  Complete the ED Grant Performance Report (ED 524B) 3

O S E P Why Is This Important? High quality objectives and measures …  Make it easier for you to measure your progress  Allow you to report progress easily and quantitatively  Allow OSEP staff to gather evidence of program effectiveness 4 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P Goal – Objectives - Measures 5 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P Project Objectives What are you trying to accomplish? 6

O S E P High Quality Project Objectives  Relevance How relevant is the project objective to the overall goal of the program and/or the goal of your project? How relevant is the project objective to the overall goal of the program and/or the goal of your project?  Applicability How applicable is the project objective to the specific activities that are being conducted through your particular project? How applicable is the project objective to the specific activities that are being conducted through your particular project? 7 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P High Quality Project Objectives  Focus How focused is the project objective? How focused is the project objective?  Measurability Are there concepts in the project objective that lend themselves to measurement? If so, is measurement feasible? Are there concepts in the project objective that lend themselves to measurement? If so, is measurement feasible? 8 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P Project Objectives -- Examples  Implement early childhood language models in Part C and preschool programs  Increase the capacity of SEAs to scale-up RTI programs  Increase practitioners’ knowledge of social emotional interventions  Disseminate the impact of the project at a national level

O S E P Performance Measures How are you measuring your progress in meeting your objectives? 10

O S E P Performance Measures  Measurable indicator used to determine how well objectives are being met.  How will progress be assessed?  How much progress will constitute success?  How will it be known if an objective or part of an objective has been achieved? 11 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P Performance Measures 12 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P Types of Performance Measures  Program Measures established by OSEP for the Technical Assistance and Dissemination Program. These include measures established for reporting to Congress under the Government Performance and Results Act of Measures established by OSEP for the Technical Assistance and Dissemination Program. These include measures established for reporting to Congress under the Government Performance and Results Act of Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P TA&D Program Performance Measures Program Performance Measure #1: The percentage of products and services deemed to be of high quality by an independent review panel of qualified experts or individuals with appropriate expertise to review the substantive content of the products and services. Program Performance Measure #1: The percentage of products and services deemed to be of high quality by an independent review panel of qualified experts or individuals with appropriate expertise to review the substantive content of the products and services. Program Performance Measure #2: The percentage of products and services deemed to be of high relevance to educational and early intervention policy or practice by an independent review panel of qualified members of the target audiences of the technical assistance and disseminations. Program Performance Measure #2: The percentage of products and services deemed to be of high relevance to educational and early intervention policy or practice by an independent review panel of qualified members of the target audiences of the technical assistance and disseminations. Program Performance Measure #3: The percentage of products and services deemed to be of high usefulness by target audiences to improve educational or early intervention policy or practice.Program Performance Measure #3: The percentage of products and services deemed to be of high usefulness by target audiences to improve educational or early intervention policy or practice.

O S E P TA&D Program Performance Measures Program Performance Measure #4: The federal cost per unit of technical assistance provided by the Technical Assistance and Dissemination program.Program Performance Measure #4: The federal cost per unit of technical assistance provided by the Technical Assistance and Dissemination program. Pilot Program Performance Measure: The percentage of projects that achieve proposed milestones within the approved timeline.Pilot Program Performance Measure: The percentage of projects that achieve proposed milestones within the approved timeline.

O S E P Program Performance Measures  Projects may customize the program performance measures to their project. However, the basic focus of the measure must remain the same and be clearly aligned with the program measure. 16

O S E P Program Performance Measures Program Performance Measure #1: The percentage of products and services deemed to be of high quality by an independent review panel of qualified experts or individuals with appropriate expertise to review the substantive content of the products and services.Program Performance Measure #1: The percentage of products and services deemed to be of high quality by an independent review panel of qualified experts or individuals with appropriate expertise to review the substantive content of the products and services. Example:Example: Each year of the project, an expert advisory group will rate a minimum of 90% of the center’s products as “high quality” (i.e. conceptually sound, based on evidence, and clearly communicated). 17

O S E P Program Performance Measures Program Performance Measure #3: The percentage of products and services deemed to be of high usefulness by target audiences to improve educational or early intervention policy or practice.Program Performance Measure #3: The percentage of products and services deemed to be of high usefulness by target audiences to improve educational or early intervention policy or practice. Example:Example: At the end of the second year of the project, 80% of families who participated in a webinar series will rate it as “useful” or “very useful” on written evaluation forms distributed within 3 months of the end of the series. 18

O S E P Program Performance Measures Program Performance Measure #4: The federal cost per unit of technical assistance provided by the Technical Assistance and Dissemination program.Program Performance Measure #4: The federal cost per unit of technical assistance provided by the Technical Assistance and Dissemination program. Choose a specific TA service rendered or a specific TA product that was developedChoose a specific TA service rendered or a specific TA product that was developed Calculate the cost to render or develop the service or product selected (direct costs + indirect cost)Calculate the cost to render or develop the service or product selected (direct costs + indirect cost) Identify the number of target audience units directly reached by the product or serviceIdentify the number of target audience units directly reached by the product or service Divide the cost by the number of target audience units reachedDivide the cost by the number of target audience units reached 19

O S E P Program Performance Measure Program Performance Measure #4: The federal cost per unit of technical assistance provided by the Technical Assistance and Dissemination program.Program Performance Measure #4: The federal cost per unit of technical assistance provided by the Technical Assistance and Dissemination program. Example:Example: The cost for developing and delivering a workshop on evidence-based math instruction will cost no more than $50.00 per attendee. 20

O S E P Program Performance Measures Pilot Program Performance Measure: The percentage of projects that achieve proposed milestones within the approved timeline.Pilot Program Performance Measure: The percentage of projects that achieve proposed milestones within the approved timeline. Example:Example: During the reporting period, project staff will achieve nine of the nine milestones (100%) within 2 weeks of their proposed due dates. 21

O S E P Types of Performance Measures  Project Measures that the grantee establishes to meet their project objectives Measures that the grantee establishes to meet their project objectives Project performance measures can address both the process of working towards an objective and the outcome related to meeting the objective Project performance measures can address both the process of working towards an objective and the outcome related to meeting the objective Ensure a mix of both process and outcome measures Ensure a mix of both process and outcome measures 22 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P High Quality Performance Measures High quality performance measures show:  What will change  How much change you expect  Who will achieve the change  When the change will take place 23 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P Project Performance Measure Examples (e.g.) Process measure (e.g.) In each of the 2 nd through 5 th years of the grant (when), project staff (who) will provide 15 (how much) training sessions (what) via distance education to paraprofessionals/interveners on providing individualized supports to children who are deaf-blind.

O S E P Project Performance Measure Examples Outcome measure (e.g.) At the end of the training sessions and practicum (when), 100% (how many) of paraprofessionals/interveners (who) will be highly qualified to provide individualized supports to children who are deaf-blind (what).

O S E P Project Performance Measure Examples Outcome measure (e.g.) At the end of each workshop (when), 85% (how many) of participants (who) will report increased knowledge of literacy practices (what) as measured on a post-training survey.

O S E P Project Performance Measure Examples Outcome measure (e.g.) At the end of the 5 th year of the project (when), State Leadership Teams in 3 states (who) will have overseen the full implementation of the intervention model (what) in a minimum of 3 districts (how many).

O S E P Project Performance Measure Examples Outcome measure (e.g.) By year 3 of the project (when), 75% (how much) of network members (who) will use the project’s website at least 3 times per year (what) for communication within the network.

O S E P Project Performance Measure Example (e.g.) Process measure (e.g.) By the end of the project period (when), project staff (who) will present on activities and outcomes of the project (what) at a minimum of 9 national meetings (how much).

O S E P Common Problems  Activities are NOT performance measures  If the best response is “Yes, we did that,” it is likely an activity (not a performance measure)  Activities:  Establish a coaching program  Hold an advisory board meeting  Conduct a workshop 30 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P Common Problems  Performance measures need to be measurable  Not measurable  Collaborative partnerships will be maintained  Evaluation will gauge teachers’ knowledge and project effectiveness  To increase the sustainability of the professional development model across the state 31 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P Additional Information on Writing Performance Measures  Grantees are encouraged to seek additional training on writing performance measures.  For further information on developing performance measures as well as logic models, see

O S E P Summary  Projects should have a few clear objectives that explain what the project is doing to support the overall goal  Each objective should have a few, specific performance measures to demonstrate how progress toward meeting the objective is being measured 33 Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University

O S E P Completing the 524B The ED 524B is a required reporting form with specific instructions.  The form is used by all ED grants and has been approved by the Office of Management and Budget (OMB). Project Directors must follow the directions listed in the Dear Colleague letter and ED 524B Instructions provided by OSEP  The 524B form can be found at: orms.html 34

x OMB No Exp. 02/28/

Reporting Period: For first year grants, the date is the beginning of the project year to February 28 th. For grants in years 2-4, it is the date from the end of the previous reporting period to February 28 th. ANNUAL PERFORMANCE REPORTS 36

ANNUAL PERFORMANCE REPORTS Budget Expenditures: Report the expenditures during the “Reporting Period.” Must be data or information from the business or grants office. 37

Signatory must have authority to sign on behalf of the institution since the grant is from the Department to the institution and not to an individual. Performance Measure Status: This will be checked “No” since OSEP is asking for data for the reporting period, not for the budget period. The date entered here will be the due date for your Final Performance Report; which is 90 days after the end of the grant. ANNUAL PERFORMANCE REPORTS 38

OMB No Exp U.S. Department of Education Grant Performance Report (ED 524B) Executive Summary (See Instructions) *** Provide highlights of the project's activities and the extent to which the expected outcomes and performance measures were achieved during the reporting period. Do NOT include the project abstract. PR/ Number # (11 characters)_______ ED 524BPage 2 of 11 EXECUTIVE SUMMARY SHEET 39

PROJECT STATUS CHART 40

Enter one of the project’s objectives; on subsequent pages, you will enter additional project objectives as submitted in your grant application. PROJECT STATUS CHART 41

Enter your PROJECT performance measures that show you are measuring progress toward meeting the objective. In addition, enter PROGRAM performance measures that align with the objective. PROJECT STATUS CHART 42

Here you identify if the performance measure is a project measure, “PROJ,” or a program measure, “PRGM.” Note: Program Measures refer to OSEP Measures required in all personnel development grants. Project Measures are unique to your grant. PROJECT STATUS CHART 43

Depending on your measure, enter either a raw number or a ratio and percentage. Enter the target number identified in the performance measure and then the actual data for this year. PROJECT STATUS CHART QUANTITATIVE DATA In this area of the page, enter information to explain the quantitative data, as well as activities the project engaged in to meet the objective. 44

QUALITATIVE DATA If the measure requires the collection of qualitative data, then leave the “Quantitative Data” section blank. In this area of the page, referencing the performance measure by number, report applicable qualitative data along with other information you wish to include. PROJECT STATUS CHART 45

Final Page of the Report Section B: Refer to the instructions for Section B in the ED 524B Instructions Section C: Include additional information (recruitment material, syllabi, evaluation instruments, journal articles) 46

O S E P Section B – Budget Information This section should not be blank. Please include: Actual expenditures for the reporting period.Actual expenditures for the reporting period. An explanation if you did not expend funds at the expected rate.An explanation if you did not expend funds at the expected rate. A description of significant changes to your budget resulting from modifications of project activities.A description of significant changes to your budget resulting from modifications of project activities. A description of any changes to your budget that affected your ability to achieve your approved project activities and/or project objectives.A description of any changes to your budget that affected your ability to achieve your approved project activities and/or project objectives. 47

O S E P Section B – Budget Information An explanation for any unexpended funds at the end of the current budget period and how you plan to use the unexpended funds [carryover] in the next budget period.An explanation for any unexpended funds at the end of the current budget period and how you plan to use the unexpended funds [carryover] in the next budget period. A description of any anticipated changes in your budget for the next budget period that require prior approval from the Department.A description of any anticipated changes in your budget for the next budget period that require prior approval from the Department. 48

O S E P Section C – Additional Information   If applicable, please provide a list of current partners on your grant and indicate if any partners changed during the reporting period or if you anticipate any change in partners during the next budget period.   Describe any changes that you wish to make in the grant’s activities for the next budget period that are consistent with the scope and objectives of your approved application. 49

O S E P Section C – Additional Information   If you are requesting changes to the approved Project Director and/or to other key personnel, please indicate the name, title and percentage of time of the requested key personnel. Attach a curriculum vitae for the proposed key personnel when you submit your performance report. Note: Note: Do not report on any key personnel changes that were already made during the current or previous budget period(s). Departmental approval must be requested and received prior to making key personnel changes.   Provide any other information about your project including unanticipated outcomes or benefits. 50

O S E P Submitting the ED 524B   Submit the ED 524B at   Signed ED 524B Cover Sheet must be scanned and ed to Kimberly Savoy-Brown at or faxed to her at   Two conditions require or hard copy submission rather than submission through e-Grants –   Grants in no-cost extension-submit to your Project Officer   Grants that have been front-loaded/forward funded last year need to be submitted to Kimberly Savoy-Brown with a copy to your Project Officer 51

O S E P Submitting the ED 524B Regular postal service: U.S. Department of Education, Office of Special Education Programs Attn: Kimberly Savoy-Brown, PCP – Room 5077 LBJ Basement Level Maryland Avenue, SW Washington, DC Hand delivery/Commercial Carrier: U.S. Department of Education, Office of Special Education Programs Kimberly Savoy-Brown th Street, SW Room 5077, Potomac Center Plaza Washington, DC Electronic submission is strongly encouraged; but mail delivery is allowed…

Contact your OSEP Project Officer with any questions! Due Date: April 15, 2011 Thank You!