Pouring a Foundation for Program Improvement with Quality SPP/APR Data OSEP’s message regarding Indicators 1, 2, 13 and 14 - data collection and improvement strategies Ruth Ryder USDE/OSEP/MSIP

Updates Status of revisions to information collection (Indicator/Measurement Table) –Received recommendations that Indicator 13 be revised Longer Shorter Tweaked –In process

Updates Status of OSEP’s review of the APR and revised SPP submissions –Opportunity for Clarification –Response Table –Determinations –Letters in early June

OSEP Review Process State contacts did initial review Division did facilitated review Division leadership “triaged” all Status Tables Opportunity for Clarification Developing Response Tables

Indicators 1 and 2 Only a few States continued to do the comparison to all youth Many States are using 618 State-reported data Many States revised their improvement activities, usually adding more specific activities for “out years”

Indicator 1 and 2 Issues Some States could not provide data (05-06 data were provided) Great variations in calculation methodologies (more using cohort) Not even close to meeting targets Improvement activities – (“kitchen sink” approach or minimalist approach)

Indicator 13 All States submitted data –We questioned the validity and reliability of a few States State compliance ranged from 4.9% to 100% –About 15 States were below 50% compliance –Many States could not demonstrate timely correction of previously identified noncompliance

Indicator 13 Issues What exactly are States reporting to us? –More than half of the States are using the NSTTAC checklist or some variation –Remaining States are using their own checklists and it’s often hard to tell what requirements they are evaluating What does timely correction look like for this indicator?

Indicator 14 With a few exceptions, States were able to give us data About 8 States did not provide valid and reliable data –Denominator –Only graduates

Indicator 14 Issues What do the reported data represent? –Many States did not describe the respondent group –Can’t determine if the respondent group is representative of the population Small sample sizes Improvement activities focus on data collection

The Challenges: 2007 From our review of the Feb 2007 submissions we identified patterns of challenges – –The Basics – Data –Compliance –Improvement

The Successes and Challenges: 2008 Successes –The Basics – Much better, States provided the required information, etc. –Data – Much better, correct measurement, correct year –Compliance – More accurate data, more evidence of timely correction –Improvement Activities – Many States revised and/or added

The Successes and Challenges: 2008 Challenges –The Basics – Keep up the good work! –Data – Reconciling database data with monitoring system data, calculation methodologies for 1 and 2 –Compliance – Documenting timely correction, improving performance –Improvement Activities – Purposeful, linked, sequenced, evidence-based

Improvement Activities: External TA Analysis Categories Improve data collection and reporting Improve systems administration and monitoring Build systems and infrastructures of technical assistance and support Provide technical assistance/training/ professional development

(Continued) Clarify/examine/develop policies and procedures Program development Collaboration/coordination Evaluation Increase/adjust FTE

One State’s Perspective on Making the Grade with the SPP/APR Attend as many OSEP-funded TA offerings as possible Provide accurate and reliable data and if can’t, explain why and what you’re doing about it Analyze data by local programs Develop standard headings, stems and data formats to use for all indicators

One State’s Perspective on Making the Grade with the SPP/APR Maintain documentation that you: –Identify noncompliance at the local level –Identify research-based improvement activities that are a match to the identified problems –Require and approve corrective action plans with appropriate timelines –Oversee timelines and require proof of correction (evidence of success)

Examples

What You’re Doing is Working! From 1987 to 2003: –Postsecondary enrollment rose from 15% to 32% –4-year college enrollment rose from 1% to 9%

What You’re Doing is Working! More academic coursework More above-average grades More congruency between age and grade level More support services

What You’re Doing is Working! More students with disabilities are exiting with a standard diploma 1996 to 2006, rates rose from 42% to 56% Fewer students with disabilities are dropping out From , rates declined from 47% to 26%