Standardization in Perkins: Data Quality Institute Why and What ? The Perspective from Secondary Education Data Quality Institute June 14, 2005 Don Hilber Education Associate Data Collection and Analysis South Carolina Department of Education

Major Questions and Issues Why Standardize ? PART and NAVE Findings Relevant to Secondary Education What to Standardize ? Number Served Nationally Critical Performance Indicators [How to Best Standardize ?] Covered During Remainder of Institute Related Issues No Child Left Behind Standardization versus Alignment

“ Ineffective programs don’t get cut . . . they just get ignored ! “ WHY STANDARDIZE ? “ Ineffective programs don’t get cut . . . they just get ignored ! “ Effective programs stand better chance of: Additional Resources Greater Leverage (state or private $) Less Constant Scrutiny Less Defection

Standardization in Perkins: Why? (Secondary Education) Program Assessment Rating Tool Findings Multiple and Potentially Overlapping Objectives Deficiencies in Performance Reporting Lack of Numerical Targets That can be Aggregated State-Defined Targets Not Appropriately Rigorous States set Own Thresholds and have Different Definitions for Who is Voc Ed Student No Long-Term Numerical Targets Established, nor the Data to Report on Those Targets

Standardization in Perkins: Why? (Secondary Education) National Assessment of Vocational Education Findings Lack of Clarity Over Perkins Purpose and Goals = Too Many Indicators Present Conflicted Picture Secondary Voc Ed Confers Little Advantage in: High School Completion Academic Achievement Post-secondary Enrollment Post-Secondary Completion Secondary Voc Ed May Have Positive Impact on: Medium-Term Earnings Academic Course-Taking

Standardization in Perkins: Why? (Secondary Education) Observations on PART and NAVE Findings PART may be Outdated in Some Particulars -- US Dept of Education Now Has Targets, Reports Data PART & NAVE Ignore Provisions in Perkins that Specify a Large Number of Performance Indicators; NAVE Introduces More Secondary Indicators Neither Mentions “Alignment” with No Child Left Behind, the NCLB Lack of Similar Uniformity or the Need to Share Performance Indicators Another Dilemma: State Uniformity Means Delay in Baseline Setting and Hence Demonstrated Results

Standardization in Perkins: What? (Secondary Education) Key Data Shortcoming From PART Analysis = Lack of a Common Concentrator Definition Fosters Confusion over Who Perkins Serves Impedes Ability to Aggregate States’ Data Does Allow for Cost versus Performance Assessment Creates Undue Variability in State Baselines Performance Indicators Mentioned in PART Attainment of Diploma, Certificate or GED Entry in Employment or Post-Secondary Attainment of Literacy and Numeracy Skills

Standardization in Perkins: What? (Secondary Education) Key Finding From NAVE Analysis = Perkins Must Show Meaningful Outcomes Who Does Secondary Career Ed Serve? Occupational Investors versus Course-Takers Concentrators and/or Explorers = 3+ Units Perkins Performance Indicators in NAVE Academic Achievement = Courses Taken Academic Achievement = NAEP 12th Grade Test High School Completion Post-Secondary Enrollment (Short & Medium)

Standardization in Perkins: What? (Secondary Education) Other Performance Indicators in NAVE Post-Secondary Completion (of Secondary Students) Eventual Earnings (Seven Years After Graduation) Additional NAVE Implications No Concern for Non-Traditional Participation and Little Concern for Immediate Employment Remedial Education and Years to Complete Post-Secondary Might be Secondary Issues Also Tech Prep Increasingly Losing Distinctiveness Technical Courses Matter Once Academic Courses Met – So Why Not Standardize Technical Skill Also?

Related Issues for Secondary Ed Standardization Does Standardization = Alignment ? Two Views States Use Perkins Measures Consistent Among Each Other States Also Mesh These Measures with Other Federal Programs: NCLB &/or WIA To Meet NAVE & PART Criticisms only # 1 is necessary! (if true outcomes) To Meet Other External Pressures # 2 might also be desirable

Related Issues for Secondary Ed Standardization NCLB Alignment States Establish Own Tests and Levels States Set Own Intermediate Goals States Vary in Graduation Measures Possible to Align with Long-Term Goals of 100% Proficiency by 2014 (Target-Setting) WIA Alignment Youth Programs not Central to HS Reform Future Jobs Demand Post-HS Education Wage-Record Tracking Not Suited for Part-Time Earners, Continuing Students

Wrap Up Points for Discussion and Consensus Why Standardize ? To Become an Effective Program = Common Service; Additive Indicators; # Goals What to Standardize in Secondary Ed? CONCENTRATOR DEFINITION Academics and Graduation: YES Non-traditional: NO Technical Skill: WHY NOT? Placement – Immediate Education & Jobs: ? Additional Alignment ? NCLB in Spirit, not Specifics; WIA Low Priority