1. Technical Skill Assessments Overarching Issues Common Concerns Intended State Approaches Critical Definitions Data Quality Institute Savannah, Georgia May 19, 2007

2. Overarching Issues: Topics Don Hilber, DTI –Technical Skills Assessment Study Group: advice for DQI agenda / associated research –Results of Questions for States: issues of greatest importance; stage of development; intentions in transition plan & beyond –Framework for approaching tech skills indicators Bob Sheets, U of IL –Recap recommendations from 2005-2006 DQI –Perkins IV realities – what still applies and what does not –Facilitated discussion; Q & A

3. Technical Skill Assessment Study Group Spun off of Next Steps Working Group 25 states, OVAE, associations and consultants – about 50 get notices Accountability / data experts and curriculum / assessment expert; some administrators Eleven calls starting Jan 10, 2007 Invited private companies to address group and take questions Informed DQI; desired other research = knowledge of others, (deferred) inventory

4. Study Group: Most Active States AlaskaSecondaryNorth CarolinaSecondary ArizonaSecondaryOhioBoth ArkansasBothOklahomaSecondary FloridaSecondaryOregonSecondary HawaiiSecondarySouth CarolinaSecondary IndianaPostsecondaryTennesseePostsecondary IowaSecondaryTexasBoth MichiganSecondaryUtahBoth MinnesotaPostsecondaryVirginiaSecondary New YorkPostsecondaryWisconsinBoth

5. Experts Invited to Address Technical Skill Attainment Study Group NOCTI (Foster +) ACT (Klemme +) VTECS (McCage, Hattaway) XPAND Corp (Osman, Butler) OSU (Mahlman +) ABG / Skills USA (McClelland +) Others Post-DQI ?

6. Questions Asked of States Level of Experience with Technical Skill Assessments – Secondary and Postsecondary Details from Experienced States Importance of 16 Issues Intentions as to: Methods, Types of Skills, When Assessment Will Occur Numerator / Denominator in Transition Plan – if Different than Guidance

7. Experience Among States Using Technical Skill Assessments SecondaryPostsecondary Very Experienced; no need to make alterations 51 Somewhat Experienced; will need to make alterations 52 Experienced – with some methods or program areas 814 Just Starting to Gain Experience or Familiarity 1210 No Experience Whatsoever 57 Did Not Submit 17

8. Alternative to Guidance 23 of 35 States (65%) Following Guidance on Technical Skill Attainment for Both Secondary and Postsecondary Secondary Exceptions = 7 States –2 completion and 2 GPA (permanent or interim?) –3 specific changes – year or numerator Postsecondary Exceptions = 9 States –2 completion and 1 GPA (permanent or interim?) –2 waiting for data from colleges –3 specific changes to student definitions, year or numerator

9. Issues of Importance – Top Tier Definition of Technical Skill Determining Phase-in / Interim Reporting Defining / Specifying Validity and Reliability Data Collection and Reporting End of Course / End of Program Assessments Development / Administrative Costs Value of Assessment to Student Student Definitions (Concentrator) Suitability of Assessment to Perkins Alignment to Standards All of these 60%+ Critical or Very Important

10. Issues of Importance – Second Tier Distinguishing Among Foundational, Generic or Occupational-Specific Skills (may have confused w/ “Definition of Technical Skill”) Ability of and Devices for States to Collaborate and Pool Resources Dealing with Local v State Development or Selection of Assessments Developing a Seamless System for Secondary and Postsecondary Relationship Among Perkins Indicators Clarifying Clusters, Pathways and Programs of Study

11. Technical Skill Attainment: A Framework WHO needs to be measured? WHAT is to be measured? WHEN should measurement occur? HOW can measurement be accomplished? In general order of most uniformity to most flexibility

12. WHO Needs to be Measured? Student Definitions: –Concentrator –Numerator / Denominator (also WHEN) Presented: Day One (OVAE) Discussed: Day One (Definitions) and Overarching Issues (Tech Skill Attainment Measure)

13. WHAT is to be Measured? Definition of Technical Skill Relationship of Technical Skill Attainment to Other Perkins Indicators: Academic, Completion Distinguishing Among Foundational, Generic and Occupational Skills Clarifying Clusters, Pathways and Programs of Study Presented: Day One (OVAE) and Two Discussed: Breakouts One (S & PS) and Two (Similar States)

14. WHEN Should Measurement Occur? Data Collection and Reporting Determining the Phase-In Period and Interim Reporting End of Course versus End of Program Developing a Seamless System for Secondary and Postsecondary (also HOW) Presented: Day Two - Panels Discussed: Breakouts One (S & PS) and Two (Similar States)

15. HOW Can Measurement be Accomplished? Defining Validity / Reliability Alignment to Standards Suitability to Perkins Value to the Student Local v State Development / Selection Development / Administrative Costs Pooling / Sharing Resources Presented: Day Two - Panels Discussed: Breakouts One and Two

16. DQI: Past and Present Quick Review of 05-06 Data Quality Institute Perkins IV Impact on: –Student Definition = Concentrator –Secondary Technical Skill Attainment Numerator and Denominator –Postsecondary Technical Skill Attainment Numerator and Denominator Clarification on Overarching Issues –Student Definitions (Recap of Day One) –Phase-In Period and Interim Reporting: How? –Definition of Technical Skill Versus Academic Skill and Completion (Secondary) Versus Credential, Certificate or Degree (Postsecondary)