The Minnesota CTE Assessment Project Building a CTE Assessment System for Student Results and Program Improvement

Minnesota CTE Assessment Meeting Agenda March 6, 2009 Review of Project GoalsReview of Project Goals Review of Overall Project PlanReview of Overall Project Plan –Process –Timeline Areas for Minnesota InputAreas for Minnesota Input QuestionsQuestions

Minnesota CTE Assessment Project Goals Build consensus around shared expectations as to what students should know and be able to do in Minnesota’s CTE programsBuild consensus around shared expectations as to what students should know and be able to do in Minnesota’s CTE programs Develop valid and reliable CTE “Assessment Blueprint” for CTE programsDevelop valid and reliable CTE “Assessment Blueprint” for CTE programs Ensure the system provides useful, timely and accurate feedback to teachers, administrators, students and employersEnsure the system provides useful, timely and accurate feedback to teachers, administrators, students and employers

Carl D. Perkins Career and Technical Education Act of 2006 Each state established a performance accountability system with multiple measures of student learning, program completion, and transitions to further education, employment and the militaryEach state established a performance accountability system with multiple measures of student learning, program completion, and transitions to further education, employment and the military Perkins III allowed wide flexibility in how to measure “technical skill attainment”Perkins III allowed wide flexibility in how to measure “technical skill attainment” Perkins IV requires a more focused assessment approach for technical skill attainmentPerkins IV requires a more focused assessment approach for technical skill attainment

Technical Skill Attainment -- Secondary Sec 113 (b)(2)(A)Sec 113 (b)(2)(A) …”core indicators of performance…that are valid and reliable… measures of each of the following:”…”core indicators of performance…that are valid and reliable… measures of each of the following:” “Student attainment of career and technical skill proficiencies, including student achievement on technical assessments, that are aligned with industry-recognized standards, if available and appropriate.”“Student attainment of career and technical skill proficiencies, including student achievement on technical assessments, that are aligned with industry-recognized standards, if available and appropriate.”

Technical Skill Attainment -- Postsecondary  Sec 113 (b)(2)(B)  …”core indicators of performance…that are valid and reliable…measures of each of the following:”  “Student attainment of career and technical skill proficiencies, including student achievement on technical assessments, that are aligned with industry-recognized standards, if available and appropriate.”  “Student attainment of an industry-recognized credential, a certificate, or a degree.”

Options to fulfill Perkins IV are: Use Industry-Based CertificationsUse Industry-Based Certifications Use other developed standardized- assessments (NOCTI, Skills/USA, etc.)Use other developed standardized- assessments (NOCTI, Skills/USA, etc.) Develop valid and reliable assessments or assessment processes through a statewide collaborative processDevelop valid and reliable assessments or assessment processes through a statewide collaborative process

Putting First Things First FIRST, decide WHAT to assess THEN decide HOW to assess

Skill Layers in Career Development Courtesy of Steve Klein & MPR Associates

Can One Assessment Measure It All? No matter the approach, it is inherent that the program will include applied academic skills, employability skills, cluster- and pathway-level skills, and program/occupation skills. When considering assessments, consider if one assessment can adequately measure all those skills. Industry certification test State test from National Item Banks Commercial employability skills test Program of Study

Evaluating Assessmen t Mechanisms How to measure and when?How to measure and when? a) Review of existing assessments/item banks b) Adopt and/or develop assessments as necessary c) Document technical quality of assessments Critical features:Critical features: a) Measure what is important b) Useful and timely feedback to stakeholders c) Fair and accurate measures (e.g., reliable & valid)

Project Plan: The Process Gather input and insight from key CTE leaders and teachers across the state.Gather input and insight from key CTE leaders and teachers across the state. Launch on-line survey to identify current methods of skill assessmentLaunch on-line survey to identify current methods of skill assessment Identify options for reporting on technical skill attainment in the short-term and long-term.Identify options for reporting on technical skill attainment in the short-term and long-term. Decide on how best to engage business and industry representatives for each of the relevant Career Clusters.Decide on how best to engage business and industry representatives for each of the relevant Career Clusters.

Project Plan: The Process (con't.) Convene subject matter expert groups of teachers and college faculty to identify core competencies in each of 4 designated career pathways/programs of study.Convene subject matter expert groups of teachers and college faculty to identify core competencies in each of 4 designated career pathways/programs of study. Develop an “Assessment Blueprint” for each of 4 selected career pathwaysDevelop an “Assessment Blueprint” for each of 4 selected career pathways Provide recommendations on how the State should proceed in developing an assessment system.Provide recommendations on how the State should proceed in developing an assessment system.

Initial Programs of Study for Review and Development Plant Systems (part of the Agriculture, Food and Natural Resources Cluster)Plant Systems (part of the Agriculture, Food and Natural Resources Cluster) Business, Financial Management and Accounting (part of the Business, Management and Administration Cluster)Business, Financial Management and Accounting (part of the Business, Management and Administration Cluster) Therapeutic Services (part of the Health Science Cluster)Therapeutic Services (part of the Health Science Cluster) Manufacturing Production and Process Development (part of the Manufacturing Cluster)Manufacturing Production and Process Development (part of the Manufacturing Cluster)

Phase 2: Programs of Study for Review and Development The second four Working Groups will be convened later in Network Systems (part of the Information Technology Cluster)Network Systems (part of the Information Technology Cluster) Diagnostic Services (part of the Health Science Cluster)Diagnostic Services (part of the Health Science Cluster) Law Enforcement Services (part of the Law and Public Safety Cluster)Law Enforcement Services (part of the Law and Public Safety Cluster) Restaurant and Food/Beverage Services (part of the Hospitality and Tourism Cluster)Restaurant and Food/Beverage Services (part of the Hospitality and Tourism Cluster)

Questions for Discussion and Survey

What type(s) of assessments are currently being used in your program? Teacher-developed tests based on local curriculumTeacher-developed tests based on local curriculum Teacher-developed tests based on national association and/or industry standardsTeacher-developed tests based on national association and/or industry standards Pathway Assessments -- developed by national associations, aligned with industry standardsPathway Assessments -- developed by national associations, aligned with industry standards Certification exams developed by Business and Industry (companies and organizations)Certification exams developed by Business and Industry (companies and organizations) State-licensing exams or assessmentsState-licensing exams or assessments

What percentage of the assessments you use involve: OnlineOnline Paper / pencilPaper / pencil Demonstration of skillsDemonstration of skills What percentage of your programs are currently covered by pathway assessments, industry certifications and/or licensing exams? Do your business/industry advisory boards advise you on assessments that should be used?

Do any of your programs use skill assessments as a measure to award postsecondary credit for high school students (similar to the College Board AP test?) Should secondary assessments be used in this way?

Do teachers currently review the results of external assessments? Have teachers ever received training in test item analysis? Do teachers modify their teaching / curriculum based on assessment results?

Thank you for your participation and input! Questions and comments? What have we overlooked? Other advice?

For more information, contact: Hans Meeder: Hans Jennifer Grams: