1. © 2003 VTECS 2003 NAWI Annual Conference: Leading Change in Career & Technical Education 2003 NAWI Annual Conference: Leading Change in Career & Technical Education General Session II Panel – What’s New in Assessment Pilot Testing Cross Cluster Skills General Session II Panel – What’s New in Assessment Pilot Testing Cross Cluster Skills Presenter: Brenda Hattaway Assistant Executive Director

2. © 2003 VTECS ANY COMMON KNOWLEDGE & SKILLS? Arts, Audio-Video Technology, & Communications Health Science Information Technology Manufacturing Transportation, Distribution & Logistics “The Eleven” Arts, Audio-Video Technology, & Communications Health Science Information Technology Manufacturing Transportation, Distribution & Logistics “The Eleven” We decided to find out! We decided to find out!

3. © 2003 VTECS WHERE WILL WE FIND COMMON KNOWLEDGE & SKILLS? WHERE WILL WE FIND COMMON KNOWLEDGE & SKILLS? Communications Communications Employability & Career Development Employability & Career Development Information Technology Information Technology Communications Communications Employability & Career Development Employability & Career Development Information Technology Information Technology Foundation Knowledge & Skills... Common Across All Clusters In... Common Across All Clusters In... Foundation Knowledge & Skills... Common Across All Clusters In... Common Across All Clusters In...

4. © 2003 VTECS YES! We identified 23 skills that are consistently addressed in all 16 career clusters. YES! We identified 23 skills that are consistently addressed in all 16 career clusters. DID WE FIND ANY COMMON KNOWDEDGE & SKILLS?

5. © 2003 VTECS Develop and interpret tables, charts, and figures to support written and oral communications. Develop and interpret tables, charts, and figures to support written and oral communications. AN EXAMPLE from COMMUNICATIONS COMMUNICATIONS Skill F02.7

6. © 2003 VTECS How can we DocumentProficiencyAttainment? DocumentProficiencyAttainment?

7. © 2003 VTECS Let’sDevelopATEST!!Let’sDevelopATEST!!

8. And That’s How The Great Adventure Began. And That’s How The Great Adventure Began. AssessmentAssessment AssessmentAssessment AssessmentAssessment

9. © 2003 VTECS WHAT ITEM FORMAT?

10. © 2003 VTECS DevelopItemsDevelopItems Scenarios (Performance) Scenarios (Performance) with Rubrics with Rubrics Situation Situation Multiple Choice Multiple Choice Scenarios (Performance) Scenarios (Performance) with Rubrics with Rubrics Situation Situation Multiple Choice Multiple Choice

11. © 2003 VTECS Establish Validity...... of the Assessment Instruments by: ~ Linking Each Item to a Validated ~ Linking Each Item to a Validated Knowledge/Skills List Knowledge/Skills List... and of the Assessment Procedures by: ~ Providing a Test Administration ~ Providing a Test Administration Manual & Materials Manual & Materials

12. © 2003 VTECS Career Cluster Assessment Employability F09.3 Career Cluster Assessment Communications F02.7 Career Cluster Assessment Communications F02.7 Then…. We Designed the Materials We Designed the Materials For the Pilot Test Package. For the Pilot Test Package.Then…. We Designed the Materials We Designed the Materials For the Pilot Test Package. For the Pilot Test Package.

13. © 2003 VTECS What’s in the pilot test package? Packing Slip Test Administration Manual Pre-paid “Label” for Material Return For each Common Skill Assessment: Security Checklist Test Administrator’s Report Form Test Booklets Answer Sheets Rubric Sheets for Scenarios Packing Slip Test Administration Manual Pre-paid “Label” for Material Return For each Common Skill Assessment: Security Checklist Test Administrator’s Report Form Test Booklets Answer Sheets Rubric Sheets for Scenarios

14. © 2003 VTECS Test Booklets & Answer Sheets Test Booklets & Answer Sheets

15. © 2003 VTECS A Rubric For Each Scenario Criteria NoviceApproaching Proficiency ProficiencyAbove Proficiency Develop tables, charts and figures to support written and oral communication.  Facts were grouped in a logical order but charted in line or pie chart rather than a bar chart.  Did not document sources of data.  Chart only used lines and was not labeled.  Chart did not include most of the elements to be published—bar chart, color coded with labels, horizontal and vertical axis in readable increments, footnotes and no errors in data interpretation or in spelling.  Compiled facts and arranged in chart but did not use a bar chart.  One source of data was documented.  Bar chart was completed in shades of one color. Axis and bars were not all labeled.  Chart included at part of the elements to be published—bar chart, color coded with labels, horizontal and vertical axis in readable increments, footnotes and no errors in data interpretation or in spelling. Had to be returned for additional work before it met publication standards.  Compiled facts and arranged in an organized manner in a bar chart.  All sources of data were documented.  Bar chart was color coded with labels for each bar. Axis was labeled.  Chart included all the elements to be published— bar chart, color coded with labels, horizontal and vertical axis in readable increments, footnotes and no errors in data interpretation or in spelling.  Compiled facts and arranged in an organized and eye-catching manner in a bar chart.  All sources of data were documented with footnotes or references indicated on chart.  Bar chart was color coded with labels for each bar. Axis were identified in easy to read increments.  Chart included all the elements to be published— bar chart, color coded with labels, horizontal and vertical axis in readable increments, footnotes and no errors in data interpretation or in spelling. Used clever and interesting business style. COMMUNICATIONS Skill F02.7

16. © 2003 VTECS TEST ADMINISTRATION MANUAL TEST ADMINISTRATION MANUAL

17. © 2003 VTECS Package&Deliver Test Materials To Pilot Sites Package&Deliver Test Materials To Pilot Sites Conduct Pilot Test Conduct

18. © 2003 VTECS Score pilot testsScore pilot tests and compile scores. and compile scores. Score pilot testsScore pilot tests and compile scores. and compile scores. Score Pilot Test Score Review ResultsReview Results Perform appropriate statistical analyses.Perform appropriate statistical analyses.

19. © 2003 VTECS Review/revise/delete itemsReview/revise/delete items Review/Revise TestReview/Revise Test Administration Procedures Administration Procedures Review/revise/delete itemsReview/revise/delete items Review/Revise TestReview/Revise Test Administration Procedures Administration Procedures Review/ReviseReview/Revise

20. © 2003 VTECS A Session on the Pilot Test of the Common Skills will be presented at: For More Information & to Register On-Line Go to www.v-tecs.org A Session on the Pilot Test of the Common Skills will be presented at: For More Information & to Register On-Line Go to www.v-tecs.org