1. Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data

2. Overview of NOCTI Over 40 years serving the CTE community Non-profit organization Owned and operated by the 50 states Works with both secondary and post-secondary populations The Whitener Group (a NOCTI subsidiary) provides services to industry

3. NOCTI Customer Breakdown

4. Industry Connections

5. NOCTI Products Supplemental Products Over 80 Job Ready tests offered 30 Pathway tests in 10 clusters available Over 90 private industry employment and/or advancement tests delivered 20 teacher-level assessments Assessments Study guides for all Job Ready and Pathway assessments Countdown Kits offered for 40 titles Successful Schools-Successful Programs Guide

6. NOCTI Job Ready Assessments Assess technical skills at the occupation level (e.g., Accounting, Carpentry, Pre-Engineering) Measure aspects of occupational competence such as factual and theoretical knowledge Target secondary and post-secondary program completers

7. Assessment Blueprints Blueprints include: Competency List Content Distribution Sample Items and Jobs

8. NOCTI Job Ready Assessment Blueprint Number of Questions: 222 Areas Covered:Accounting

9. Assessment Blueprints to Prepare How can I prepare my students for the NOCTI assessment? Assessment Blueprints!!! Ideas for using blueprints in the classroom: Use throughout the school year as a syllabus and guide for review Involve the students: have them calculate the number of questions in each content area Send a copy home with students to share with parent

10. Assessment Results NOCTIs Standard Score Reporting Package: Individual Score Reports Group Score Reports Comparative Data Other Reports Available: Task-Link Reports Integrated Academic Reports

11. NOCTI Standard Score Reports Individual Score Reports Provided for each test taker Scores by duty and total Pre-/post-test comparisons Comparative data Individual Score Reports Provided for each test taker Scores by duty and total Pre-/post-test comparisons Comparative data

12. NOCTI Standard Score Reports Group Score Report Provided for the testing group Duty and total scores for each participant Group Score Report Provided for the testing group Duty and total scores for each participant

13. NOCTI Standard Score Reports Analysis of Scores Report Provided for the testing group Comparative data Criterion- referenced cut score Other statistical information Analysis of Scores Report Provided for the testing group Comparative data Criterion- referenced cut score Other statistical information

14. Using Data to Improve Instruction

15. Collect Data: Become familiar with your data

16. Understanding Your Data School Program Teacher Class Student Standards National State District/System

17. What the Data Reveal Achievement gaps Differences between subgroups Trends Progress

18. Assessment Includes process of collecting, analyzing, and interpreting information Determines the degree to which students are becoming competent and meeting standards Occurs at many levels (e.g., classroom and program assessments to state and national assessments)

19. Good Assessments Valid Test items measure what they are supposed to measure: the important content or concepts that were taught and that the student must know Test is relevant and based on current standards Subject matter experts develop and evaluate test items

20. Good Assessments Reliable Test and its items produce a similar score over time Scores are consistent, not contradictory Scores are not affected by chance

21. Student Assessment On-going with frequent opportunities for feedback Use a variety of different types of assessment (e.g., written and performance, formative and summative) during the year

22. Summative Assessment Pretests (covering all major content areas) are used to identify entry skills and measure gain End-of-program or annual test Broader in scope, for comparison of groups

23. Formative Assessment Continuous correction from quizzes, homework, projects, participation Use practice items for instructional revision before summative assessment

24. Building an Assessment Test specification matrix or Blueprint National Standards Duties and Tasks Written/Performance Weighting Panel of Experts

25. Step 2 Analyze Data: Draw conclusions from data

26. Classroom Example A Carpentry Class

27. Areas on a NOCTI Carpentry Test Safety Tools and Accessories Blueprint Reading and Estimation Foundations, Forms and Concrete Rough Framing Exterior Finish Interior Systems Installation Interior Finish Carpentry Related Mathematics

28. Individual Carpentry Pre-Test Carpentry Pre-test Areas Pre-test % Correct Student 1 Post-test % Correct Student 1 Safety 54.6 -- Tools and Accessories 30.0 -- Blueprint Reading and Estimation 41.1 -- Foundations, Forms, and Concrete 36.4 -- Rough Framing 53.8 -- Exterior Finish 36.4 -- Interior Systems Installation 40.0 -- Interior Finish 27.3 -- Carpentry Related Mathematics 35.9 -- Total 40.0

29. Carpentry Class Pre-Test Carpentry Pre-test Areas Student 1 % Correct Student 2 % Correct Student 3 % Correct Student 4 % Correct Student 5 % Correct Student 6 % Correct Class Average % Correct Safety 54.645.536.427.3 54.540.9 Tools and Accessories 30.020.010.020.010.030.020.0 Blueprint Reading and Estimation 41.117.623.5 29.435.328.4 Foundations, Forms, and Concrete 36.454.545.5 36.454.545.5 Rough Framing 53.838.530.838.530.853.841.0 Exterior Finish 36.4 27.336.4 45.536.4 Interior Systems Installation 40.060.040.060.0 80.056.7 Interior Finish 27.39.118.227.39.127.319.7 Carpentry Related Mathematics 35.945.527.336.4 54.539.4 Total 40.034.028.033.029.046.035.0

30. Carpentry Class Post-Test Carpentry Post-test Areas ParticipantGroupSiteStateNation Safety 100.086.479.572.575.5 Tools and Accessories 91.779.278.270.770.6 Blueprint Reading and Estimation 100.095.083.864.366.4 Foundations, Forms, and Concrete 92.369.265.851.652.9 Rough Framing 97.471.869.863.164.1 Exterior Finish 91.764.667.658.157.8 Interior Systems Installation 87.553.161.955.656.5 Interior Finish 87.579.775.666.667.0 Carpentry Related Mathematics 100.056.368.266.167.0 Total 94.372.071.462.763.4

31. Thank You! Questions? www.nocti.org