1. Measuring Academic Achievement in Career- Technical Education Presented by: Sandra Pritz, Senior Consultant National Occupational Competency Testing Institute

2. Rationale for Standardized Assessments— technical and academic External objective data of student achievement Comparative data for schools, districts, states, and across states Perkins and NCLB core measures May be used by employers and postsecondary institutions

3. Value Added To Change What Is Taught and How to Teach “The administrator sits down with teachers and goes over the results to determine changes that may be needed in the curriculum.” (Teacher Sandra Himes, PA) “I love the data and detail to be able to improve instructional strategies.The results are charted, show technical gains. If the group shows highs and lows, you can work on the lows. Teachers see where they missed and work on different approaches.” (Administrator Ed Haynor, MI)

4. Why should career-technical programs assess academic skills? State/national academic standards demand attention. Career-tech is accountable for academic gains. High school students must pass academic tests for graduation. Academics in context tend to be student-friendly Workplace demands for technical literacy are increasing.

5. Technical Literacy: integrated academics and technical content at basic, proficient,and advanced levels Read, understand, and communicate in the language of a career field Understand technical concepts and principles Use academic knowledge and skills to solve problems Use basic technology

6. Academic Skills Measured in Context Academic Skills + Technical Skills + Employability Skills Comprehensive Assessment Academic Skills Employability Skills Technical Skills

7. How Should Career-Technical Programs Assess Academic Skills? In the context of the students’ career- technical program In alignment with state and national academic standards In alignment with post-secondary program requirements Embedded In their career-technical tests, such as end-of-course or program

8. Testing Academic Skills in Context Math, science, and language arts are required to complete most occupational items and performance tasks. NOCTI tests occupational competencies, but also tests embedded academics. Embedded academics are correlated to national standards (NCTE, NCTM,NSES). Student scores can be isolated by discipline.

9. Academics Identified Written tests contain embedded academics that match standards in all three disciplines. Performance tests contain a higher percentage of embedded academics than written tests. Students tend to score lower on the written than on the performance test, and the items most often missed involve technical literacy.

10. Ways NOCTI can help By providing performance as well as written tests, NOCTI can help assess academics at higher levels. NOCTI can respond to the need to reinforce academic instruction through assessment, can customize tests and test reports. NOCTI can provide states the service of cross-correlating its tests to their state standards.

11. Agricultural Mechanics Example from the Agricultural Electrical Power & Processing Duty: The completed path through which electricity flows is called the A. fuse B. circuit C.circuit breaker D. ground wire -NCTE Language Arts: Apply strategies to comprehend, interpret, evaluate, and appreciate texts. Use experience, interactions with readers/writers, meaning of words, word identification, and features, e.g., sound-letter correspondence, sentence structure, context, graphics. -National Science Education (NSES) Standards:Transfer of energy

12. Workplace Readiness Example: You earn $8/hour and work 40 hours/week. You receive time and a half for overtime and you worked five overtime hours last week. What was your gross pay last week? A.$350 B.$360 C.$380 D.$390 -NCTM Math: Select/use various types of reasoning and methods of proofs; Apply/adapt a variety of appropriate strategies to solve problems. -South Dakota Math, grades 9-12:Use numbers in a variety of equivalent forms to solve problems; Use properties of the number system to solve problems. -South Dakota Language Arts, grades 9-12 :Use various reading cues/strategies to comprehend text

13. PA Assessments… Math Standards Alignment –8 th, 11 th, and National Standards –Standardized paper and pencil doesn’t fit everyone—use written and performance. –Students show improvement using CTE methodologies. Criterion Referenced Scores –Advanced, Proficient, Basic, Below Basic –Aligns with State Assessment –Top 20 Tests & 200 Pennsylvania Employers

14. Other IA Assessments in demonstration projects… South Dakota –Workplace Readiness and Carpentry –Language arts, math, and science –State and national standards – 8 th and 9-12 th grade standards –Written and performance Oklahoma –27 different tests on 11 campuses –14 math and 10 science selected –State standards with national as available –9-12 th grade standards –Written at all sites, performance at sites per their selection

15. What Do I Receive from NOCTI? Score Reports –Coordinator’s Report –Individual Student Reports –Customized Reporting Available— e.g. embedded academics Statistical Information –Group Data –Site Data –State Data –National Data Student Certificates Personalized Contact Assistance All results, including integrated academics, are broken down into duty, category and total scores. Note:

16. Pre and Post Test Gain Analysis

19. Program Improvement Focus on industry standards –Plan program and curriculum changes –Assign “just-in-time” lab projects Expand emphasis on technical literacy at proficient levels –Reading career -technical material –Interpreting technical information –Calculating in technical problems

20. Student Achievement Improvement Identify gaps—group and individual-- and target instruction to them Heighten awareness of responsibility for learning per strengths/weaknesses Focus on academic and employability skills

21. Workplace Readiness Test NOCTI Workplace Readiness Test measuring employability skills is now up-scaled. A large and stellar Subject Matter Expert Team has culled and rated the critical competencies. Career Cluster Foundation Skills are assessed.

22. Workplace Readiness Assessment CommunicationsSystems Information Technology Applications Problem Solving and Critical Thinking Safety, Health and Environmental Leadership, Management and Teamwork Ethics and/or Legal Responsibilities Employability and/or Career Development

23. Workplace Readiness Assessment 82 item multiple-choice assessment; parallel forms with.94 internal consistency/reliability 41-item version to accommodate single-period schedules; no sub- scores;.96 reliability Career Skills middle school 41-item version Online delivery format available for all versions

24. To discuss your interests,visit www.nocti.org, call 1-800-334-6283, or write spritz@nocti.org Thank You