1. Career-Technical Education VoCATS: WHAT’S IT ALL ABOUT? IT ALL ABOUT? VoCATS

2. This project is funded through federal Perkins legislation and assures equal opportunity regarding race, color, national origin, gender and disability. For North Carolina Educators Only February 2003 ©

4. The NC Instructional Management System for Career-Technical Education

5. What is the term, Career-Technical Education? Replaces the name Workforce Development Officially changed by NC State Board of Education January 2003

6. Keys for Student Improvement High expectations for all Alignment among: Instruction Assessment Curriculum

7. Who writes those test bank questions and other curriculum materials?

8. Who are curriculum team members?

9. Veteran teachers from across state Veteran teachers from across state Recommendation of Director Recommendation of Director From schools of varying size From schools of varying size & wealth & wealth Good writers with technology skills Good writers with technology skills Diversity considerations Diversity considerations Release by principal Release by principal

10. What do the curriculum teams develop for a course?

11. Competencies/Objectives Curriculum Support Materials Correlated Assessment Classroom Assessment Test Bank Classroom Assessment Test Bank Accountability Assessment Test Bank Accountability Assessment Test Bank ALL ALIGNED ALL COURSES NEED ALL PARTS

12. What does alignment of curriculum mean?

13. Blueprint

14. BLUEPRINT Basic structure of course Cognitive level of objectives to be taught Percentage of course for each objective (converts to time)

15. A - Art E - English, Language Arts CD - Career Development H - Healthful Living M - Math SC - Science SS - Social Studies

16. Blueprint Content Outline

17. CONTENT OUTLINE Bridge between blueprint and curriculum product and assessment Only outline topics will be assessed May contain other information References Performance requirements National standard correlation

18. Blueprint Curriculum Product Content Outline

19. CURRICULUM PRODUCT Developed or adopted Varies greatly by program area /course

20. Assessment Blueprint Content Outline Curriculum Product

21. ASSESSMENT Tests ONLY what is on content outline Accountability assessment at cognitive level of objective Classroom banks can have variety of cognitive levels

22. 1. Develop blueprint, outline & perhaps other evaluation materials 3. Pilot course if appropriate - revise 4. Develop or adopt curriculum product 5. Develop test item bank – 550+ questions 2. Validate/review by business and industry What is the process for developing a new course or revising an old course? (Process varies some with course and program area)

23. Test Item Development Process Teacher Modifies, Adds, & Removes Questions 300+ Classroom Assessment Questions CMS Conversion Completed CD Provided to Classroom Teachers 250+ Accountability Assessment Questions 550+ Test Items Changes reviewed by program area consultants Each question validated by 3-4 teachers Changes Made Test Administered CMS Conversion Completed Each Question VALIDATED by 3-4 Teachers & Content Consultant Changes Made Test Administered Changes Made RELIABILITY Studied Test Made & Edited by Consultants

24. How do we know the test questions are good?

25. All questions (accountability and classroom) are evaluated for validity by content experts. Who are content experts?

26. Truthfulness Does What We Say It Does What is Validity?

27. All Test Items Are Validated One at a time By at least three teachers/ subject matter experts For accountability assessment banks, participants see a maximum of 1/3 of the bank, in a secure environment.

28. Validation Follows Guidelines Does not contradict state-adopted texts Content valid (or correct) Tests blueprint & outline Appropriate reading level for grade One & only one correct answer Written clearly Language fair to all students Correct cognitive level for accountability banks

29. Validation evaluates test item difficulty. Too Easy Easy Appropriate Challenging Too Difficult All cognitive levels can have all levels of difficulty. Questions that are too easy or too difficult are eliminated.

30. Reliability is also studied for accountability assessment questions. Do classroom teachers do this? NO This is done by an internationally recognized testing expert.

31. What Is Reliability? Ability to Measure Consistently What kind of information is this? How is it used?

32. Q # Correct Answer How Students Answered Correct Incorrect Correct Incorrect Total % correct Reliability Coefficient Masters Non-Masters

33. Q # Answer How Answered Total % correct Reliability Coefficient Many students answered A & C Most masters not correct 20% correct Low # Delete Question Correct answer B Masters Non-masters

34. Correct answer C Most answer C All correct High coefficient Q # Answer How Answered Total % correct Reliability Coefficient Masters Non-masters 2/3 correct 83% correct

35. 1. Correct answer:____ 2. Most answered:____ 3. Masters answered:____ 4. Non-masters answered:____ 5. % correct:____ What do you know about question 5? Incorr 41% Corr & Incorr A B 6. What should be done? Q # Answer How Answered % correct Reliability Coefficient Masters Non-masters Delete Question

36. Q # Correct Answer How Students Answered Correct IncorrectCorrectlyIncorrect Total % correct Reliability Coefficient MastersNon-Masters

37. What about performance assessment ? Is this important??? Performance assessment of students is a critical part of evaluation. YES

38. C Current classroom assessment banks have almost 2000 performance evaluation items. lassroom assessment banks will continue to have at least one performance item for each objective with weight in the blueprint column 5.

39. Performance items are useful: Performance items are useful: evaluation tools. evaluation tools. activities for individual students activities for individual students or groups of students. or groups of students. Items have rubrics. Items have rubrics. Using performance assessment as part of student evaluation is a as part of student evaluation is a LOCAL DECISION. LOCAL DECISION.

40. How are CTSO activities incorporated into the curriculum?

41. CTSO Activities Teaching/Learning Strategy Integral Part of Curriculum

42. Show me how curriculum is correlated.

52. Classroom assessment questions and accountability assessment questions should directly test the blueprint and outline. They should also be correlated to student learning through instruction and other activities. Health Occupation Education 7222 Medical Sciences II Correlated VoCATS Classroom Assessment Questions

53. Which situation is malpractice if the patient is injured as a result of a health care workers actions? A A health care taker fails to wipe up some water that he/she spilled on the floor. B A health care worker forgets to put up the patient’s bedside rail. C A health care worker performs a special procedure on a patient that he/she is not instructed to perform. D A heating pad with a defective cord is used on a patient even though the health care worker knows it is defective.

54. A health assistant wishes to watch an operation on a patient, even though the assistant is not directly involved in the patient’s care. Whose permission is necessary in order for the assistant to be present during the operation? A Both the doctor and the patient B Doctor C No one D Patient

63. The menu above is monotonous because what is repeated? A. Color B. Texture C. Flavor D. Shape MENU Broiled Chicken Breast Pear Salad Cooked Carrots Stewed Squash Cornmeal Muffin with Butter Jell-O with Fruit Cocktail Milk How does the “temperature rule” of meals apply to the menu above? A. All foods should be served at the same temperature. B. All foods should be served on the same plate. C. The menu contains both hot and cold foods. D. The menu should be served only in the summer.

64. Who Decides How CTE Post-Assessment Scores Are Used In Student Grades? Use of student scores is an issue for: CLASSROOM TEACHERS SCHOOL ADMINISTRATORS LOCAL SCHOOL BOARDS USE OF SCORES IS A LOCAL DECISION

65. What is a teacher allowed to do in CMS ? Add questions to the bank Make improvements to questions How is this done?

66. Access login From your VoCATS Coordinator Teachers need administrator-level access to fully use the classroom assessment bank for instructional management.

67. To change the answer click on the correct answer Must be in DEVELOPMENT tab Select the question To edit the question, double click on this panel

68. After you double click the question appears in Word. Make changes. Close screen. Close screen when finished making changes

69. Remember the “Gold Standard” IS IT BEST FOR STUDENTS??

70. For Questions Contact: Rhonda Welfare, VoCATS Consultant 919.807.3876 rwelfare@dpi.state.nc.us rwelfare@dpi.state.nc.us Mary Jo Nason, VoCATS Consultant 919.807.3822 mnason@dpi.state.nc.us mnason@dpi.state.nc.us North Carolina Department of Public Instruction 6358 Mail Service Center Raleigh, NC 27699-6358 http://www.NCPublicSchools.org/workforce_development/vocats