1. Southern Regional Education Board 1 Preparing for the 2014 HSTW Assessment Mike Ross Consultant, HSTW / MMGW SW Ohio Susanne Davis HSTW Assessment Coordinator, Ohio

2. Southern Regional Education Board 2 Topics Covered What is the HSTW Assessment? Administration Nuts & Bolts Best Practices for Successful Assessments

3. Southern Regional Education Board 3 What is the HSTW Assessment? Required in even-numbered years Subject tests in Reading, Mathematics and Science based on NAEP frameworks Student Survey Course-taking patterns School and classroom experiences Teacher Survey Provides schools with comprehensive school-level data breaking out student achievement by experiences

4. Southern Regional Education Board Examples of How Schools Use the Results Identify improvement needs Serves as basis for Site Review Process (SRP) Gain support for school improvement Engage the entire faculty in school improvement Set priorities and goals for school improvement Improve classroom practices 4

5. Southern Regional Education Board 5 New for 2014 There will be no individual Student Reports!

6. Southern Regional Education Board 6 Five Components Reading test Mathematics test Science test Student survey Course-taking patterns School and classroom experiences Teacher survey (online)

7. Southern Regional Education Board Testing Times Reading 75 min. Mathematics 80 min. Science 55 min. Total 3.5 hours Student Survey Section 1 - staff person completes Sections 2 & 3 - untimed (about 35-45 min.) 7

8. Southern Regional Education Board 8 Test Content Used 2009 NAEP frameworks as a starting point Modified to reflect college and career readiness goals Reading – focus on informational texts Mathematics – topics through Algebra II and some pre-calculus Science – life, physical and earth science

9. Southern Regional Education Board 9 Reading Test 75 minutes 80% informational texts; 20% literary nonfiction Includes meaning vocabulary Includes paired texts (to assess critical analysis) Does not include literary fiction or poetry

10. Southern Regional Education Board 10 Reading Test Passage length: 500-1500 words Cognitive targets 15% locate/recall 55% integrate/interpret 30% critique/evaluate

11. Southern Regional Education Board 11 Mathematics Test 80 minutes Scientific or graphing calculators permitted for one section Type of calculator will neither advantage nor disadvantage student Topics Pre-algebra through Pre-calculus, including Algebra II

12. Southern Regional Education Board 12 Mathematics Test Content 11% - Number Properties and Operations 29% - Measurement/Geometry 24% - Data Analysis, Statistics and Probability 36% - Algebra

13. Southern Regional Education Board 13 Mathematics Test Cognitive Targets 50% - Low Complexity recall routine procedures 45% - Moderate Complexity make connections within mathematics or between mathematics and real world 5% - High Complexity analyze a problem mathematically

14. Southern Regional Education Board 14 Science Test 55 minutes Content 43% - Life science 42% - Physical science 15% - Earth and space science

15. Southern Regional Education Board 15 Science Test Science Practices 70% Identifying science principles Using science principles 30% Using scientific inquiry Using technological design

16. Southern Regional Education Board 16 Student Survey Section 1 – Course Experience Staff member should complete this section for each student Based on Course Matching and Transcript Analysis Section 2 – School Experience Students complete this section School and classroom experiences Based on HSTW key practices Section 3 – General Information Students complete this section

17. Southern Regional Education Board 17 Teacher Survey Online survey! School and classroom experiences School mission High expectations and extra help Guiding and supporting students Curriculum content and engaging students in learning Transitions Leadership and professional growth

18. Southern Regional Education Board 18 Results Reporting 1.School Report (June, 2014) PDF file on CD 2.HSTW Award of Educational Achievement (by May 2, 2014) 3.Student Reports - Discontinued

19. Southern Regional Education Board 19 HSTW Assessment Report Mailed to schools June, 2014 Executive Summary -“3-minute report” One page summary of performance and implementation Overview -“30-minute report” Slightly longer summary Full report -“3-hour report” Tables breaking out achievement by experiences Teacher survey results

20. Southern Regional Education Board 20 School Report Executive Summary One-page snapshot Graphs and charts Includes: Implementation summary Percentage of students meeting readiness goals: 250, 257, 258 Percentage of students at each proficiency level Blank quadrant for state data

21. Southern Regional Education Board 21

22. Southern Regional Education Board 22 School Report The Overview Approximately 15 pages Graphs and charts Includes: Mean scores Percent meeting readiness goals Percent at each proficiency level Performance on HSTW indices More detailed implementation summary

23. Southern Regional Education Board 23

24. Southern Regional Education Board 24 School Reports Full report All those tables! Student achievement broken out by survey questions Teacher survey results

25. Southern Regional Education Board 25

26. Southern Regional Education Board 26 HSTW Award of Educational Achievement Award certificates mailed to schools by May 2, 2014 Awarded to students who: Meet all three readiness goals (reading 250, math 257, science 258) Complete at least 2 of 3 HSTW recommended curriculum goals English/language arts, mathematics, science Complete a concentration Career/technical, mathematics/science, humanities

27. Southern Regional Education Board 27 HSTW Award of Educational Achievement

28. Southern Regional Education Board 28 Award of Educational Achievement Opportunities for High Schools Local community colleges Local employers Additional incentives for students

29. Southern Regional Education Board 29 HSTW Assessment Administration Nuts & Bolts

30. Southern Regional Education Board 30 2014 Student Assessment and Teacher Survey Window January 24– February 14 (3 weeks) Plan ahead to get the best results!

31. Southern Regional Education Board 31 Conducting the Assessment Process Ordering Preparing Administering Returning materials

32. Southern Regional Education Board 32 Submit orders today if possible, but no later than Friday, Sept. 27. Do NOT order online! Do NOT send order to SREB! The good news : There is no cost to your school! HSTW SWO pays for the entire Assessment! Ordering the Assessment

33. Southern Regional Education Board 33 Unless there are special reasons, we encourage you to test no more than 60 students. You will receive extra test sets. However, when submitting scan sheets to be scored, do not exceed the number ordered! Schools testing more than 60 must pay for each additional student: $44 per student. A cancellation fee may be charged if you cancel testing or miss the return deadline. Other Ordering Information

34. Southern Regional Education Board 34 Test Administration Guide Sampling Instructions Student Roster Form Checklist Test Description and Sample Question Booklet Tasks and Timelines Course Description Guide Course Matching Chart Administration Materials Online www.sreb.org

35. Southern Regional Education Board Preparing for the Assessment Administration materials are now available online at: www.sreb.org Booklets & scan sheets will arrive by January 17, 2014 35

36. Southern Regional Education Board 36 Tasks & Timeline Per Regional Office instructions

37. Southern Regional Education Board 37 Identify Students for the Assessment Select one of two options: 1.RANDOM SAMPLE OF 60 SENIORS - OR - 2.ALL SENIORS (ENTIRE CLASS) Select a true random sample by using the instructions provided. You must also select five alternates at random. Preparing for the Assessment

38. Southern Regional Education Board 38 Other Tips for Selecting Students Only include currently enrolled seniors. Only exclude students if their IEP exempts them from state assessments. Same accommodations provided for state assessments should be provided for this assessment.

39. Southern Regional Education Board 39 Other Tips for Selecting Students Administrator Guide will contain suggested accommodations for English-language learners. Contact ETS by October 25 to request large-print or audio CD versions. Be reasonably assured that all students can complete ALL 4 components – reading, math, science, AND student survey.

40. Southern Regional Education Board 40 Random Sampling Options What is a “random sample”? What is the easiest way to draw a random sample? How to use random number tables

41. Southern Regional Education Board Random Sampling Simple random sample - every time a selection is made, every student on the list has an equal chance of being the one selected. Selection Bias – anything that contaminates or distorts the process.

42. Southern Regional Education Board What’s the easiest way to select a random sample? Draw names from a hat, box, bowl, or drum! Draw until you have selected 60 + 5 names! 42

43. Southern Regional Education Board 43 Group Activity Random Sampling Procedures: Random Number Table 1.Develop a list of all eligible students (delete names of students who moved or have IEP’s stating they may not participate). 2.Number the remaining names of students on the list 3.Use the random number sheet to select students, starting with column one and working down. 4.The table provided works for classes of up to 350. For larger classes, obtain a larger random number table from SREB. 5.Add student names to the roster until you have the appropriate number.

44. Southern Regional Education Board CAREER CENTERS and HOME SCHOOLS: How to handle “double-selected” students 44 Career Center selects random sample. Home School selects random sample. Center and home school should compare lists to see if any students were “double-selected” (i.e., randomly selected by BOTH schools). If a student is double-selected, test the student at the site where he/she takes senior academic courses. Contact ETS (Lisa Rion – lrion@ets.org 609- 734-5646) to obtain a special student code. Enter that special code on the double-selected student’s answer document. That student’s results will go into both reports.

45. Southern Regional Education Board 45 Preparing for the Assessment Complete the Student Roster - VERY IMPORTANT!! Fully complete form including assessment answer document ID numbers. File with school testing records for five years! Also give one copy to principal. You will need this fully completed roster form in Spring 2015 to conduct the follow- up survey of graduates Also needed to identify students who fail to include their names (for student reports and awards)

46. Southern Regional Education Board Student Roster 46

47. Southern Regional Education Board 47 Preparing for the Assessment Notification Notify students, parents and teachers Provide information on assessment and/or schedule informational meeting Refer to your Board of Education policy where applicable Scheduling Schedule time and place (SREB recommends administering over 2-3 days) Include make-up sessions Identify assistants (follow recommended ratios)

48. Southern Regional Education Board 48 Preparing for the Assessment Course-Taking Preparation Work with department chairs to complete course-matching chart (may update previous work) Prepare student transcripts (including Fall grades and Spring schedule)

49. Southern Regional Education Board 49

50. Southern Regional Education Board 50 Other Tips for Course-Matching 1.Get each department head for English, math and science to help with this process. 2.Include any courses the seniors have taken from grades 9-12 within English, math, science or career-technical areas. 3.Using the course description guide, match each course from your school’s master schedule to the SREB course title on the course-matching chart. Example from the course-matching guide Regular, Advanced, or College Prep Algebra I: “Algebra I courses include the study of properties and operations of the real number system, evaluating rational algebraic expressions …”

51. Southern Regional Education Board 51 Preparing for the Assessment Receive Assessment Materials Booklets & scan sheets arrive at site -by January 17, 2014 Verify contents according to instructions Keep materials secure (locked)

52. Southern Regional Education Board 52 Administering the Assessment Prepare Assessment Materials Assign a numbered answer document to each student Record the i.d. numbers on the student roster A staff person should complete Student Survey Section One – “Course Experience” for each senior –Reduces testing time –More accurate

53. Southern Regional Education Board Administering the Teacher Survey Detailed instructions will be provided, including: Sample letter to teachers Survey and log-on instructions Teachers must enter the correct HSTW site code. Ohio site codes are 5-digit numbers beginning with “3 6” 53

54. Southern Regional Education Board 54 Administering the Assessment Review Test Arrangements Confirm dates, times, locations and assistants Remind students, teachers and parents Prepare Students Practice items and use of calculators

55. Southern Regional Education Board Idea Exchange #1 What are some of the most useful or effective things your site has done in previous years (or plans to do this year) to prepare for the Assessment? 55

56. Southern Regional Education Board 56 Administering the Assessment January 24 – February 14, 2014 Schedule make-up sessions for absent students. It is crucial that students complete ALL FOUR components!!

57. Southern Regional Education Board 57 Administration Procedures Assemble materials and set up the room. Supervise student check-in. Supervise the session using the administration scripts. You may answer questions about the content of the student survey, but not the three subject tests. Keep materials secure at all times. Schedule makeup sessions. Check i.d. information on scan sheets.

58. Southern Regional Education Board 58

59. Southern Regional Education Board 59 Administering the Teacher Survey January 24– February 14, 2014 Schedule early in the window! Include all full- and part-time classroom teachers 9 th -12 th RECOMMENDED – Administer during a staff meeting or other function to guarantee completion. Use computer lab. Plan on one hour (although most teachers will finish in about 45 minutes). Surveys are anonymous. Teachers shared by MMGW & HSTW should take survey version for site at which they are most involved.

60. Southern Regional Education Board Idea Exchange #2 What are some of the most useful or effective things your site has done in previous years (or plans to do this year) to administer the Assessment? 60

61. Southern Regional Education Board Educational Testing Service Test Processing Center Ewing, New Jersey ETS processes thousands of tests every day! Misplaced materials can be very hard to locate. Use the pre-paid, bar-coded shipping labels to ensure that your materials are delivered to the right location! Returning Materials

62. Southern Regional Education Board 62 Returning Materials Pack according to instructions! Use the original ETS boxes. Use the ETS pre-paid mailing labels. Return items in their designated box or envelope: White Box Brown Box “To Be Scored” “NOT To Be Scored” 60 answer sheets All test booklets plus header sheet

63. Southern Regional Education Board Returning Materials VERY IMPORTANT Do not return more student tests than you ordered! For example, if you ordered 60, do not return more than 60 in the white box. Your school will be responsible for paying $44 for each additional answer document sent to ETS for scoring. 63

64. Southern Regional Education Board Returning Materials Include control forms Use the pre-paid shipping labels Keep receipts and UPS tracking numbers 64

65. Southern Regional Education Board Returning Student Materials 1.Postmark deadline to mail materials to ETS: February 14, 2014 2.Use the prepaid labels, original boxes and the envelope provided. Include the completed survey control form. 3.If you misplace mailing materials, contact your Regional Office as soon as possible. 4.Return everything! Pack all test books AND extra scan sheets (used or unused) in original boxes and return as instructed. 5.Make sure your materials are picked up. **KEEP YOUR UPS RECEIPT- this is the only method of tracking your materials.** 65

66. Southern Regional Education Board 66 Teacher Surveys All Teacher Surveys must be completed and submitted online by February 14, 2014.

67. Southern Regional Education Board 67 Dates to Remember Orders to Regional Office TODAY! (no later than Friday, Sept. 27) Administration materials available online Now! Requests for large print or CD versions to Educational Testing Service by October 25, 2014 Testing materials arrive at sites by January 17, 2014 Student & Teacher administration window January 24 – February 14, 2014

68. Southern Regional Education Board 68 Dates to Remember Student materials postmark deadline February 14, 2014 Award certificates and student reports mailed to sites by May 2, 2014 School reports mailed to sites June, 2014

69. Southern Regional Education Board 69 Top Ten Mistakes (in no particular order) 1.Returning extra student scan sheets (i.e., more than ordered) 2.Missed deadlines holding make-up tests late in window shipping too close to deadline forgetting to have materials picked up 3.Site code errors Site code  zip code ! All Ohio site codes are 5-digit numbers beginning with “3 6” 4.Incorrect shipping shipping to wrong address shipping in wrong box or envelope damaging scan sheets by using tape, rubber bands, or paper clips

70. Southern Regional Education Board 70 Top Ten Mistakes (continued) 5.Failure to administer the Student Survey No subject tests can be scored without the Student Survey! 6.Incorrectly marking the “Assessment Status” box on student answer sheets Student tests marked “absent” will not be read by the scanner 7.Random sampling errors - “Frankensamples” Adjusting the sample to match site demographics, programs, feeders, etc. Picking every 3 rd or 4 th student Asking for volunteers Drawing sample from selected classes only

71. Southern Regional Education Board 71 Top Ten Mistakes (continued) 8.Using incorrect test materials using Xeroxed copies of scan sheets using old scan sheets 9.Lost items student roster pre-paid mailing labels original shipping cartons UPS tracking numbers 10.Incomplete student information names left blank on scan sheets incorrect demographic information

72. Southern Regional Education Board 72 Make the 2014 Assessment a Success! Communication Letters to students and their parents Announcements to faculty and staff Cooperation Importance of the assessment to the school’s improvement goals Importance of students, administration and faculty taking it seriously Handling make-up work Cooperation from teachers whose classes might be interrupted by the assessment

73. Southern Regional Education Board 73 Make the 2014 Assessment a Success! Encourage all students to do their best. Build enthusiasm Be honest – explain how this helps the school and future students Make students feel special for participating in the assessment and thank them for their participation

74. Southern Regional Education Board Recommended Next Steps to Get Started 1.Place order. 2.Download and review materials from www.sreb.org. 3.Assemble a small planning team. 4.Schedule dates, times, locations. 5.Announce preliminary plans to faculty and staff. 74

75. Southern Regional Education Board 75 Previous participation in an assessment Name, phone and email for the assessment contact person (will be receiving materials for a year!) Complete shipping address (no P.O. boxes) Principal’s name, phone and email Number of students to be tested and sampling method SWO Region pays for 60 seniors at your site. You must pay for any additional students beyond 60 or special composites. Week you plan to administer the assessment Number of teachers to be surveyed Ordering Information Needed from You

76. Southern Regional Education Board 76 Contact Information Susanne Davis, HSTW SW Ohio msdavis1225@cinci.rr.com (513) 677-5641 Missy Kramer, HSTW SW Ohio kramer@hstw.org (513) 619-2330 Fran Cowart, SREB fran.cowart@sreb.org (404) 879-5611 Lisa Rion, Educational Testing Service lrion@ets.org (609) 734-5646