1. Testing and Accountability in North Carolina Charter Schools Curtis Sonneman Accountability Services, NCDPI December 16, 2013

2. 2 Overview North Carolina Testing Program –Test Program Overview –Test Coordinator Responsibilities Uses of Testing Program Data –READY Accountability Model –Federal Annual Measurable Objectives (AMO) and State Targets –School Performance Grades

3. What is North Carolina’s Testing Program?

4. 4 Classroom Assessments: Formative and Diagnostic Information Balanced Assessment System Formative Assessment (Classroom) Formative Assessment (Classroom) Benchmark Assessment (Classroom, School, District) Benchmark Assessment (Classroom, School, District) Summative Assessment (Classroom, Statewide) Summative Assessment (Classroom, Statewide)

5. K - 2 3 4 5 6 7 8 9 10 11 Diagnostic and Formative Assessments Student  Teacher School Accountability Assessments State  LEA  School K-5 Diagnostics ACT Explore ACT Plan The ACT 12 Measures of Student Learning to be included in Student Growth component in teacher evaluation Teacher  School EOG (Math, ELA) End of Grade (Math, English Language Arts, Science) EOG (Math, ELA) EOG (Math, ELA, Science) EOG (Math, ELA) State & Local Instructional Improvement Tools Formative & Benchmark Assessments and Data Analysis for Instruction High School NC Final Exams in core academic non-tested subjects 3 End of Course (EOC) Biology, Math I and Eng II Elementary and Middle School NC Final Exams in currently non- tested subjects Lower Elementary Measures of Student Learning The ACT ACT WorkKeys

6. Current Testing Program Assessments Administered in North Carolina End of Grade (Grades 3-8) ELA/Reading Mathematics Science (Grades 5 & 8) Alternate Assessments End of Course Math I English II Biology Alternate Assessments Field Tests LEP/ESL Assessments W-APT ACCESS for ELLs ACT Suite of Assessments –Dependent Upon State Funding ACT Explore - Grade 8 ACT Plan - Grade 10 The ACT - Grade 11 ACT WorkKeys - Grade 12 (CTE Concentrators) NAEP Testing

7. State Assessments 2013-14 Aligned to the new standards in 2012-13 Inclusion of item types other than Multiple Choice Math/Math I—Gridded Response (All) Science/Biology and English II—Technology Enhanced (Online Only) English II—Constructed Response (All) Moving towards full online administration in 2014-15

8. How do we deliver the assessments?

9. Online Administration – All End of Course Assessments (Math I, English II, & Biology) – End of Grade 5 & 8 Science – NCEXTEND2 Alternate Assessments (Final Year 2013-14) NCEXTEND2 ELA/Reading and Math 3-8 NCEXTEND2 Science 5 & 8 NCEXTEND2 Math I, English II & Biology Paper/Pencil Administration – ELA/Reading and Math 3-8 – All online assessments will be available in this format NCDPI recommends online administration, but paper/pencil will be available Assessment Delivery Format

10. Grade 5 Science TE Item 5.P.1.2 Infer the motion of objects in terms of how far they travel in a certain amount of time and the directions in which they travel (RBT = Conceptual/Understand; DOK = 2) This distance/time graph shows the distance covered by an insect crawling across a table. Using the information on the graph, place (click and drag) the remaining distances to complete the table below. Distance traveled in 10 secondsDistance traveled in 20 secondsDistance traveled in 25 seconds 20 cm 30 cm40 cm50 cm Distance Time Graph

11. Grade 8 Science TE Item 8.L.4.1 Summarize the use of evidence drawn from geology, fossils, and comparative anatomy to form the basis for biological classification systems and the theory of evolution (RBT = Conceptual/ Understand; DOK = 2) List of Species and Their Characteristics Estuarine species inhabit coastal areas; they are adapted to a mix of salt and fresh water Desert species are adapted to hot, arid areas which have large temperature fluctuations Very rare tree species inhabit the Amazon basin Reef-building coral inhabit coastal areas; their health and growth are negatively affected by pollutants Salamander species in the southern U.S. depend on flowing water to keep oxygen levels high enough for survival If the changing conditions in the chart below occur, which species listed above will likely go extinct? Drag and drop the species that will most likely go extinct for each changing condition. Changing ConditionsSpecies Likely to Go Extinct Sea levels rise rapidlyEstuarine Prolonged drought Deforestation in South America Catastrophic oil spill in a coastal region Amazon rare treeDesert Reef-building coralSalamander

12. Grade 6 Gridded Response Item Calculator: Inactive DOK: Skill/Concept Domain: Expressions and Equations Standard: 6.EE.7 Solve real- world and mathematical problems by writing and solving equations of the form x + p = q and px = q for cases in which p, q and x are all nonnegative rational numbers. A company is having a picnic. The expenses will be for music and refreshments. The music will cost $150. The refreshments will cost $125. Tickets will be sold for $2.50 per employee. What is the minimum number of tickets that must be sold to pay for the picnic expenses? Note that students receive instructions and practice for gridding answers.

13. Math I Gridded (Online) Response Item Calculator: Inactive DOK: Skill/Concept Domain: Interpreting Functions Standard: F.IF.4 For a function that models a relationship between two quantities, interpret key features of graphs and tables in terms of the quantities, and sketch graphs showing key features given a verbal description of the relationship. Key features include: intercepts; intervals where the function is increasing, decreasing, positive, or negative; relative maximums and minimums; symmetries; end behavior; and periodicity. The daily profit, P, of a business that sells x units of a product each day is given by the function P = – 2x 2 + 200x + 3,000. The number of units sold on Tuesday was 10 less than the number needed for maximum possible profit. What is the difference between the actual profit on Tuesday and the maximum possible profit? Enter your response here: Only 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,., -, and / are allowed.

14. English II Multiple-Choice Item Moonrise by Jenette Purcell City night sky gives itself to me again when I have so little left to receive it. I am dark, crumbling and you are rivers and trees away searching your own night sky for a sign. The strong gates of your heart are wide open to me always, but, if only. So I wait, as seasons before, decades before, fathers and mothers before me still inside watch and listen. Suddenly, bamboo, bones, fiber, fences, water, glistening koi, * all the tiny rooms, paths and places I hold your memories relax in audible, reverent wonder at the fullness forming on this horizon’s edge. * koi: colorful fish that symbolize love and friendship Which line from the poem describes the speaker’s feelings about loving someone? “when I have so little left to receive it” “are wide open to me always, but” “paths and places I hold your memories “at the fullness forming” DOK: Skill/Concept Standard: Reading for Literature (RL.1) Key Ideas and Details: Cite strong and thorough textual evidence to support analysis of what the text says explicitly as well as inferences drawn from the text.

15. English II Constructed-Response Item Moonrise by Jenette Purcell City night sky gives itself to me again when I have so little left to receive it. I am dark, crumbling and you are rivers and trees away searching your own night sky for a sign. The strong gates of your heart are wide open to me always, but, if only. So I wait, as seasons before, decades before, fathers and mothers before me still inside watch and listen. Suddenly, bamboo, bones, fiber, fences, water, glistening koi, * all the tiny rooms, paths and places I hold your memories relax in audible, reverent wonder at the fullness forming on this horizon’s edge. * koi: colorful fish that symbolize love and friendship In Moonrise, explain how the theme is developed throughout the poem. Use specific details to support your answer. DOK: Strategic Thinking Standard: Reading for Literature (RL.2) Key Ideas and Details: Determine a theme or central idea of a text and analyze in detail its development over the course of the text, including how it emerges and is shaped and refined by specific details; provide an objective summary of the text.

16. English II Technology-Enhanced Item Excerpt from Moonrise by Jenette Purcell Suddenly, bamboo, bones, fiber, fences, water, glistening koi, * all the tiny rooms, paths and places I hold your memories relax in audible, reverent wonder at the fullness forming on this horizon’s edge. Select (by clicking) the synonym that can replace reverent in the poem. respectful redundant amazed significant DOK: Skill/Concept Standard: Reading for Literature (RL.4) Craft and Structure: Determine the meaning of words and phrases as they are used in the text, including figurative and connotative meanings; analyze the cumulative impact of specific word choices on meaning and tone (e.g., how the language evokes a sense of time and place; how it sets a formal or informal tone).

17. 2013-14 Online Assessment System – NCTest will continue to be the online assessment delivery system – Requires a locked down browser or app 2014-15 Online Assessment System (Transitional Year) – Home Base will be new system for delivering summative assessments online (this will include delivery of Smarter Balanced Assessments) Test Nav – NCTest may continue to provide online assessment delivery as transition to Test Nav moves forward Online Assessment Delivery Plan

18. 18 Some Additional Tests CTE Post-Assessments –Used for CTE certifications –Used for Educator Effectiveness NC Final Exams (aka Common Exams 12-13) –Used for Educator Effectiveness

19. 5-day window for semester-long courses – Includes EOC, NC Final Exams, CTE post assessments, teacher made finals. 10-day window for year-long courses – Includes EOG, EOC, NC Final Exams, CTE post assessments, teacher made finals. Testing Window-GA Legislation

20. What about NC Final Exams?

21. NC Final Exams Library of NC Final Exams When: End of Year or Semester; Same window as EOG/EOC Used For: Providing an EVAAS score Only for those teachers whose subjects are currently non-tested For professional growth, and as a required component of employment decisions* Logistics: Once a year 1 or 2 class periods Replaces teacher- made final exam * Note: Not part of School Accountability Model (“A-F”) Educator Effectiveness Tests to measure student growth as a part of educator evaluation

22. NC Final Exams Charter Requirement Must participate in the NC Final Exams administration if-- Received Race To the Top funds Have beginning teachers who are seeking continuing licensure (Year 1) Must have 3 years of growth data Charters may elect to participate in NC Final Exams

23. 23 Test Coordinator Responsibilities

24. 24 Testing and Accountability Responsibilities Principals/Directors/Board Members are ultimately responsible for Testing and Accountability Duties North Carolina Testing Code of Ethics Responsible for Test Security –Materials must be stored in a secure locked facility (plan your facilities accordingly) Accessible by 1-2 staff (Test Coordinator and Principal) Room, closet, or locked cabinet that cannot be removed

25. 25 School Test Coordinators Each charter school must have an assigned Testing Coordinator –Must be employee of charter school Recommend a certified staff, but not regular classroom teacher if at all possible (not enough time to do both) –Must be available to attend all trainings (off-site) –Have no other responsibilities during test administration days –Available for training and updates during summer months (accountability reporting ends June 30)

26. 26 Test Coordinator Responsibilities Ensure test security with leadership –Test materials –Testing violations –Distribution of test materials –Ensure test materials are on copied or reproduced in any way –Order/Request test materials Attend and conduct training –NC Train the Trainer Model NCDPI Regional Staff Train School Test Coordinators School Test Coordinators Train School Staff

27. 27 Test Coordinator Responsibilities Create school testing plan Schedule test administrations Develop local policies and procedures to ensure proper test administration Submit materials for scanning/scoring With assistance from RAC determine testing irregularities Check online systems for updates

28. 28 Test Coordinator Responsibilities Maintain confidentiality of student test scores and teacher information –Will have access to all test results and accountability for the school Submit data requested by RAC or NCDPI Ensure data accuracy

29. 29 School Test Coordinator Skills Strong Computer Skills –Microsoft Excel –Microsoft Word –Internet savvy –Able to learn new programs easily Secure File Transfer System NCEducation online system HomeBase Able to send/receive information via email/phone and respond in a timely manner Strong organizational skills

30. 30 School Test Coordinator Skills Available to attend regularly scheduled meetings (generally 1 time each month) Strong interpersonal relationship skills –Training staff –Work closely with school leadership and NCDPI staff –Work closely with other staff to obtain data –Meet/talk with parents/staff to answer questions Good listening skills Not afraid to ask questions or address issues

31. Testing Questions

32. How are schools held accountable?

33. 33 Accountability Accountability of Test Scores –All public schools must participate in the North Carolina Testing Program Schools are expected to test all eligible students –95% Tested Rule »Failure to meet results in non-compliance notification »Additional requirements for not testing 95% of students for federal reporting

34. 34 Accountability Accountability of Test Scores (cont.) –Data Collections Economically Disadvantaged Students (EDS) –Whether or not you have a school lunch program Power School Limited English Proficient—Consolidated Federal Data Collection CECAS Test Administration Data –Data Validation

35. 35 Accountability Accountability Model –READY Accountability Model 2012-2013 Yearly Performance—Status Performance over time—Progress (targets) Student Growth—EVAAS Growth Model

36. 36 Elementary and Middle School Accountability End of Grade 3-8 Math End of Grade 3-8 ELA End of Grade 5 & 8 Science Δ End of Grade 3-8 Math Δ End of Grade 3-8 ELA Δ End of Grade 5 & 8 Science EVAAS Growth School-wide Status Indicators “this year” Progress Indicators “trend over time for groups of students” Growth Indicators “combined individual student’s growth”

37. 37 High School Accountability 37 End of Course ACT Graduation Rates Math Course Rigor ACT WorkKeys Graduation Project Δ End of Course Δ ACT Δ Graduation Rates Δ Math Course Rigor Δ ACT WorkKeys EVAAS Growth School-wide Status Indicators “this year” Progress Indicators “trend over time for groups of students” Growth Indicators “combined individual student’s growth”

38. READY Accountability Model Reporting and School Performance Grades Indicators End-of-Grade and End-of-Course Performance Composite READY Broken out by assessment SPG Percent of proficient tests in a school −All EOG/EOC tests, subjects, and grade levels (Including alternate assessments) The ACT The ACT Composite Score of 17 as meeting standard −17 is UNC System minimum for admission Math Course Rigor Percent of graduates who earn credit in Alg. II, Int. Math III, or Math III −Excludes the 1% and OCS populations ACT WorkKeys Percent of graduates who are CTE concentrators and who achieve a Silver certificate, or better, on the ACT WorkKeys assessment Graduation Rate Percent of students that graduate within 4 years (4-year cohort graduation rate) READY and SPG Percent of students that graduate within 5 years (5-year cohort graduation rate) READY

39. 2012-2013 Ready Accountability Model Reporting Accountability Indicators  Indicator will display data if school has 5 or more assessments/students Federal Targets  English language arts/reading, math, graduation rate, and attendance State Targets  Science, The ACT, and ACT WorkKeys EVAAS Growth (School-wide Accountability Growth)  Reported as Exceeds, Meets, or Does Not Meet on website  Linked to EVAAS public reporting website Graduation Project  Reported as Yes, No or N/A Ability to drill down into data for analysis

40. How will the School Performance Grades be calculated?

41. School Performance Grades First passed by the General Assembly in 2012 Assigns a letter grade A-F for each public school in North Carolina Includes consequences for “F” schools

42. School Performance Grades Legislation Implemented: 2013-14 school yearReported: August 2014 Components: Performance (School Achievement Score) and Growth

43. School Performance Grades School Achievement Score Growth School Performance Grade

44. School Performance Grades Break Out Elem/Middle EOG Mathematics EOG ELA/Reading EOG Science Math I (when applicable) English II (when applicable) Biology (when applicable) High Schools Math I English II Biology The ACT Math Course Rigor ACT WorkKeys Graduation Rate

45. School Achievement Scores Components of the School Achievement Scores (80%) Test Scores: Percent of students who score at or above Level 3  End-of-Grade Tests  End-of-Course Tests Graduation Rate: Percent of students who graduate in four years Math Course Rigor: Percent of graduates who successfully complete Math III (Algebra II, Integrated Math III, Math III) The ACT: Percent of 11 th grade students who score 17 or above (the UNC System’s minimum composite score requirement) ACT WorkKeys: Percent of CTE concentrator graduates who achieve a Silver Certificate or above

46. READY Accountability Model Reporting and School Performance Grades Indicators End-of-Grade and End-of-Course Performance Composite READY Broken out by assessment SPG Percent of proficient tests in a school −All EOG/EOC tests, subjects, and grade levels (Including alternate assessments) The ACT The ACT Composite Score of 17 as meeting standard −17 is UNC System minimum for admission Math Course Rigor Percent of graduates who earn credit in Alg. II, Int. Math III, or Math III −Excludes the 1% and OCS populations ACT WorkKeys Percent of graduates who are CTE concentrators and who achieve a Silver certificate, or better, on the ACT WorkKeys assessment Graduation Rate Percent of students that graduate within 4 years (4-year cohort graduation rate) READY and SPG Percent of students that graduate within 5 years (5-year cohort graduation rate) READY

47. School Growth EVAAS School-wide Accountability Growth (20%) Includes  End-of-Grade Tests  End-of-Course Tests Reported for each school as  Exceeds Expected Growth  Meets Expected Growth  Does Not Meet Expected Growth Not included in School Performance Grades for schools that have a School Achievement Score at or above 80 % and Meets Expected Growth

48. School Growth Meets growth: No additional calculation; growth not included in School Performance Grade Did not meet or exceeded growth: Include growth in calculation of School Performance Grade School Achievement Score at or above 80 % Include growth in calculation of School Performance Grade School Achievement Score 79 % or below

49. Growth Conversion 808590506070 0+2+4+6+8-10-8-6-4-2  Index range constrained at -10 to +10  Index value converted to 50-100 point scale Did Not Meet Growth Exceeded Growth Met Growth 100 +10

50. Elementary Achievement Calculations 5&8 Science % Proficient 3-8 Math % Proficient 3-8 Reading % Proficient Biology % Proficient English II % Proficient Math I % Proficient # of Available Indicators ++ ++ + = Points 60.1% (98/163) 83.6% (336/402) 73.1% (294/402) 0% (0/0) 0% (0/0) 93.8% (30/32) 4 ++ ++ + = 77.7 73.1 + 83.6 + 60.1 + 93.8 = 310.6 and 310.6 ÷ 4 = 77.65 ≈ 77.7

51. Elementary Grade Calculation (62.2)+(18.3)= 80.5 80.5pts=letter grade of “B” Achievement Score under 80 OR Achievement Score at least 80 AND “Did Not Meet Growth” or “Exceeded Growth Achievement Score = School Performance Grade Points 90-100pts=A 80-89pts=B 70-79pts=C 60-69pts=D Under 60pts=F = 80.5 + X 0.2 X 0.8 Growth Points ( ) ( ) + X 0.2 X 0.8 ( ( ) ) 91.3 77.7

52. High School Achievement Calculation Biology % Proficient English II % Proficient Math I % Proficient The ACT % Met 17 ACT WorkKeys % Silver MCR % Math III # of Available Indicators ++ ++ + = Points 82.8% (77/93) 89.7% (78/87) 85.7% (90/105) 69.1% (56/81) 85.7% (42/49) 63.4% (52/82) 7 ++ ++ + = 82.2 85.7 + 89.7 + 82.8 + 63.4 + 69.1 + 85.7 + 98.8 = 575.2 and 575.2 ÷ 7 = 82.17 ≈ 82.2 98.8% (80/81) + Grade Rate % 4yr +

53. High School Grade Calculation Achievement Score 80 or above AND “Met Growth” Achievement Score = School Performance Grade Points 90-100pts=A 80-89pts=B 70-79pts=C 60-69pts=D Under 60pts=F 82.2 = 82.2pts=letter grade of “B”

54. High School Grade Calculation (65.8)+(13.1)= 78.9 78.9pts=letter grade of “C” Achievement Score under 80 OR Achievement Score at least 80 AND “Did Not Meet Growth” or “Exceeded Growth Achievement Score = School Performance Grade Points 90-100pts=A 80-89pts=B 70-79pts=C 60-69pts=D Under 60pts=F = 78.9 + X 0.2 X 0.8 Growth Points ( ) ( ) + X 0.2 X 0.8 ( ( ) ) 65.6 82.2

55.  Growth will be reported for each school: exceeded, met, or did not meet expected growth  A separate achievement score for math and reading will be reported for schools serving grades K-8  The report card for schools serving 3rd graders will contain information on the number and percentage of third graders who are retained or promoted based on reading performance Additional Reporting

56. What is new in testing in 2014-15 and beyond?

57. 57 Testing Read to Achieve (Beginning 2013-14) –End-of-grade test form administered at beginning of 3 rd grade (BOG3) –End-of-grade retesting for end of 3 rd grade ELA/Reading administrations or alternative assessment ACT –HB 587: requires an alternative for ACT in Spring 2014 and for ACT Plan in Fall 2014

58. Draft – March 2012. Check http://www.ncpublicschools.org/ready/resources/ for Updates to this Presentationhttp://www.ncpublicschools.org/ready/resources/ Testing Online Assessments for 2014-15 EOCs Science- Grades 5 and 8 NCEXTEND2: Not available in 2014-15 Will no longer provide based on ESEA Waiver requirement

59. School Accountability Assessments: Timeline Test2012-20132013-20142014-15 on New State Developed Math 3-8 ELA 3-8 English II Math I Biology Science 5 & 8 ACT SMARTER Balanced Math 3-8* ELA 3-8* 11 th Grade Math* 11 th Grade ELA* In Development SBE Decision *2014-15 implementation of SMARTER assessments contingent upon board approval of assessments to replace the State-developed ELA and Math assessments.

60. Questions? Every Student READY