Summer Institutes 2013 Changing Teacher Practice Changing Student Outcomes

North Carolina Summer Institutes Learning Path WHAT Focus: Internalizing a new SCOS HOW Focus: Planning how instruction needs to change IMPROVING PRACTICE Essential Standards Focus: Reflecting, adjusting and improving after year one of implementation Summer Institutes

2013 Summer Institutes |Changing Teacher Practice  Changing Student Outcomes Design Studio Session Understanding School-Based Accountability and Supporting a Balanced System of Assessment Presenters: Jami Inman, Julian Wilson, Curtis Sonneman

2013 Summer Institutes |Changing Teacher Practice  Changing Student Outcomes Design Studio Session Understanding School-Based Accountability and Supporting a Balanced System of Assessment Presenters: Lisa McIntosh, Jennifer Northrup, Nancy Barbour, and Hope Lung (Charlotte), Scott Beaudry (Greensboro), Jeff Payne (Hendersonville)

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Design Studio Session Agenda – Session Overview School Accountability Use: School Performance Grades, ESEA and Public Reporting Formative and Diagnostic Information Use: To assist in teaching and learning Educator Effectiveness Use: To ensure that part of every educator’s evaluation is based on student growth SCHOOLS CLASSROOMS EDUCATORS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Design Studio Session By the end of this session participants will: Understand the Use of Assessments and Data for School Accountability Understand the Importance of Formative and Diagnostic Information Understand Common Exams, Summative Assessment Data and Their Relationship to Teacher Effectiveness SCHOOLS CLASSROOMS EDUCATORS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes K Diagnostic and Formative Assessments Student  Teacher School Accountability Assessments State  LEA  School K-5 Diagnostics EXPLORE PLAN ACT 12 Measures of Student Learning to be included in Student Growth component in teacher evaluation Teacher  School EOG (Math, ELA) End of Grade (Math, English Language Arts, Science) EOG (Math, ELA) EOG (Math, ELA, Science) EOG (Math, ELA) State & Local Instructional Improvement Tools Formative & Benchmark Assessments and Data Analysis for Instruction High School Measures of Student Learning (Common Exams) in core academic non- tested subjects 3 End of Course (EOC) Biology, Math I and Eng II Elementary and Middle School Common Exams in currently non- tested subjects Lower Elementary Measures of Student Learning ACT WorkKeys

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments Formative and Diagnostic Information Use: To assist in teaching and learning CLASSROOMS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Balanced Assessment System Formative Assessment (Classroom) Formative Assessment (Classroom) Benchmark Assessment (Classroom, School, District) Benchmark Assessment (Classroom, School, District) Summative Assessment (Classroom, Statewide) Summative Assessment (Classroom, Statewide)

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS NC FALCON NC FALCON North Carolina’s Formative Assessment Learning Community’s Online Network

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Indicate how often you do the following: 5 - Daily 4 - Weekly 3 - Monthly 2 - Quarterly 1- Rarely/Never  I use checklists when gathering information about student learning.  I use rubrics for assessing my students.  I write learning targets in student-friendly language on the board and go over them with my students.  I provide students specific information (without using grades) about where they are in meeting the learning targets.  I plan or modify classroom instruction based on the information I receive from students during class.  I give students opportunities to self-assess and set goals for future learning.  I give students opportunities to assess their peers. Self-Assessment Survey

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information What is Formative Assessment? CLASSROOMS Formative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to help students improve their achievement of intended instructional outcomes. The Council of Chief State School Officers (CCSSO, 2008)

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS 4 Key Words: Process Feedback During Students Feedback

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information Formative Assessment Model CLASSROOMS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Research shows that if students are formatively assessed, learning will improve. When learning is improved, students are able to demonstrate that learning in a variety of ways including scoring well on standardized assessments like the EOG and EOC. Black and Wiliam (1998)

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Standard IV: Teachers Facilitate Learning For Their Students Teachers use a variety of methods to assess what each student has learned. Teachers use multiple indicators, including formative and summative assessment to evaluate student progress and growth. Teachers provide opportunities, methods, feedback, and tools for students to assess themselves and each other. Teachers use 21 st Century assessment systems to inform instruction and demonstrate evidence of students’ 21 st Century knowledge, skills, performance, and dispositions.

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Attributes of Effective Formative Assessment Providing students with learning goals and targets in language they can understand Clearly describing the criteria for successfully meeting the target through examples Effectively using learning progressions to scaffold learning Providing descriptive feedback that helps the student know what to do next in their learning Actively engaging students in self-assessment as well as peer-assessment.

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Grade/ Subject: Big Idea/Objective/Standard: Learning TargetCriteria for SuccessCollecting EvidenceDocumenting Evidence 1.What misconceptions do you think students might have? 2.What will you do to address the misconceptions to move learning forward (e.g., how will you adjust instruction, what descriptive feedback will you provide)? Formative Assessment Plan

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Establishing Clear Learning Targets

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Criteria for Success Where am I going?

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Collecting Evidence Where am I now?

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Documenting Evidence The documentation should reflect the learning adequately and appropriately. The documented evidence of learning should provide enough information to make sound decisions that inform instruction and improve student learning.

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Sample Strategies for Collecting and Documenting Evidence of Student Learning: Mental Notes – Giving special attention while listening to student discourse with intent to remember and provide descriptive feedback. Symbolic Indicators – Record any symbol, understood by you and your students, on your class roster that indicates where the student is in reaching the learning target. Matrix – A chart with names down the left side and the learning targets written across the top. As students reach each target, check it off or record short phrases to indicate where each student is in the learning process. Audio/Video Recordings – Live documented footage of what a student does and/or says which indicates where they are in reaching the learning target. The advantage of this strategy is that growth can be documented at different intervals.

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS FORMATIVE ASSESSMENT CONTENT EXAMPLES NC FALCON NC FALCON

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Home Base Components CLASSROOMS Assessment Student Information and Learner Profile Instructional Design, Practice & Resources Data Analysis and Reporting Information a simpler, better information system to replace NC WISE Integrated Instructional Solution a new standards-aligned tool for instruction (e.g. lesson plans, unit plans), assessment and data analysis Effectiveness a simpler, better online evaluation system and new professional development system InformationInstruction Educator Effectiveness: Evaluation and Professional Development Test Nav Summative Assessment OpenClass Collaboration Schoolnet Instructional Tools and Assessment PowerSchool Student Information Truenorthlogic Evaluation and PD

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Home Base Resources CLASSROOMS Develop aligned assessments for formative, interim or summative purposes Create an Express Test Create a Test Manually Deliver assessments Online Print options available for paper/pencil delivery Data analysis and reporting tools integrate with assessment tools and student information to deliver data on student progress to teachers, parents and students in an easy-to-understand, actionable way

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Reflection – Mix Music Meet Using sticky notes…. Select one of the tools we just discussed and describe how it might be used for diagnostic/pre-assessment or summative assessment. Music provided by:

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Classroom Assessments: Formative and Diagnostic Information CLASSROOMS Balanced Assessment System How do I keep my balance?

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Design Studio Session EDUCATORS Break

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Common Exams Educator Effectiveness Use: To ensure that part of every educator’s evaluation is based on student growth EDUCATORS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Common Exams: Focusing on the “Why” EDUCATORS So why have statewide Measures of Student Learning/Common Exams? 1.North Carolina has a statewide evaluation system to ensure that every teacher receives a fair and consistent evaluation, regardless of his or her employing LEA 2.Teachers in all content areas should receive a Standard Six rating based on the growth of their own students on their content-specific standards 3.Most LEAs do not have the capacity to design their own assessments for all non state-tested grades and subjects

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Common Exams Measures of Student Learning: Library of Common Exams When: End of Year or Semester; Flexible window Used For: Providing an EVAAS score Only for those teachers whose subjects are currently non-tested For professional growth, and as a required component of employment decisions* Logistics: Once a year 1 or 2 class periods Should replace teacher- made final exam * Note: Not part of School Accountability Model (“A-F”) Educator Effectiveness Tests to measure student growth as a part of educator evaluation EDUCATORS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Common Exams Following Local Superintendents’ Recommendation Local Superintendents recommended that State, not districts, develop measures of student growth for non-tested grades and subjects Designed and Developed with Teachers Over 800 teachers are involved in the design and development Using EVAAS Employ EVAAS to measure student growth facilitated by educators Focuses Attention on All Content Areas (not just Math & English) Ensuring teachers of all content areas receive feedback on how well they facilitate student growth Allows District Flexibility around Administration of Exams Districts make choices regarding logistics and who takes which exams EDUCATORS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Common Exams: District Flexibility Logistics Handled by District No set State testing window – districts schedule at their discretion 90 minute test administration (actual student testing time) Districts can choose online or paper-pencil versions, or a combination Districts choose whether or not to have exam proctors Districts choose methods for scoring short-answer (non-multiple choice) exam questions Common exams should replace teacher-made final exams (therefore, should not create additional testing for students) EDUCATORS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Common Exams: District Flexibility Policy Decisions Handled by District Districts choose whether or not to count common exam scores as part of students’ final course grades If district includes common exam score in students’ final grade, district chooses how much it counts toward final grade Districts set policies locally regarding what qualifies as a “mis- administration” (for example, cheating) Districts must ensure that each teacher has a measure of how he or she is facilitating student growth, but there is no requirement to administer all of the common exams in the library Districts use State-provided “decision-tree” to determine which exams must be administered, but can give any exam they choose EDUCATORS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Common Exams: State-provided Decision Tree Does the teacher administer an End-of-Course (EOC) assessment, End-of-Grade (EOG) assessment, or Career and Technical Education (CTE) Post-Assessment to ALL of his or her students? YES No State requirement to administer a Common Exam.* NO Does the teacher instruct a course or grade/subject with a Common Exam? YES State requirement to administer the Exam. NO No State requirement for 2012 – * Note: Districts may still choose to administer Common Exams EDUCATORS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Common Exams: Reaction to First Administration EDUCATORS Use an index card to record the following: Three things that went well with your LEA’s implementation of Common Exams Three things you will do differently next time Share your responses with your group

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Common Exams: Reaction to First Administration Looking for suggestions? Revisit the Google Doc for your region – Take a few minutes to review the areas of concern for others in your region. In the right-hand column, please provide strategies that might help another LEA. EDUCATORS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Common Exams EDUCATORS Break

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes School Accountability Assessments School Accountability Use: School Performance Grades, ESEA and Public Reporting SCHOOLS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes School Accountability Assessments: For School Performance Grades, School Accountability and ESEA (formerly NCLB) SCHOOLS End-of-Grade (EOG), End-of-Course (EOC ) 3-8 English and Math 5 & 8 Science 1 each Math, English and Science in High School WorkKeys 12 th ACT 11 th When: EOC/EOGs - May or June (Dec/Jan for Semester I) ACT – March WorkKeys - February Used For: School accountability (state and required federal) School data made public and used to target support Logistics: Once a year; proctored

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes School Accountability Assessments: Timeline Test on New State Developed Math 3-8 ELA 3-8 English II Math I Biology Science 5 & 8 ACT SMARTER Balanced Math 3-8* ELA 3-8* 11 th Grade Math* 11 th Grade ELA* In Development SBE Decision SCHOOLS * implementation of SMARTER assessments contingent upon board approval of assessments to replace the State-developed ELA and Math assessments.

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes State Assessments and Demo of New Assessments Aligned to the Common Core and Essential Standards Inclusion of New Items Types Moving towards full online administration in SCHOOLS

Online Administration – All End of Course Assessments (Math I, English II, & Biology) – End of Grade 5 & 8 Science – NCEXTEND2 Alternate Assessments NCEXTEND2 Reading and Math 3-8 NCEXTEND2 Science 5 & 8 NCEXTEND2 Math I, English II & Biology Paper/Pencil Administration – Reading and Math 3-8 – All online assessments will be available in this format NCDPI recommends online administration, but paper/pencil will be available 2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Assessment Delivery Format SCHOOLS

Online Assessment System – NCTest will continue to be the online assessment delivery system – Will require a locked down browser or app Online Assessment System – Home Base will be the new system for delivering summative assessments online (this will include delivery of Smarter Balanced Assessments) Test Nav 2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Online Assessment Delivery Plan SCHOOLS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Standard Setting ScheduleDeliverable June 28, 2013 o All test data from due to NCDPI Accountability July 2013 o Development of scale for all assessments Late June and July 2013 o Standard Setting for all assessments August 2013 o Data Analysis Mid-September 2013 o Recommendations prepared from Standard Setting for SBE approval October 3, 2013 Presented to SBE for Approval Cut scores Achievement Levels and Descriptors READY Accountability results SCHOOLS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Standard Setting Administration of New Tests May/June Compile Data and Build Developmental Scale July into August Achievement Standards Presented to SBE October 4 Educator Groups Recommend Achievement Levels June into July SCHOOLS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes College and Career Readiness It is important to remember that we have raised expectations significantly in the school year. Claims in the Past: Grade-level Proficiency Only Claims in the Future: Grade-level Proficiency and Career- and College-Ready SCHOOLS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes READY Accountability and School Performance Grades Indicators Performance Composite (Elementary and High) Percent of proficient tests in a school −All EOG/EOC tests, subjects, and grade levels (Including alternate assessments) Math Course Rigor Percent of FRC graduates who earn credit in Alg. II, Int. Math III, or Common Core Math III −Excludes the 1% and FRC9 populations Graduation Rate Percent of students that graduate within 4 years (4-year cohort graduation rate) WorkKeys Percent of graduates who are CTE concentrators and who achieve a Silver certificate, or better, on the WorkKeys assessment ACT Percent of college-ready benchmarks met (READY Accountability Model) School Performance Grades may use different criteria SCHOOLS

House and Senate have 2 different bills Senate version in budget proposal being discussed in conference House version currently in Senate for consideration 2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes School Performance Grades Legislation SCHOOLS

Operational in The SBE must respond to the General Assembly “…annually by January 15 on recommended adjustments to the school performance grade elements and scales for award of scores and grades.” Must be reported publicly 2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Reporting School Performance Grades SCHOOLS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Calculating School Performance Grades Per HB 435 Performance Composite Math Course Rigor Graduation Rate WorkKeys ACT High School + Growth {state mean} Elementary and Middle Performance Composite + Growth {Schools that meet or exceed growth are raised one letter grade} {state mean} SCHOOLS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes HB 435 School Performance Grades: Guiding Principles 1)Using the state mean, which means raising standards will not create undifferentiated grades 2)Each indicator has a different mean and scores are distributed around that mean 3)Set criteria against the state average 46% 87% Example Indicator A Example Indicator B CC SCHOOLS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes HB 435 School Performance Grades: Guiding Principles 4)Monitor data for continuous improvement of the accountability system 5)Making or exceeding EVAAS growth will improve school’s grade by one letter grade 6)Each met ACT Benchmark counts (ex: 3 out of 5 counts more than 1 out of 5) 7)WorkKeys based on the state average % of CTE graduates meeting silver level SCHOOLS

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes SB 402 School Performance Grades: Guiding Principles Uses the same indicators as House version, but in a different way. Weights the indicators according to how many students participated. For example, WorkKeys would have less weight due to the smaller N count. ACT criteria (tentatively): Composite score meeting the UNC System minimum (currently set at 17). Growth could be up to 20% of final grade. SCHOOLS

School Calendar - Draft Aug Sep Oct Nov Dec Jan Feb Mar Apr May June July Aug Sept Oct State Tests (3-8 ELA and Math; 5 & 8 Science; 3 High School Exams) and Common Exams ACT, PLAN, EXPLORE, WORKKEYS SMARTER Balanced (Development Work) PLAN EXPLORE ACT March 4, 2014 March 18, 2014 (make-up) March 4-18 (accommodations testing) WorkKeys 3-8 and High School Assessment Administration Compilation of New Data Standard Setting READY Report to SBE Field test of summative and interim assessment items and performance tasks 5th Tech Readiness Tool Collection Windows WorkKeys for Early Graduates in Dec High School Assessment Administration READY Report to SBE

Facilitated Team Time Preparation To prepare for Facilitated Team Time, complete the brief reflection to identify the “big ideas” gained from this session that you will share with your Summer Institute team. To access the reflection document, visit or scan the QR code on the next slide. To access the reflection responses during Facilitated Team Time, visit

Facilitated Team Time Preparation

2013 Summer Institutes | Changing Teacher Practice  Changing Student Outcomes Design Studio Session QUESTIONS