21st Century Balanced Assessment System Lisa Youell Division of Educator Quality and System Support West Virginia Department of Education

Today’s Learning Targets Assessment Literacy Journey Examine the need for a balanced assessment system Explain the key differences between assessment for and of learning Develop common assessment language (vocabulary) Recognize the profound influence of formative classroom assessment on student learning Today’s Learning Targets are Understand the purpose and structure of the Classroom Assessment Network (CAN) Examine the need for a balanced assessment system Explain the key differences between assessment for and of learning Develop common assessment language (vocabulary) Recognize the profound influence of formative classroom assessment on student learning

Once a Year… Physical Exam Yields indicators of one’s health Concerns about the final numbers Doctor makes recommendations You have concerns about your Body Mass Index (BMI), your blood pressure, or maybe your cholesterol.

Doctor’s Recommendations? Focus on Healthy Practices Daily Focus on the Exam Once a Year Practice good habits on a daily basis Drink water every day Eat more fiber Eat fruits and vegetables Eat less fat Exercise/walk Avoid tobacco For 3 months before the next exam do the following: Exercise Reduce salt intake Get more sleep Eat oats everyday You have concerns about your Body Mass Index (BMI), your blood pressure, or maybe your cholesterol.

School’s Annual Checkup Once a Year… School’s Annual Checkup Yields indicators of school’s progress Concerns about the final numbers Principal makes recommendations

School’s Annual Checkup Focus on Daily Practices Focus on the Test Once a Year For 3 months before the test do the following: Look at the data Determine weaknesses Practice test items with standardized format Become assessment literate Use day to day classroom assessment practices that involve students directly in self-assessment, goal setting, and communicating about their own learning Collect a continuous stream of evidence of student learning Use assessment information formatively to plan further instruction

We are living in the old system while we are trying to build a new system. Teachers are caught in a situation that deserves our help and understanding. The dilemma they face is how to raise scores on state tests without sacrificing real learning in the classroom. We need to make clear the critical role that classroom assessment can play in improving student performance on standardized assessments OF learning.

In an Effective School System… Continuous Formative Classroom Assessment Periodic Benchmark Assessments Annual Accountability Testing (State Summative Test) Classroom Level Users (Students, Teachers (teams) and Parents) Program Level Users (Teacher Teams and Leaders) Institutional/Policy Users (School, District and State Leadership)

Research on Assessment & Student Achievement British researchers Paul Black and Dylan Wiliam completed a comprehensive review of 250 international studies exploring the connection between formative assessment practices and student achievement (1998) *****IMPORTANT***** It is critical that you read “Inside the Black Box” by Black and Wiliam before you present the information on slides ______. The article is in your resource section. In 1984 two British researchers, Paul Black and Dylan Wiliam, examined the research literature on assessment worldwide, asking if there is evidence that improving the quality and effectiveness of use of formative (classroom) assessments raises student achievement. iNTERNATIONAL They collected and then synthesized over 250 studies that addressed these issues. Their review examined studies that collectively encompassed kindergarteners to college students, represented a range of subject areas and were conducted in numerous countries throughout the world, including the United States.

Research on Assessment & Student Achievement Does improved formative assessment cause better learning? Do formative assessment practices need improving? Is there evidence about how to improve formative assessment? If so, what kinds of improvement in classroom assessment practices are likely to yield gains in achievement as reflected in summative assessments? The review focused on the following questions. Does improved formative assessment cause better learning? Do formative assessment practices need improving? Is there evidence about how to improve formative assessment?

Black & Wiliam Research on Effects of Classroom Assessment for Learning: .4 to .7 Gain .7 Standard Deviation Score Gain = 25 Percentile Points on ITBS (Iowa Test of Basic Skills) 70 SAT Score Points 4 ACT Score Points Largest Gain for Low Achievers The gains reported in the studies they reviewed are among the largest found for any educational intervention. Pooling the information on the estimated effects of improved formative (classroom) assessment on summative test scores, they discovered unprecedented positive effects on student achievement. There is significant research showing that assessment for learning practices increase student motivation and achievement. Paul Black and Dylan Wiliam found effect sizes of between .4 and .7 standard deviations.. To understand the significance of these numbers, we can translate them as follows: A .7 standard deviation gain on the ITBS (Iowa Test of Basic Skills) equals a gain of 25 percentile points, a gain of 70 points on each portion of the SAT (formerly Scholastic Aptitude Test and Scholastic Assessment Test), and a gain of 4 points on the ACT (college entrance and placement exam) composite score. An additional outcome common among the studies they analyzed is that formative (classroom) practices greatly increased the achievement of low-performing student, in some cases to the point of approaching that of high-achieving students. © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060

Needed Improvements to Realize Gains Increased commitment to high-quality classroom assessments Increased descriptive feedback; reduced evaluative feedback Increased student involvement in the assessment process Black and Wiliam, 1989

Dylan Wiliam Formative Assessment: http://www.ltscotland.org.uk/video/f/formativeassessmentdylanwiliam.asp

Classroom Assessment For Learning Common Language? Formative Assessment Summative Assessment Assessment FOR Learning Benchmark Assessments Assessment OF Learning Turn to your neighbor and discuss the differences in formative and benchmark assessment. Classroom Assessment For Learning

A Balanced Assessment System Assessment of Learning Summative Assessment An event after learning Benchmark Assessment Assessment for Learning Formative Assessment A process during learning Classroom Assessment For Learning Assessment of learning is for accountability while assessment for learning is used to support learning. Both are important ,but they are different because they serve fundamentally different proposes. The key to success as educators is to balance the two – to find the synergy between them. (Rick Stiggins) All three levels are important. One level cannot replace another.

What is the primary aim of assessment? Who will use the information? Critical Questions What is the primary aim of assessment? Who will use the information? What decisions will they make? Ask participants to jot down their thoughts on this question. Assessments provide results that inform decisions.

Assessment of Learning In our current accountability environment, assessment of learning especially summative tests are used for ranking students and school. Periodic (3 or 4 times during the year Annual

Primary Users Who are the decision makers? Benchmark Summative

Typical Uses What are the typical uses of the assessment? Summative Benchmark

Assess What? What is assessed? Benchmark Summative

Assess How? How is the assessment designed? Summative Benchmark

Assess When? When and how often is the assessment given? Benchmark Summative Periodic (3 or 4 times during the year) Annual Assess When

Assessment OF Learning Users  Uses  What  How  When Turn to your neighbor and discuss the key differences in benchmark and summative assessment.

Assessment of Learning An event after learning that determines how much a student has learned at a particular point in time. Summative/Benchmark Assessment is used for accountability used to determine if learning has occurred a snapshot of student performance at a given point in time Summative Assessment is Used to inform multiple levels of accountability To determine if learning has occurred and to check if students are meeting standards A snapshot of student performance at a given point in time, involves judging and interpreting data, assigning a grade, and making a judgment about program and instructional effectiveness.

Assessment of Learning An event after learning that determines how much a student has learned at a particular point in time. Examples of Summative/Benchmark Assessments are Standardized tests like state assessments (WESTEST 2, Golden Horseshoes Test, ACT Plan and Explore) College admissions tests (ACT, SAT) District-wide interim and benchmark tests (Acuity) End of unit or semester tests Assessments used to assign report-card grades All three levels are important. One level cannot replace another.

Assessment for Learning Refer to the Assessment Brochure

Primary Users Who are the decision makers? Classroom Assessment for Learning Formative Assessment Teachers use assessment information formatively when they do such things as identify which concepts or skills students need more work with to plan further instruction; investigate the effectiveness of their own teaching practices and provide regular feedback to students on their strengths and areas for improvement. Students engage in assessment for learning when they use assessment information to learn how to judge the quality of their own work and set goals for their own improvement. Student as Decision Maker

Typical Uses What are the typical uses of the assessment? Classroom Assessment for Learning Formative Assessment Descriptive Feedback

Typical Uses What are the typical uses of the assessment? Classroom Assessment for Learning Formative Assessment The student involvement component is what differentiates formative assessment from classroom assessment for learning. Student Involvement

Assess What? What is being assessed? Formative Assessment Classroom Assessment for Learning Assessment for Learning is about assessing the learning targets that are embedded in the learning objective. Assessment of Learning is about assessing the overall objective. Learning Targets

Performance Skill Targets Summative Assessment Objective/Benchmark: First Grade Reading /English Language Arts Produce writing to communicate with different audiences for a variety of purposes. Overall Target Type:  Knowledge  Reasoning  Performance Skill  Product Learning Targets What are the knowledge, reasoning, performance skill or product targets underpinning the standard/objective? Knowledge Targets Reasoning Targets Performance Skill Targets Product Targets Distinguish the uses or meanings of a variety of words (word choice) Holds a pencil correctly Print letters correctly Space words Use lines & margins Stretch out sounds in words to create a temporary spelling of a word Write sentences with varied beginnings Know what a sentence is Understand concept of word choice Overall Target Type: Product Underlying targets include: knowledge, reasoning and performance Formative/Classroom Assessment for Learning

Performance Skill Targets Summative Assessment Standard/Objective: Drive with skill. Type:  Knowledge  Reasoning  Performance Skill  Product Learning Targets What are the knowledge, reasoning, skill or product targets underpinning the standard/objective? Knowledge Targets Reasoning Targets Performance Skill Targets Product Targets Know the law Understand informal rules of the road Understand what different parts of the car do Read signs and understand what they mean Understand what “creating a danger” means Understand what “creating a hazard” means Other? Analyze road conditions, vehicle performance, and other driver’s actions Compare/contrast this information with knowledge and past experience Synthesize information and evaluate options to make decisions on what to do next Evaluate “Am I safe?” and synthesize information to take action if needed. Other? Driving actions such as: steering, shifting, parallel parking, looking, signaling, backing up, braking, accelerating, etc. Fluidity/automaticity in performance driving actions. Other? None Since the ultimate type of target is a performance skill, there are no embedded product targets Back to the example of Driving with Skill…the summative assessment is a performance. Formative/Classroom Assessment for Learning

Assess How? How is the assessment designed? Classroom Assessment for Learning Formative Assessment Let’s not forget the power of personal communication as a method of assessment.

Assess When? When and how often is the assessment given? Classroom Assessment for Learning Formative Assessment

An Analogy A pilot guides a plane or boat toward its destination by taking constant readings and making careful adjustments in response to wind, currents, weather, etc. A teacher using formative classroom assessment practices does the same: Plans a carefully chosen route ahead of time Takes numerous readings along the way - Changes course as conditions dictate

Classroom Assessment for Learning The only difference in Formative Assessment (as described above) and Classroom Assessment for Learning is the student involvement component. Teachers make all of the decisions in formative assessment. CAfL involves the students in self-assessment, record-keeping, tracking their own learning and communicating about their learning.

Student Involvement Once students become involved, assessment for learning looks more like teaching than it does testing. It takes advantage of the power of assessment as an instructional tool that promotes learning rather than an event designed solely for the purpose of evaluating and assigning grades (Davies 2000). Students engage in assessment for learning when they use assessment information to learn how to judge the quality of their own work and set goals for their own improvement.

Assessment for Learning A process during learning that provides students and teachers with a continuing stream of evidence of student progress in mastering the intended learning. Assessment for learning results in a purposeful, ongoing collection of information about how students are learning while there is still time to improve. Formative Classroom Assessment is a collection of practices used by teachers and students that provide information they need to make decisions that promote continued learning occurs while the learning is still happening and throughout the learning process Is used to diagnose students’ needs and support students’ learning All three levels are important. One level cannot replace another.

Assessment for Learning A process during learning that provides students and teachers with a continuing stream of evidence of student progress in mastering the intended learning. Assessment for learning results in a purposeful, ongoing collection of information about how students are learning while there is still time to improve. Formative Classroom Assessment is day-to-day, minute- by-minute activities that involve students in their learning gathered through observation of skills and products not products that can be purchased not used to assign grades All three levels are important. One level cannot replace another.

Review of Research Literature …achievement gains from using such assessment-for-learning strategies were “among the largest ever reported for educational interventions.” -Black and Wiliam (1998) More frequent testing does not necessarily mean greater gains. The strategies Black and Wiliam refer to involve students in the entire process.

A Balanced Approach “Teachers involve their students in classroom assessment, record-keeping, and communication during learning. But, when it’s time for students to be accountable for what they have learned, the teacher takes the lead in conducting assessments OF learning.” -Richard J. Stiggins

Formative or Summative It isn’t the method of assessment that tells us whether it is an assessment of learning (summative) or assessment for learning (formative). Many assessment methods – tests, quizzes, performance tasks, writing essays and data gathered through observation of skills and product - can be used either way. How the results are used tells us if the assessment is of or for learning.

Formative or Summative ? Type of assessment What is the purpose? Who will use the information? How will it be used? Is the use formative or summative? State Test Measure level of achievement on state content standards State Determine AYP District, School, Teacher Teams Determine program effectiveness Identify percentage of students meeting performance standards on state content standards Comparison of schools/districts Develop programs/interventions for groups or individuals District Benchmark, Interim, or Common Assessment Measure level of achievement toward state content standards Identify program needs Identify students needing additional help District, School, Teacher Teams, Teachers Plan interventions for groups or individuals

Formative or Summative ? Classroom Assessment Measure level of achievement on learning targets taught Teachers Determine report card grade Diagnose student strengths and areas needing re-teaching Teacher Teams, Teachers Revise teaching plans for next year/semester Plan further instruction/differentiate instruction for these students Teachers, Students Provide feedback to students Understand strengths and areas needing work Students Self-assess, set goals for further study/work

Ongoing Formative Classroom Assessment Cycle Team Processes for Balanced Assessment Practices Assess and monitor for progress Repeat intervention loop as needed Periodic School-Based Benchmark Assessments Periodic District-Wide Benchmark Assessments Instruction and ongoing Assessment Intervention and Enrichment Cycle School Year Summative Assessment Cycle Establish school SMART goals (for school year) based on analysis of summative data Revise curriculum, instruction and assessments as needed Administer WESTEST2 - State Accountability Create an intervention Plan Analyze WESTEST2 Ongoing Formative Classroom Assessment Cycle Analyze assessment results Collaborative teams establish SMART goals based on grade-level/content data Balanced assessment literacy begins by understanding the overall process and the role each piece plays in effectively collaborating and using common assessments. The team work that PLCs engage in initially is described in the boxes to the left of the graphic. The yellow star is where we begin to establish the common assessment process. Monitor learning for progress Identify learning targets and learning targets Instruction and ongoing Assessment Design formative assessments Starting points West Virginia Department of Education Developed by the Office of District Leadership Development Contact Lisa Youell Or Linda Bragg at 304-558-3199 Collaborative Teamwork 46 46

Ongoing Formative Classroom Assessment Cycle Assess and monitor for results Team Processes for Formative Classroom Assessment Practices Repeat intervention loop as needed Instruction and ongoing Assessment Intervention and Enrichment Cycle Establish school SMART goals (for school year) based on analysis of summative data Revise curriculum, instruction and assessments as needed Create an intervention Plan Ongoing Formative Classroom Assessment Cycle Analyze assessment results Collaborative teams establish SMART goals based on grade-level/content data Balanced assessment literacy begins by understanding the overall process and the role each piece plays in effectively collaborating and using common assessments. The team work that PLCs engage in initially is described in the boxes to the left of the graphic. The yellow star is where we begin to establish the common assessment process. Monitor learning for results Identify learning targets and learning targets Instruction and ongoing Assessment Design formative assessments Starting points West Virginia Department of Education Developed by the Office of District Leadership Development Contact Lisa Youell Or Linda Bragg at 304-558-3199 Collaborative Teamwork 47 47

Balanced Assessment Practices Professional Learning Community Assessment FOR Learning FORMATIVE CLASSROOM Assessment OF Learning SUMMATIVE Balanced Assessment Practices through Professional Learning Community Processes School Collaborative Teams examine student specific data and establish team SMART goals Assessment processes and practices teachers and students use to gather evidence for the purpose of improving Monitor progress of School SMART goals Leadership Team examines school data and establishes school SMART goals based on summative data Key Decisions What comes next in the learning? How can instruction be adjusted based on results/data? Which learning targets require additional attention? Uses Provides descriptive feedback to students Enables students to take responsibility for the learning Supports student motivation and growth Are enough students meeting standards? What changes need to be made about district and school programs and resources? Measures achievement status at a point in time for purpose of reporting and accountability Communicates evidence of performance to parents and community Assessments that provide evidence of student achievement for the purpose of making a judgment about student competence or program effectiveness SCHOOL-WIDE FOCUS ON LEARNING Addressing the Four Critical Questions What do we want students to learn? What knowledge, skills and dispositions do we expect them to acquire? How will we know if students are learning the essential skills, concepts and dispositions we have deemed most essential? How will we respond when some of our students do not learn? What processes are put in place to ensure students receive additional time and support for learning in a timely, directive and systematic way? 4. How do we enrich and extend the learning for students who are already proficient? Monitor progress of Team SMART goals HO p. 80 The first part of team literacy is understanding the overall process and the role each piece plays in effectively collaborating and using common assessments. The team work that PLCs engage in initially is described in the boxes to the left of the graphic. The yellow star is where we begin to establish the common assessment process. 48 48

School Year Summative Assessment Cycle Team Processes for Summative Assessment Practices Periodic District-Wide Benchmark Assessments School Year Summative Assessment Cycle Administer WESTEST2 - State Accountability Analyze WESTEST2 Periodic School-Based Benchmark Assessments Establish school SMART goals (for school year) based on analysis of summative data Balanced assessment literacy begins by understanding the overall process and the role each piece plays in effectively collaborating and using common assessments. The team work that PLCs engage in initially is described in the boxes to the left of the graphic. The yellow star is where we begin to establish the common assessment process. Starting points West Virginia Department of Education Developed by the Office of District Leadership Development Contact Lisa Youell Or Linda Bragg at 304-558-3199 Collaborative Teamwork 49 49

SMART Goals Strategic and specific Measurable Attainable Results-oriented Time bound

Are These SMART Goals? Strategically aligned with the school-wide goal of improving student performance in language arts, by the end of the 2008/09 school year we will: Develop and administer four common assessments in the area of writing. Increase the use of cooperative learning activities in our reading classrooms by 25%. Increase the number of students achieving the target score (80% or higher) on the district reading assessment from 81% to 90%. But never to evaluate teachers…. West VA Strategic Plan

Strategic Middle School Writing Assessment indicates a problem at every grade level. All content area teams set goals to improve student writing. Elementary students are having trouble with reading non-fiction, informational text. High School sees that content vocabulary is an issue.

Goal Alignment Translate district initiatives into school goals. Identify a limited number of very focused school goals. Use well-designed processes to drive those goals into every classroom in your school. Link school goals to team goals. Focus on results, not activities. Page 10

Ongoing Formative Classroom Assessment Cycle Team Processes for Balanced Assessment Practices Assess and monitor for progress Repeat intervention loop as needed Periodic School-Based Benchmark Assessments Periodic District-Wide Benchmark Assessments Instruction and ongoing Assessment Intervention and Enrichment Cycle School Year Summative Assessment Cycle Establish school SMART goals (for school year) based on analysis of summative data Revise curriculum, instruction and assessments as needed Administer WESTEST2 - State Accountability Create an intervention Plan Analyze WESTEST2 Ongoing Formative Classroom Assessment Cycle Analyze assessment results Collaborative teams establish SMART goals based on grade-level/content data Balanced assessment literacy begins by understanding the overall process and the role each piece plays in effectively collaborating and using common assessments. The team work that PLCs engage in initially is described in the boxes to the left of the graphic. The yellow star is where we begin to establish the common assessment process. Monitor learning for progress Identify learning targets and learning targets Instruction and ongoing Assessment Design formative assessments Starting points West Virginia Department of Education Developed by the Office of District Leadership Development Contact Lisa Youell Or Linda Bragg at 304-558-3199 Collaborative Teamwork 54 54

Lisa Youell Division of Educator Quality and System Support West Virginia Department of Education lyouell@access.k12.wv.us 304-558-3199