Benchmarking International Standards Audience: The audience for this PowerPoint is for all educators. Instructor Notes: This presentation is a broad look at the international benchmarking of standards. Much of the information was found in the document found on the Core Standards website: Benchmarking for Success: Ensuring U.S. Students Receive a World-Class Education. National Governors Association, Council of Chief State School Officers, and Achieve, Inc., 2008. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Benchmarking Definitions “A process of comparing the quality of a specific process or method to another that is widely considered to be an industry standard or best practice.” “Benchmarking is the practice of being humble enough to admit that someone else has a better process and wise enough to learn how to match or even surpass them.” American Productivity and Quality Center from Benchmarking for Success: Ensuring U.S. Students Receive a World-Class Education a report by the NGA, CCSSO and Achieve Instructor notes: These two quotes on benchmarking come from the document cited. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Goals of International Comparisons Gain insight to improve educational systems Access global knowledge and best practices Necessary to compete in our global economy Compare educational performance over time Instructor notes: The Introduction of the Benchmarking document sums up the need for the U.S. to use international benchmarking in the development of its education policies and standards: “We are living in a world without borders. To meet the realities of the 21st century global economy and maintain America’s competitive edge into the future, we need students who are prepared to compete not only with their American peers, but with students from all across the globe for the jobs of tomorrow. States have voluntarily taken the lead in developing standards-based education, but policymakers lack a critical tool for moving forward—international benchmarking. This report is intended to help states take the next steps toward ensuring that American students receive a world-class education that positions them to compete and innovate in the 21st century. International benchmarking will help state policymakers identify the qualities and characteristics of education systems that best prepare students for success in the global marketplace. The stakes are high, and improving our educational system will require commitment and insight not just from state leaders but many other stakeholders as well. With this in mind, the National Governors Association, the Council of Chief State School Officers, and Achieve, Inc. have joined to provide to states a roadmap for benchmarking their K-12 education systems to those of top-performing nations.” By internationally benchmarking our standards, we also have a structure to compare our achievement over time, with other countries through international assessments. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

International Comparison: Areas of Educational Benchmarking Standards Accountability Educator workforce Assessment Instructor notes: It is important to realize that international benchmarking covers more that just the standards. All parts of the education system need to be included in the comparisons. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Characteristics of Standards in the best-performing nations Focus: Deeper Rigor: Complex and challenging Coherence: Orderly progression Fluency: Proficient and automatic Instructor notes: For standards’ benchmarking, a number of characteristics were identified as being common to the standards of high-performing countries. These characteristics also come from the Benchmarking document. Focus: In the U.S. content standards cover a large number of topics ending up with a curriculum that is “a mile wide and an inch deep.” Rigor: The curriculum studied by the typical American eighth-grader is two full years behind the curriculum being studied by eighth-graders in high-performing countries. Coherence: Needs to be an orderly progression of topics that follow the logic of the discipline, not a laundry list of topics which result in too much repetition across grades. According to Bill Schmidt, a Michigan State University researcher and expert on international benchmarking says, “In the United States the principle that seems to guide our curriculum development is that you teach everything everywhere because then somehow somebody will learn something somewhere.” © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Instructor notes: This chart is a summary of the the three international assessments of educational progress. The United States participates in all three. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Benchmarking International Standards Mathematics Benchmarking International Standards © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Focus for Math Assessments Same skills assessed across countries International data provided since 1995 Focus on international economics & employment Instructor notes: The international assessments have been providing data since 1995. However, since the Common Core standards have now been internationally benchmarked, it will make future achievement data comparisons with other countries much more relevant. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

International Assessments: Mathematics Programme for International Student Assessment (PISA) Assessments: Focus on determining if students can use what they have learned in the situations they are likely to encounter in their daily life Have students analyze, reason, and communicate effectively as they pose, formulate, solve and interpret mathematical problems in a variety of situations Test how students apply skills to real-world problems Instructor notes: PISA is coordinated by the Organization for Economic Cooperation and Development (OECD), an intergovernmental organization of industrialized countries. Three areas are assessed each administration (reading literacy, mathematics literacy and science literacy. From the PISA website FAQs http://nces.ed.gov/surveys/pisa/faq.asp: “Each PISA data collection effort assesses one of these three subject areas in depth, although all three are assessed in each cycle. The subject covered in depth is considered the major subject area, and the other two subjects are considered minor subject areas for that assessment year. Assessing all three areas allows participating jurisdictions to have an ongoing source of achievement data in every subject area, while rotating one area as the main focus over the years.” In 2003, the major focus area was mathematics. It will be the focus area again in 2012. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

PISA Sample Problem Instructor notes: This is a sample of a PISA problem. Have participants do the problem and discuss how the questions assess both content and application. Share out ideas with the large group. What is the approximate distance from the starting line to the beginning of the longest straight section of the track? What is the lowest speed recorded during the second lap? What can you say about the speed of the car between the 2.6 km and 2.8 km marks? © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

International Assessments: Mathematics Trends in International Mathematics and Science Study (TIMSS) Assessments: Place emphasis on questions and tasks that offer better insight into students’ analytical, problem solving and inquiry skills and capabilities Test both content and cognitive domains Content: Number, Geometry, Algebra, Data and Chance, etc. Cognitive: Knowing, Applying, Reasoning Instructor notes: The following statements are from the TIMSS website overview http://nces.ed.gov/timss/index.asp The Trends in International Mathematics and Science Study (TIMSS) provides reliable and timely data on the mathematics and science achievement of U.S. 4th- and 8th-grade students compared to that of students in other countries. TIMSS data have been collected in 1995, 1999, 2003, and 2007. In 2011, more than 60 countries and jurisdictions, including the United States, will participate in TIMSS. More than 20,000 students in more than 1,000 schools across the United States will take the assessment in spring 2011, joining almost 500,000 other students around the world taking part in TIMSS. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

TIMSS Sample Problem John and Cathy were told to divide a number by 100. By mistake John multiplied the number by 100 and obtained an answer of 450. Cathy correctly divided the number by 100. What was her answer? If n is a negative integer, which of these is the largest number? a) 3 + n b) 3 × n c) 3 − n d) 3 ÷ n Instructor notes: These are two sample problems from the TIMSS eighth grade assessment. Have participants do the problems and discuss what implications these types of assessments have on current instructional and assessment practices in their school/district setting. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

International Benchmarking: Composite Standards for Mathematics A model to compare US standards with three high-performing countries: Hong Kong, Singapore and Korea. Focus on learning progressions Transform the Asian standards into an organization of mathematics content Create a set of composite standards by topic and grade. Informing Grades 1-6 Mathematics Standards Development: What Can be Learned from High-performing Hong Kong, Korea and Singapore. American Institutes for Research 2009 Instructor notes: The document listed described a process for making a set “composite” mathematics standards based on the standards of Hong Kong, Singapore and Korea. These standards were used in helping develop the Common Core standards. The process: 1. Focused on learning progressions—how systematically the mathematical content progresses across the grades within a broad mathematical topic. 2. Took the content and organized it by strand across the grade levels © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Composite Standard Development Identify the core mathematics topics taught within each strand across countries Identify each country’s grade-by-grade sequence of mathematical competencies for each core topic Create composite standards for each core topic by drawing from the learning progressions in the standards in each of the three countries Instructor notes: The process followed these steps in the creation of the “composite” standards. Identify the core mathematics topics for each strand. These topics are generally apparent from the topic structure of the standards. Look at the grade by grade sequence for each topic. Create composite standards. The rules that applied were as follows: If core topic contents were similar for the three standards grade by grade, the content of the standard that was judged to offer the clearest learning progression was chosen. • If core topic contents were similar overall across standards but differed in some respects grade by grade, the composite standard reflected a judgment as to which standard offered the most in-depth or clearest learning progression. • If core topic contents differed on some competency that was not in the other standards, that competency was included only if it added to and was consistent with the overall learning progression. Direct participants to pages 1-4 of the Comparison packet to highlight standards comparison for Composite, Common Core and current California state standards. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Benchmarking International Standards Reading Literacy Benchmarking International Standards Instructor notes: Next, we are going to explore international benchmarking for Reading Literacy. As we move through the presentation, I will also refer to the Comparison packet to share and highlight some correlations. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Focus on Reading Literacy Assessments Inclusion of reading, mathematical & science literacy International data provided since 2001 Emphasis on reading comprehension in country’s native language Instructor notes: We’ll begin with a brief overview of what we know about the focus of international assessments for Reading Literacy. So far, we know that we have reading literacy assessments that includes not just reading, but also math and science literacy. We also know that we have had international data since 2001 and all reading comprehension assessments are given in the country’s native language. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Introduction: Reading Literacy Major international assessments: The Progress in Reading Literacy Study (PIRLS) Programme for International Student Assessment (PISA) Instructor notes: Reading literacy is one of the most important abilities students acquire as they progress through their school years. It is foundational to learning across all subject areas, personal growth, and it equips students with the ability to participate fully in the larger society. Therefore, to evaluate the quality and alignment of standards, it is important to consider the assessments that will be used to compare the performance of our students with students from other nations. Currently, there are two major international assessments used to measure student progress in reading literacy. The Progress in Reading Literacy Study (PIRLS) and Programme for International Student Assessment ( PISA). Let’s begin with a closer look at The Progress in Reading Literacy Study also known as PIRLS. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

The Progress in Reading Literacy Study: PIRLS Instructor notes: PIRLS was established in 2001 by the International Association for the Evaluation of Educational Achievement (IEA) in response to the need for data on trends in reading literacy around the world. It is conducted every 5 years, previous PIRLS assessments took place in 2001 and 2006. 2011 will be the third cycle for PIRLS. Many countries, who also participated in 2001 and 2006 study cycles, will now have comprehensive data regarding reading literacy achievement across three time points. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

PIRLS: Purpose Measure trends in reading literacy achievement. Improve the teaching of reading and the acquisition of reading skills around the world. Assess reading achievement for students in their fourth year of school. Instructor notes: PIRLS was designed to measure progress and trends in students’ reading achievement as well as the experiences they have at home and at school in learning to read. The data PIRLS provide will not only identify reading literacy trends, but also help inform how well students acquired the reading skills and improve the teaching of reading literacy. PIRLS assesses students in fourth grade. Fourth grade was chosen because at this juncture, students are transitioning from learning to read to reading to learn. Assessing reading literacy at this point will provide an insight to whether students can apply the skills they have gained in the previous years of school. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

PIRLS: Reading Literacy Definition PIRLS defines reading literacy as the ability to understand and use those written language forms required by society and/or valued by the individual. Young readers can construct meaning from a variety of texts. They read to learn, to participate in communities of readers in school and everyday life, and for enjoyment. Instructor notes: To better understand how PIRLS assessments are structured, it is important to know how PIRLS defines reading literacy. In 2001, the PIRLS framework for assessing reading literacy was developed along with its definition of reading literacy. PIRLS definition joins the terms reading and literacy to convey a broad notion that includes the ability to reflect on what is read and to be able to use it as a tool for attaining individual and societal goals. In PIRLS 2006, the definition of reading literacy was refined to highlight the widespread importance of reading in school and everyday life. This includes actively constructing meaning and reflecting as they interact with different types of texts and for different purposes—to learn, to participate in society, and for enjoyment. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

PIRLS: Purposes for Reading The PIRLS assessment focuses on the two overarching purposes for reading : Reading for literary experience (50%) Reading to acquire and use information (50%) Instructor notes: Next, we are going to look at the structure of the PIRLS assessment. Purposes for reading and processes of comprehension are the foundational aspects of PIRLS written assessment of reading comprehension. PIRLS focuses on two overarching purposes: Reading for literary experience- the reader engages with text to become involved in imagined events, setting, actions just to name a few. The main form of literary texts used in PIRLS is narrative fiction Reading to acquire and use information-reader engages with aspects of the real world they read to acquire information and use it or act in accordance with what is read. Informational text varies in organization so it how readers read it varies as well. Procedural—instructions, biographies, reading for cause/effect or compare/contrast. Because both types of reading are important at this age, the PIRLS assessment contains an equal proportion (50%) of material assessing each purpose. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

PIRLS: Comprehension Processes Focus on and retrieve explicitly stated information (20%) Make straightforward inferences (30%) Interpret and integrate ideas and information and textual elements (30%) Examine and evaluate content, language, and textual elements (20%) Instructor notes: In addition to the two overarching purposes, PIRLS also takes into consideration how readers construct meaning as they read. Four types of comprehension processes are: Focus on and retrieve explicitly stated information: readers use various ways to locate and understand content that is relevant to the question posed. Some reading tasks that may exemplify this type of processing include– looking for specific ideas, identifying information that is relevant to the specific goal of reading or relevant to the question posed. Make straightforward inferences- readers must use the information in the text and connect two or more ideas or pieces of information. Although the ideas may be explicitly stated, the connection between them is not, and thus must be inferred. Tasks includes- inferring that one event caused another event and identifying generalizations made in the text. Interpret and integrate ideas and information and textual elements involves using the ideas and information from the text and also drawing on their understanding of the world. Tasks includes discerning the overall message or theme of a text and comparing and contrasting text information. Examine and evaluate content, language, and textual elements involves a shift from constructing meaning to critically considering the text itself by examining the structure and language to determine how meaning is presented. Tasks includes judging the completeness or clarity of information in the text and determining an author’s perspective on the central topic. The four processes are assessed within each purpose for reading. As you can see, each process is accounted differently on the assessment. Direct participants to pages 6-8 of Comparison packet to highlight standards comparison for Common Core and current California state standards. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

Programme for International Student Assessment: PISA Instructor notes: Now, let’s take a look at the other major international reading literacy assessment- The Programme for International Student Assessment or PISA. The Programme for International Student Assessment is developed jointly by member countries of the Organization for Economic-Co-operation and Development (OECD). United States is currently a member of OECD. PISA assessments takes place every three years and focuses on three domains- reading, mathematical, and scientific literacy. While the three domains form the core of each cycle, 2/3 of the assessment time is devoted to a “major” domain. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

PISA: Purpose Measure how well students are prepared to meet the challenges they may encounter in future life. Tested Age-15 Instructor notes: PISA is an international assessment of the skills and knowledge of 15 year- olds. PISA’s goal is to assess whether students approaching the end of their mandatory education have acquired the skills that are essential for full participation and successfully meeting the challenges in society Let’s take a look at how PISA defines Reading Literacy. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

PISA: Reading Literacy Definition Reading literacy is understanding, using, reflecting on and engaging with written texts, in order to achieve one’s goals, to develop one’s knowledge and potential, and to participate in society. Instructor notes: Because PISA assesses students at the end of mandatory education, PISA defines Reading Literacy as understanding, using, reflecting on and engaging with written texts. Engagement with text was added in PISA 2009 as an important aspect of reading literacy. There is a strong emphasis on how readers apply the skills as they are interacting, reflecting, and engage with the different types of text in order to build their knowledge and participate in society. In addition- the PISA 2009 Reading framework was updated to include assessments of reading electronic texts. PISA 2009 results are now available at http://www.oecd.org © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

PISA Skills Access and retrieve Integrate and interpret Reflect and evaluate Instructor notes: Similar to PIRLS, PISA also assess very specific skills or aspects– the mental strategies, approaches or purposes that readers use to negotiate their way into, around, and between texts. Access and retrieve- readers being able to locate and retrieve one or more distinct pieces of information. Tasks include-locating details required by an employer from a job advertisement or finding a particular fact to support or disprove a claim someone has made. Integrate and interpret- readers are able to process what is read to make internal sense of text. This will require readers to be able to make meaning by reading between the lines, make connections within the text and come to an understanding of what was read. Basically, they are inferring as they are reading. Tasks include- identifying a theme of a literary text, or explaining the purpose. Reflect and evaluate- readers must be able to draw upon outside sources and related it to the information provided in the text as well as compare it to their own beliefs and experiences. Tasks include-Assessing the claims make in the text against their own knowledge of the world, and also be able to defend their own point of views. It may also include tasks that requires students to evaluate how an author portrays a characteristic or persuades the reader. These skills are aligned with NAEP skills- Locate and Recall, Integrate and Interpret, and Critique and Evaluate Direct participants to the handout pgs. 9-11 for examples from Common Core Standards and current California standards. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association

References Benchmarking for Success: Ensuring U.S. Students Receive a World-Class Education. National Governors Association, Council of Chief State School Officers, and Achieve, Inc., 2008 (http://www.achieve.org/benchmarkingforsuccess) Comparative Indicators of Education in the United States and Other G-8 Counties: 2009 (http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2009039) IEA’s Progress in International Reading Literacy Study in Primary School in 40 Countries (http://timss.bc.edu/pirls2006/intl_rpt.html) Overview of IEA’s PIRLS Assessment 2011 http://www.timss.bc.edu/pirls2011/download PIRLS2011_Framework.pdf) PISA 2009 Assessment Framework: Key competencies in reading, mathematics and science (http://www.oecd.org/dataoecd/11/40/44455820.pdf) Informing Grades 1 – 6 Mathematics Standards Development: What Can Be Learned From High-Performing Hong Kong, Korea , and Singapore? (http://www.air.org/focus-area/education/index.cfm?fa=viewContent&content_id=670) The Second Derivative: International Benchmarks in Mathematics for U.S. States and School Districts (http://hub.mspnet.org/index.cfm/20749) U. S. Performance Across International Assessments of Student Achievement: Special Supplement to The Condition of Education 2009 National Center for Education Statistics (http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2009083) Foundations for Success: The Final Report of the National Mathematics Advisory Panel. US Department of Education, 2008 (http://www2.ed.gov/about/bdscomm/list/mathpanel/report/final-report.pdf ) Instructor notes: For more information, we have provided you with a list of the references that we used. Point out the 3rd and 4th reference for sample items. © 2011 California County Superintendents Educational Services Association 9/20/2018 © 2011 California County Superintendents Educational Services Association