ED /Supporting English Language Learners in Literacy and Content Knowledge Development (SELL) December 9, 2010

Outcomes By the end of today’s class we will have: 1. Reviewed the nature of high stake’s assessments 2. Examined popular media perceptions about “What’s wrong with American education,” 3. Considered arguments that contradict media reports

High-stakes tests (e.g. MSA, SAT, NAEP*, TIMSS*) give us valuable information for improving our education system. Yes No *The National Assessment of Educational Progress (NAEP) is the largest nationally representative and continuing assessment of what America's students know and can do in various subject areas. *The Trends in International Mathematics and Science Study (TIMSS) provides reliable and timely data on the mathematics and science achievement of U.S. 4th- and 8th-grade students compared to that of students in other countries. TIMSS data have been collected in 1995, 1999, 2003, and 2007.

Chapter 14 Understanding and Using Standardized Tests Viewing recommendations for Windows: Use the Arial TrueType font and set your screen area to at least 800 by 600 pixels with Colors set to Hi Color (16 bit). Viewing recommendations for Macintosh: Use the Arial TrueType font and set your monitor resolution to at least 800 by 600 pixels with Color Depth set to thousands of colors.

Overview Standardized tests Using standardized tests for accountability purposes: High-stakes testing Standardized testing and technology Copyright © Houghton Mifflin Company14–5

Characteristics of standardized tests Designed by people with specialized knowledge and training in test construction Every person who takes the test responds to the same items under the same conditions The answers are evaluated according to the same scoring standards The scores are interpreted through comparison to the scores obtained from a group that took the same test under the same conditions or through comparison to a predetermined standard Copyright © Houghton Mifflin Company14–6

Criteria for evaluating standardized tests Reliability Stability in test performance Validity Test accuracy Content validity, predictive validity, construct validity Normed excellence Norm group representation compared to the general population Examinee appropriateness Appropriateness for a particular group of students Copyright © Houghton Mifflin Company14–7

Validity Content validity evidence How well a test’s items reflect a particular body of knowledge and skill Predictive validity evidence How well a test predicts a student’s future behavior Construct validity evidence How well a test measures some internal attribute of a person Copyright © Houghton Mifflin Company14–8

Types of standardized tests Achievement tests Reveal how much of a subject or skill has been learned Aptitude tests Reveal how much knowledge and skill a student could acquire with effective instruction Copyright © Houghton Mifflin Company14–9

Types of achievement tests Norm-referenced tests &Criterion-referenced Norm-referenced test Test where one’s performance is evaluated with reference to a norming group Criterion-referenced test Test where one’s performance is evaluated with reference to some standard or criterion Copyright © Houghton Mifflin Company14–10

Interpreting standardized test scores Grade equivalent scores Interprets test performance in terms of grade level Percentile ranks Score that indicates the percentage of students who are at or below a given student’s score Standard scores Score that is expressed in terms of standard deviations Stanine score Type of standard score that divides a population into nine groups Copyright © Houghton Mifflin Company14–11

What is high-stakes testing? High stakes testing is… using standardized test scores, either by themselves or in conjunction with other data, to determine whether students get promoted to the next grade or graduate from high school, whether teachers and administrators receive financial rewards or demotions, and whether school districts receive additional state funds or lose their accreditation. Copyright © Houghton Mifflin Company14–12

Current status of high-stakes testing Type of test used (Education Week on the Web, 2001) 40 states use criterion-referenced tests in English 34 states use criterion-referenced tests in math Remaining states use norm-referenced tests in these subjects Copyright © Houghton Mifflin Company14–13

Current status of high-stakes testing Use of test scores (Education Week on the Web, 2001) 11 states hold schools accountable for student learning solely on the basis of students’ test scores 20 states provide financial rewards to schools whose students perform at an acceptable level 14 states can close, restaff, or overhaul schools with low test scores 18 states require students to pass a state-mandated test in order to graduate 3 states require students in certain grades to pass a state- mandated test to be promoted Copyright © Houghton Mifflin Company14–14

Problems with high-stakes testing programs The place of tests in educational reform Characteristics of standardized tests Relationship of tests to state standards Breadth of assessment Use of test results to support remediation Use of test results to support improvements in instruction Impact on curriculum and instructional methods Effects on students Copyright © Houghton Mifflin Company14–15

Standardized testing and technology Using technology to prepare students for assessments Using technology to assess mastery of standards e.g., American College Testing (ACT); Project Essay Grade (PEG) Computer adaptive testing (CAT) computers determine sequence and difficulty level of test items Copyright © Houghton Mifflin Company14–16

What happens… When Tests Go Wild!

'Wake-up call': U.S. students trail global leaders Out of 34 countries assessed, U.S. ranked 14th in reading, 17th in science, 25th in math -Christine Armario, The Associated Press, 12/7/2010 “The three-yearly OECD Programme for International Student Assessment (PISA) report, which compares the knowledge and skills of 15- year-olds in 70 countries around the world, ranked the United States 14th out of 34 OECD countries for reading skills, 17th for science and a below-average 25th for mathematics.”

Read a summary of the Report Read one of the three content summaries of the report and share with the class: Reading: Math: Science:

The Reaction: "This is an absolute wake-up call for America," U.S. Education Secretary Arne Duncan said in an interview with The Associated Press. "The results are extraordinarily challenging to us and we have to deal with the brutal truth. We have to get much more serious about investing in education."

“The wake up call is that neither Secretary of Education Arne Duncan, nor Chester E. Finn Jr., former assistant secretary at the Department of Education, recognizes a bogus study when they read one. You can draw no conclusions from the study because is methodology is flawed. They are using the Mean to compare a subset of a population, Shanghai students, to an entire population, students in the US. But then both Ms Duncan and Mr. Finn are products of a US education and that adds some credulity to the study’s conclusions.” Other responses:

And other Considerations : The problem is poverty: Evidence from Gerald Bracey & Stephen Krashen “The entire basis for the national standards/testing movement is our low scores on international tests when compared to other countries. Our scores, however, are only low because we have such a high percentage of children in poverty, compared to other countries that participate in international tests. When we consider only middle-class children who attend well-funded schools, our math scores are near the top of the world (Payne and Biddle, 1999).”

Stephen Krashen, professor emeritus at the University of Southern California, wrote the following, which was posted on the Schools Matter blog. I have not yet seen an analysis of the impact of poverty on overall PISA scores But data available now tells us that poverty, as usual, had a huge impact on PISA reading test scores for American students. American students in schools with less than 10% of students on free and reduced lunch averaged 551, higher than the overall average of any OECD [Organization for Economic Cooperation and Development country. Those in schools with 10% to 25% of students qualifying for free and reduced lunch averaged 527, which was behind only Korea and Finland.In contrast, American students in schools with 75% or more of children in poverty averaged 446, second to last among the 34 OECD countries.

Save the Last Word for Me Read “The Bracey Report on the Condition of Public Education.” Highlight for further discussion three or four of the most salient or thought-provoking passages you might like to discuss with your classmates.