Highlights from PIRLS and TIMSS 2011 Jack Buckley National Center for Education Statistics Washington, DC December 11, 2012

Overview  What are PIRLS and TIMSS and which education systems participate in them?  How are U.S. students performing compared to their international peers in:  4th-grade reading?  4th- and 8th-grade mathematics?  4th- and 8th-grade science?  How are students performing in participating U.S. states? 2

What are…? 3 Progress in International Reading Literacy Study (PIRLS) 4th-grade reading assessment * In 1999, no grade 4 assessment. Trends in International Mathematics and Science Study (TIMSS) 4th- and 8th-grade mathematics and science assessment Developed by the TIMSS & PIRLS International Study Center at Boston College, under contract to the International Association for the Evaluation of Educational Achievement (IEA)

4 26 education systems education systems 53 education systems

5 44 education systems 37 education systems 47 education systems57 education systems 74 education systems 2011

Participating states 6 9 states participated as separate entities to obtain scores in PIRLS or TIMSS or both Each participated as part of the nation and on its own NCES funded participation at grade 8 of all states except Florida as part of a study to statistically link NAEP (National Assessment of Educational Progress) and TIMSS (Florida received other Education Department funding) PIRLSTIMSS grade 4TIMSS grade 8 Florida North Carolina Florida North Carolina Alabama California Colorado Connecticut Indiana Massachusetts Minnesota

U.S. national sample size 4th grade Schools: 370 Students: 12,726 4th grade Schools: 369 Students: 12,569 8th grade Schools: 501 Students: 10,477 7

U.S. state sample sizes (public schools only) 4th grade (FL) Schools: 77 Students: 2,598 4th grade (FL, NC) Schools: 46 – 77 Students: 1,792 – 2,661 8th grade (AL, CA, CO, CT, FL, IN, MA, MN, NC) Schools: 53 – 82 Students: 1,712 – 2,614 8

READING

PIRLS Definition The ability to understand and use those written language forms required by society and/or valued by the individual. Young readers can construct meaning from a variety of texts. They read to learn, to participate in communities of readers in school and everyday life, and for enjoyment. Content Literary and informational texts Cognitive dimensions Focus on and retrieve Make straightforward inferences Interpret and integrate Examine and evaluate 10 PIRLS 2011 framework

What is on the PIRLS assessment? 11 Percentage of 4th-graders answering correctly: U.S.: 90% Int’l Avg.: 89%

What is on the PIRLS assessment? 12 Percentage of 4th-graders answering correctly: U.S.: 42% Int’l Avg.: 29%

13 U.S. average score (556) higher than the PIRLS scale average (500) Higher than U.S. Not measurably different than U.S. Lower than U.S.

14 U.S. average score (556) lower than in 5 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

15 U.S. average (556) not measurably different than in 7 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

16 U.S. average score (556) higher than in 40 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

17 *p <.05. Change in average scores is significant. NOTE: Education systems ordered according to average reading score in Change in average score

18 *p <.05. Change in average scores is significant. NOTE: Education systems ordered according to average reading score in 2011.

PIRLS international reading benchmarks 19 Grade 4 Advanced (625) Students can interpret figurative language, distinguish and interpret complex information from different parts of text, and integrate ideas across text to provide interpretations about characters’ feelings and behaviors. High (550) Students can recognize some textual features, make inferences on the basis of abstract or embedded information, and integrate information to recognize main ideas and provide explanations. Intermediate (475) Students can identify central events, make straightforward inferences from the text, and begin to make connections across parts of the text. Low (400) Students can retrieve explicitly stated details from literary and informational texts.

Percentages of U.S. 4th-grade students reaching PIRLS reading benchmarks were higher than international medians in NOTE: All U.S. percentages are significantly higher than the corresponding PIRLS international median at the.05 level of statistical significance.

NOTE: Education systems with lower percentages of students reaching the Advanced benchmark than the percentage of U.S. students reaching the Advanced benchmark are not included in figure. 21 Two systems had higher percentages of 4th-grade students reaching Advanced than the U.S. Higher than U.S. (p <.05) Not measurably different than U.S. (p <.05)

MATHEMATICS

TIMSS 2011 mathematics framework 23 TIMSS mathematics Content dimensions Grade 4 Number Geometric Shapes and Measures Data Display Grade 8 Number Algebra Geometry Data and Chance Cognitive dimensions Knowing Applying Reasoning

24 Percentage of 4th-graders answering correctly: U.S.: 80% Int’l Avg.: 54% What is on the TIMSS mathematics assessment?

25 Percentage of 4th- graders answering correctly: U.S.: 76% Int’l Avg.: 65%

26 What is on the TIMSS mathematics assessment? Percentage of 8th- graders answering correctly: U.S.: 87% Int’l Avg.: 70%

27 What is on the TIMSS mathematics assessment? Percentage of 8th- graders answering correctly: U.S.: 66% Int’l Avg.: 45%

28 U.S. average score (541) higher than the TIMSS scale average (500) Higher than U.S. Not measurably different than U.S. Lower than U.S.

29 U.S. average score (541) lower than in 8 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

30 U.S. average (541) not measurably different than in 6 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

31 U.S. average score (541) higher than in 42 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

32 Higher than U.S. Not measurably different than U.S. Lower than U.S. U.S. average score (509) higher than the TIMSS scale average (500)

33 Higher than U.S. Not measurably different than U.S. Lower than U.S. U.S. average score (509) lower than in 11 education systems

34 U.S. average (509) not measurably different than in 12 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

35 U.S. average score (509) higher than in 32 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

36 *p <.05. Change in average scores is significant. NOTE: Education systems ordered according to average mathematics score in 2011.

37 *p <.05. Change in average scores is significant. NOTE: Education systems ordered according to average mathematics score in 2011.

38 *p <.05. Change in average scores is significant. NOTE: Education systems ordered according to average mathematics score in 2011.

TIMSS international mathematics benchmarks Grade 4Grade 8 Advanced (625) Students can apply their understanding and knowledge in a variety of relatively complex situations and explain their reasoning. Students can reason with information, draw conclusions, make generalizations, and solve linear equations. High (550) Students can apply their knowledge and understanding to solve problems. Students can apply their understanding and knowledge in a variety of relatively complex situations. Intermediate (475) Students can apply basic mathematical knowledge in straightforward situations. Low (400) Students have some basic mathematical knowledge. Students have some knowledge of whole numbers and decimals, operations, and basic graphs. 39

Percentages of U.S. 4th-graders reaching TIMSS mathematics benchmarks were higher than international medians in NOTE: All U.S. percentages are significantly higher than the corresponding TIMSS international median at the.05 level of statistical significance.

Higher than U.S. (p <.05) Not measurably different than U.S. (p <.05) NOTE: Education systems with lower percentages of students reaching the Advanced benchmark than the percentage of U.S. students reaching the Advanced benchmark are not included in figure. 41 Seven systems had higher percentages of 4th- grade students reaching Advanced than the U.S.

42 NOTE: All U.S. percentages are significantly higher than the corresponding TIMSS international median at the.05 level of statistical significance. Percentages of U.S. 8th-graders reaching TIMSS mathematics benchmarks were higher than international medians in 2011

NOTE: Education systems with lower percentages of students reaching the Advanced benchmark than the percentage of U.S. students reaching the Advanced benchmark are not included in figure. 43 Eleven systems had higher percentages of 8th- graders reaching Advanced than the U.S. Higher than U.S. (p <.05) Not measurably different than U.S. (p <.05)

SCIENCE

TIMSS 2011 science framework 45 TIMSS science Content dimensionsGrade 4 Earth science Life science Physical science Grade 8 Biology Chemistry Earth science Physics Cognitive dimensions Knowing Applying Reasoning

What is on the TIMSS science assessment? 46 Percentage of 4th- graders answering correctly: U.S.: 96% Int’l Avg.: 83%

What is on the TIMSS science assessment? 47 Percentage of 4th- graders answering correctly: U.S.: 90% Int’l Avg.: 73%

What is on the TIMSS science assessment? 48 Percentage of 8th- graders answering correctly: U.S.: 28% Int’l Avg.: 27%

What is on the TIMSS science assessment? 49 Percentage of 8th- graders answering correctly: U.S.: 76% Int’l Avg.: 61%

U.S. average score (544) higher than the TIMSS scale average (500) 50 Higher than U.S. Not measurably different than U.S. Lower than U.S.

51 Higher than U.S. Not measurably different than U.S. Lower than U.S. U.S. average score (544) lower than in 6 education systems

52 U.S. average (544) not measurably different than in 3 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

53 U.S. average score (544) higher than in 47 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

54 U.S. average score (525) higher than the TIMSS scale average (500) Higher than U.S. Not measurably different than U.S. Lower than U.S.

55 U.S. average score (525) lower than in 12 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

56 U.S. average (525) not measurably different than in 10 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

57 U.S. average score (525) higher than in 33 education systems Higher than U.S. Not measurably different than U.S. Lower than U.S.

58 *p <.05. Change in average scores is significant. NOTE: Education systems ordered according to average science score in 2011.

59 *p <.05. Change in average scores is significant. NOTE: Education systems ordered according to average science score in 2011.

60 *p <.05. Change in average scores is significant. NOTE: Education systems ordered according to average science score in 2011.

61 *p <.05. Change in average scores is significant. NOTE: Education systems ordered according to average science score in 2011.

TIMSS international science benchmarks 62 Grade 4Grade 8 Advanced (625) Students can apply knowledge and understanding of scientific processes and relationships and show some knowledge of the process of scientific inquiry. Students communicate an understanding of complex and abstract concepts in biology, chemistry, physics, and earth science. High (550) Students apply their knowledge and understanding of the sciences to explain phenomena in everyday and abstract contexts. Students demonstrate understanding of concepts related to science cycles, systems, and principles. Intermediate (475) Students have basic knowledge and understanding of practical situations in the sciences. Students recognize and apply their understanding of basic scientific knowledge in various contexts. Low (400) Students have some elementary knowledge of life science and physical science. Students can recognize some basic facts from the life and physical sciences.

63 NOTE: All U.S. percentages are significantly higher than the corresponding TIMSS international median at the.05 level of statistical significance. Percentages of U.S. 4th-graders reaching TIMSS science benchmarks were higher than international medians in 2011

NOTE: Education systems with lower percentages of students reaching the Advanced benchmark than the percentage of U.S. students reaching the Advanced benchmark are not included in figure. 64 Higher than U.S. (p <.05) Not measurably different than U.S. (p <.05) Three systems had higher percentages of 4th- grade students reaching Advanced than the U.S.

65 Percentages of U.S. 8th-graders reaching TIMSS science benchmarks were higher than international medians in 2011 NOTE: All U.S. percentages are significantly higher than the corresponding TIMSS international median at the.05 level of statistical significance.

NOTE: Education systems with lower percentages of students reaching the Advanced benchmark than the percentage of U.S. students reaching the Advanced benchmark are not included in figure. 66 Higher than U.S. (p <.05) Not measurably different than U.S. (p <.05) Twelve systems had higher percentages of 8th- graders reaching Advanced than the U.S.

Summary of change in average U.S. scores over time 67 PIRLS Reading TIMSS Mathematics TIMSS Science TIMSS Mathematics TIMSS Science Grade 4 Grade 8 Change over time in U.S. average scores was statistically significant. Change over time in U.S. average scores was not measurably different.

For more information PIRLS at NCES: International Data Explorer: Contact: Sheila Thompson NCES TIMSS at NCES: International Data Explorer: Contact: Stephen Provasnik NCES