1. The Ohio State University College of Education School of Educational Policy and Leadership Policy Implications of PISA, TIMSS, & World-Class University Rankings The Ohio State University College of Education School of Educational Policy and Leadership Policy Implications of PISA, TIMSS, & World-Class University Rankings Chuing Prudence Chou ( 周祝瑛 ) Professor, Cheng-chi University, Taiwan Email: iaezcpc@nccu.edu.tw May 17th, 2007 iaezcpc@nccu.edu.tw

2. YOU ARE GOING TO SHARE ABOUT… What are the Policy Implications of PISA, TIMSS and the World-class University Rankings? What are the Policy Implications of PISA, TIMSS and the World-class University Rankings? What lessons can we learn from these global rankings from a comparative perspective? What lessons can we learn from these global rankings from a comparative perspective?

3. ARE YOU IN OR NOT? PISA (OECD) PISA (OECD) TIMSS (IEA) TIMSS (IEA) World-class University Rankings World-class University Rankings

4. IS THIS EVERYBODY’S BUSINESS? The Finnish education phenomenon The Finnish education phenomenon The German PISA shock The German PISA shock Mexican ’ s indifferent attitude Mexican ’ s indifferent attitude (For PISA and TIMSS) (For PISA and TIMSS) Brain Korea 21 (BK21) Brain Korea 21 (BK21) Japanese public university incorporation Japanese public university incorporation China ’ s 211 and 985 Project China ’ s 211 and 985 Project Taiwanese world-class university funding program, etc Taiwanese world-class university funding program, etc (For World-class University Rankings) (For World-class University Rankings)

5. WHAT ARE THESE BENCHMARKS? PISA (Program for International Student Assessment) (OECD), first in 2000 and then held every 3 years. PISA (Program for International Student Assessment) (OECD), first in 2000 and then held every 3 years. Three domains: Three domains: reading literacy, reading literacy, mathematical literacy, & mathematical literacy, & scientific literacy. scientific literacy.

6. TIMSS ( Trends in International Mathematics and Science Study ) IEA IEA Focus on international mathematics and science achievement assessment. Focus on international mathematics and science achievement assessment. First data collected in 1995, and 1999, 2003 and 2007. The United States is collecting TIMSS data now. First data collected in 1995, and 1999, 2003 and 2007. The United States is collecting TIMSS data now.

7. PISA PISA TIMSS TIMSS Sponsor OECD OECD IEA IEA Framework Expert-driven Expert-driven Mathematics Mathematics curriculum-driven curriculum-driven Top-down approach Bottom-up Bottom-up approach approach Real-world Real-world application application Classroom math Classroom math knowledge knowledge Problem-solving Problem-solving Curriculum content Curriculum content

8. Target Target group group An age-based An age-based sample sample A grade-based A grade-based sample sample 15-year- olds 15-year- olds Grade 3 & 4, 7 & 8 7 & 8 + high school seniors + high school seniors

9. WHAT ARE THE TEST OUTCOMES? There is a greater difference between Asian countries/regions and other countries in TIMSS rather than in PISA. There is a greater difference between Asian countries/regions and other countries in TIMSS rather than in PISA. Most English-speaking countries are ahead of Eastern European counterparts in. PISA, but fall behind in TIMSS. Most English-speaking countries are ahead of Eastern European counterparts in. PISA, but fall behind in TIMSS.

10. WHY SUCH DIFFERENCES EXIT? TIMSS items tend to be more curriculum-based (more universal), TIMSS items tend to be more curriculum-based (more universal), PISA items are more application-based (more real-life experience and cultural-bound) PISA items are more application-based (more real-life experience and cultural-bound)

12. TIMSSTIMSS (1997) TIMSS (13-year-old's average score, Third International Math and Science Study) Countries: (sample) GlobalrankMathScience ScoreRankScoreRank Singapore 1 643 643 1 607 607 1 Japan 2 605 605 3 571 571 3 South Korea South Korea 3 607 607 2 565 565 4 Czech Republic Czech Republic 4 564 564 6 574 574 2 England 18 18 506 506 25 25 552 552 10 10 Thailand 20 20 522 522 20 20 525 525 21 21 Germany 22 22 509 509 23 23 531 531 19 19 France 23 23 538 538 13 13 498 498 28 28 United States United States 24 24 500 500 28 28 534 534 17 17 Source: 1997 TIMSS, in The Economist, March 29th 1997. TIMSSThe EconomistTIMSSThe Economist

13. TIMSSTIMSS (2003) TIMSS (13-year-old's average score, Third International Math and Science Study) Countries: (sample) GlobalrankMathScience ScoreRankScoreRank Singapore 1 605 605 1 578 578 1 Taiwan 2 585 585 4 571 571 2 South Korea South Korea 3 589 589 2 558 558 3 Hong Kong Hong Kong 4 586 586 3 556 556 4 Japan 5 570 570 5 552 552 5 Netherlands 7 536 536 7 9 England 10 10 498 498 18 18 544 544 7 United States United States 12 12 504 504 15 15 527 527 11 11 Malaysia 18 18 508 508 10 10 510 510 21 21 Italy 23 23 484 484 22 22 491 491 22 22 Sources:TIMSS Math 2003 and TIMSS Science 2003 TIMSS Math 2003TIMSS Science 2003TIMSS Math 2003TIMSS Science 2003

14. Trends in PISA math achievement 20002003 Japan 557 557 534 534 Korea 547 547 542 542 Netherland 537 537 538 538 Finland 536 536 544 544 Australia 533 533 524 524 Canada 533 533 532 532 Switzerland 529 529 527 527 UK 529 529 Belgium 520 520 529 529 France 517 517 OECD 500 500 Germany 490 490 503 503 Russia 478 478 New Zealand US 493 523 523

15. There were 38 participating countries in 2003 PISA US ranked ： 7th in reading literacy 7th in reading literacy (15th in 2000 out of 32 countries) (15th in 2000 out of 32 countries) 24th in math literacy 24th in math literacy (19th in 2000) (19th in 2000) 9th in science literacy 9th in science literacy (14th in 2000) (14th in 2000) 20th in problem solving 20th in problem solving

16. Trends in PISA science achievement 20002003 Japan 548 548 550 550 Finland 548 548 538 538 Korea 552 552 Australia 525 525 528 528 Netherland 524 524 528 528 New Zealand 521 521 Canada 519 519 529 529 Switzerland 513 513 France 511 511 Belgium 509 509 Germany 502 502 Ireland 513 513 UK 532 532 Swiss 512 512 AustriaUS 499 519 519

17. Trends in PISA reading literacy achievement 2003 2003 2003 Finland 543 543 Korea 534 534 Canada 528 528 Australia 525 525 New Zealand 522 522 Ireland 515 515 Swiss 514 514 Netherland 513 513 Belgium 507 507 Norway 500 500 Switzerland 499 499 Japan 498 498 Poland 497 497 France 496 496

18. There were 38 participating countries in 2003 PISA US ranked ： 7th in reading literacy 7th in reading literacy (15th in 2000 out of 32 countries) (15th in 2000 out of 32 countries) 24th in math literacy 24th in math literacy (19th in 2000) (19th in 2000) 9th in science literacy 9th in science literacy (14th in 2000) (14th in 2000) 20th in problem solving 20th in problem solving

19. As the provision of higher educational As the provision of higher educational opportunities becomes increasingly opportunities becomes increasingly international, institutional comparison international, institutional comparison becomes more in demand. The two most becomes more in demand. The two most frequently cited rankings ： frequently cited rankings ： 1. Shanghai Jiaotong University, 1. Shanghai Jiaotong University, Academic Ranking of World Universities Academic Ranking of World UniversitiesAcademic Ranking of World UniversitiesAcademic Ranking of World Universities 2. Times Higher Education Supplement, London, Times Higher University World Rankings 2. Times Higher Education Supplement, London, Times Higher University World Rankings Times Higher University World Rankings Times Higher University World Rankings

20. WHO NEEDS WORLD-CLASS UNIVERSITIES? FROM WHOES PERSPECTIVES? Becoming international, becoming Becoming international, becoming competition-driven competition-driven More pressure from domestic and abroad More pressure from domestic and abroad Who can afford joining the "World-Class Club"? Who can afford joining the "World-Class Club"? Who benefits from these global, world-class competition? Who benefits from these global, world-class competition?

21. MYSTERIES UNRESOLVED? What does that mean by the "World Class University"? What does that mean by the "World Class University"? How many world class universities do we need? How many world class universities do we need? Does every country need one? Does every country need one? How could universities be evaluated and compared in an international scope? How could universities be evaluated and compared in an international scope? What measures should be taken in order to establish world class universities? What measures should be taken in order to establish world class universities? Who share the cost and who benefit? Who share the cost and who benefit?

22. World’s top 15 universities as an average of THES & SJTU 2005 rankings Rank Av. Rank InstitutionCountryTimesShanghai 11 Harvard Harvard U.S.11 22.5CambridgeU.K.32 33.5 Massachusetts Inst. of Technology U.S.25 44StanfordU.S.53 55 University of California, Berkeley U.S.64 67 California Inst. of Technology U.S.86 67OxfordU.K.410 88.5PrincetonU.S.98 99YaleU.S.711 1013CornellU.S.1412 1113.5ChicagoU.S.189 1113.5ColumbiaU.S.207 1317.5 University of California, S.F. U.S.1718 1418TokyoJapan1620 1418 Imperial College London U.K.1323 Sources: Shanghai Jiaotong Academic Ranking of World Universities – 2005; Times Higher Education Supplement World University Rankings – 2005.

23. Rank Av. Rank InstitutionCountry Tim es Sha ngh ai 1621.5DukeU.S.1132 1723Johns HopkinsU.S.2719 1823.5University of PennsylvaniaU.S.3215 1924Swiss Federal Institute of Technology, ZurichSwitzerland2127 2025.5University of California, L.A.U.S.3714

24. WHAT CAUSES THE GLOBAL RANKINGS AND WHAT THE EFFECTS ARE? The "Impact of Globalization " The "Impact of Globalization " Borrowing and lending reform strategies Borrowing and lending reform strategies In comparative education, there is a great need to examine how local reactions are to these external global forces. In comparative education, there is a great need to examine how local reactions are to these external global forces. In policy studies, a new paradigm for educational policy-making and school reform is expected to be under way. In policy studies, a new paradigm for educational policy-making and school reform is expected to be under way.

25. What can be learned and imported from elsewhere? (borrowing) What can be learned and imported from elsewhere? (borrowing) What can be taught and exported elsewhere? (lending), What can be taught and exported elsewhere? (lending), The phenomena of cross-national policy attraction and educational borrowing The phenomena of cross-national policy attraction and educational borrowing Global and international convergence in education at the expense local needs and cultural diversity Global and international convergence in education at the expense local needs and cultural diversity

26. Three types of National Policy Responses According to Steiner-Khamsi (2007), three types of policy reactions after international comparisons: According to Steiner-Khamsi (2007), three types of policy reactions after international comparisons: 1. Scandalization (to blame one's own weak educational system), weak educational system), 2. Glorification (to praise one's good educational system), educational system), 3. Indifference to ranking results.

27. Typology of Political Reactions to International Comparative Studies TIMSS TIMSS PISA PISA World-Class University Rankings Scandalization Scandalization USA USA Germany, Germany, Israel Israel Korea, Japan, Korea, Japan, Taiwan Taiwan Glorification Glorification Japan, Finland Japan, Finland UK, Finland UK, Finland USA,UK USA,UK Indifference Indifference Mexico Mexico

28. IS IT A FAIR GMAE FOR EVERYBODY TO PLAY? What makes countries to react so differently? What makes countries to react so differently? Has scandalization led to increased policy import from other educational systems? Has scandalization led to increased policy import from other educational systems? Has glorification led to increased policy export to other educational systems? Has glorification led to increased policy export to other educational systems? Why some remain indifferent to these global comparisons? Why some remain indifferent to these global comparisons?

29. DO POLICY MAKERS BENEFIT FROM THESE GLOBAL RANKINGS? 1. More evidence-based research in public policy studies. 1. More evidence-based research in public policy studies. 2. More Outcome-based and standard-based criteria 2. More Outcome-based and standard-based criteria in monitoring school reforms in monitoring school reforms 3. A "scientific rationality" in education to appeal to the tax payers the tax payers 4. A legitimate “ self-referential system “ acting as policy 4. A legitimate “ self-referential system “ acting as policy borrowing ( a lesson learned from elsewhere) and a form borrowing ( a lesson learned from elsewhere) and a form of externalization. of externalization.

30. WHY GERMAN KIDS FELL BEHIND? In the PISA study released in 2001, German students scored below the OECD-average with regard to reading literacy. In the PISA study released in 2001, German students scored below the OECD-average with regard to reading literacy. The release of the PISA study led to a major uproar in the media, and calls for policy change and school transformation. The release of the PISA study led to a major uproar in the media, and calls for policy change and school transformation.

31. The Finnish Education Phenomenon The Finnish Education Phenomenon German PISA Shock German PISA Shock Brain Korea 21 (BK21) Brain Korea 21 (BK21) Japanese university incorporation Japanese university incorporation China ’ s 211 and 985 Project China ’ s 211 and 985 Project Taiwanese fifty-trillion dollars within five year ’ s world-class university funding program, etc. Taiwanese fifty-trillion dollars within five year ’ s world-class university funding program, etc. India ’ s brain drain of the university faculty India ’ s brain drain of the university faculty

32. DARE TO SAY NO?! Problems with global rankings : 1. Credibility and reliability issues of PISA 1. Credibility and reliability issues of PISA & TIMSS & TIMSS 2. Danger of the international convergence 2. Danger of the international convergence of educational institutions and policies of educational institutions and policies 3. Near-sighted government reform policies & 3. Near-sighted government reform policies & projects aiming at policy borrowing projects aiming at policy borrowing across national boundaries. across national boundaries.

33. 4. The assumption of “ standardized and universal qualifications ” within universities universal qualifications ” within universities 5. Diagnosed rather than prescribed formula approach, not aiming for solving local educational problems. formula approach, not aiming for solving local educational problems. 6. Indifference to the ranking impact on students, employers, teachers, and school officials.

34. 7. Criticism about these global rankings as the “ fast capitalism ” which benefits ranking benchmarks 8. The advantage of English-speaking countries in the global university rankings 9. Only the richest and the best survive in the world- class university rankings 10. The rich get richer, and the poor get poorer. 11. Increasing polarization of institutional resource generation, public funding, and educational quality nation-wide and world-wide.

35. 12. The cumulative advantage vs. equal opportunities for all opportunities for all 13. Most university ranking criteria focus on academic research productivity rather academic research productivity rather than human aspect in cultivation and than human aspect in cultivation and collegiality. collegiality.

36. 14. Higher education shifted from public good 14. Higher education shifted from public good to private commodity. to private commodity. 15. Many courses and degree programs are 15. Many courses and degree programs are packaged and marketed just like consumer goods, and students are seen as customers with a world of choice in front of them. packaged and marketed just like consumer goods, and students are seen as customers with a world of choice in front of them. 16. “ Exporting ” university services to less developed countries.

37. THE RANKING GAME IS NOT OVER YET, BUT THE LECTURE IS GETTING... International rankings and comparisons will continue in spite of criticism and protest. International rankings and comparisons will continue in spite of criticism and protest. Support from many governments, mass media, educational institutes and the general public. Support from many governments, mass media, educational institutes and the general public. Benchmarks create huge benefits. Benchmarks create huge benefits.

38. QUESTIONS REMAIN… Why the US has not so satisfactory PISA and TIMSS results, but has established so many world-class universities in the countries? Why the US has not so satisfactory PISA and TIMSS results, but has established so many world-class universities in the countries? Does this global ranking competition bring real change or improvement to education at home? Does this global ranking competition bring real change or improvement to education at home? What is the politics of comparison in global ranking? What is the politics of comparison in global ranking?

39. Does Taiwan need to be part of the game? Why? Does Taiwan need to be part of the game? Why?

40. THANK YOU ( 謝謝） THANK YOU ( 謝謝） & GOD BLESS YOU!! GOD BLESS YOU!! Comments are welcome! Comments are welcome! Eamil: iaezcpc@nccu.edu.tw Eamil: iaezcpc@nccu.edu.tw