The Ohio State University College of Education School of Educational Policy and Leadership Policy Implications of PISA, TIMSS, & World-Class University Rankings The Ohio State University College of Education School of Educational Policy and Leadership Policy Implications of PISA, TIMSS, & World-Class University Rankings Chuing Prudence Chou ( 周祝瑛 ) Professor, Cheng-chi University, Taiwan May 17th, 2007

YOU ARE GOING TO SHARE ABOUT… What are the Policy Implications of PISA, TIMSS and the World-class University Rankings? What are the Policy Implications of PISA, TIMSS and the World-class University Rankings? What lessons can we learn from these global rankings from a comparative perspective? What lessons can we learn from these global rankings from a comparative perspective?

ARE YOU IN OR NOT? PISA (OECD) PISA (OECD) TIMSS (IEA) TIMSS (IEA) World-class University Rankings World-class University Rankings

IS THIS EVERYBODY’S BUSINESS? The Finnish education phenomenon The Finnish education phenomenon The German PISA shock The German PISA shock Mexican ’ s indifferent attitude Mexican ’ s indifferent attitude (For PISA and TIMSS) (For PISA and TIMSS) Brain Korea 21 (BK21) Brain Korea 21 (BK21) Japanese public university incorporation Japanese public university incorporation China ’ s 211 and 985 Project China ’ s 211 and 985 Project Taiwanese world-class university funding program, etc Taiwanese world-class university funding program, etc (For World-class University Rankings) (For World-class University Rankings)

WHAT ARE THESE BENCHMARKS? PISA (Program for International Student Assessment) (OECD), first in 2000 and then held every 3 years. PISA (Program for International Student Assessment) (OECD), first in 2000 and then held every 3 years. Three domains: Three domains: reading literacy, reading literacy, mathematical literacy, & mathematical literacy, & scientific literacy. scientific literacy.

TIMSS ( Trends in International Mathematics and Science Study ) IEA IEA Focus on international mathematics and science achievement assessment. Focus on international mathematics and science achievement assessment. First data collected in 1995, and 1999, 2003 and The United States is collecting TIMSS data now. First data collected in 1995, and 1999, 2003 and The United States is collecting TIMSS data now.

PISA PISA TIMSS TIMSS Sponsor OECD OECD IEA IEA Framework Expert-driven Expert-driven Mathematics Mathematics curriculum-driven curriculum-driven Top-down approach Bottom-up Bottom-up approach approach Real-world Real-world application application Classroom math Classroom math knowledge knowledge Problem-solving Problem-solving Curriculum content Curriculum content

Target Target group group An age-based An age-based sample sample A grade-based A grade-based sample sample 15-year- olds 15-year- olds Grade 3 & 4, 7 & 8 7 & 8 + high school seniors + high school seniors

WHAT ARE THE TEST OUTCOMES? There is a greater difference between Asian countries/regions and other countries in TIMSS rather than in PISA. There is a greater difference between Asian countries/regions and other countries in TIMSS rather than in PISA. Most English-speaking countries are ahead of Eastern European counterparts in. PISA, but fall behind in TIMSS. Most English-speaking countries are ahead of Eastern European counterparts in. PISA, but fall behind in TIMSS.

WHY SUCH DIFFERENCES EXIT? TIMSS items tend to be more curriculum-based (more universal), TIMSS items tend to be more curriculum-based (more universal), PISA items are more application-based (more real-life experience and cultural-bound) PISA items are more application-based (more real-life experience and cultural-bound)

TIMSSTIMSS (1997) TIMSS (13-year-old's average score, Third International Math and Science Study) Countries: (sample) GlobalrankMathScience ScoreRankScoreRank Singapore Japan South Korea South Korea Czech Republic Czech Republic England Thailand Germany France United States United States Source: 1997 TIMSS, in The Economist, March 29th TIMSSThe EconomistTIMSSThe Economist

TIMSSTIMSS (2003) TIMSS (13-year-old's average score, Third International Math and Science Study) Countries: (sample) GlobalrankMathScience ScoreRankScoreRank Singapore Taiwan South Korea South Korea Hong Kong Hong Kong Japan Netherlands England United States United States Malaysia Italy Sources:TIMSS Math 2003 and TIMSS Science 2003 TIMSS Math 2003TIMSS Science 2003TIMSS Math 2003TIMSS Science 2003

Trends in PISA math achievement Japan Korea Netherland Finland Australia Canada Switzerland UK Belgium France OECD Germany Russia New Zealand US

There were 38 participating countries in 2003 PISA US ranked ： 7th in reading literacy 7th in reading literacy (15th in 2000 out of 32 countries) (15th in 2000 out of 32 countries) 24th in math literacy 24th in math literacy (19th in 2000) (19th in 2000) 9th in science literacy 9th in science literacy (14th in 2000) (14th in 2000) 20th in problem solving 20th in problem solving

Trends in PISA science achievement Japan Finland Korea Australia Netherland New Zealand Canada Switzerland France Belgium Germany Ireland UK Swiss AustriaUS

Trends in PISA reading literacy achievement Finland Korea Canada Australia New Zealand Ireland Swiss Netherland Belgium Norway Switzerland Japan Poland France

There were 38 participating countries in 2003 PISA US ranked ： 7th in reading literacy 7th in reading literacy (15th in 2000 out of 32 countries) (15th in 2000 out of 32 countries) 24th in math literacy 24th in math literacy (19th in 2000) (19th in 2000) 9th in science literacy 9th in science literacy (14th in 2000) (14th in 2000) 20th in problem solving 20th in problem solving

As the provision of higher educational As the provision of higher educational opportunities becomes increasingly opportunities becomes increasingly international, institutional comparison international, institutional comparison becomes more in demand. The two most becomes more in demand. The two most frequently cited rankings ： frequently cited rankings ： 1. Shanghai Jiaotong University, 1. Shanghai Jiaotong University, Academic Ranking of World Universities Academic Ranking of World UniversitiesAcademic Ranking of World UniversitiesAcademic Ranking of World Universities 2. Times Higher Education Supplement, London, Times Higher University World Rankings 2. Times Higher Education Supplement, London, Times Higher University World Rankings Times Higher University World Rankings Times Higher University World Rankings

WHO NEEDS WORLD-CLASS UNIVERSITIES? FROM WHOES PERSPECTIVES? Becoming international, becoming Becoming international, becoming competition-driven competition-driven More pressure from domestic and abroad More pressure from domestic and abroad Who can afford joining the "World-Class Club"? Who can afford joining the "World-Class Club"? Who benefits from these global, world-class competition? Who benefits from these global, world-class competition?

MYSTERIES UNRESOLVED? What does that mean by the "World Class University"? What does that mean by the "World Class University"? How many world class universities do we need? How many world class universities do we need? Does every country need one? Does every country need one? How could universities be evaluated and compared in an international scope? How could universities be evaluated and compared in an international scope? What measures should be taken in order to establish world class universities? What measures should be taken in order to establish world class universities? Who share the cost and who benefit? Who share the cost and who benefit?

World’s top 15 universities as an average of THES & SJTU 2005 rankings Rank Av. Rank InstitutionCountryTimesShanghai 11 Harvard Harvard U.S CambridgeU.K Massachusetts Inst. of Technology U.S.25 44StanfordU.S University of California, Berkeley U.S California Inst. of Technology U.S.86 67OxfordU.K PrincetonU.S.98 99YaleU.S CornellU.S ChicagoU.S ColumbiaU.S University of California, S.F. U.S TokyoJapan Imperial College London U.K.1323 Sources: Shanghai Jiaotong Academic Ranking of World Universities – 2005; Times Higher Education Supplement World University Rankings – 2005.

Rank Av. Rank InstitutionCountry Tim es Sha ngh ai DukeU.S Johns HopkinsU.S University of PennsylvaniaU.S Swiss Federal Institute of Technology, ZurichSwitzerland University of California, L.A.U.S.3714

WHAT CAUSES THE GLOBAL RANKINGS AND WHAT THE EFFECTS ARE? The "Impact of Globalization " The "Impact of Globalization " Borrowing and lending reform strategies Borrowing and lending reform strategies In comparative education, there is a great need to examine how local reactions are to these external global forces. In comparative education, there is a great need to examine how local reactions are to these external global forces. In policy studies, a new paradigm for educational policy-making and school reform is expected to be under way. In policy studies, a new paradigm for educational policy-making and school reform is expected to be under way.

What can be learned and imported from elsewhere? (borrowing) What can be learned and imported from elsewhere? (borrowing) What can be taught and exported elsewhere? (lending), What can be taught and exported elsewhere? (lending), The phenomena of cross-national policy attraction and educational borrowing The phenomena of cross-national policy attraction and educational borrowing Global and international convergence in education at the expense local needs and cultural diversity Global and international convergence in education at the expense local needs and cultural diversity

Three types of National Policy Responses According to Steiner-Khamsi (2007), three types of policy reactions after international comparisons: According to Steiner-Khamsi (2007), three types of policy reactions after international comparisons: 1. Scandalization (to blame one's own weak educational system), weak educational system), 2. Glorification (to praise one's good educational system), educational system), 3. Indifference to ranking results.

Typology of Political Reactions to International Comparative Studies TIMSS TIMSS PISA PISA World-Class University Rankings Scandalization Scandalization USA USA Germany, Germany, Israel Israel Korea, Japan, Korea, Japan, Taiwan Taiwan Glorification Glorification Japan, Finland Japan, Finland UK, Finland UK, Finland USA,UK USA,UK Indifference Indifference Mexico Mexico

IS IT A FAIR GMAE FOR EVERYBODY TO PLAY? What makes countries to react so differently? What makes countries to react so differently? Has scandalization led to increased policy import from other educational systems? Has scandalization led to increased policy import from other educational systems? Has glorification led to increased policy export to other educational systems? Has glorification led to increased policy export to other educational systems? Why some remain indifferent to these global comparisons? Why some remain indifferent to these global comparisons?

DO POLICY MAKERS BENEFIT FROM THESE GLOBAL RANKINGS? 1. More evidence-based research in public policy studies. 1. More evidence-based research in public policy studies. 2. More Outcome-based and standard-based criteria 2. More Outcome-based and standard-based criteria in monitoring school reforms in monitoring school reforms 3. A "scientific rationality" in education to appeal to the tax payers the tax payers 4. A legitimate “ self-referential system “ acting as policy 4. A legitimate “ self-referential system “ acting as policy borrowing ( a lesson learned from elsewhere) and a form borrowing ( a lesson learned from elsewhere) and a form of externalization. of externalization.

WHY GERMAN KIDS FELL BEHIND? In the PISA study released in 2001, German students scored below the OECD-average with regard to reading literacy. In the PISA study released in 2001, German students scored below the OECD-average with regard to reading literacy. The release of the PISA study led to a major uproar in the media, and calls for policy change and school transformation. The release of the PISA study led to a major uproar in the media, and calls for policy change and school transformation.

The Finnish Education Phenomenon The Finnish Education Phenomenon German PISA Shock German PISA Shock Brain Korea 21 (BK21) Brain Korea 21 (BK21) Japanese university incorporation Japanese university incorporation China ’ s 211 and 985 Project China ’ s 211 and 985 Project Taiwanese fifty-trillion dollars within five year ’ s world-class university funding program, etc. Taiwanese fifty-trillion dollars within five year ’ s world-class university funding program, etc. India ’ s brain drain of the university faculty India ’ s brain drain of the university faculty

DARE TO SAY NO?! Problems with global rankings : 1. Credibility and reliability issues of PISA 1. Credibility and reliability issues of PISA & TIMSS & TIMSS 2. Danger of the international convergence 2. Danger of the international convergence of educational institutions and policies of educational institutions and policies 3. Near-sighted government reform policies & 3. Near-sighted government reform policies & projects aiming at policy borrowing projects aiming at policy borrowing across national boundaries. across national boundaries.

4. The assumption of “ standardized and universal qualifications ” within universities universal qualifications ” within universities 5. Diagnosed rather than prescribed formula approach, not aiming for solving local educational problems. formula approach, not aiming for solving local educational problems. 6. Indifference to the ranking impact on students, employers, teachers, and school officials.

7. Criticism about these global rankings as the “ fast capitalism ” which benefits ranking benchmarks 8. The advantage of English-speaking countries in the global university rankings 9. Only the richest and the best survive in the world- class university rankings 10. The rich get richer, and the poor get poorer. 11. Increasing polarization of institutional resource generation, public funding, and educational quality nation-wide and world-wide.

12. The cumulative advantage vs. equal opportunities for all opportunities for all 13. Most university ranking criteria focus on academic research productivity rather academic research productivity rather than human aspect in cultivation and than human aspect in cultivation and collegiality. collegiality.

14. Higher education shifted from public good 14. Higher education shifted from public good to private commodity. to private commodity. 15. Many courses and degree programs are 15. Many courses and degree programs are packaged and marketed just like consumer goods, and students are seen as customers with a world of choice in front of them. packaged and marketed just like consumer goods, and students are seen as customers with a world of choice in front of them. 16. “ Exporting ” university services to less developed countries.

THE RANKING GAME IS NOT OVER YET, BUT THE LECTURE IS GETTING... International rankings and comparisons will continue in spite of criticism and protest. International rankings and comparisons will continue in spite of criticism and protest. Support from many governments, mass media, educational institutes and the general public. Support from many governments, mass media, educational institutes and the general public. Benchmarks create huge benefits. Benchmarks create huge benefits.

QUESTIONS REMAIN… Why the US has not so satisfactory PISA and TIMSS results, but has established so many world-class universities in the countries? Why the US has not so satisfactory PISA and TIMSS results, but has established so many world-class universities in the countries? Does this global ranking competition bring real change or improvement to education at home? Does this global ranking competition bring real change or improvement to education at home? What is the politics of comparison in global ranking? What is the politics of comparison in global ranking?

Does Taiwan need to be part of the game? Why? Does Taiwan need to be part of the game? Why?

THANK YOU ( 謝謝） THANK YOU ( 謝謝） & GOD BLESS YOU!! GOD BLESS YOU!! Comments are welcome! Comments are welcome! Eamil: Eamil: