PISA Programme for International Student Assessment PISA team Department of Education – Ghent University – Belgium Beijing – July 24-25, 2009

Slides:



Advertisements
Similar presentations
EU Presidency Conference Effective policies for the development of competencies of youth in Europe Warsaw, November 2011 Improving basic skills in.
Advertisements

The Reasons behind the Results Markku Linna
Second Information Technology in Education Study (SITES) A Project of the International Association for the Evaluation of Educational Achievement (IEA)
OECD Programme for International Student Assessment (PISA)
OECD World Forum Statistics, Knowledge and Policy, Palermo, November
International Reading Association
The OECD PISA Study in Romania.
Potential impact of PISA
Lifelong Guidance: A Key to Lifelong Learning – EU Policy Perspective John McCarthy European Commission DG EAC Vocational Training Policy Unit.
Managing the Statutory Requirements for Assessment April 2011.
Equity - Research Reveals the What, the Where and the How November 21, 2011.
School Based Assessment and Reporting Unit Curriculum Directorate
Programme for International Student Assessment PISA 2009 Competences of Portuguese students Warsaw, February 10, 2011 Carlos Pinto-Ferreira, Anabela Serrão.
PISA OECD Programme for International Student Assessment Learning trends Maciej Jakubowski 10 February 2011 Learning Trends Changes in student performance.
Victorian Curriculum and Assessment Authority
European Frameworks of Reference for Language Competences Waldemar Martyniuk Language Policy Division, Council of Europe / Jagiellonian University, Poland.
An introduction to PISA and the type of assessments used Module 1 1.
1 © 2006 Curriculum K-12 Directorate, NSW Department of Education and Training English K-6 Syllabus Using the syllabus for consistency of assessment.
1 Building human capital and social cohesion through schools Cape Town, South Africa July 2005 Barry McGaw Director for Education Organisation for.
System of training lower secondary school teachers in Flanders and the case of… GROUP T – Leuven Education College Stijn Dhert Liesbeth Hens 06/10/2009.
International Federation of Accountants International Education Standards for Professional Accountants Mark Allison, Executive Director Institute of Chartered.
Supporting Teachers to make Overall Teacher Judgments The Consortium for Professional Learning.
Consistency of Assessment
Southeastern Louisiana University College of Education & Human Development Conceptual Framework: Setting the Standard for Excellence through Best Practice.
Organisation for Economic Cooperation and Development (OECD)
What do international assessments measure: PISA Raymond J. Adams Washington DC, May This paper is intended to promote the exchange of ideas among.
Good teaching, good teachers and comparative analysis Fernando Reimers.
Title I Needs Assessment and Program Evaluation
Maths matters: the Northern Ireland experience Katrina Godfrey Department of Education.
OECD Programme for International Student Assessment (PISA)
PISA OECD Programme for International Student Assessment Science Competencies for Tomorrow’s World Science competencies for tomorrow’s world Seeing school.
PISA FOR DEVELOPMENT Technical Workshops Contextual questionnaires 10 th April 2014 OECD Secretariat 1.
American Diploma Project 11 September 2009 Andreas Schleicher International Benchmarking International Benchmarking What it means – what it takes Washington,
1 Productivity and participation: a national perspective Barry McGaw Chair, National Curriculum Board Director, University of Melbourne Education Research.
UNESCO INSTITUTE for STATISTICS New Initiatives and Challenges UIS Education Statistics Programme Douglas Lynd, Senior Programme Coordinator.
Raising Academic Standards for all School Development Planning Initiative.
Raising the Quality of Educational Outcomes and Improving E-learning: Cross-national Evidence on Challenges and Opportunities Judit Kádár-Fülöp Indicators.
Improving School Leadership 1 st Workshop of Participating Countries Beatriz Pont Education and Training Policy Division Workshop of National Coordinators.
International Outcomes of Learning in Mathematics and Problem Solving: PISA 2003 Results from the U.S. Perspective Commissioner Robert Lerner National.
PISA: Behind the headlines and past the rankings Sue Thomson Director, Educational Monitoring and Research, ACER National Project Manager PISA National.
Connecting Teachers Can there be models of effective practice for teachers with ICT? Chair: Christine Vincent, Becta Presenter: Margaret Cox King’s College.
PISA OECD Programme for International Student Assessment PISA for Development Andreas Schleicher Paris, 27 June 2013 Welcome PISA for Development Andreas.
OECD World Forum “Statistics, Knowledge and Policy”, Palermo, November
PISA AND THE OECD TEST FOR SCHOOLS International benchmarking for schools and school districts Atlanta, October 2, 2014 Tue Halgreen Directorate for Education.
Institute for School Development Research (IFS) Researching the influence of educational policy on school- and teaching quality – The European Collaborative.
Student learning outcomes from a gender perspective What do international assessments tell us? Organisation for Economic Cooperation and Development (OECD)
Quality in schools: a case to develop school based policies and approaches PISA team Department of Education – Ghent University – Belgium Beijing – July.
Israel Accession Seminar PIAAC: Programme for International assessment of Adult Competencies Skills strategy in OECD Programme for the International Assessment.
Knowledge Economy Forum World Bank Conference 21 February 2002 Ian Whitman -- OECD
Student Name Student Number ePortfolio Demonstrating my achievement of the NSW Institute of Teachers Graduate Teacher Stage of the Professional Teacher.
Changes in the context of evaluation and assessment: the impact of the European Lifelong Learning strategy Romuald Normand, Institute of Education Lyon,
Andreas Schleicher, OECD Directorate for Education, 2003 Organisation for Economic Co-Operation and Development Literacy Skills for the World of Tomorrow.
Briar Hill School Data Summary Fall Student Diversity: ESL, Special Education Based on 194 students % of Population Gifted 5% Other Mild/moderate.
Andreas Schleicher, Quality of Education – Teachers’ Professional Training and Development, Athens, 2003 Organisation for Economic Co-Operation and Development.
School leadership matters in OECD countries Beatriz Pont Education and Training Policy Division OECD Education Directorate International Conference School.
Development Team Day 5a October Aim To explore approaches to evaluating the impact of the curriculum on pupil learning.
1 Perspectives on the Achievements of Irish 15-Year-Olds in the OECD PISA Assessment
1 Main achievement outcomes continued.... Performance on mathematics and reading (minor domains) in PISA 2006, including performance by gender Performance.
1 PISA 2006 Main achievement outcomes and factors associated with performance on science Eemer Eivers, Gerry Shiel & Rachel Cunningham Educational Research.
CEPT HF Background monitoring (Automatic data collection in the HF range) Update March 2012 Paul Dingenouts Radio Agency The Netherlands Copenhagen,
John Jerrim UCL Institute of Education
What is PIAAC?.
World Beaters or Underachievers
CEPT HF Background monitoring (Automatic data collection in the HF range) ‘the 5th anniversary’ Update November 2012 Paul Dingenouts Radio Agency.
PISA 2009 – New Approaches to Assessing Reading Literacy
PISA 2015 Excellence and Equity in Education Peter Adams
Australian Council for Educational Research in the International Context Marion Meiers.
Jenny Bradshaw NCETM National CPD Conference 23rd March 2011
Exploring the dimensions of quality in education - OECD perspective -
Organisation for Economic Co-Operation and Development Indicators on the Quality of Educational Performance Quality of Education Teachers’ Professional.
Presentation transcript:

PISA Programme for International Student Assessment PISA team Department of Education – Ghent University – Belgium Beijing – July 24-25, ( Partly based on Schleicher, A., 2006)

Structure Starting grounds Objectives Samples – Population Quality benchmarks Framework Typical assessment approach National Centres

PISA: starting grounds OECD: Organisation for Economical Organisation and Development Basis of PISA: original work of OECD related to statistics and indicators about education –late 1980s –voluntary contributions –Member engagement through networks

PISA: starting grounds Network on educational outcomes –Proposal to study educational outcomes –Formally started with 11 members in 1996 –Expanded in 1997 New name PISA Members bear costs and risks –Number of countries grows every PISA edition: 2000, 2003, 2006, 2009, …

PISA 2000 OESO-landen: Australië, Oostenrijk, België, Canada, Tsjechische Republiek, Denemarken, Finland, Frankrijk, Duitsland, Griekenland, Hongarije, Ijsland, Ierland, Italië, Japan, Korea, Liechtenstein, Luxemburg, Mexico, Nederland, Nieuw-Zeeland, Noorwegen, Polen, Portugal, Spanje, Zweden, Zwitserland, Verenigd Koninkrijk, Verenigde Staten Partnerlanden (niet-OESO): Albanië, Argentinië, Brazilië, Bulgarije, Chili, Hong Kong-China, Indonesië, Israël, Letland, Voormalige Joegoslavische Republiek Macedonië, Peru, Roemenië, Rusland, Thailand

PISA 2003 OESO-landen: Australië, Oostenrijk, België, Canada, Tsjechische Republiek, Denemarken, Finland, Frankrijk, Duitsland, Griekenland, Hongarije, Ijsland, Ierland, Italië, Japan, Korea, Liechtenstein, Luxemburg, Mexico, Nederland, Nieuw-Zeeland, Noorwegen, Polen, Portugal, Slovakije, Spanje, Zweden, Zwitserland, Turkije, Verenigd Koninkrijk, Verenigde Staten Partnerlanden (niet-OESO): Brazilië, Hong Kong-China, Indonesië, Letland, Macao-China, Rusland, Servië & Montenegro, Thailand, Tunesië, Urugay

PISA 2006 OESO-landen: Australië, Oostenrijk, België, Canada, Tsjechische Republiek, Denemarken, Finland, Frankrijk, Duitsland, Griekenland, Hongarije, Ijsland, Ierland, Italië, Japan, Korea, Luxemburg, Mexico, Nederland, Nieuw- Zeeland, Noorwegen, Polen, Portugal, Slovakije, Spanje, Zweden, Zwitserland, Turkije, Verenigd Koninkrijk, Verenigde Staten Partnerlanden (niet-OESO): Argentinië, Azerbeidzjan, Brazilië, Bulgarije, Chili, Colombia, Kroatië, Estland, Hong Kong-China, Indonesië, Israël, Jordanië, Kyrgyzstan, Letland, Liechtenstein, Litouwen, Macao-China, Qatar, Montenegro, Servië, Roemenië, Rusland, Slovenië, Taipei, Thailand, Tunesië, Uruguay

OECD countries participating from PISA 2000 OECD countries participating from PISA from 2003 OECD partner countries participating from PISA 2000 OECD partner countries participating from PISA 2003 OECD partner countries participating from PISA 2006 PISA country participation PISA 2000 (32 countries) PISA 2003 (41 countries) PISA 2006 (57 countries)

A three-yearly global educational assessment What did they learn? Performance of 15-year-olds key subject areas, and a range of educational outcomes Additionally: student attitudes to learning, self efficacy beliefs, and learning strategies contextual data from students, schools, parents and systems: policy levers PISA: Objectives See further

Comparing performance within and between countries Cross-cultural study Central concept: LITERACY Mathematical literacy Scientific literacy Reading literacy PISA: Objectives

Example: scientific literacy Is defined in terms of an individual’s: Scientific knowledge and use of that knowledge to… …identify scientific issues, …explain scientific phenomena, and …draw evidence-based conclusions about science-related issues Understanding of the characteristic features of science as a form of human knowledge and enquiry Awareness of how science and technology shape our material, intellectual and cultural environments Willingness to engage with science-related issues. PISA:Objectives

Population of 15 year old pupils National samples Representative samples between 3, ,000 pupils Most federal countries: regional samples e.g., Flanders versus Wallonia within Belgium PISA covers roughly 90% of the world economy. PISA: Population versus samples

13 German speaking commmunity Wallonia French speaking commmunity Flanders Dutch speaking commmunity Belgium Federal state (10, inhabitants)

PISA sampling requirements Population: all 15-year-olds in school –excludes 15-year-olds out of school –includes 15-year-olds in special education institutions –could exclude up to 5% of 15-year-olds in school difficult to reach (e.g. remote schools) non-participation –few countries fail to reach required sample in 2003 NZ (5.1%), Denmark (5.3%), UK (5.4%), Canada (6.8%), Spain (7.3%), US (7.3%) Sample –minimum of 150 schools per country –two random samples: schools and replacement schools –if school declines, replacement school is invited –requirements set by countries

International expertise network building on the participating countries… –Instruments ~ input of > 40 countries (see next presentation) –Cross-national and cross-cultural validity –Analysis of results International, natiopnal, regional analyses and reports Country reviews Consortium of research institutions ACER, CITO, ETS, NIER, WESTAT Coordinated by OECD and international organisations PISA: Networks

Focus on performance in subject areas: –Languages: Reading literacy Using, interpreting and reflecting on written material. –Mathematics: Mathematical literacy Recognising problems that can be solved mathematically, representing them mathematically, solving them. –Sciences: Scientific literacy Identifying scientific questions, recognising what counts as scientific evidence, using evidence to draw conclusions about the natural world. PISA: Objectives

PISA Objectives: cycles Reading Maths Science

Focus on performance in additional domains subject areas: –2000: Problem Solving –2003: ICT literacy –2006: Attitudes towards science –2009: ERA PISA: Objectives

PISA key quality benchmarks Overall performance of education systems Equity in the distribution of learning opportunities –Measured by the impact students’ and schools’ socio-economic background has on performance… …not merely by the distribution of learning outcomes Consistency of performance standards across schools Gender differences Foundations for lifelong learning

PISA Reading Maths Science Problem solving ICT literacy Attitudes towards sciences ERA

PISA framework Outputs and Outcomes impact of learning Individual learner Level A Domain 1 Quality and distribution of knowledge & skills

PISA framework Student background variables Mediating variables Literacy

PISA framework: complex interplay variables Student background variables Mediating variables Literacy System level Class-school level Student level

PISA framework Social & economic outcomes of education Output and performance of institutions Quality of instructional delivery Quality and distribution of knowledge & skills Outputs and Outcomes impact of learning Individual learner Level A Instructional settings Level B Schools, other institutions Level C Country or system Level D Domain 1

High Student performance Advantage PISA Index of social background Disadvantage Low

PISA framework Structures, resource allocation and policies Social & economic outcomes of education The school learning environment Teaching, learning practices and classroom climate Individual attitudes, engagement and behaviour Output and performance of institutions Quality of instructional delivery Quality and distribution of knowledge & skills Policy Levers shape educational outcomes Outputs and Outcomes impact of learning Individual learner Level A Instructional settings Level B Schools, other institutions Level C Country or system Level D Domain 2 Domain 1

PISA framework National education, social & economic context Structures, resource allocation and policies Social & economic outcomes of education Community and school characteristics Student learning, teacher working conditions Socio-economic background of learners Antecedents contextualise or constrain ed policy The school learning environment Teaching, learning practices and classroom climate Individual attitudes, engagement and behaviour Output and performance of institutions Quality of instructional delivery Quality and distribution of knowledge & skills Policy Levers shape educational outcomes Outputs and Outcomes impact of learning Individual learner Level A Instructional settings Level B Schools, other institutions Level C Country or system Level D Domain 3Domain 2 Domain 1

Typical PISA assessment Information collection –From students 3½ hours of main domain assessment 1 hour in relation to other subdomain 2 hours on paper-and-pencil tasks (subset of all questions) ½ hour for questionnaire on background, learning habits, learning environment, engagement and motivation –From school principals Questionnaire (school demography, learning environment quality) Indirect assssment of classroom variables (teacher, class) BEWARE!! Only adequate if grade based sampling has been applied

PISA National Centre Linking with international consortium Implementation of framework ~national level Reporting to consortium Representation during international meetings

PISA International Strong prescriptive framework Framework, timing, procedures, tools National data gathering International processing of data Priority in international release of results National (regional) processing of data next priority level Secondary analysis of data: data available

New developments Towards electronic assessment 2009: first full scale trial –ERA Electronic Reading Assessment

PISA Programme for International Student Assessment PISA team Department of Education – Ghent University – Belgium Beijing – July 24-25, ( Partly based on Schleicher, A., 2006)