Martin Ripley – World Class Arena National PISA and TIMSS centre OECD

What exactly are the international comparison studies? PISATIMSSPIRLSTALIS

For more information: About PISA 2015: About TIMSS 2015: In general: -_pisa_timss_and_pirls/archive/2014/12/22/what-are-these- international-surveys-and-why-do-we-take-part.aspx

What exactly are the international comparison studies? Name… PISATIMSSPIRLSTALIS …in fullProgramme for International Student Assessment Trends in International Mathematics and Science Study Progress in International Reading Literacy Study Teaching and Learning International Survey Target group15-year oldsPupils in international grades 4 and 8 (year 5 & 9) Pupils in international grade 4 (year 5) Teachers of lower secondary pupils (Key Stage 3) Subject focusScience, maths and reading skills Science and maths knowledge Reading abilityProfessional development & working conditions Regularity3-yearly Most recent: yearly Most recent: yearly Most recent: yearly Most recent: 2013 What are they testing/ comparing? Application of core skills Knowledge of an international curriculum Ability to read for different purposes in different contexts Investment in teacher workforce, deployment and training PISATIMSSPIRLSTALIS

What is PISA? PISA Background  Assessment of science, maths and reading competence  Sample of 15 year-olds across 70 countries  Contextual teacher and pupil questionnaires  Most recently run in 2012, next study in 2015 PISA competencies What are they? How are they assessed? The meaning of “scientific literacy” Teaching PISA competencies TIMSS (year 5)TIMSS (year 9)PISA (age 15)

“....literacy is not an attribute that an individual either has or does not have; rather, it is a skill that can be developed over a lifetime.” “Scientific literacy requires not just knowledge of the concepts and theories of science but also a knowledge of the common procedures and practices associated with scientific enquiry and how these 4 enable science to advance.” “...knowledge of the facts, concepts, ideas and theories about the natural world that science has established.” “...knowledge of the practices and concepts on which empirical enquiry is based such as repeating measurements to minimise error and reduce uncertainty, the control of variables, and standard procedures for representing and communicating data.” “...understanding of the function that questions, observations, theories, hypotheses, models, and arguments play in science..” “..attitudes or disposition towards science will determine their level of interest, sustain their engagement, and may motivate them to take action.” “Scientific literacy”

PISA Cycles 2000 Reading 2003 Maths Science Reading 2006 Science Maths Reading 2009 Reading Maths Science 2012 Maths Science Reading Financial Literacy Problem Solving 2015 Science Collaborative Problem Solving Maths Reading Financial Literacy

Main Study Trans.& Adapt. Review Field Trial Translation & Adaptation Item Development Framework Development PISA Test Development Expert Group Meetings Marker training Field Trial Review 2015 Marker training 2013 Country Proposes; Reviewed; Independent linguistic review International training Reliability scripts.92 threshold Quality Monitors International standardisation

Strict set of “standards” Representative sample of schools are drawn and then a random in-school sample National adaptation of questionnaires Localisation (and translation) of test content International marking training; international standardisation of marking 0.92r required

PISA Questions

TIMSSPISA TALIS Thinking about the teachers at your school: to what extent do you agree with the following statements? (Please select one response in each row.) I get along well with most of my teachers. Most of my teachers are interested in my well-being. Most of my teachers really listen to what I have to say. If I need extra help, I will receive it from my teachers. Most of my teachers treat me fairly. My teachers are interested in my school activities. My teachers are supportive of my educational efforts and achievements. My teachers support me when I am facing difficulties in school. My teachers encourage me to be confident. My teachers encourage me to learn new skills. My teachers give me feedback on my performance in school. My teachers suggest ways that I can move forward when I'm stuck. My teachers tell me what to do to complete a task and explain why I should do so. My teachers give me detailed information and clarification about the learning task and strategies. My teachers demonstrate particular skills that are important for solving a task. My teachers ask me questions to initiate a deeper understanding of the content. Students have a voice in decision making at my school. Adults at this school listen to students' suggestions. PISA Background Questions

What is TIMSS? Background  Assessment of year 5 & year 9 pupils’ maths and science knowledge/application  Most recently run in 2011, next study in 2015 TIMSS competencies TIMSS ‘curriculum’ TIMSS (year 5)TIMSS (year 9)PISA (age 15)

TIMSS Grade 4 Question

TIMSS Grade 4 Question

TIMSS Grade 4 Question

TIMSS Grade 8 Question - biology

TIMSS Grade 8 Question - chemistry

TIMSS Grade 8 question - physics

TALIS

In Spring 2013, just under 4,000 lower secondary (KS3) teachers in England filled in a questionnaire about their: characteristics; working environments; CPD activities and initial training; practices and beliefs; self-efficacy and job satisfaction. Their head teachers also filled in a questionnaire. As did lower secondary teachers and head teachers in 33 other countries across the world. The results were published in June 2014 TALIS is a partnership between An international research consortium OECD Governments in 33 countries European Commission Teachers’ unions TALIS What? Who? When? How? Where?

 Strengths in the UK and England: primary; science; problem solving  Shanghai and the Far East tend to do well in maths  The Netherlands have world-leading accountability systems  The USA has strength in reforming school systems  Japan is strong in supporting and developing teachers  In the UK, men and women with high literacy, numeracy and ICT skills earn 8 times more than those with weakest skills  13% of KS3 teachers are in schools whose headteachers report a shortage of materials (26% across all TALIS schools)  46% in schools where headteachers reported a shortage of qualified and well-performing teachers (38% across all TALIS schools)

Thank you for listening! For more information: * About PISA 2015: * About TIMSS 2015: * In general: _pisa_timss_and_pirls/archive/2014/12/22/what-are-these-international-surveys-and-why-do- we-take-part.aspx _pisa_timss_and_pirls/archive/2014/12/22/what-are-these-international-surveys-and-why-do- we-take-part.aspx

Debunking the Myths Do Shanghai only select their brightest pupils to take part? Isn’t it all cultural differences? Do we really want to be like Shanghai? Unreliable - different students answering different questions? Does the translation work? Is it fair to assess only that which can be measured across countries? Black box - the rankings are too simplistic The results are not very useful for individual schools/teachers