Challenges in International Large-Scale Assessments Higher School of Economics, Moscow, Russia, May 16, 2013.

Slides:



Advertisements
Similar presentations
Second Information Technology in Education Study (SITES) A Project of the International Association for the Evaluation of Educational Achievement (IEA)
Advertisements

21st Century Social Studies Session ID 86 Presented by Christy G. Keeler, Ph.D.
NARUC Presentation – July 2008 NARUC Presentation – July 2008 Brenda Kempster, KEMPSTER GROUP ICT Digital Literacy Initiative.
Somer Lewis, MA NBCT Teacher-In-Residence UNCW Watson School of Education.
Linking the Fairs to the 2013 Ontario Curriculum Social Studies 1 to 6 and History and Geography 7 and 8.
What is PIAAC?. About PIAAC PIAAC is an international large-scale assessment administered in in 23 countries It assessed 16 - to 65-year-olds,
5/16/ © 2011 California County Superintendents Educational Services Association.
Highlights from PIRLS and TIMSS 2011 Jack Buckley National Center for Education Statistics Washington, DC December 11, 2012.
Enhancing 21 st Century Skills: Implications for Community Colleges Rebecca Rahschulte, M.Ed.
Creative Mobile Multitasking Collaborative Producers.
Millennials “What we resolve to do in schools only makes sense when considered in the broader context of what the society intends to accomplish through.
Preparing Indiana Students for the 21st Century Ken Kay, President Partnership for 21 st Century Skills Indiana Education Roundtable Indianapolis, IN May.
Bridging the Gap from Implementation to Attainment: Utilising Results from International Comparative Studies. Surette van Staden PIRLS 2011 Co National.
P21 framework OPV 362.
Why Science Fair?. Promoting Positive Attitudes Towards Science Heightening student interest in science and allow for the exploration of personal interest.
Framework for 21 st Century Learning Hendersonville Middle School Curriculum Night November 20, 2012 Rena Nisbet.
+ 21 st Century Skills and Academic Standards Kimberly Hetrick Berry Creek Middle School Eagle County School District.
Signs and Symbols of Our World How do we make meaning with symbols?
Challenges in Developing a University Admissions Test & a National Assessment A Presentation at the Conference On University & Test Development in Central.
Curriculum for Excellence Aberdeenshire November 2008.
21st Century Skills Initiatives
1 1 EQAO Reporting of 21 st Century Skills Spring Conference 2013 Richard Jones & Carol Ricker-Wilson Presentation to AERO.
Student Learning Outcomes: Interpretations, Validity, and Factor Development Krista Soria and Laura Gorny This project was funded by the Undergraduate.
What should a Housy graduate be able to do? December 4, 2014 J. Martinez.
Preparing Every Ohio Student for the 21 st Century: The Role of P-16 Education Ken Kay, President Partnership for 21 st Century Skills STEMM/21 st Century.
Rethinking Computers and Instruction.  2007 report released by ISTE (International Society for Technology in Education).  Indicates ALL students, regardless.
21 ST CENTURY- THE AGE FOR LIFE (SKILLS) “In which greater attention will be paid to human beings and their health and happiness.” Linus Pauling.
Marion H. Martinez, Ed.D. Associate Commissioner for Teaching, Learning and Instructional Leadership August 25,
21st Century Skills: Just what are they?. Student Outcomes.
Building 21 st Century Skills with ICTs What does it mean? GEORGE SCHARFFENBERGER 6 September 2006.
By Karen Diaz TechKNOW Associates.
Preparing Every Child for the 21 st Century: What can teachers do? Partnership for 21 st Century Skills Nevada Department of Education Stateline, Nevada.
1 21C. learning in the what is it? 3 4 but first,
Using Understanding by Design
A Curriculum for Excellence Routes for Learning study day February 2007 Jessie Wojciechowski Professional Adviser.
21st Century Skills Framework. CORE SUBJECTS AND 21st CENTURY THEMES Mastery of core subjects and 21st century themes is essential for all students in.
What is a 21st Century Learner?
A Focus on Health and Wellbeing Wendy Halliday Learning and Teaching Scotland.
World Religions PreAP World History Mr. Weissler.
Lott Middle School Parent Information 21 st CENTURY Learning Skills.
Preparing Students for the for 21 st Century Instruction April 18, 2013 Dr. Lin Yu-Lan
The Nuts and Bolts of Service-Learning SCHOOLS + AGENCIES = SCHOOLS + AGENCIES = SERVICE-LEARNING SERVICE-LEARNING March 10, 2006 March 10, 2006.
Programming the New Syllabuses (incorporating the Australian Curriculum)
Ken Kay, President Partnership for 21 st Century Skills Webinar August 5, 2008 Presentation for Ohio Advisory Group.
A Sample of 21 st Century Skills Strong base of knowledge in core disciplines Global awareness, social & cross-cultural skills Financial, economic, business,
21 st Century Skills for Education Leaders: How do we prepare our students for the new global economy? Partnership for 21 st Century Skills Teaching &
21 st Century Skills and Content Paisley IB Magnet School Staff Development October 15, 2010.
International Large-Scale Assessments – Best practice and what are they good for? Dirk Hastedt, IEA Moscow, October 2015.
21 st Century Skills and Mathematics: Preparing Every Child for the 21st Century ASSM 2009 Annual Meeting.
21 st Century Skills Jason McLaughlin Kean University EMSE
Mathematics Experiences and Outcomes Curriculum for Excellence Support for Trialling.
International Baccalaureate Middle Years Programme IB MYP.
SSA – Technical Cooperation Fund End of Project Conference The Role of International Achievement Studies (OECD PISA, IEA TIMSS, PIRLS…) Importance of Large-scale.
11 PIRLS The Trinidad and Tobago Experience Regional Policy Dialogue on Education 2-3 December 2008 Harrilal Seecharan Ministry of Education Trinidad.
Lisa P. Smith, Henderson Library and Phyllis S. Dallas, Department of Writing & Linguistics Georgia Southern University Statesboro, GA Critical Thinking.
OECD EMPLOYER BRAND Playbook 1 What makes schools and school systems successful? Programme for International Student Assessment (PISA) Organisation for.
Defining 21st Century Skills: A Frameworks for Norfolk Public Schools NORFOLK BOARD OF EDUCATION Fall 2009.
1 Perspectives on the Achievements of Irish 15-Year-Olds in the OECD PISA Assessment
The audience will be appealed forcing them to participate in the activities that each station has and asking parents to provide succinct expressions.
 Continue to develop a common understanding of what STEM education is/could/should be here at Killip.
21st Centruy Approaches to Teaching Physics
21st Century Tools to Teach, Learn, Collaborate and Advocate for Languages 2009.
INTERNATIONAL ASSESSMENTS IN QATAR TIMSS
國立臺灣師範大學英語系陳秋蘭 英語閱讀與會考命題趨勢 國立臺灣師範大學英語系陳秋蘭
國立臺灣師範大學英語系陳秋蘭 PISA 與英語閱讀素養 國立臺灣師範大學英語系陳秋蘭
21st Century Skills
Oregon Literacy Plan Framework
Skills frameworks New Approaches to Statistical Capacity Development
21st Century Skills.
Presentation transcript:

Challenges in International Large-Scale Assessments Higher School of Economics, Moscow, Russia, May 16, 2013

Challenges 1.Using information to improve quality 2.Measuring beyond the academics 3.Reaching out of school population 4.What to measure 5.Difficulty level of the tests

Challenge 1: Using Assessment information to improve education quality

Challenge 1: Using ILSA to improve education quality A success Story: The case of Poland

Monitoring impact of reforms in Poland Source: The World Bank. Knowledge Brief. Nov 2010, Vol 34

Poland: PISA Reading Scores

Challenges for effective use of assessments Political environment Dissemination Local capacity Assessment culture

Challenge 2: Measuring beyond the academics Success requires a combination of academic, personality and social skills

Personality is important to predict success Personality Psychology and Economics Almlund, M., Duckworth, A., Heckman, J., Kautz, T.

Self-Discipline Outdoes IQ in Predicting Academic Performance of Adolescents Duckworth, A. L. & Seligman, M. E. P. (2005). Psychological Science, 16,

Grit Scale GRIT: Perseverance for achieving long-term goals

Grit Scale Men 1st-9th percentile: < th-19th percentile: th-29th percentile: th-39th percentile: th-49th percentile: th-59th percentile: th-69th percentile: th-79th percentile: th-89th percentile: th-99th percentile: Women 1st-9th percentile: < th-19th percentile: th-29th percentile: th-39th percentile: th-49th percentile: th-59th percentile: th-69th percentile: th-79th percentile: th-89th percentile: th-99th percentile:

Background Questionnaires TIMSS 2011 – Attitudes towards math and science: Like, value, confidence Results – Younger students have more positive attitudes – Correlation with achievement Strong and positive within countries Weak among countries

Discussion Should we move towards measuring personality traits? How?

Challenge 3: Reaching out of school population

73% 72% 84% 91% 96% n.a. The PISA 2009 profiles by country/economy.

PISA 2006 Sample Coverage Ferreira, F. & Gignoux, J. (2011). The Measurement of Educational Inequality. Achievement and Opportunity. Policy Research Paper Development Research Group. The World Bank Covered Pop / Total Pop

PISA 2006 Sample Coverage Ferreira, F. & Gignoux, J. (2011). The Measurement of Educational Inequality. Achievement and Opportunity. Policy Research Paper Development Research Group. The World Bank Covered Pop / Total Pop Science Scores

Which country did better? Country Y: Country X: 70% 90% 2ary school enrollment

Did the country stay the same? % 70% 2ary school enrollment 400 Country X

Discussion Should we compare countries with different proportions of out-of-school population?

Challenge 4: What to measure?

The traditional view in assessment Emphasis on opportunities to learn

The “new” view in assessment Society PISA approach: to what extent students can apply their knowledge to real-life situations and be equipped for full participation in society.

Measuring 21 st Century Skills

21 st Century Learning Framework Core subjects and interdisciplinary themes – Language, math, sciences, – Finances, health, civics Skills – Creativity and innovation – Critical thinking and problem solving – Communication and collaboration – Information, media, and ICT literacy

Challenge 5: Difficulty level of the tests Measuring what student should know and be able to do? Measuring what student actually know and can do?

Difficulty Level of the tests

Difficulty Level of the Tests Focus on and Retrieve Explicitly Stated Information Make Straightforward Inferences Interpret and Integrate Ideas and Information Examine and Evaluate Content, Language, and Textual Elements Recognize letters Recognize words Read short sentences PIRLS & PrePIRLS EGRA, Provinha

PrePIRLS Example questions Retrieve explicitly stated information

Provinha Example Question Identify Letters Source: Provinha March Caderno do Professor. X

Administrator reads test questions to the students. Test questions are only included in the Test Administrator booklet

Provinha Example question Read words Source: Provinha March Caderno do Professor. X

Difficulty Level of the Tests Focus on and Retrieve Explicitly Stated Information Make Straightforward Inferences Interpret and Integrate Ideas and Information Examine and Evaluate Content, Language, and Textual Elements Recognize letters Recognize words Read short sentences PIRLS & PrePIRLS EGRA, Provinha

Opposed Forces More low ability students expected to take the tests in more developing countries Push for more difficult “higher order thinking skills” questions

How to address this? new PISA for Development PrePIRLS

Universal scale

Discussion What challenges do you foresee? How can they be addressed?