QATAR IN PISA 2006 AND IN OTHER INTERNATIONAL STUDIES PISA National Report Preparatory Briefing November 28 th, 2007 Dr. J. Enrique Froemel Director Student Assessment Office

ISSUES TO BE COVERED Why do international assessments matter? – Purpose. – Why is Qatar participating? – In which international studies is Qatar going to participate? – How will the information be used? Explanation and comparison of PISA, PIRLS and TIMSS. – Periodicity. – Domain focus. – Some similarities and differences. – Populations included, in Qatar and number of countries participating. – Basic structure and administration.

ISSUES TO BE COVERED… Program for International Student Assessment (PISA). – Countries participating in PISA – More in-depth information. – Some example items. Progress in International Reading Literacy Study (PIRLS). – Background. – Results from international and national reports. – Press release. Trends in Mathematics and Science Study (TIMMS). – Some advance information. What information will the media have available, about international studies, in the coming twelve months. – PISA. – PIRLS. – TIMSS. Questions and discussion.

WHY DO INTERNATIONAL ASSESSMENTS MATTER? – Purpose. Standardized international studies results, are to be used by system level decision-makers and sometimes by principals, to answer the following questions: Where do we stand internationally and/or nationally? Why we are where we are? – Why is Qatar participating? International studies, when census based, allow verifying national assessments results (QCEA). Results of international studies, when crossed with data on students, teachers, schools and the country, (national [QNEDS] and international) provide keys to the country’s performance. Qatar’s participation in international studies allows building baselines and trends, through subsequent cycles, to monitor the progress of the reform. Qatar results in these studies gauge student’s performance in key competencies and contents, at critical stages of their school life.

WHY DO INTERNATIONAL ASSESSMENTS MATTER?… – In which international studies is Qatar participating? PISA : Run by OECD, participated in 2006 and is preparing participation in PIRLS : Run by IEA, participated in 2006 and is considering participation in TIMSS : Run by IEA, participated in 2007 and is preparing to release results in – How will the information be used. National reports of each study will be available for policy- makers (SEC and Ministry) to confirm or adjust policies accordingly. For PISA and most probably the other two studies, individual school reports will be distributed, for principals and teachers to use first-hand information about their school’s performance, with scale and performance level scores, for overall tests and sub-domains. For PISA, a web-based feedback system is to operate soon for teachers, on specific contents and skills that students showed to master or not.

EXPLANATION AND COMPARISON OF PISA, PIRLS AND TIMSS – Periodicity. PISA. Every three years. PIRLS. Every five years. TIMSS. Every four years. – Domain focus. PISA. A lternating main focus: reading, mathematics or science. PIRLS. Permanent focus on reading literacy. TIMSS. Permanent foci on mathematics and science. – Some similarities and differences. All use rotating booklet designs. All include cognitive tests, as well as questionnaires. In all cases, administration is through proctors. PISA is age based: 15 year olds. The other two are grade based: PIRLS > 4 th graders and TIMSS > 4 th and 8 th graders. PISA focuses both, in contents and skills (curriculum and competencies). PIRLS does it on skills (competencies). TIMSS focuses on content (curriculum).

EXPLANATION AND COMPARISON OF PISA, PIRLS AND TIMSS – Populations included, in Qatar and number of countries participating. PISA. Census based. All 15 year olds in independent, Ministry of Education, private Arabic and international schools. In total 7,734, belonging into 131 schools, participated. Total of 56 countries. PIRLS. Census based. All 4 th graders in independent, Ministry of Education and private Arabic schools. In total 6,680 students from 4 th grade, belonging into 119 schools, participated. Total of 40 countries and 5 territories from Canada. TIMSS. Census based. All 4 th and 8 th graders in independent, Ministry of Education and private Arabic schools. In total 7,260 students from 4 th grade and 7,332 from 8 th grade, belonging into 114 and 67 schools respectively, participated. Total of over 68 countries, 15 of those being Arab. – Basic structure and administration. In all cases, the test administration takes approximately two hours and questionnaires are sent to principals, teachers ( PIRLS and TIMSS ), parents and students. PISA is administered in Arabic to independent, MoE and private Arabic school students and in English to those of international schools. PIRLS and TIMSS only in Arabic to all schools involved.

COUNTRIES PARTICIPATING IN PISA 2006 OECD countries Australia Korea Austria Luxemburg Belgium Mexico Canada Netherlands Czech Republic New Zealand Denmark Norway Finland Poland France Portugal Germany Slovak Republic Greece Spain HungarySweden Iceland Switzerland Ireland Turkey Italy United Kingdom Japan United States PISA partner countries Argentina Kyrgyzstan Azerbaijan Latvia Brazil Liechtenstein Bulgaria Lithuania Chile Macao China Chinese Taipei Qatar Colombia Republic of Serbia and Montenegro Croatia Romania EstoniaRussian Federation Hong Kong-China Slovenia Indonesia Thailand Israel Tunisia Jordan Uruguay

PROGRAM FOR INTERNATIONAL STUDENT ASSESSMENT (PISA) More in-depth information. The PISA framework for scientific literacy.

PROGRAM FOR INTERNATIONAL STUDENT ASSESSMENT (PISA)… – Some example items. Identifying Scientific Issues. Question 3: GENETICALLY MODIFIED CROPS S508Q03 Corn was planted in 200 fields across the country. Why did the scientists use more than one site? A So that many farmers could try the new GM corn. B To see how much GM corn they could grow. C To cover as much land as possible with the GM crop. D To include various growth conditions for corn.

PROGRAM FOR INTERNATIONAL STUDENT ASSESSMENT (PISA)… – Some example items… Explaining Phenomena Scientifically. PHYSICAL EXERCISE Regular but moderate physical exercise is good for our health. Question 5: PHYSICAL EXERCISE S493Q05 – Why do you have to breathe more heavily when you’re doing physical exercise than when your body is resting?

PROGRAM FOR INTERNATIONAL STUDENT ASSESSMENT (PISA)… – Some example items… Using Scientific Evidence. ACID RAIN Below is a photo of statues called Caryatids that were built on the Acropolis in Athens more than 2500 years ago. The statues are made of a type of rock called marble. Marble is composed of calcium carbonate.

PROGRAM FOR INTERNATIONAL STUDENT ASSESSMENT (PISA)… − Some example items… Using Scientific Evidence… Question 3: ACID RAIN S485Q03… A marble chip has a mass of 2.0 grams before being immersed in vinegar overnight. The chip is removed and dried the next day. What will the mass of the dried marble chip be? A Less than 2.0 C grams B Exactly 2.0 grams C Between 2.3 and 2.4 grams 3 More than 2.4 grams

PROGRESS IN INTERNATIONAL READING LITERACY STUDY (PIRLS) – Background. In the spring of 2006, more than 6,500 primary school students, in 4th grade, in Qatar, participated in the Progress in International Literacy Study (PIRLS) – Results from international and national reports. International results and analyses are going to be released by IEA, in Boston, Massachusetts, USA, on November 28 th, at 6PM (Doha Time). National Qatar results and analyses, are going to be issued, by the Evaluation Institute, in Qatar, in a National Report, in Arabic and English versions, shortly afterwards. – Press-release. A press-release on Qatar student results is to be issued by the Communications Office, of the SEC, on December 4 th, at 12Noon (Doha Time).

TRENDS IN MATHEMATICS AND SCIENCE STUDY (TIMMS) – Some advance information. TIMSS 2007 international analyses are currently under way and countries are to be informed, under embargo, of preliminary findings, next December. As soon as the international database is available, the Evaluation Institute is going to select a consulting group to carry out, under the leadership of the Student Assessment Office, the analyses of the Qatar data and to generate the Qatar TIMSS 2007 National Report. Late in 2008 both, the TIMSS international and national reports will be issued, by IEA and the Evaluation Institute, respectively.

WHAT INFORMATION WILL THE MEDIA HAVE AVAILABLE, ABOUT INTERNATIONAL STUDIES, IN THE COMING TWELVE MONTHS – PISA. Qatar National Report, in Arabic and English versions, after December 4th. Summary of the Qatar National Report, in Arabic and English versions, on December 4 th. Press-conference on the international and Qatar results, by the Assistant Director of the Evaluation Institute, Dr. Hamda Al sutaiti, on December 4 th, at 11AM (Doha Time). Press-release, by the Communications Office, of the SEC, on December 4 th, at 12Noon (Doha Time).

WHAT INFORMATION WILL THE MEDIA HAVE AVAILABLE, ABOUT INTERNATIONAL STUDIES, IN THE COMING TWELVE MONTHS… – PIRLS. Qatar National Report, in Arabic and English versions, shortly in Press-release, by the Communications Office, of the SEC, on December 4 th, 12Noon (Doha Time).

WHAT INFORMATION WILL THE MEDIA HAVE AVAILABLE, ABOUT INTERNATIONAL STUDIES, IN THE COMING TWELVE MONTHS… – TIMSS. TIMSS Qatar National Report, at the end of Connected summary, press-release and press- conference, at a similar time.

QUESTIONS AND DISCUSSION