Using assessment to improve the teaching of reading 18 September 2013 David Carroll and Keith Prenton

The Basic Education Program (BEP) is a five-year initiative designed to benefit all public primary and lower secondary schools in Kosovo, (serving grades 1 – 9). It began on 1 October 2010 and is scheduled to end on 30 September BEP’s motto is “Developing students’ 21st Century skills with schools and communities”. To this end, BEP seeks to improve the capacity of Kosovo’s schools to provide relevant skills for its students. BEP’s overarching goal is to improve the Government of Kosovo’s institutional capacity in the education sector and improve the quality of primary education. BEP is funded by USAID and the Government of Kosovo. FHI 360 is implementing BEP on behalf of USAID in partnership with the Ministry of Education, Science and Technology and Kosovo Education Center (KEC) which manages Component 3 of the Program. USAID/GoK Basic Education Program

CLICK TO ADD TITLE

1.Inviting all Kosovo’s primary school teachers to complete a three day, “Assessment for Learning” (AfL) course, which aims to increase the quality of teaching and learning process and enhance the progress of individual students. To date, over 7500 teachers have attended the course. 2.Assisting the MEST Assessment Division to improve the quality of test items, using PISA models. 3.Developing an Early Grades Reading Assessment (EGRA) in Albanian language; conducting a baseline study of 800 students using this EGRA; and, based on the findings,developing a training course and supplementary reading materials for Grade 1 – 5 teachers. Using Assessment to Improve Student Competencies

Deconstructing the PISA Reading Literacy Framework Getting from where we are to where we want to be

Predicting Kosovo’s 2015 Performance Country2009 Reading Mean Score Albania385 Montenegro408 Romania424 Bulgaria429 Serbia442 Croatia476

The PISA Reading Literacy Framework Texts: medium, environment, format, type Aspects: access and retrieve; integrate and interpret; reflect and evaluate Situations: personal, public, educational, occupational

PISA Proficiency Levels Six levels are defined: Average percent of students able to perform at the level or above is given Characteristics of tasks at each level Examples of released questions at the level, with their scaled difficulty.

PISA 2009 – Cumulative % of test takers at or above each level Level 1aLevel 2Level 3 Albania Montenegro Romania Bulgaria Serbia Croatia OECD avg

Level 3 – the Median OECD Test Taker Access and retrieveIntegrate and interpretReflect and evaluate Locate several pieces of information, each of which may need to meet multiple criteria. Combine pieces of information within a text. Deal with competing information. Integrate several parts of a text in order to identify the main idea, understand a relationship or construe the meaning of a word or phrase. Compare, contrast or categorise taking many criteria into account. Deal with competing information. Make connections or comparisons, give explanations, or evaluate a feature of a text. Demonstrate a detailed understanding of the text in relation to familiar, everyday knowledge, or draw on less common knowledge.

Comparing Level 2 and Level 3 Level 3Level 2  Locate several pieces of information, each of which may need to meet multiple criteria.  Locate one or more pieces of information, each of which may need to meet multiple criteria.  Combine pieces of information within a text.   Identify the main idea, understand a relationship or construe the meaning of a word or phrase based on integration of parts of a text.  Identify the main idea, understand a relationship, or construe the meaning of a word or phrase within a limited part of the text when the information is not prominent and low-level inferences are required.  Compare, contrast or categorise taking many criteria into account.  Form or apply simple categories  Make connections or comparisons, give explanations, or evaluate a feature of a text.  Make connections or comparisons between the text and outside knowledge  Demonstrate detailed understanding of the text in relation to familiar, everyday knowledge  Explain a feature of the text by drawing on personal experience or attitudes.  Draw on less common knowledge.   Deal with competing information.  Deal with some competing information

What to do? Two broad options: Focus on training grade 8 and 9 students in the required competences. Design teaching of reading to develop the required competences from early grades on. These are not mutually exclusive, of course.

Deconstructing the PISA Competences PISA describes exit competences. The national education system to determines how the competences will be achieved. The BEP aimed to develop a sequence of competences across grades, based on the PISA proficiency level descriptors. We used the sequence of complexity across the levels to guide us in envisaging what level 3 might look like at grade 5 and at grades 2/3.

The “PISA-based” Model of Reading Comprehension at Early Grades Locate one or more independent pieces of information, where the information is explicitly stated, each piece of information meets a single criterion and there is little or no competing information. Locating the information may or may not require making a synonymous match; and the target information may or may not be prominent in the text. Recognise the main theme or author’s purpose in a text where the topic of the text is familiar and the required information is prominent in the text. Make very simple inferences Make a simple connection between information in the text and common, everyday knowledge. Relate a simple feature of the text to their personal experience or attitudes.

Current Practice Ishte njëherë një djalosh i quajtur Pikalosh, që jetonte me babain, nënën dhe qenin e tij Bupin. Ata jetonin në një kasolle afër pyllit. Kur po darkonin pranë oxhakut, jashtë dëgjuan të lehurat e Bupit. Pikaloshi dhe babai i tij dolën dhe panë Bupin duke u përleshur me një ujk. Pikaloshi zuri të qante. Por, kur pa ujkun që po ia mbathte, kuptoi që Bupi ishte heroi i tij i vërtetë. 1)Me kë jetonte Pikaloshi?(Me babain, nënën dhe qenin e tij, Bupin). 1)Çfarë dëgjoi kur po darkonte Pikaloshi? (Ai dëgjoi të lehurat e Bupit). 1)Çfarë panë Pikaloshi dhe babai i tij? (Ata panë Bupin duke u përleshur me një ujk). 1)Ç’kuptoi Pikaloshi?(Ai kuptoi që Bupi ishte heroi i tij i vërtetë).

A “PISA-based” model Questions might have the following features: Retrieval of explicitly-stated information might involve some linguistic transformation or retrieval of less obviously salient information There would be questions requiring simple inferences There might be a question about the author’s purpose or the main theme of the text There would be one or more questions requiring the reader to make a connection with their own knowledge of the world or attitudes.

What to do? Differences in PISA performance between different nations reflect broad characteristics of student learning at age 15. These characteristics matter. The PISA frameworks are the state of the art in specifying exit competences. But they define goals, not how to achieve them. If PISA performance is to change significantly, the place to start is not grade 8. It is grade 1. You need a thorough review of objectives and instruction, and it needs to be guided by the exit competences, as defined by PISA