1. PISA International Conference

2. Reading Performance of Hong Kong’s 15-Year-Old Students in PISA

3. Contents I.Design of Reading Assessment The construct of reading literacy Design of PISA assessment tasks II.The findings The overall and specific performance of HK students on reading literacy III. Some Observations

4. The Construct of Reading Literacy in PISA

5. “the ability to understand, use and reflect on written texts in order to achieve one’s goals, to develop one’s knowledge and potential, and to participate effectively in society” (OECD, 1999)

6. Reading Literacy in PISA Process of ReadingContent of ReadingContext of Reading 3 dimensions of reading

7. The Process - Five aspects of reading Retrieving information Broad understanding Developing an interpretation Reflecting on content Reflecting on form Retrieving information Interpreting Reflecting and Evaluating content and form

8. The Content – 2 categories, 11 types Text ContinuousNon-Continuous Descriptive Narrative Expositive Argumentative Injunctive Charts and Graphs Tables Diagrams Maps Forms Advertisements

9. The Context Personal Public Occupational Educational

10. The Construct of Reading Literacy for PISA Content (Format, Type) Context (Personal, Public, Occupation, Education) Process (retrieving, interpreting, reflecting)

11. The Design of PISA Literacy Assessment

12. The Hong Kong Sample PISA sampling procedure closely followed 140 schools 4405 15-year-old students

13. The assessment materials 37 stimulus texts 141 reading tasks 9 equal sets Chinese version verified by PISA

14. Formats of Response Multiple choice questions Complex multiple-choice items Closed-constructed responses Short responses Open-ended responses

15. Reporting of Reading Literacy Performance Four different proficiency scales Retrieving information sub-scale Interpreting text sub-scale Reflecting on and evaluating text sub-scale Combined scale These scores then mapped on to a five-level proficiency scale

16. Descriptors of skills demonstrated at each level of the three reading proficiency sub-scales (Appendix 3) Proficiency Level Retrieving InformationInterpretingReflecting Level 5 Locate and possibly sequence or combine multiple pieces of deeply embedded information, some of which may be outside the main body of the text. Infer which information in the text is relevant to the task. Deal with highly plausible and/or extensive competing information. Either construe the meaning of nuanced language or demonstrate a full and detailed understanding of a text. Critically evaluate or hypothesise, drawing on specialised knowledge. Deal with concepts that are contrary to expectations and draw on a deep understanding of long or complex texts. Level 3 Locate, and in some cases recognize the relationship between pieces of information, each of which may need to meet multiple criteria. Deal with prominent competing information. Integrate several parts of a text in order to identify a main idea, understand a relationship or construe the meaning of a word or phrase. Compare, contrast or categorize taking many criteria into account. Deal with competing information. Make connections or comparisons, give explanations, or evaluate a feature of text. Demonstrate a detailed understanding of the text in relation to familiar, everyday knowledge, or draw on less common knowledge. Level 1 Locate one or more independent pieces of explicitly stated information, typically meeting a single criterion, with little or no competing information in the text. Recognize the main theme or author's purpose in a text about a familiar topic, when the required information in the text is not prominent. Make a simple connection between information in the text and common, everyday knowledge.

17. Table 5: Reading literacy score range of the reading proficiency levels Reading Literacy Scores Proficiency Level 5More than 625 Proficiency Level 4553 – 625 Proficiency Level 3481 – 552 Proficiency Level 2408 – 480 Proficiency Level 1335 – 407 Proficiency Level Below 1Below 335

18. The testing material – A sample item Graffiti Question 1: The purpose of each of these letters is to: A explain what graffiti is. B present an opinion about graffiti. C demonstrate the popularity of graffiti. D tell people how much is spent removing graffiti. Answer:B Situation:Public Text format:Continuous Aspect:Interpreting texts Level:2 PISA scale score:421

19. HK Students’ Performance Overall

20. Table 6: Combined Reading Literacy Mean scores in reading literacy of participating countries/regions MeanS.E. ＊ significant difference ○ insignificant difference Finland546(2.6) ＊ Canada534(1.6) ○ New Zealand529(2.8) ○ Australia528(3.5) ○ Ireland527(3.2) ○ Hong Kong, China525(2.9) - Korea525(2.4) ○ United Kingdom523(2.6) ○ Japan522(5.2) ○ Sweden516(2.2) ○ Austria507(2.4) ＊

21. Below Level 1 At Level 1 At Level 2 At Level 3 At Level 4 At Level 5 Fig. 2 Percentage of students performing at each of the proficiency levels on the combined reading literacy scale

23. Fig. 3 A comparison of students performance on the combined reading literacy scale 300 350 400 450 500 550 600 650 700 0102030405060708090100 Percentile Combined reading score Hong Kong OECD average

24. Descriptors of performance by students at Level 1 and Below Level 1 Level 1: “capable of completing only the least complex reading tasks, such as locating a single piece of information, identifying the main theme of a text or making a simple connection with everyday knowledge.” Below Level 1 : “not capable of the most basic type of reading that PISA seeks to measure……Such students have serious difficulties in using reading literacy as an effective tool to advance and extend their skills in other areas.”

27. Summary of HK students’ overall performance on reading literacy HK has a relatively high performance (525, Proficiency Level 3, 6th) in reading literacy comparing with other countries Comparing with other countries, HK has a relatively small percentage (10%) of very proficient readers and poor readers The high rank of HK is largely due to the large percentage of students achieving Level 3 (33%) & 4 (31%)

28. HK Students’ Performance Specific

29. Table 7a: HK students’ performance at Retrieving Information Proficiency levelRetrieving Information HKOECDDifference Below 1 4%8% -4% Level 1 8%12% -4% Level 2 19%21% -2% Sub-total (Level 2 and below) 31%41% -10% Level 3 30%26% +4% Level 4 28%21% +7% Level 5 12% 0% Sub-total (Level 3 and above) 70%59% +11% Note: There may be rounding errors.

30. Table 7b: HK students’ performance at Interpreting Proficiency levelInterpreting HKOECDDifference Below 1 2%6% -4% Level 1 7%12% -5% Level 2 18%22% -4% Sub-total (Level 2 and below) 27%40% -13% Level 3 34%28% +6% Level 4 30%22% +8% Level 5 8%10% -2% Sub-total (Level 3 and above) 72%60% +12% Note: There may be rounding errors.

31. Table 7c: HK Students’ performance at Reflecting and Evaluation Proficiency levelReflecting and Evaluation HKOECDDifference Below 1 3%7% -4% Level 1 6%11% -5% Level 2 15%21% -6% Sub-total (Level 2 and below) 24%39% -15% Level 3 29%28% +1% Level 4 32%23% +8% Level 5 15%11% +4% Sub-total (Level 3 and above) 76%62% +14% Note: There may be rounding errors.

32. Table 7: Proportion of students at each proficiency level of the reading sub-scales Proficiency level Retrieving InformationInterpretingReflecting and Evaluation HKOECDDifferenceHKOECDDifferenceHKOECDDifference Below 1 4%8% -4% 2%6% -4% 3%7% -4% Level 1 8%12% -4% 7%12% -5% 6%11% -5% Level 2 19%21% -2% 18%22% -4% 15%21% -6% Sub-total (Level 2 and below) 31%41% -10% 27%40% -13% 24%39% -15% Level 3 30%26% +4% 34%28% +6% 29%28% +1% Level 4 28%21% +7% 30%22% +8% 32%23% +8% Level 5 12% 0% 8%10% -2% 15%11% +4% Sub-total (Level 3 and above) 70%59% +11% 72%60% +12% 76%62% +14% Note: There may be rounding errors.

33. Summary of HK students’ performance on the reading sub-scales HK students generally do better than OECD countries on the sub-scales HK students maintain their good performance on “reflecting and evaluation” HK high-achieving students gradually improve their performance from retrieving to evaluating, while low achievers showed a reversed pattern of results HK students’ performances show a greater disparity across the sub-scales than that of the OECD countries

34. Table 8: Mean percentage scores across different text formats Mean Percentage Scores No of items HKOECD* Continuous texts856661 Non- Continuous texts 426461 *2 items were deleted from the Hong Kong data set for subsequent analysis. This OECD average is calculated based on 127 corresponding items.

35. Table 9: Mean percentage scores across different text types Mean Percentage Score (%) TEXT TYPENo. of tasksHong KongOECD Continuous text Narrative176461 Argumentative/Persuasive176960 Expository316863 Descriptive116360 Injunctive96258 Non-continuous text Advertisements12029 Forms56962 Maps475 Charts/Graphs156968 Diagrams56256 Tables125751

36. Summary of HK students’ performance on text types HK students outperformed OECD countries on all types of texts except in the case of “advertisement” HK students show an advantage in argumentative text in the order of performance HK students score exceptionally better on “Forms”, “Diagrams” and “Tables” than that of OECD HK has a 7% variation across different types of continuous texts

37. Table 10: Mean percentage scores across different contexts of use Mean Percentage Score (%) ContextNo. of itemsHKOECD Educational366864 Occupational196958 Personal256560 Public476361

38. Summary of HK students’ Performance across contexts of use HK students are better at texts written for educational (68%) and occupational Purposes (69%) than for personal (65%) and public uses (63%) HK students perform exceptionally better than OECD average on “Occupational” texts

39. Conclusion and Discussion

40. Comparatively speaking, HK lags behind in the number of very proficient readers. HK students are relatively good at high order reading skills. HK students are particularly good at argumentative text, but weak at advertisement. HK students perform better on “educational”, “occupational” than on “personal” and “public” context of reading. HK education is successful in producing a majority of students with an above-average reading literacy. Are the results contradictory to our perception? How good do we want our students to be?