1. 政府統計處 Census and Statistics Department Use of administrative databases in the compilation of academic value-added of schools in Hong Kong LEUNG Chun-wai, Chris Census and Statistics Department, Hong Kong, China

2. 政府統計處 Census and Statistics Department Introduction Public examination results are important indicators of absolute standards. They may not be good indicators of school effectiveness as they are often influenced by the characteristics of the students, most notably their prior attainment. The Education Bureau (EDB) has developed a value-added system to measure the extent to which schools have contributed to the levels of attainment of their students.

3. 政府統計處 Census and Statistics Department Value-added System OECD Definition: –“the contribution of a school to students’ progress towards stated or prescribed education objectives (e.g. cognitive achievement). The contribution is net of other factors that contribute to students’ educational progress.”

4. 政府統計處 Census and Statistics Department Concept of Value-added Student actual attainment = Student predicted attainment + (school residual + student residual)

5. 政府統計處 Census and Statistics Department Mathematical Form of the Model y ij = β 0 + β 1 x 1ij + β 2 x 2ij + … + β n x nij + (u j + e ij ) y ij : Normalised HKDSE Examination score x 1ij : Academic Ability Index (AAI) score, which measures the prior attainment. x 2ij … x nij : Other control variables: –School average AAI –Gender –All girls school –All boys school 5

6. 政府統計處 Census and Statistics Department Mathematical Form of the Model y ij = β 0 + β 1 x 1ij + β 2 x 2ij + … + β n x nij + (u j + e ij ) u j : A residual term for school j, indicating the school effect on performance (i.e. school value added) u j ~N(0, σ u 2 ) e ij : A residual term for student i in school j, indicating that part of the student's performance that could not be explained by factors already included in the model e ij ~N(0, σ e 2 ) 6

7. 政府統計處 Census and Statistics Department Mathematical Form of the Model Given that the examination scores have been adjusted statistically for the factors known to influence student performance, the differences among schools can reasonably be assumed to reflect differences in their effectiveness. 7

8. 政府統計處 Census and Statistics Department Interpretation of value-added Value-added is also presented in stanines (groups from 1 to 9) so that schools can quickly compare their value-added with that of others. 8 Stanine% 94 87 712 617 520 417 312 27 14 High Performance

9. 政府統計處 Census and Statistics Department 9 Use of administrative databases Hong Kong Diploma of Secondary Education (HKDSE) Examination Database –Taken by most Hong Kong students at the end of Secondary 6. –Standards-referenced reporting: Written examinations and school-based assessment (SBA) –HKDSE qualification is recognised by the Universities and Colleges Admissions Service (UCAS) in the UK

10. 政府統計處 Census and Statistics Department 10 Use of administrative databases Hong Kong Diploma of Secondary Education (HKDSE) Examination Database –Some 65 000 day school students took part in the HKDSE Examination in 2014. –Database contains the scores and grades of students in every subject entered. –Value-added scores can be compiled for individual subjects as well as for subject groups such as the “Core 4” subjects (viz. Chinese, English, Mathematics and Liberal Studies) and the “Best 5” subjects.

11. 政府統計處 Census and Statistics Department 11 Use of administrative databases Student Database –contains students’ Academic Ability Index (AAI) at Secondary 1 as their measure of prior attainment. –AAI is compiled based on the internal assessments of students in their primary schools, moderated by the academic performance of the primary schools in the territory.

12. 政府統計處 Census and Statistics Department 12 Use of administrative databases Student Database –To capture the effect of peers on HKDSE performance, the average AAI of all students in a secondary school is computed and entered into the value-added model. –Other information present includes demographic variables, such as gender, date of birth, nationality and language spoken, as well as the records of student changing schools.

13. 政府統計處 Census and Statistics Department 13 Use of administrative databases School Database –Information includes the mode of finance, geographical location and whether the school is a single-sex or a mixed-sex school.

14. 政府統計處 Census and Statistics Department 14 Potential source of control variables Socio-economic status (SES) is a commonly adopted control variable in value-added models worldwide. In Hong Kong, financially disadvantaged families can apply for the Comprehensive Social Security Assistance (CSSA) Scheme and the School Textbook Assistance (STA) Scheme. These databases serve as potential sources of control variables for the value-added model.

15. 政府統計處 Census and Statistics Department 15 Potential source of control variables Financial aid data are sensitive, and consent from the financial aid recipients needs to be obtained. Financial aid data can only identify students with low SES, but are unable to differentiate students at the high end of the SES spectrum.

16. 政府統計處 Census and Statistics Department 16 Potential source of control variables Alternatively, the median monthly household income of the District Council Constituency Area (with an average population of around 17 000 each) that a student lives in can serve as a proxy of SES. Drawbacks: –In areas with diverse household income, the median monthly household income is a poor proxy of SES. –With the construction of new housing estates, the profile of household income in an area may experience great changes in the 5-year inter- census / by-census period.

17. 政府統計處 Census and Statistics Department 17 Conclusion Value-added provides an alternative indicator of school effectiveness in addition to the conventional public examination results. To further enhance the quality of the value-added model, it would be desirable to review the value-added model, including its control variables from time to time.

18. 政府統計處 Census and Statistics Department References Education Bureau (2013). “Schools Value-added Information System: Technical Manual” OECD (2008). “Measuring Improvements in Learning Outcomes Best Practices to Assess the Value-Added of Schools: Best Practices to Assess the Value-Added of Schools” Hong Kong Examinations and Assessment Authority (2011). “Grading Procedures and Standards-referenced Reporting in the HKDSE Examination” Hong Kong Examinations and Assessment Authority (2013). “Information on School-based Assessment” Value-Added Research Center, Wisconsin Center for Education Research (2010). “Academic Growth over Time: Technical Report on the LAUSD School-Level Model 2009-2010” 18

19. 政府統計處 Census and Statistics Department Thank you