1. 1 STANDARD ERRORS ON DIFFERENCES

2. How to compute the standard error of the difference between : – Two countries; – An OECD country and the OECD total or the OECD average – A partner country and the OECD total or the OECD average – Two groups of students (e.g. boys versus girls, natives versus non natives) within countries? Standard Errors on Differences

3. School IDSchool meanBoys meanGirls mean 01 400350450 02 450410490 03 500470530 04 550530570 05 600590610 Mean 500470530 06 500470530 Mean if 01 replaced by 06 520494546 Mean if 05 replaced by 06 480446514

4. The expected value of the covariance between the two estimates: – should be equal to 0 if the two samples are independent, i.e. Two countries A partner country and the OECD Total or OECD Average Two explicit strata within a country – should be different from 0 if the two samples are not independent Two groups within a country if the group variable was not used as explicit stratification variable An OECD country and the OECD Total or OECD Average Standard Errors on Differences

5. How important is this covariance? – Country correlation between school performance for boys and school performance for girls, and country intraclass correlation Standard Errors on Differences

11. These two macros can also be used to compute the SE on the difference for STD, Variance, percentiles (sas), quartiles…

12. Differences in STD : reading Study NberNS(%)NNS(%)PNS(%)PS(%)VR PIRLS01 COMBINED SCALE36285617 1.09 INFO36226414 1.08 LIT36225819 1.08 PIRLS06 COMBINED SCALE4524679 1.08 INFO4522699 1.08 LIT45276013 1.08 AVERAGE PIRLS 24.2%62.3%13.5%0.0% PISA00 COMBINED SCALE4262335 1.16 RETRIEVE4264332 1.17 INTERPRET4260365 1.14 EVALUATE4260382 1.16 PISA03 COMBINED SCALE417129 1.20 PISA06 COMBINED SCALE5675232 1.20 PISA09 COMBINED SCALE6585115 1.18 RETRIEVE6585115 1.22 INTERPRET6569265 1.15 EVALUATE65752222 1.17 DIGITAL1958375 1.08 AVERAGE PISA 69.3%27.1%3.4% 0.2% AVERAGE 53.4%39.5%7.0% 0.1%1.14

13. Study NP5P10MEANP90P95 PIRLS01 COMBINED SCALE 360.340.330.250.20 INFO 360.290.280.220.170.16 LIT 360.34 0.270.23 PIRLS06 COMBINED SCALE 450.280.270.210.160.15 INFO 450.25 0.190.140.12 LIT 450.28 0.210.170.16 AVERAGE PIRLS 0.300.290.220.18 0.17 PISA00 COMBINED SCALE 420.470.460.350.240.22 RETRIEVE 420.380.360.250.140.12 INTERPRET 420.41 0.320.220.20 EVALUATE 420.580.570.460.350.33 PISA03 COMBINED SCALE 410.520.500.360.250.23 PISA06 COMBINED SCALE 560.560.540.420.300.27 PISA09 COMBINED SCALE 650.56 0.450.330.31 RETRIEVE 650.560.540.400.270.25 INTERPRET 650.510.500.420.310.29 EVALUATE 650.59 0.480.370.35 DIGITAL 190.430.420.320.230.22 AVERAGE PISA 0.510.500.39 0.270.25 Differences in STD : reading

14. STUDYNNENNPNPO PrimaryIEATIMSS95 P1 LG2425%67%8%0% TIMSS95 P1 UG2627%65%8%0% TIMSS03 P12941%48%10%0% TIMSS07 P14439%55%7%0% OECD------ Total math primary12334%58%8%0% SecondaryIEATIMSS95 P2 LG3926%67%8%0% TIMSS95 P2 UG4129%46%24%0% TIMSS99 P23834%58%5%3% TIMSS03 p25139%55%6%0% TIMSS07 p25749%39%12%0% OECDPISA004240%52%7%0% PISA034166%27%7%0% PISA065760%32%9%0% PISA096560%34%6%0% Total math secondary43146%44%9%0% Differences in STD: mathematics

15. STUDYNVRP5P10MEANP90P95P95-P5 PrimaryIEATIMSS95 P1 LG241,100,00-0,01-0,08-0,14-0,16-0,15 TIMSS95 P1 UG261,100,060,04-0,03-0,08-0,10-0,15 TIMSS03 P1291,100,070,05-0,02-0,09-0,10-0,17 TIMSS07 P1441,120,090,06-0,02-0,09-0,10-0,19 OECD--------- Mean math primary1231,110,050,03-0,04-0,10-0,11-0,17 SecondaryIEATIMSS95 P2 LG391,100,00-0,01-0,07-0,14-0,16 TIMSS95 P2 UG411,120,01-0,01-0,08-0,15 -0,16 TIMSS99 P2381,120,030,01-0,05-0,12-0,13-0,16 TIMSS03 p2511,110,07 0,00-0,07-0,09-0,16 TIMSS07 p2571,140,130,120,03-0,05-0,07-0,20 OECDPISA00421,12-0,01-0,02-0,09-0,17-0,19 PISA03411,170,01-0,01-0,11-0,21-0,24-0,25 PISA06571,13-0,01-0,03-0,10-0,19-0,21-0,20 PISA09651,12-0,01-0,02-0,10-0,18-0,20-0,19 Mean math secondary4311,130,020,01-0,06-0,14-0,16-0,19 Differences in STD: mathematics

16. STUDYNNENNPNPO PrimaryIEATIMSS95 P1 LG2425%75%0% TIMSS95 P1 UG2638%58%4%0% TIMSS03 P12941%48%10%0% TIMSS07 P14441%45%14%0% OECD------ Total science primary12337%54%8%0% SecondaryIEATIMSS95 P2 LG3933%59%8%0% TIMSS95 P2 UG4132%51%17%0% TIMSS99 P23842%50%8%0% TIMSS03 p25139%47%14%0% TIMSS07 p25754%32%14%0% OECDPISA004243%55%2%0% PISA034159%39%2%0% PISA065768%26%5%0% PISA096574%22%3%2% Total science secondary43152%40%8%0% Differences in STD: scienes

17. Standard Errors on Differences