1 Main achievement outcomes continued.... Performance on mathematics and reading (minor domains) in PISA 2006, including performance by gender Performance on mathematics and reading (minor domains) in PISA 2006, including performance by gender Performance across domains in PISA 2006 Performance across domains in PISA 2006 Trends in performance in Ireland and across OECD countries ( Trends in performance in Ireland and across OECD countries (

2 PISA Mathematics Major domain in PISA 2003 (85 items) Major domain in PISA 2003 (85 items) Minor domain in PISA 2006 (48 items) Minor domain in PISA 2006 (48 items) Overall scale in 2006; overall proficiency levels, but no subscales Overall scale in 2006; overall proficiency levels, but no subscales No new items No new items

3

4

5 Sample Question 15 Quantity – Closed constructed response Quantity – Closed constructed response Full credit – both maximum (137 zeds) and minimum (80 zeds) correct Full credit – both maximum (137 zeds) and minimum (80 zeds) correct Partial credit –maximum or minimum correct Partial credit –maximum or minimum correct Item Stats % OECD % Irel Full correct 6769 Partial corr. 118 Incorrect1821 Missing52 PISA Item Difficulty Scale Score: 464 (PC) 496 (FC) Proficiency Level: 2 (PC) 3 (FC)

6 Sample Question 16 Quantity – Traditional multiple choice Quantity – Traditional multiple choice Correct answer: D (12) Correct answer: D (12) Item Statistics % OECD % Ireland Correct4630 Incorrect Missing43 PISA Item Difficulty Scale Score 570 Proficiency Level 4

7 Sample Question 17 Quantity – Short constructed response Quantity – Short constructed response Item Statistics % OECD % Ireland Correct5050 Incorrect4548 Missing52 PISA Item Difficulty Scale Score 554 Proficiency Level 4

8 Performance on PISA Mathematics (2006) Mean score for Ireland – Mean score for Ireland – Not significantly different from OECD average of Not significantly different from OECD average of Rank: 16th among OECD countries (range: 12th-17th) Rank: 16th among OECD countries (range: 12th-17th) Rank: 22nd among 57 participation countries (range: 17th-23rd) Rank: 22nd among 57 participation countries (range: 17th-23rd)

10 Mathematics: Performance at the 95 th Percentile (2006)

11 Mathematics: Performance at the 5 th Percentile (2006)

12 Spread of Achievement in Mathematics Can be estimated by finding the difference between scores at the 95 th and 5 th percentile Can be estimated by finding the difference between scores at the 95 th and 5 th percentile Difference in Ireland: 268 points. Difference in Ireland: 268 points. OECD average: 299 OECD average: 299 Only Finland (267) and Wales (270) have differences close to Ireland’s. Only Finland (267) and Wales (270) have differences close to Ireland’s.

13 Mathematics – Performance by Proficiency Levels – Ireland and OECD Average

14 Gender Differences in PISA Mathematics (2006) – Ireland and OECD Average

15 Gender Differences in Ireland– Mathematics Proficiency Levels (2006)

16 Explaining Gender Differences in Mathematics Structure of test (e.g., impact of Space & Shape items) Structure of test (e.g., impact of Space & Shape items) Item type (multiple-choice vs. constructed response) Item type (multiple-choice vs. constructed response) Nature of the PISA mathematics tasks Nature of the PISA mathematics tasks Contrast with Junior Certificate examination where females achieve more A and B grades in mathematics at higher and ordinary levels Contrast with Junior Certificate examination where females achieve more A and B grades in mathematics at higher and ordinary levels Presentation by Seán Close this afternoon to examine gender differences in detail. Presentation by Seán Close this afternoon to examine gender differences in detail.

17 Trends in Achievement ( ) Complexity in establishing trends over time as domains shift from major to minor Complexity in establishing trends over time as domains shift from major to minor Mix of items (e.g., effect on reading score of many reading vs. many science items) Mix of items (e.g., effect on reading score of many reading vs. many science items) Reconfiguration of items into new clusters to meet framework specifications Reconfiguration of items into new clusters to meet framework specifications Characteristics of the particular set of linking items used (country interaction) Characteristics of the particular set of linking items used (country interaction)

18 Trends in Mathematics ( ) Major to minor domain from 2003 to 2006 Major to minor domain from 2003 to ‘common items’ 48 ‘common items’ Neither Ireland’s overall mean scores, nor the OECD country average scores, changed between the two years. Neither Ireland’s overall mean scores, nor the OECD country average scores, changed between the two years. No significant differences in Ireland at the 5 th or 95 th percentiles, though the score of students in Ireland at the 95 th was 7 points lower in No significant differences in Ireland at the 5 th or 95 th percentiles, though the score of students in Ireland at the 95 th was 7 points lower in 2006.

19 Trends in Mathematics ( ) – Mean Scores, Ireland and OECD Average

20 Trends in Mathematics ( ) Students in Mexico (+20) and Greece (+14) registered significantly higher mean scores in Students in Mexico (+20) and Greece (+14) registered significantly higher mean scores in Students in France (-15), Iceland (-10) and Belgium (-9) had significantly lower mean scores. Students in France (-15), Iceland (-10) and Belgium (-9) had significantly lower mean scores. Students in Denmark and The Netherlands scoring at the 95 th percentile had significantly lower scores in Students in Denmark and The Netherlands scoring at the 95 th percentile had significantly lower scores in 2006.

21 PISA Reading Literacy Major domain in PISA 2000 (141 items) Major domain in PISA 2000 (141 items) Minor domain in PISA 2003/2006 (28 items) Minor domain in PISA 2003/2006 (28 items) Overall scale in 2006; overall proficiency levels, but no subscales Overall scale in 2006; overall proficiency levels, but no subscales Items drawn from 3 reading processes: retrieve (25%), interpret (50%), and reflect/evaluate (25%) Items drawn from 3 reading processes: retrieve (25%), interpret (50%), and reflect/evaluate (25%) No new items No new items

22 Sample Reading Literacy Item – Q. 1 Item Stats % OECD % Irel Correct4858 Incorrect3636 Missing166 PISA Item Difficulty Scale Score 558 Proficiency Level 4

23 Sample Reading Literacy Item – Q. 2 List two examples... Interpretation of text. List two examples... Interpretation of text. Answer – two of 3 possible answers Answer – two of 3 possible answers High level of difficulty High level of difficulty PISA Item Difficulty Scale Score 669 Proficiency Level 5 Item Stats % OECD % Irel Correct76 Incorrect5265 Missing4129

24 Performance on PISA Reading Literacy (2006) Mean score for Ireland – Mean score for Ireland – Significantly higher than OECD average of Significantly higher than OECD average of Rank: 5 th among OECD countries (range: 4-6) Rank: 5 th among OECD countries (range: 4-6) 6th among 56 participating countries (range: 5-8) (US missing for reading) 6th among 56 participating countries (range: 5-8) (US missing for reading)

26 Reading: Performance at the 95 th Percentile (2006)

27 Reading Literacy: Performance at the 5 th Percentile (2006)

28 Spread of Achievement in Reading Literacy Based on difference between scores at the 95 th and 5 th percentile Based on difference between scores at the 95 th and 5 th percentile Difference in Ireland: 303 points. Difference in Ireland: 303 points. OECD average: 324 OECD average: 324 Korea (289), Finland (265) and Denmark (293) have smaller gaps, indicating a more homogeneous spread of achievement in those countries. Korea (289), Finland (265) and Denmark (293) have smaller gaps, indicating a more homogeneous spread of achievement in those countries.

29 Reading Literacy – Performance by Proficiency Levels (2006) – Ireland and OECD Average

30 Gender Differences and Reading Literacy (2006)

31 Gender Differences in Ireland on Reading Proficiency Levels

32 Explanation of Gender Differences in Reading Literacy Emphasis on text type (continuous vs. non- continuous) Emphasis on text type (continuous vs. non- continuous) Item type (multiple-choice vs. constructed response) Item type (multiple-choice vs. constructed response) Reading process (retrieve, interpret, reflect/evaluate) Reading process (retrieve, interpret, reflect/evaluate) Presentation by Brian Murphy this afternoon will examine differences in more detail Presentation by Brian Murphy this afternoon will examine differences in more detail

33 Trends in Reading Literacy ( ) – Mean Scores, Ireland and OECD Average

34 Trends in Reading Literacy ( ) – Mean Scores Korea’s mean score increased by 31 points between 2000 and 2006 Korea’s mean score increased by 31 points between 2000 and 2006 Poland’s mean score increased by 29 points between the same years. Poland’s mean score increased by 29 points between the same years. Significant declines in mean reading scores in Australia (-15), France (-17), Greece (-14), Iceland (-22), Italy (-19), Japan (-24), Norway (-21), and Spain (-32). Significant declines in mean reading scores in Australia (-15), France (-17), Greece (-14), Iceland (-22), Italy (-19), Japan (-24), Norway (-21), and Spain (-32).

35 Trends in Reading Literacy ( ) – Performance at Key Benchmarks Significant (22 point) drop in performance of students in Ireland at 95 th percentile in 2003 (relative to 2000), though some of the difference (14 points) was made up in 2006 Significant (22 point) drop in performance of students in Ireland at 95 th percentile in 2003 (relative to 2000), though some of the difference (14 points) was made up in 2006 No significant differences at 5th percentile. No significant differences at 5th percentile. More comprehensive trend data for reading literacy in 2009, when it becomes a major domain again - the first time a major domain is repeated in PISA. More comprehensive trend data for reading literacy in 2009, when it becomes a major domain again - the first time a major domain is repeated in PISA.

36 Trends in Science Performance ( ) Difficulty in developing trend data going from minor domain Decline in performance on link items in Ireland between 2003 and 2006, but difference not significant. Difficulty in developing trend data going from minor domain Decline in performance on link items in Ireland between 2003 and 2006, but difference not significant. Significant increases in Mexico (+22.7) and Greece (20.5) and a drop in France (-16) Significant increases in Mexico (+22.7) and Greece (20.5) and a drop in France (-16) Ireland performed at about the same level on all three assessments to date – comfortably above the OECD average in 2000, and just above it in 2003 and Ireland performed at about the same level on all three assessments to date – comfortably above the OECD average in 2000, and just above it in 2003 and 2006.

37

38 Consistency in Performance across Domains Most countries tend to perform at a similar level on each domain Most countries tend to perform at a similar level on each domain –The top 4 spots in science, reading and maths are shared between 5 countries –The bottom 4 spots are shared between 6 countries Ireland shows a more mixed performance, with relatively lower performance in mathematics Ireland shows a more mixed performance, with relatively lower performance in mathematics