1. International Survey of Adult Skills (ISAS) Policy Results 14th October 2013

2. Victor Dukelow Principal Economist Analytical Services Department for Employment & Learning

3. International Survey of Adult Skills Northern Ireland October 2013

4. 2012 International Survey of Adult Skills The 2012 International Survey of Adult Skills was carried out on behalf of the Northern Ireland Governments by the National Foundation for Educational Research (NFER) in partnership with TNS-BMRB, NatCen Social Research (NatCen) and Northern Ireland Statistics and Research Agency (NISRA). It is an OECD survey of adults aged between 16 and 65 in 25 countries (incl. England and NI). In Northern Ireland, data was collected between August 11 and April 12 and 3,761 adults participated in the survey. The response rate was 65 per cent. The interviews and assessments are carried out in adults’ homes. It measures numeracy, literacy and problem solving in technology-rich environments. It aims to be able to make comparisons over time, and has some linking questions with previous international surveys. It has a particular focus on measuring skills in real-world situations.

5. How does it work? Like most population level surveys it uses a multi-stage sampling design. It includes a background questionnaire and a cognitive assessment and both are conducted in the respondent’s home at least in part by Computer-Assisted Personal Interviewing (CAPI). Computer based assessments of the three skill areas. A paper based assessment of literacy and numeracy is available for those who are unable to carry out the assessment on computer.

6. How do the assessments work? Computer based tests are adaptive in that respondents are presented with items appropriate to their skill level based on responses to previous groups of items. All respondents see the most basic questions and are presented with more difficult questions based on their response to these. This allows a broad ability range to be effectively assessed.

7. Skills assessed Literacy ‘the ability to understand, evaluate, use and engage with written texts to participate in society, to achieve goals and to develop one’s knowledge and potential.’ Numeracy ‘the ability to access, use, interpret, and communicate mathematical information and ideas, in order to engage in and manage the mathematical demands of a range of situations in adult life.’ Problem solving in a technology-rich environment ‘the ability to use digital technology, communication tools and networks to acquire and evaluate information, communicate with others and perform practical tasks’

8. Levels OECD’s International Survey of Adult Skills Literacy and Numeracy Levels Rough equivalent National Qualification Framework (NQF) Levels (Literacy) Rough equivalent National Qualification Framework (NQF) Levels (Numeracy) Below Level 1 Entry Level 1 Entry Level 2 Level 1Entry Level 3 Entry Level 2 Level 2Level 1 Entry Level 3 Level 3 Level 2 and above Level 1 Level 4 Level 2 and above Level 5

9. Findings

10. Achievement scores DomainNorthern Ireland average OECD average Literacy 269*273 Numeracy 259*269 Problem solving in technology-rich environments 275*283 Table 2.1 Average scores for Northern Ireland in each domain Source: PIAAC (2012) * Significant at 5 per cent level compared with OECD average Scores in each of the domains are on a scale from 0 to 500.

11. Literacy Northern Ireland performed similarly to neighbouring English-speaking countries of England and the Republic of Ireland. Literacy levels in Northern Ireland have increased significantly since last measured in 1996, but are still below the OECD average. Although below the OECD average for literacy skills, Northern Ireland still compares favourably to France, Italy and Spain in this area. Northern Ireland had a similar spread of attainment compared with the OECD average, indicating a similar range of skills compared with other countries. However, there was a higher proportion of adults at Level 2 or below compared with the OECD average.

12. International average literacy scores for IALS and the International Survey of Adult Skills Country IALS (1996) ISAS (2012) Average Average of participating countries **275(0.4)273(0.2) Australia*272(1.0)280(0.9) Belgium277(3.5)275(0.8) Canada**279(2.7)273(0.6) Czech Republic**277(1.0)274(1.0) Denmark**289(0.8)271(0.6) Finland287(0.9)288(0.7) Germany**282(1.0)270(0.9) Republic of Ireland264(3.2)267(0.9) Italy*243(1.9)250(1.1) Netherlands286(0.9)284(0.7) Norway**294(1.1)278(0.6) Poland*232(1.1)267(0.6) Sweden**306(1.0)279(0.7) United States**273(1.4)270(1.0) England (UK)*267(2.0)273(1.1) Northern Ireland (UK)*264(1.2)269(1.9)

14. Literacy scores in participating countries CountryAverage score Countries significantly above Northern Ireland in literacy Japan296(0.7) Finland288(0.7) The Netherlands284(0.7) Australia280(0.9) Sweden279(0.7) Norway278(0.6) Estonia276(0.7) Flanders (Belgium)275(0.8) Czech Republic274(1.0) Slovak Republic274(0.6) Canada273(0.6) OECD Average273(0.2) Countries not significantly different from Northern Ireland in literacy England273(1.1) Korea273(0.6) Denmark271(0.6) United States270(0.9) Germany270(1.0) Austria269(0.7) Cyprus269(0.8) Northern Ireland269(1.9) Poland267(0.6) Republic of Ireland267(0.9) Countries significantly below Northern Ireland in literacy France262(0.6) Spain252(0.7) Italy250(1.1) Source: PIAAC (2012) Differences are significant at 5 per cent level. () Standard errors appear in parentheses.

15. Numeracy Northern Ireland performed similarly to neighbouring English- speaking countries of England and the Republic of Ireland. Although below the OECD average for numeracy skills, Northern Ireland still compares favourably to France, Italy, Spain and the US in this area. Northern Ireland had a similar spread of attainment compared with the OECD average, indicating a similar range of skills compared with other countries. However, there was a higher proportion of adults at Level 2 or below compared with the OECD average.

16. Numeracy scores in participating countries CountryAverage score Countries significantly above Northern Ireland in numeracy Japan288(0.7) Finland282(0.7) Flanders (Belgium)280(0.8) The Netherlands280(0.7) Sweden279(0.8) Norway278(0.8) Denmark278(0.7) Slovak Republic276(0.8) Czech Republic276(0.9) Austria275(0.9) Estonia273(0.5) Germany272(1.0) OECD Average269(0.2) Australia268(0.9) Canada265(0.7) Cyprus265(0.8) Countries not significantly different from Northern Ireland in numeracy Korea263(0.7) England262(1.1) Poland260(0.8) Northern Ireland259(1.8) Republic of Ireland256(1.0) Countries significantly below Northern Ireland in numeracy France254(0.6) United States253(1.2) Italy247(1.1) Spain246(0.6) Source: PIAAC (2012) Differences are significant at 5 per cent level. () Standard errors appear in parentheses.

17. Problem solving in technology-rich environments Northern Ireland’s average score for problem solving in technology-rich environments was significantly lower than the OECD average and there were no countries that Northern Ireland significantly outperformed. The Republic of Ireland performed similarly, but England performed significantly better. A higher proportion of the population in Northern Ireland compared to the OECD average had sufficient experience of computers and basic computer skills to complete the problem solving assessment.

18. Problem solving scores in participating countries CountryAverage score Countries significantly above Northern Ireland in problem solving Japan294(1.2) Finland289(0.8) Australia289(0.9) Sweden288(0.6) Norway286(0.6) The Netherlands286(0.8) Austria284(0.7) OECD Average283(0.2) Denmark283(0.7) Czech Republic283(1.1) Korea283(0.8) Germany283(1.0) Canada282(0.7) Slovak Republic281(0.8) Flanders (Belgium)281(0.8) England281(1.0) Countries not significantly different from in problem solving Estonia278(1.0) United States277(1.1) Republic of Ireland277(1.0) Northern Ireland275(2.0) Poland275(1.3) Countries significantly below in problem solving

19. Adult skills compared with International Adult Literacy Survey (IALS) in 1996 There have been large changes in the makeup of the population between these two surveys and these changes will impact on literacy scores since 1996. The most profound demographic change has been the large increase in those people who have obtained some secondary school qualification. The Northern Ireland average literacy score has increased significantly between IALS and the International Survey of Adult Skills. The proportion of adults at the two lowest levels of proficiency (Level 1 and below) has decreased significantly. However, the gender gap has widened and is now statistically significant in favour of men.

20. Spread of proficiency – Percentage of adults at each level Domain Below Level 1 (below 176) At Level 1 (176 to below 226) At Level 2 (226 to below 276) At Level 3 (276 to below 326) At Level 4 (326 to below 376) At Level 5 (at or above 376) Missing Literacy Northern Ireland 2.5(0.5)14.9*(0.9)36.2*(1.5)34.3*(1.6)9.4*(0.6)0.5(0.2)2.2 (0.3) cumulative %2.517.453.688.097.397.8100 OECD Average 3.3(0.1)12.2(0.1)33.3(0.2)38.2(0.2)11.1(0.1)0.7(0.0)1.2 (0.0) cumulative %3.315.548.887.098.198.8100 Numeracy Northern Ireland 5.6(0.8)18.7*(1.2)35.9*(1.1)29.0*(1.1)7.8*(0.7)0.7(0.2)2.2 (0.3) cumulative %5.624.460.389.397.197.8100 OECD Average 5.0(0.1)14.0(0.1)33.0(0.2)34.4(0.2)11.4(0.1)1.1(0.0)1.2 (0.0) cumulative %5.019.052.086.397.798.8100

21. Spread of proficiency – Percentage of adults at each level Domain No computer experience Opted out of computer based assessment Failed ICT core Below Level 1 (below 241) At Level 1 (241 to below 291) At Level 2 (291 to below 341) At Level 3 (at or above 341) Missing Problem solving Northern Ireland 10.0(0.6)2.3(0.3)5.8(0.4)16.4*(1.5) 34.5 * (1.2)25.0*(1.2)3.7*(0.6)2.2(0.3) cumulative % 10.012.318.116.451.076.079.72.2 OECD Average 9.3(0.1)10.2(0.1)4.9(0.1)12.3(0.1)29.4(0.2)28.2(0.2)5.8(0.1)1.5(0.0) cumulative % 9.319.524.412.341.769.975.71.5

22. Gender differences Average scores for all three skills for men were statistically significantly higher than those for women, in common with the majority of other countries. The gender difference in Northern Ireland was higher than the OECD average.

23. Age differences In all three skill areas performance by age in Northern Ireland followed the same general pattern as other OECD countries, with young people in Northern Ireland (16-24) achieving a significantly higher average score than those in the oldest age group (55-65).

24. Links between adult skills and industry Adults who worked in professional, scientific and technical activities had the highest average scores in literacy, numeracy and problem solving compared with other industry sectors. Other sectors which had particularly high scores across the three domains include financial and insurance activities; public administration and defence; and compulsory social security. At the other end of the spectrum, adults working in construction had the lowest scores in literacy, numeracy and problem solving. Other industry sectors with low scores across the domains of literacy and numeracy were transportation and storage; accommodation and food services; and manufacturing.

25. Adult skills and salary There is a clear relationship between salary and proficiency in the three domains (for the top half of earners). However, for the bottom half of earners this relationship levels out. The majority of part-time workers and adults still in education are in these groups. It is also likely that these groups include younger workers, who tend to earn less in the early stages of their working lives.

26. Skills use in the workplace For all of these skills, more frequent use was associated with higher average scores in literacy, numeracy and problem solving in technology-rich environments compared with adults who use these skills infrequently, matching the OECD average pattern. This provides strong evidence for the importance of developing and assisting workers to extend and utilise their skills in the workplace.

27. Characteristics of adults with low proficiency The characteristics most likely to be associated with low proficiency were having lower levels of education, having parents who have lower levels of education, not having computer experience in everyday life, or working in certain industries and occupations.

28. Probability of achieving Level 1 or below in literacy, controlling for different characteristics

29. Probability of achieving Level 1 or below in numeracy, controlling for different characteristics

30. Probability of achieving Below Level 1 in problem solving, controlling for different characteristics

38. QUESTIONS?

39. Plausible values Proficiency scores in literacy and numeracy were imputed only for those respondents who had completed at least five questions in the background questionnaire – their skill level was estimated based on the information they had given in the form of ‘plausible values’. Similarly, plausible values were generated for those who did not take any literacy or numeracy items following the core because of the way they were routed through the assessment. For instance, if they were routed through two problem solving components or a numeracy and problem solving component they would have taken only the core literacy items. Those who completed fewer than five of the background questions were not assigned plausible values but were included as part of the weighted population totals. Plausible values are a suitable method of imputing scores because we are interested in population estimates not individual scores.

40. Routes through the assessment