1. Tools Strategies and Methodologies for measuring impact of ICTs in EDucation Simon Ellis UNESCO Institute for Statistics

2. Some key observations l Impact studies è Every country has a ‘pilot’ impact study on a few schools è Few (no?) countries have nationwide impact studies l The major determinants of school achievement are è Intelligence IQ è Education of parents è NOT computers l ICT financing è OECD; ICTs are a marginal cost so ICTs are ubiquitous in home and school è Developing countries; introducing ICTs in ALL schools in developing countries requires MAJOR investment è Measuring impact is a major issue in developing countries

3. Asia l Many Asian countries are on the threshold of ubiquitous ICT in particular through wireless l Asian countries recognise that technological revolution is necessary for growth and this has to start in schools l ICTs, especially wireless networks, can help bridge urban/rural divide l BUT introducing ICTs everywhere remains a very big investment l Asian pedagogy often consists of direct instruction rather than situational or constructivist learning

4. What must be done l Prove benefit of national programme to put ICTs in all schools given that è Marginal effect on exam results è Ubiquitous ICTs necessary for growth (having a computer at home more important than having one at school) l Need to è Justify major financial investment è Encourage further investment

5. Mobilising a partnership l UNESCO; Bangkok/UIS l KERIS l World Bank l OECD l EU

6. UNESCO Institute for Statistics (UIS) global mandate l WSIS 2003/5 set up Partnership for Measurement of ICTs for Development è UNCTAD, ITU, UNESCO, WBank, OECD, EU, UNSD è Partnership assigned global leadership for ICTs in Education to UIS l UNESCO intersectoral meeting July 2008 (CI and Ed) assigned UNESCO lead on measuring ICTs to UIS

7. UIS value-added l The official international lead agency for education indicators for EFA and MDGs l Existing global data collection è In support of EFA è Put ICTs in context of education system indicators l Able to develop and apply global international statistical standards è Eg ISCED, FCS, core ICT indicators all for endorsement by UNESCO General Conference and UN Statistics Commission after extensive international consultation l Strong network of national agencies officially responsible for managing education indicators

8. Levels of ICT indicator 1. Infrastructure è Presence of ICT ( –only used by headmaster/accounts 2. Access è Availability è Time use –How many hours per week per pupil –Shared PCs –email 3. How used? è Skills è Type of use; research, report preparation, presentations

9. Measuring Level 3 – skills and impact l Use of email (student output onto the internet) l Embedding of downloaded image and sound in written work and presentations (use of new media) l Posting of web pages, Facebook, Blogs etc (pro-active internet-based work) l Overall level of generic skills in use of technology è Communications skills with ICTs è Problem-solving skills with ICTs

10. Survey instruments l Admin data »Used to manage education system, assess spread of new technology, costs l School survey »Skills assessment eg PISA »School conditions eg SPS l Household survey »Children not in school »Home environment –Pc, internet access, parents literacy –Eg. national census, LAMP

11. UIS actions in Asia l Organizes annual/6 month meetings of regional EFA Co-ordinators and partners (with UNESCO APPEAL team) l Worked with APEID on previous UNESCO Bangkok initiative on ICT in Education statistics l WISE group (in Asia, Thailand, Korea, Malaysia) implementing new survey to collect 48 indicators l Long-term statistical capacity building activities è Lao, Brunei, Cambodia, Myanmar, India, Nepal, Bangladesh, Bhutan è Data planning and DQAF è EMIS and data collection techniques l Data collected Admin data levels 1 and 2

12. KERIS initiative l Data collected from Thailand, Philippines, Vietnam, Sri Lanka, Mongolia l Includes more contextual data l More analytical approach

13. Risks l Duplication; two or more teams collecting same data »Conflict between two national teams over ‘official mandate’ »Response ‘fatigue’ means no-one gets data »Human/financial resources spent twice to get same data l Confusion; unclear whether data requested by KERIS or UIS »Questionnaire sent to wrong agency l Inconsistency; slightly different data sent to UIS and KERIS »More recent data »Errors due to response fatigue »Different reference period »National or international definitions (eg ISCED mapping)

14. Co-ordination More international co-ordination of work on ICT in Education is needed to achieve l Sustainability l National capacity building l New indicator development

15. How to ensure sustainable impact of KERIS and UIS programmes? l National ICT indicators (levels 1 and 2) must be embedded in EMIS systems and collected regularly to monitor è progressive expansion of ICT use in schools è integration of ICTs in teacher training l BUT EMIS cannot capture integration into teaching practices

16. How to ensure that KERIS/UIS build national capacity to monitor ICTs? l Co-ordinate all regional programmes è to develop nationally implementable measures of ICT impact è To help countries to collect data collection to meet national needs l Co-ordinate interventions in particular countries to build on each others successes and not duplicate same activity