1. Demand for Internationally Comparable Education Data
My name is Albert Motivans and I head the analytical and policy unit at the UNESCO Institute for Statistics, or UIS, which is responsible for putting to use the data which we collect from member states, like the Russian Federation. While my main work is on the side of data use, I represent UNESCO as one of the stakeholders with an interest in how education statistics develops in Russia, particularly from the international perspective. But I want to stress from the beginning, that I feel this kind of work is a big benefit for countries being able to understand and improve their own systems, and not just a burden that international organisations place on countries in order to publish a statistical yearbook.
I would like to thank Mark Agranovich for this opportunity to meet with all of you. And to thank him and those in the Ministry of Education, National statistical offices, and others who have recognised the strategic importance and made a strong commitment to Russia’s participation in international education work.
Albert Motivans Education Statistics Seminar, Korolev
14 September 2003

2. Who collects global education data?
Agency
UIS
UIS & OECD
UIS, OECD & Eurostat
Instrument
WEI
UOE
Coverage
# countries
+/-125 18
30/15
I’d like to very briefly tell you about how the international collection of education data is organised.
It is basically my institute that has the mandate to collect and disseminate internationally comparable education data
OECD covers its own member states most developed countries and drives much of the indicator innovation through the INES (Indicators of National Education Systems) group and its networks
Eurostat is responsible for the EU, but in education statistics take the lead role with other European countries, especially accession states
UIS leads the WEI or World Education Indicators project jointly with the OECD in 18 middle-income countries, which includes the Russian Federation and others like Brazil, China, Egypt, India and others.
Both the OECD and Eurostat cooperate with UNESCO by providing relevant data to UNESCO database
Finally, there are the rest of the UNESCO member states, roughly 125 countries that maintain bilateral relations with the UIS.
So, you can see it is quite a large mandate – collecting comparable education statistics from every country in the world.
You will also notice that there are three different, but largely similar, instruments that are used to collect education data from countries, the UIS questionnaire, which I have copies of here in Russian; the WEI (which stands for World Education Indicators) questionnaire which Russia completes and the UOE or UNESCO, OECD, Eurostat questionnaire. I’ll discuss the differences between the questionnaires in a moment.
We also use other international data and to calculate indicators such as population estimates from the UN population division and GDP estimates from the World Bank. Finally, it is UIS which distributes the indicators based on the collected data to other UN organisations where it becomes incorporated in the World Bank database, the UNDP human development index and so on. So the data provided to UIS reach a very wide audience.
The UIS can refuse to publish national data where there is a clear issue of data quality. In case of missing data, the UIS can provide its own staff estimates.
UNESCO distributes globally (World Bank, UNDP, UNICEF et al)

3. What kinds of data are collected?
Relies almost entirely on administrative sources of data that cover, broadly speaking, the inputs into the education system:
Students
Teachers
Expenditure
WEI/UOE collects additional, mostly statutory, data on personnel, instructional hours, curricula and also LFS data on earnings, employment and educational attainment
The different data collection instruments cover common areas – the basic characteristics of the students and teachers engaged in different levels of the educational system, from pre-primary to advanced tertiary.

4. How do we ensure comparabiliy?
Systematic use of common frameworks
ISCED for classifying education programmes
UNESCO/OECD/EUROSTAT (UOE) methodology
Assess the magnitude of missing data
Set out an agenda for addressing gaps
easily resolvable
require a longer-term strategy
To ensure international comparability of indicators, there are basically three steps.
First is the systematic use of common frameworks, all comparable data start from a robust mapping of national programmes into the ISCE classification system. There are also specific methodologies used to ensure that like are measured with like, in terms of international definitions for a student, a teacher or an education-related expenditure.
Second is to assess the magnitude of missing data. For example, in many countries, there are problems covering the private sector, both as a provider of education (as there isn’t the same access to data) and as a funder of education (we know much less how much families and communities fund educational institutions). For example, lack of age by grade data, as is the case in the Russian Federation, doesn’t allow the calculation of a number of indicators of student progression.
Third is to set out an agenda for addressing gaps – those most easily resolvable in the short-term, for example, which may mean getting a figure from a different ministry or from financial records or collective bargaining agreements or using the results of a study to generalise and those that are much more complex, such as the area of private education expenditure, which will require a much longer-term strategy
In terms of data quality, we have started with a critical mass of countries that are committed to improving education data from an international perspective, including the Russian Federation, but there is still a very long way to go. We’re building a global map for example in the case of education finance, and are at the stage of refining boundaries in some regions, but with a much rougher map of other regions, where work is much more difficult.

5. The importance of ISCED97
The quality of any international education indicator depends on an ISCED mapping of the Russian educational system that is accepted at both the national and international levels.
Changes in the 1997 ISCED revision, implemented in the same year, mean a potential break in the time series.
The inclusion or exclusion of an educational programme at one ISCED level or another has an impact on all indicators relating to students, teachers and expenditure. So a definitive ISCED mapping is extremely important and should be based on dialogue and consensus with all pertinent stakeholders and should be widely disseminated.
It is important note that the comparability of trend data is an issue, as a revised ISCED classification system was introduced in The old system, called ISCED76 was applied to data prior to The key differences between the two are an additional category in the later version.

6. National v. international tensions
Why should we invest in international comparisons? Is it worth the effort?
How do they help our national planning?
How are international indicators useful when they look different from our national indicators?
There are some hard questions that will eventually need to be asked.

7. The power of international comparisons
Indicators where comparisons add value help to…
benchmark progress (EFA, MDGs)
drive/inform reform
highlight practices in other countries that can enhance performance
Data
Need an analogy for a system that works better, that provides better opportunities for all children
Use the German example from PISA
Policy

8. For student achievement, money matters, but so does how it is used
I’d like to give an example, which combines both inputs (financial resources) and learning outcomes (combined literacy score) based on the PISA study, which the Russian Federation took part in, but which wasn’t able to provide the data on cumulative education expenditure.
These comparative data
More broadly the comparisons in performance of Germany and other countries highlighted that the early grouping of students by ability may lead to lower overall achievement levels among all German students
Source: OECD, 2001

9. Moving from what the state spends to what a society invests in education
The bars represent spending on education as a proportion of GDP
The light blue bars public spending on primary and secondary
The dark blue bars public spending on tertiary
And the red bars represent private spending at any level of education
In the case of Russia, we have a combined figure for public spending and no data on private spending

10. How are priorities for international comparisons changing?
Outcomes
Policies
Country or system
System performance
Institutional organisation and resource allocation
Schools
Performance and quality
Learning environment
Classrooms
Teaching and learning practices
Learners
Knowledge and skills
Student attitude/behaviour
How do we organise all of these disparate elements – into a framework which takes into account the different levels of policy information needs and the different dimensions
Where are we in terms of the international agenda?
What does it imply for sources of data?
Comparing student literacy and links to home and school contexts (PISA, IEA et al)
Some early steps towards opening the “black box” of schooling – what happens at the instructional setting
As a result, other sources of child/student-based data (student, school, household surveys) are used to complement traditional administrative data on inputs

11. Comparable data is hard work but worth the view
The summit:
good coverage of ancillary services, private sources of education funding
Middle camp: solid accounting of public expenditure on education
I would like to close with the thought that comparable data is hard work but brings with it rewards. For example, in the area of education finance data, a country with limited information about the overall size of the education budget might be considered as being at “base camp”,
Building knowledge and using tools gets one to middle camp where there is a solid accounting of public expenditure on education. For many countries this is sufficient, but it becomes clearer that a better accounting of private expenditure gets one to the summit.
It may be considered a luxury, but in strategic terms it is essential for a country like Russia, its economic competitors are certainly engaged.
Base camp:
basic knowledge about budget