1. Multivariate analysis of the Kyrgyz data from CAPSA
Presentation at the II Academic Conference „Assessment in Education: Experience and Future Development “ Astana, Kazakhstan 29 September, 2016
Prof. Dr. Rainer Lehmann Humboldt University, Berlin, Germany

2. Multivariate analysis of the Kyrgyz data from CAPSA 29 September 2016
In September 2012, educational experts and officials from five Central Asian countries
Kasakhstan
Kyrgyzstan
Tajikistan
Turkmenistan
Uzbekistan1
agreed to conduct in their countries parallel surveys of student achievement in reading comprehension and mathematics, by way of representative random samples of fourth-grade pupils (intact classrooms) with maximum coverage. This joint effort was labelled „Central Asian Program of Student Assessment“ (CAPSA).
1 decided to withdraw at an early stage of the Program

3. Multivariate analysis of the Kyrgyz data from CAPSA 29 September 2016
General Aims
Within this common strategy, it was further agreed
to base the assessment on domain-specific frames of reference, compatible with all national curricula involved;
to define these frames of reference in a hierarchical manner so as to guide the development and selection of common test items;
to standardize data collection procedures across all populations (including pupils instructed in the national language and in Russian in all participating countries);
to conduct pilot tests in order to ascertain nationally the quality of the tests employed, to achieve high response rates and to estimate the sample sizes required to attain an effective sample size of n* > 400

4. Multivariate analysis of the Kyrgyz data from CAPSA 29 September 2016
Procedures
National Working Groups were trained to work with the statistical package SPSS (© IBM) and the program for probabilistic test analyses ConQuest (© ACER). These activities were significantly supported by Dipl. Psych. Julia Levin (Senior CIM Expert, Academy of Education of Kyrgyzstan).
In each country, the main data collection consisted of two independent sub-samples (national language and Russian), in which both reading comprehension and mathematics was tested.
Tajikistan and Turkmenistan could not adhere to the international sampling guidelines.

5. Multivariate analysis of the Kyrgyz data from CAPSA 29 September 2016
Procedures
In Kyrgyzstan a total sample size of N ~ 4030 was achieved. Tajik and Uzbek speaking students had to be excluded.
The configuration of the background questionnaire and the data collection were accomplished by the Kyrgyz working group:
Samarkul Umralieva
Mira Mykyeva
Zeinep Zhamakeeva
Dinara Dautova
Bekzhan Kubanychbek uulu
Julia Levin
The work was supervised by a steering committee, consisting of
Dr. Abakir Mamytov
Artur Bakirov.

6. Multivariate analysis of the Kyrgyz data from CAPSA 29 September 2016
Properties of the Kyrgyz sample
Total n Kyrgyztan
Design effect
(Deff)
Effective sample size
Standard-error of sampling
Mathematics
N(Kyrgyz)
3120
10,97
285
0,059
N(Russian)
906
7,86
115
0,093
Reading
3123
9,63
324
0,058
907
8,34
109
0,096

7. Multivariate analysis of the Kyrgyz data from CAPSA 29 September 2016
Intermediate Findings
Significant differences in terms of mean achievement were observed between the two sub-samples (Kyrgyz vs. Russian language of instruction; see below). Consequently, the aspired sampling accuracy was not attained.
Moreover, the clientele of classrooms with Kyrgyz as the language of instruction proved to be significantly more heterogeneous than that of classrooms with Russian as the language of instruction.
Gender differences followed the almost universal pattern of girls performing better than boys in reading comprehension whereas the opposite held true in mathematics. It is noteworthy that the reading effect was more pronounced than the mathematics one.

8. Multivariate analysis of the Kyrgyz data from CAPSA 29 September 2016
Further Findings
Absence of one or both parents from home for a significant period – presumably due to work – coincided with reduced achievement levels of their children.
The presence of educationally relevant resources (own room for study, computer applied to learning, books) as well as instruction perceived as motivating were conducive to achievement.
Regional disparities in favor of pupils from the capital (Bishkek) were striking. The only exception appeared to be the City of Osh in the domain of mathematics.
These preliminary conclusions need to be tested, however, by way of multivariate analyses in which indirect effects are accounted for.

9. Model explaining differences in reading comprehension among fourth-grade pupils in the Republic of Kyrgyzstan (unstandardized path coeff.)
Бишкек
Иссык-Куль
-34
-78
Reading R2 = 0,123
Ош область
-29
Чуй
-31
Талас
-27
Нарын
-55
Джалал-Абад
-47
-15
2,6
5,6
4,1 25
- 4,8
Баткен
-81
г. Ош
parent(s) absent (Q5)
resources (Q7)
books (Q9)
reading instruction (Q14)
Female gender
Russian language

10. Model explaining differences in mathematics among fourth-grade pupils in the Republic of Kyrgyzstan (unstandardized path coefficients)
Бишкек
Иссык-Куль
-30
-39
Mathematics R2 = 0,345
Ош область
-32
Чуй
-32
Талас
-31
Нарын
-26
Джалал-Абад
-36
,56 
-7,9
-3,3
- 8,8
Баткен
+ 2
г. Ош
Reading
Mathem. instruction (Q14)
Female gender
Russian language

11. Multivariate analysis of the Kyrgyz data from CAPSA 29 September 2016
Conclusions
The purpose of educational monitoring is to call attention to shortcomings in an educational system, such as overall or sectoral achievement deficiencies or inequalities of educational opportunities. In the case of Kyrgyzstan, two aspects stand out.
There are very large regional disparities, especially in terms of reading comprehension: Already in fourth grade, the pupils of some oblasts have fallen behind by almost a standard deviation.
There are also noticeable social disparities: If one or both parents work in a distant location, often in Russia, their children‘s academic performance levels are lower than would be expected under different circumstances.

12. Multivariate analysis of the Kyrgyz data from CAPSA 29 September 2016
Thank-you very much for your attention!