1. Learning assessment among indigenous students in Latin America: Challenges of measuring, interpretation and study of factors associated to learning y context of cultural diversity and social inequality Ernesto Treviño Senior Advisor LLECE September 21th, 2011- Cartagena de Indias, Colombia

2. Background Use of standardized tests in contexts of cultural diversity }Purposes and uses of evaluation }Tensions between standardized evaluation and cultural diversity }Knowledge, teaching and learning among indigenous groups }Validity, culture and equity when measuring learning among indigenous populations }Equity and measuring in indigenous contexts Factors associated to learning and intercultural bilingual education }CIPP model for Latin America and the Caribbean }CIPP model and intercultural bilingual education Agenda

3. Latin America }Deep social inequalities }Marginalization of indigenous groups }Stereotyping and prejudice of indigenous groups }Constitutional recognition of cultural, ethnic and linguistic diversity in several countries }Expansion on the application and use of standardized tests without necessarily considering this diversity Challenges – Peru assesses four indigenous languages Background

4.  Tension between standardized evaluation and cultural diversity }Recognition of diversity }Extensive use of standardized evaluations }Admission tests to different educational levels may seriously affect educational opportunities }Limited scientific and educational assessment development for considering cultural diversity High levels of investment for developing tests Disagreement on the writing of indigenous languages Lack of specialized personnel Use of standardized tests

5. Purposes of standardized tests }To know the levels of student achievement in different areas (low stakes or no consecquences) }To make decisions that affect directly individuals or institutions based on the results of the tests (high stakes or with consequences) Uses of standardized tests }Monitoring public policy }Program evaluation }Identification of schools that require support }Uses with problems of causality Asignation of incentives and penalties Stimulation of competition among schools Social sanctions for low performing schools Use of standardized tests

6.  Knowledge, teaching and learning within indigenous groups }Transcultural psychology }Formal education: Individual work Analytic thought Intensive language use Strict time framework Learning of scholastic contents Individual evaluation of students Use of standardized tests

7.  Knowledge…(continuation) }Teaching and learning in indigenous contexts: Collectivist vision of social organization, teaching and learning –Culturally appropriate forms for participation on learning –Pedagogic strategies –Performance measurement Holistic thought Observational strategies and practical participation for learning Lifelong learning and skill acquired through practice Non interventionist attitudes of adults Room for mistakes There is no possibility of public failure: shared responsibility for results Use of standardized tests

8.  Validity, culture and equity when measuring learning among indigenous populations }Validity Property of the inference An inference is valid if it is alinged with the aims of the tests Validity evidence –Test content –Answering process –Internal structure of the test –Relationship with other variables –Consequences of the test Cultural validity –Cultural characteristics for interpreting the world, solve problems and impact results –Universality of constructs Use of standardized tests

9.  Validity…(continuation) }Bias Systematic difference in the validity of the inference, in a way that it is less valid for one (or several groups) under comparison than others Systematic difference is necessary evidence, but not sufficient for demonstrating bias It is necessary to also have substantive evidence of test contents In Latin America indigenous students show systematic differences that affect their results Cultural and language bias in standardized tests }Adverse impact Test may be biased but not necessarily disproportionally affect low achievement groups Use of standardized tests

10.  Validity…(continuation) }Tension between diversity and equity Intercultural and bilingual education (IBE) and society Dominant language is the vehicle for educational mobility Social structures value the dominant language Advancement in language normalization Do not neglect discrimination Use of standardized tests

11.  Equity and measurement in indigenous contexts }Latin America the most unequal region in the world }Use of standardized test with consequences }Measurement for policy feedback and program evaluation }Evaluation for improvement Both standardized and classroom evaluation Use of standardized tests

12. Factors associated to learning in Latin America CIPP Model UNESCO, 2010 Context Education context at home Socioeconomic and cultural index Gender Indígenous populations Child labour Input School infrastructure Access to basic services Availability of computers Additional teacher work Learners Attendance at pre-primary education Repetition Process School climate School management Teacher satisfaction Teacher prestations Product Learning Civic education Social and economic participation School

13. Factors associated to learning in IBE: Context Demographics (Indigenous vs. Non- indigenous) Population Household size Gender distribution Age distribution Socioeconomic information (Indigenous vs. Non- indigenous) Income Quintil of autonomous income Educational indicators Illiteracy rate Years of schooling Educational coverage Number and size of schools Number of teachers per school Student-teacher ratio Sociolinguistic Context Ethnic auto-adscription Number of Competent Indigenous- speaking people Learning in indigenous language Indigenous language use IBE acceptance Principal perception of parental disposition Teacher perception of parental disposition Parental attitude/cognitive disposition Parental attitude/affective disposition Parental attitude/conative disposition

14. Pedagogic resources Texts: School/non-school Form of ellaboration Cultural contents Didactic material, audiovisual resources in language Computers Initial teacher training Teacher level of education Training and skills in cultural contents Tranining and skills in indigenous language Training and skills in didactic of the cultural contents Training and skills in didactic of indigenous language Training and skills in didactic of dominant language Monetary resources and school institution Monetary resources: quantity of funds available, timely distribution, flux of delivery process Curriculum adaptation: school project, local curriculum Actors involved in curriculum creation and adaptation Role and support from IBE educational authorities Factors associated to learning in IBE: Input

15. Teaching practices Classroom practices School organization: teachers work on IBE and update of local curriculum Implementation of teaching second languages and intercultural perspective Community-school mediation School management Internal: instruments for curriculum adaptation, committment and motivation External: relationship with educational authorities Relationship with community (level of inclusion in curriculum adaptation and implementation) Community-school relationship Participation in local practices Celebration of local traditions in school Community participation in curriculum adaptation and implementation Factors associated to learning in IBE: Processes

16. Learning Oral communication in indigenous language Written communication in indigenous language Effect in other learning outcomes Intercultural practices in the school School communit y values ethnic diversity Visibilizati on of intercultur allity in school Communi ty participati on in IBE programs Harmonic relatiohsh ps between indigenou s and non- indigenou s Valued indigenous culture Improve ment in indigenou s students self- esteem Increase in the areas of use and promotion of language( public spaces, school, etc.) Languages revitalization Number of indigenou s languages speakers (by age, gender and geographi c location Intergene rational Recovery or loss of mother language Factors associated to learning in IBE: Products

17. Standardized tests }Inferences related with the purpose of the test }Useful to know the level of achievement of the school system }Help in monitoring programs and policies Problems in culturally diverse contexts: Validity Bias Adverse impact Conclusions and recommendations

18. Educational evaluation }Cultural and linguistic differences between indigenous and non-indigenous groups }It has neither incorporated culture nor indigenous language }More extended use of high stakes standardized tests }Should we test in indigenous languages regardless of the fact that it may not be the language of instruction? Factors associated to learning }Shared factors in indigenous and non-indigenous settings }IBE Different objectives and measurement of achievement Insufficient empirical evidence on factors Conclusions and recommendations

19. Thank you! ernesto.trevino@mail.udp.cl