Challenges in Developing a University Admissions Test & a National Assessment A Presentation at the Conference On University & Test Development in Central Asia Kyrgyz Republic, 5-6 Nov Howard T. Everson Center for Advanced Study in Education City University of New York & The American Councils for International Education

Purposes of Large-Scale Assessments Ranking, selection, placement Student-centered and high-stakes Accountibility At the level of schools and school groups Monitoring At the group level (country, state, student groups)

Making Good University Selection Decisions ACCEPTREJECT SUCCEED FAIL False NegativeTrue Positive False PositiveTrue Negative Test Scores

Assessment: A Process of Reasoning from Evidence Cognition-model of how students represent knowledge Observations-tasks or situations that allow us to observe students’ performance Interpretation-method of making sense of the data Inference-judging what students’ know & can do Cognition ObservationsInterpretation Inferences

The Curriculum-Instruction- Assessment Triad Assessment Triad Assessment CurriculumInstruction Theory of Learning & Knowing

Assessment in the Service of Learning Assessment is the art & science of knowing what students’ know Assessments provide “evidence” of students’ knowledge, skills & abilities Evidence supports teachers “inferences” of what students’ know & can do Inferences guide and inform policy and instruction

Examples of Large-Scale Assessments TypeKyrgyzUSA Ranking, selectionORTSAT, ACT Accountability?State assessments MonitoringORT PISA, TIMSS NAEP 1 1 NAEP: National Assessment of Educational Progress

Assessments for Monitoring & Accountability  Purpose s: Determining students knowledge and skill profiles at group level Determining the changes in these profiles from year-to-year Providing data to educational researchers

Assessments for Monitoring & Accountability  Design Characteristics Sampling-based Wide content coverage Items distributed to students (each student takes only a portion of the items). A large portion of the items are not released in order to be able to perform trend analysis.

Assessments for Monitoring & Accountability  Characteristics of the analysis phase Individual scores are optional (reliability and validity at a the student level is poor). Analysis is done at the group level (reliability and ability at the group level is strong).

Assessments for Monitoring & Accountability  Reporting Characteristics Group level mean scores % of students at achievement levels Changes in mean scores and % at each achievement level Gaps in the performance of certain student groups and the changes in such gaps from year-to-year

U.S. Nation’s report card Trends in scale scores at the national level 4 th -grade math Basic: 214 Proficient: 249 Advanced: 282

U.S. Nation’s report card Gaps in student performance from year to year 4 th grade Math

How would a Kyrgyz Accountability Test Help?  Academic profiles of students  Can the ORT serve this purpose? Not wide enough content coverage Limited samples (self-selected)  Can TIMSS, PISA serve this purpose? Difficulty in covering the national curriculum Not administered often enough

Why Focus on Assessment for Learning ? Teachers need information to evaluate whether their teaching strategies are working. Students need feedback to monitor their own learning success and to know how to improve. Policy makers and instructional planners need to know about the levels of knowledge and understanding of individual students and groups of students so they can design the most appropriate instruction

How would a Kyrgyz Accountability Test Help?  Wider content coverage  Achievement levels: meaningful results,measurable targets  Monitoring year-to-year progress  Providing data to educational researchers, teachers, policymakers

Why Search Alternative Tests? Pressure to Reform Education Accountability Measures Integration of instruction & assessment Technological Advances Computer-Based Testing Portfolios Standards-Based Assessments Influence of Cognitive Psychology Modern Views of How People Learn New Theories of Developing Abilities