ASSESSING GRADE 5 PUPIL PERFORMANCE (Reading and Mathematics in 2001) Socialist Republic of Vietnam

Vietnam Assessing Grade 5 Reading & Mathematics Presentation Outline  Why do the study -- goals?  The research design  Different stages in the study  Teams and organization  Conduct of study  Findings  Why do the study -- goals?  The research design  Different stages in the study  Teams and organization  Conduct of study  Findings

Vietnam Assessing Grade 5 Reading & Mathematics Why do the study?  Fall in demographic demand allows the Govt. ’ s concern to move from Access to a focus on Quality  But what is current status of Quality and how can it be measured?

Vietnam Assessing Grade 5 Reading & Mathematics Study goals A clear measurement of educational quality and learning outcomes...  to document the baseline at the turn of the century  to allow measurement of the incremental impact of all future interventions in primary education (new curriculum; in-service teacher training; additional reforms in instructional methods)

Vietnam Assessing Grade 5 Reading & Mathematics Study Questions  What is the level of achievement of Grade 5 pupils overall and in the areas of reading and mathematics?  What is the level of Grade 5 teachers in these subjects?  What are the characteristics of pupils; teachers; classrooms; materials; schools?  Are predefined BENCHMARKS being met?  Have inputs been allocated to schools equitably?  What variables are associated with difference between the most and least effective schools?  What variables are most associated with achievement?

Vietnam Assessing Grade 5 Reading & Mathematics The Study Design  A cross-sectional sample survey  Aimed at Grade Five pupils (final grade of Primary)  Testing learning outcomes in two subject areas: Reading (in Vietnamese) and Mathematics  With a sample that is representative at both the NATIONAL and the PROVINCIAL levels (61 provinces)

Vietnam Assessing Grade 5 Reading & Mathematics The Eight Stages of the Study 1.Design of the Study 2.Trial and Adjustment 3.Preparation for & Collection of Data 4.Data Preparation 5.Data Analysis 6.Report Writing 7.Policy Action and Dissemination 8.Follow-up

Vietnam Assessing Grade 5 Reading & Mathematics 1.Study Design  Define the policy questions  Prepare the analysis plan with empty tables  Design the questionnaires  Develop tests in reading and maths for pupils and teachers  Define the sample  Construct the instruments  Design the process to administer the instruments & write data collection manual

Vietnam Assessing Grade 5 Reading & Mathematics 2.Trial & Adjustment  Print Instruments  Train data collectors  Administer data collection in trial area  Prepare data & check the test results of the trial items  Revise instruments and procedures

Vietnam Assessing Grade 5 Reading & Mathematics 3. Preparation for & Collection of Data  Set up the data supervisory system  Hold provincial-level data supervisor training workshops  Organize local-level data collector training  Explain sampling techniques  Supervise both sampling and data collection  Return all instruments to central office

Vietnam Assessing Grade 5 Reading & Mathematics 4. Data preparation  Define the editing rules  Carry out hand editing of data booklets  Data entry and verification (double entry for 2 schools/province ~3%)  Data cleaning and consistency checks (video training from IIEP)  Sample weighting of the data set  Prepare derived variables

Vietnam Assessing Grade 5 Reading & Mathematics 5.Data Analysis  Test scoring and Item analysis  Statistical analysis and preparation of the tables (c.f. #2)  Cross-tabulations and checks  Correlation and regression analysis  Effective school analysis

Vietnam Assessing Grade 5 Reading & Mathematics 6.Report Writing  Interpretation of the tables  Preparation of the text and descriptive and analytical sections  Review results against other existing information  Discussion of results by Govt./MoET managerial group

Vietnam Assessing Grade 5 Reading & Mathematics 7.Policy Actions & Dissemination  Preparation of policy actions, recommendations and changes  Review and adoption by MoET and Govt. leadership  Publication of results  Preparation of regional (3) workshops  Use of results in the annual meeting of Provincial Education Directors  Series of provincial level dissemination workshops

Vietnam Assessing Grade 5 Reading & Mathematics 8.Follow-up  Prepare a full archival dataset  List all additional research topics  Start supplementary analysis of the data  Write up recommendations for inclusion in the next national assessment exercise –Sampling; Focus; Level; Subjects; –Corrections; Additions

Vietnam Assessing Grade 5 Reading & Mathematics The Teams  Overall Management  Item design  Questionnaire design  Sampling  Supervision of Data collection  Data collectors  Data Editing  Data Entry  Data Cleaning & Verification  Data Analysis

Vietnam Assessing Grade 5 Reading & Mathematics The Teams (continued)  Results & Writing Team  Ministry Review team  Dissemination Team  Supplemental analysis teams  Policy recommendation team

Vietnam Assessing Grade 5 Reading & Mathematics Organizational Structure

Vietnam Assessing Grade 5 Reading & Mathematics Conduct of the Study  Design early 2000  Sample August 2000  Collect data April 2001  Data entry July/Oct 2001  Cleaning Oct/Dec 2001  Analysis Jan/May 2002  Report writing April/June 2002  Dissemination July 2002+

Vietnam Assessing Grade 5 Reading & Mathematics Results

Vietnam Planned and achieved sample

Vietnam Assessing Grade 5 Reading & Mathematics Hours/week teaching in primary schools Vietnam: 16.7 hours per week

Vietnam Assessing Grade 5 Reading & Mathematics Total school resources by school location

Vietnam Assessing Grade 5 Reading & Mathematics Total school resources by region

Vietnam Assessing Grade 5 Reading & Mathematics Classroom resources by region

Vietnam Assessing Grade 5 Reading & Mathematics Materials and textbooks by region

Vietnam Assessing Grade 5 Reading & Mathematics Type of classroom by school location

Vietnam Assessing Grade 5 Reading & Mathematics Some benchmarks  Classroom resources (8 items): blackboard, Chalk, teacher table, teacher chair, geometrical instruments, teachers ’ guide (VN&math), and pictures as teaching illustrations.  Class size: maximum 35 pupils per class.  Total enrolment: maximum 1050 pupils per school  Blackboard: all classroom should have.  School resources (12 items): library, hall, staff room, school head room, store room, first aid kit, medical room, play ground, piped water, fence, music equipment, and sports equipment.

Vietnam Assessing Grade 5 Reading & Mathematics Average class size by school location Benchmark: 35 pupils/class

Vietnam Assessing Grade 5 Reading & Mathematics Percent pupil in full day shift National average: 20%

Vietnam Assessing Grade 5 Reading & Mathematics Selected pupil characteristics of Kinh and non- Kinh pupils

Vietnam Assessing Grade 5 Reading & Mathematics Socio-economic status and pupil achievement

Vietnam Assessing Grade 5 Reading & Mathematics Means of achievement in reading and mathematics by homework corrected category

Vietnam Assessing Grade 5 Reading & Mathematics Relationship between years of teaching experience and pupil reading and mathematics achievement

Vietnam Assessing Grade 5 Reading & Mathematics Percentage passed each level

Vietnam Assessing Grade 5 Reading & Mathematics Differences in math scores between schools in Dong Nai Province

Vietnam Assessing Grade 5 Reading & Mathematics Differences in math scores between schools in Son La Province

Vietnam Assessing Grade 5 Reading & Mathematics The study dataset is stored with the Ministry of Education and Training, and a research report is available in three volumes from the World Bank