Office of the State Minister of Reforms Coordination of Georgia /GORBI Monitoring Governance Reforms Pilot Survey in Tbilisi Schools

Methodology The presented data is based on multi-component research projects; the utilized survey instruments that involved all research components were based on a methodology that was first developed at the World Bank Institute under the leadership of Dr. Daniel Kaufmann. The field work for quantitative components were conducted between June and July 2006 in Tbilisi and a following series of focus group discussions were conducted in July. Prior to each increment of fieldwork, pilot surveys were field tested and refined. Face-to-face interviews were conducted for parents and a teacher’s component along with a self-administered questionnaire was utilized during the interviewing of student respondents. Random probability sampling was the chosen method for selecting schools in all five Tbilisi districts and both public and private schools were included in the sampling frame. A sampling totaling 50 public and 10 private schools was used for the project design, which included 800 parents, 1000 students and 403 teachers; principals were included in this sampling of 403 teaching professionals. Data was process and analyzed at GORBI’ headquarters office in Tbilisi in cooperation with an invited consultant Mr. Pablo Zoido.

Quality Evaluations 89% 88% 70% 41% 81% 61% 26% 100% 70% Public Schools Private Schools % of respondents who chose “good” or “very good” quality 56% 45% 29% 57% 45% 23% 79% 80% 70% 50% Overall Teaching Management Facilities Parents Students Teachers

Detail Quality Evaluations Public Schools 64% 62% 69% 72% 85% 66% 71% 64% 79% 84% 76% 93% 88% 93% Students at my school help each other when they can I would not move to my children to another school My school's highest priority is the success of the student Students at my school feel safe on the way to and from the school The core subjects (math, language arts, social studies, science) students learn at your school are useful for them Parents Students Teachers 19% 26% 44% 34% 52% 28% 36% 21% 48% 35% 22% 38% 71% 48% 81% School facilities meet parents' and students' expectations My school provides the learning resources that meet my needs (e.g. computers, internet, library, lab) Students at my school follow the rules Textbooks used in the school meet parents' and students' expectations Teachers at my school treat me fairly Top 5 issues Bottom 5 issues

Determinants of Quality: What actually works and what does not? We constructed a regression model based on quality assessment by parents and students, including how teachers and administrators evaluated the way in which schools are operated. The main findings of this statistical model in presented below. Personal management Budget management Information management Informal payments Conclusions Results from regression model (controlling for type and size of school): Dependent variable: is quality assessment by students/parents Independent variable: evaluations by teachers/administrators

What actually works and what does not? Personnel management

What actually works and what does not? Personnel management Position vacancies announced within the school as well as announced publicly outside the school (q22.2) Strong Disagreement Strong Agreement High quality Low quality Teaching Quality (Students)

What actually works and what does not? Personnel management Formal rules high quality education is associated with the application of formal rules and procedures of human resource management that : requires many administrative steps and procedures. strictly enforced, or in another words, any non-compliance can result in negative consequences for those that do not follow the established rules. Management decisions Transparency and accountability enhance quality: High quality is associated with human resource decisions that are made in a transparent manner and are subject to appeal procedures also enhance quality. Basing hiring and promotion decisions on merit, education and seniority is also associated with greater quality. some puzzles: do illegal payments / nepotism increase quality of education?

What actually works and what does not? Personnel management-continued

What actually works and what does not? Personnel management-continued Median Reported Monthly Salary (lari) High quality Low quality Overall Quality (Parents) * Median salary for schools in VS region (420 lari) not included

What actually works and what does not? Personnel management Salary and personal satisfaction There is convincing evidence that satisfaction with the level of one’s own salary and benefits package is associated with higher teaching standards and overall educational quality. However, increased salaries and higher salary increases are not strongly associated with overall improved quality. Training The amount of training provided to teachers and administrators does not appear to Significantly impact the quality of education. More than 75% of teachers and administrators had received some kind of training in and 25% had participated in three or more training events. Disciplinary actions/sanctions Disciplinary actions does not appear to result in improved performance. Turnover There is weak evidence that excessive teacher and administrative turnover negatively impacts the overall quality of education.

What actually works and what does not? Budget management

What actually works and what does not? Budget management Decisions relating to the budget administration were announced/open to public knowledge through various legal means Strong Disagreement Strong Agreement High quality Low quality Facilities Quality (Parents)

What actually works and what does not? Budget management When budget-management decisions are made openly and transparently, the quality of services provided by Tbilisi schools are improved, especially in terms of school facilities.

What actually works and what does not? Information management

What actually works and what does not? Information management Well-defined mechanisms/procedures that considers both the feedback and needs of students and parents? (q61.1) Don’t exist Fully exist and effective High quality Low quality Management Quality (Parents)

What actually works and what does not? Information management There is a positive impact on educational quality when students are fully aware of what it is expected from them in terms of academic performance. In general, with greater transparency and accountability between school teachers/administrators and students/parents is associated with better quality education.

What actually works and what does not? Informal payments

What actually works and what does not? Informal payments Proportion of teachers that receive informal payments at your school High quality Low quality Teachers Quality (Students)

What actually works and what does not? Informal payments Those schools with lower quality are also those that experience a higher degree of informal payments. Paying for good marks appears to be particularly worrisome.

Assessment of Reforms

Q. How supportive would you be of the following types of public school reforms? Teacher’s Support to Reforms

Teacher’s Awareness of the New Scheme Q : Are you aware of a new scheme of determining salaries for teachers? Q: How effective, do you think this new scheme is?

Effectiveness of the New Scheme (Teachers) Q: Has or will your salary be recalculated according to this new scheme? Q: If yes, in your case, has or will your salary be increased or decreased?

Q: Do you think such a system would have an immediate effect on the quality of teaching? Effectiveness of the New Scheme (Teachers)

Q. To what extent do you agree or disagree that Parent’s Support to Reforms

Have you ever heard of School Boards? Do you know the Board's functions/responsibilities ? Knowledge on School Boards (Students) Q: How many members does it contain? Q. Who are the members of your School Board?

What Conclusions 1.openness and transparency in personal and budget administration is priority for improving services provided by Tbilisi schools 2.It is important to have teachers satisfied with salaries and all the benefits. \but this does not necessary means higher salaries 3.Informal payment and other forms of corruption hinder service quality 4.Quality of education and services in assessed positively by the most respondents 5.On going reforms are also assessed more positively than negatively 6.Majority of respondents are well informed about reforms in educational system