1. PRELIMINARY RESULTS JUNE 6, 2011 ARIANNA LEGOVINI MIRIAM BRUHN BILAL ZIA WORLD BANK Impact Evaluation of Brazil’s School Based Financial Education Program 1

2. Impact Evaluation Objectives (Students) Measure the impact of a three semester long program that teaches high school students financial concepts  Does the program improve students’ financial knowledge? The program also includes exercises and case studies related to the students’ daily life (e.g. making a budget)  Does the program change attitudes about financial decisions, as well as actual financial behavior and decision making? 2

3. Impact Evaluation Objectives (Parents) Some exercises are meant to be done at home, jointly with the parents  Is this enough to also improve parents’ financial knowledge and to change their attitudes and behavior? Can we achieve a greater impact by also providing a financial literacy workshop directly to parents? 3

4. Methodology Pilot test the financial education program in close to 900 schools in various states in Brazil Schools that were interested in participating in the pilot were randomly divided into two groups  Treatment group receives financial education text books for free AND receives teacher training for how to implement the material  Control group will receive the same two school years later (possibly with modified material and training, based on evaluation results) Half of the parents in treatment schools will also randomly be selected for participation in a parent financial education workshop in schools 4

5. Pilot Sample Size Total of 891 schools in six states (439 treatment and 452 control schools) Approximately 26,000 students and parents (one class per school) StateNumber of schools São Paulo372 Rio de Janeiro270 Ceará122 Distrito Federal64 Tocantins34 Minas Gerais29 5

6. Timeline TimeActivity April – May 2010Complied list of interested schools with help of State Ministries of Education; Randomly divided schools into treatment and control group May – July 2010Teacher training Early August 2010Baseline survey Mid August 2010Teachers started teaching financial education material, continuing until November 2011 Late November 2010First follow-up survey Spring 2011Parent workshops November 2011Second follow-up survey 6

7. Results Presented Today Use data from August 2010 baseline survey and November 2010 follow-up survey Students had received about 1 semester of financial education before the November 2010 follow-up survey Full program takes 3 semesters – the pilot ends in November 2011 Results presented today are preliminary since they only cover the first out of three semesters of financial education 7

8. Baseline Characteristics: Balance across Treatment and Control Group 8

9. Follow-up Data: Impact Analysis 9 At baseline, average student and parent characteristics, as well as test scores, were the same in the control and treatment group  This is because the treatment was randomly assigned  Therefore the control group is a valid comparison group for the treatment group Thus, any differences in averages between treatment and control group at follow-up are CAUSED BY the financial education program

10. To measure students’ financial knowledge, the survey firm (CAEd) developed a test tailored to the program’s material Based on this test, CAEd calculated the level of financial proficiency for each student, ranging from 0 to 100 Students’ Financial Proficiency 10

11. In the follow-up survey, the average level of financial proficiency was significantly higher in the treatment group (60.4) than in the control group (56.1) Impact on Financial Proficiency 11

12. Statistical Analysis of Impact on Financial Proficiency 12 The statistical test of these differences shows that the financial education program increased the level of proficiency on the financial knowledge test by 3.6 points (compared to 56 points in the control group)

13. How large is the effect on financial proficiency? 13 The 3.6 point increase corresponds to a 6.4% increase compared to the average in the control group (in one semester)  According to CAEd, other school programs they have worked with in Brazil tend to improve test scores by 2.5% per school year  In Mexico, providing cash transfers to parents conditional on children’s school attendance improved test scores by 8-9% for 6-14 year olds (measured 6 years after the start of the transfers) The effect size (0.25 of a standard deviation) is similar to the effect of  A program in Colombia that provided low-income children with vouchers for half the cost of private secondary school (0.2 std. dev. after 6 years)  A program in India that provided remedial education to students lagging behind in school (0.28 std. dev. after two years, the effect of the same program after one year was 0.14 std. dev)

14. Effect Size (est.) 14

15. Students’ Self-Reported Knowledge Ratings 15 The surveys also asked students to rate their knowledge about different financial concepts  On a scale from 0 to 10, how would you rate your knowledge about  Interest rates  Loans/financing  Insurance  Income tax  Minimum payment for credit cards

16. Impact on Students’ Self-Reported Knowledge Ratings (Follow-Up Data) 16 At follow-up, average self-reported knowledge ratings for all these concepts were higher in the treatment group than in the control group

17. 17 Statistical Analysis of Impact on Students’ Self-Reported Knowledge Ratings The program significantly increased self-reported knowledge ratings of several financial concepts  Size of the increase is about 0.74 to 0.92 points, compared to an average rating of about 5 to 6.4 in the control group

18. Two Questions Often Used in the Economics Literature to Measure Financial Literacy 18 1. Suppose you are interested in taking a loan of R$50,000. Bank “A” offers you a loan of R$50,000 payable in one year with an annual interest rate of 15%. Bank “B” offers you a loan of the same amount, R$50,000, but you will have to pay $60,000 within one year. Which loan would you prefer? a)Bank A b)Bank B c)Don’t know 2. Suppose the interest rate on your savings account is 1% per year and inflation is 2% per year. After a year, if you had not made any deposits or withdrawals in this account, would you be able to buy more than, exactly the same as, or less than today with the money in this account? a)More than today b)The same as today c)Less than today d)Don‘t know

19. Impact on Students’ Correct Answers to Financial Literacy Questions Used in Economics Literature (Follow-Up Data) 19 At follow-up, a higher percentage of students in the treatment group than in the control group gave the correct answer to the financial literacy questions often used in the economics literature

20. 20 The program significantly increased the percentage of correct answers to financial literacy questions often used in the economics literature The size of this increase is  2% for the first question (compared to 40% correct answers in the control group)  3% for the second question (compared to 33% correct answers in the control group) Statistical Analysis of Impact on Students’ Correct Answers to Financial Literacy Questions Used in Economics Literature

21. Student Financial Autonomy Autonomy index, designed by CAEd, based on 3 dimensions of Autonomy  Reflexive Autonomy e.g. “ I like to think carefully before deciding to buy something”  Emotional Autonomy e.g. “I am prepared to talk about money with my parents”  Functional Autonomy e.g. “I always try to save some money to do things I really like” These three components were combined into one overall index of financial autonomy, where higher values indicate more financial autonomy 21

22. Impact on Student Financial Autonomy (Follow-up Data) 22 At follow-up, students in the treatment group obtained higher scores on the autonomy index than students in the control group

23. 23 Statistical Analysis of Impact on Student Financial Autonomy The program increased the overall (final) measure of students’ financial autonomy by 1.9 points (compared to a score of 49 in the control group)

24. Students’ Intention to Save Intention to save index, designed by CAEd, based on 3 components  Attitudes towards behavior e.g. “In my opinion, saving money every month extremely beneficial  Subjective norms and expectations e.g. “My family has the habit of saving money every month”  Perception of capacity of controlling one’s behavior e.g. “I believe I can save a little money every month” These three components were combined into one overall index of intention to save, where higher values indicate higher intention to save 24

25. Impact on Students’ Intention to Save (Follow-Up Data) 25 At follow-up, students in the treatment group had a higher measured intention to save than students in the control group

26. 26 Statistical Analysis of Impact on Students’ Intention to Save The program increased the overall measure of student’s intention to have by 6.2 points (compared to a score of 97 in the control group)

27. Students’ Saving Behavior 27 In order to assess whether intention to save is also reflected in savings behavior, the survey asked students  Which fraction of their income they save  About 61% of students have non-zero income (from work or from parents)  About 37% of students work  Whether they save to be able to finance future projects

28. Impact on Students’ Saving Behavior (Follow-Up Data) 28 At follow-up, a higher percentage of students in the treatment group than in the control group  Saved at least some of their income  Saved to able to finance projects in the future

29. 29 Statistical Analysis of Impact on Students’ Saving Behavior The program significantly increased the % of students who  Saved at least some of their income (by 4.8% compared to 44% in the control group)  Saved to be able to finance projects in the future (by 2.4% compared to 16.5% in the control group)

30. Students’ Spending Behavior 30 The survey asked a number of questions to assess students’ spending behavior  Do students make a list of expenses every month?  How do students pay for purchases?  Do they use a credit card?  Do they use a booklet/installment plan in the store?

31. Impact on Students’ Spending Behavior (Follow-Up Data) 31 At follow-up, a higher percentage of students in the treatment group than in the control group made a list of expenses every month A lower percentage of students in the treatment group than in the control group  Often shopped with a credit card  Often shopped with a booklet/installment plan in the store

32. 32 Statistical Analysis of Impact on Students’ Spending Behavior The program significantly increased the % of students who make a monthly list of expenses by 2.8% (compared to 13% in the control group) The program significantly decreased the % of students who  Often shop with a credit card (by 1.3% compared to 11% in the control group)  Often shop with a store plan (by 1% compared to 10% in the control group)

33. Other Measures of Financial Behavior 33 Apart from saving and spending behavior, the survey asked about several other features of students financial behavior  Did the student learn to manage money in school?  Does the student negotiate how to pay when making a purchase?  Does the student search for other brands and model when deciding which product to purchase (i.e. do the compare options?) These are often thought to be savvy/smart financial behaviors

34. Impact on Students’ Financial Savvy (Follow-Up Data) 34 At follow-up, a higher percentage of students in the treatment group than in the control group  Say they learned to manage money in school  Negotiate how to pay  Search similar model/brands when deciding what to buy

35. 35 Statistical Analysis of Impact on Students’ Financial Savvy The program significantly increased the % of students who  Say they learned to manage money in school (by 13.5% compared to 2% in the control group)  Negotiate how to pay (by 4.3% compared to 63% in the control group)  Search similar brands (by 2.5% compared to 66% in the control group)

36. Parent Outcomes 36 Parents have not received financial education directly But the students’ financial education material includes exercises to be completed with parents (e.g. make a household budget) This can have positive “spillovers” on parents’ financial knowledge  To measure parent’s financial knowledge, the parent survey included the same two financial literacy questions often used in the economics literature that were also included in the student survey

37. Impact on Parent Outcomes (Follow-Up Data) 37 At follow-up, a higher percentage of parents in the treatment group than in the control group  Say they talk to the student about finances  Say the student helps organize the household budget But, there are only small differences in the percentage of parents who answer the financial literacy questions correctly

38. 38 Statistical Analysis of Impact on Parent Outcomes The program significantly increased the % of parents who  Say they talk to the student about finances (by 3.6% compared to 67% in the control group)  Say the student helps organize the household budget (by 4.9% compared to 48% in the control group) So far, we see no statistically significant impact of the program on parents’ financial knowledge

39. Conclusions 39 The first semester of the high school financial education program increased students’ financial knowledge The program also led to improvements in students’ financial attitudes And, the program changed financial behavior – due to the program, students are more likely to  Save and manage their expenses  Engage in “smart” financial behaviors  Students are also more likely to talk to their parents about financial matters and to help with organizing the household budget “Spillover” effects on parents’ financial knowledge are small so far  need for the parent workshop

40. Next Steps Implement the parent workshop  Currently working on this  Have received updated student lists from close to 150 schools and have randomly allocated parents in these schools into two groups  One will watch financial literacy video  Other will watch health video  Still need to receive student lists from the remaining treatment schools Start planning the November 2011 survey 40

41. World Bank Team & Contact Info TTL  Rogelio Marchetti (LCSPF) rmarchetti@worldbank.orgrmarchetti@worldbank.org Researchers  Miriam Bruhn (DECFP) mbruhn@worldbank.orgmbruhn@worldbank.org  Bilal Zia (DECFP) bzia@worldbank.orgbzia@worldbank.org Field coordinator  Luciana Leão (LCSPF) lululeao@terra.com.brlululeao@terra.com.br Advisor  Arianna Legovini (DECOS) alegovini@worldbank.orgalegovini@worldbank.org 41