Why causal answers are needed and how we can get them Dr Steven Stillman Senior Fellow, Motu Adjunct Professor of Economics, Waikato NIDEA Theme Leader,

Slides:



Advertisements
Similar presentations
Policy recommendations that may contribute to better education outcomes of immigrant children The case of Switzerland ( OECD Economic Surveys 2007 )
Advertisements

A Research-to-Practice Study. Research suggests parent training leads to positive outcomes for children; however, many studies lack methodological rigor.
Figure ES-1. How Well Do Different Strategies Meet Principles for Health Insurance Reform? Principles for Reform Tax Incentives and Individual Insurance.
Place and Economic Activity: Key issues from the area effects debate Nick Buck ISER, University of Essex.
IFS Parental Income and Childrens Smoking Behaviour: Evidence from the British Household Panel Survey Andrew Leicester Laura Blow Frank Windmeijer.
Marion Macleod Senior Policy and Parliamentary Officer.
New Commitment Devices Jon Zinman Dartmouth College (joint work with Dean Karlan) AEA 2010.
Migration, remittances, and development indicators: The economic pillar Ben Slay Team leader, regional poverty reduction practice UNDP—Europe and Central.
Area Profiles Worcestershire Partnership Management Group David Onions Corporate Business Intelligence and Performance Manager.
Who volunteers? Volunteering trends based on government data: A briefing from nfpSynergy February 2011 Telephone: (020)
Choosing the level of randomization
For learning and competence Improving the Quality of Education in Finland Belgrade 6-7 December 2012 Raakel Tiihonen Director Information and Evaluation.
Understanding relationships between education, health and well-being among young people: what can the HBSC study contribute? Professor Candace Currie HBSC.
External validity: to what populations do our study results apply?
Conclusion Epidemiology and what matters most
Group versus Individual Liability The Philippines Lecture # 20 Week 12.
Grace Piggott Junior University of Wisconsin Stout Advisor Dr. Fassil Fanta.
Girls’ scholarship program.  Often small/no impacts on actual learning in education research ◦ Inputs (textbooks, flipcharts) little impact on learning.
Cross Sectional Designs
THE URBAN INSTITUTE Genevieve Kenney 2009 ACAP Medicaid Managed Care Policy Summit Hotel Monaco – Washington, DC July 15, 2009 Health Reform for Children:
Impact of Migration on Older Age Parents A Case Study of Two Communes in Battambang Province, Cambodia Paper presented at Mekong Workshop, Salt Lake City.
Identifying Non-Cooperative Behavior Among Spouses: Child Outcomes in Migrant-Sending Households Session 4E: Growth, Jobs and Earnings May 15, 2008 Joyce.
24 July 2014 PISA 2012 Financial Literacy results – New Zealand in an international context.
EMPOWERING WOMEN: LEGAL RIGHTS AND ECONOMIC OPPORTUNITIES IN AFRICA Mary Hallward-Driemeier Office of the Chief Economist, The World Bank.
How large are returns to schooling? Hint: money isn't everything Philip Oreopoulos and Kjell G. Salvanes September 2009.
Mobilizing Savings through Products and Persuasion Nava Ashraf Harvard Business School Oct. 17, 2005.
Microfinance and Education Lecture # 17 Week 10. Structure of this class Further inquiry on adding on “human capital accumulation” in microfinance A case.
Poverty: Facts, Causes and Consequences Hilary Hoynes University of California, Davis California Symposium on Poverty October 2009.
Migration Policies and Interventions: Lessons from New Zealand and the Pacific David McKenzie, Development Research Group, World Bank.
Richer or poorer - gender, income and wealth Planning for retirement Professor Hazel Bateman School of Risk & Actuarial Studies UNSW Business School ARC.
 Introduction (Scary details)  Part I: Introduction to Stock Market Challenge (Brett) 4:30 to 5:15  Part II: What is Financial Literacy (Bill) 5:15.
Racial and Economic Segregation in Schools: Barrier to Quality and Equality in Education Baris Gumus-Dawes.
Financial inclusion: A means to an end Presentation by Dean Karlan.
The effect of migration on children’s outcomes: Evidence from Mexico and the Pacific Islands David McKenzie World Bank.
IMPACTS OF A WORK-BASED POVERTY REDUCTION PROGRAM ON CHILDREN’S ACHIEVEMENT AND SOCIAL BEHAVIOR: THE NEW HOPE PROJECT Aletha C. Huston, Greg J. Duncan,
The Scientific Method.  Theory  Hypothesis  Research  Support the theory OR Refute/Fail.
Effects of Income Imputation on Traditional Poverty Estimates The views expressed here are the authors and do not represent the official positions.
Europe and Central Asia Region, The World Bank The Global Economic Crisis, Migration, and Remittance Flows to Armenia: Implications for Poverty International.
Workshop: Impact of Poverty and Social Exclusion..., Bratislava, Slovakia, 8-9 Sept 2008 The health of children in Norwegian low-income families Jon Ivar.
©The Work Foundation Stephen Bevan Director, Centre for Workforce Effectiveness The Work Foundation & Honorary Professor Lancaster University The Clinical.
Mother’s, Household, and Community U.S. Migration Experience and Infant Mortality in Mexico Erin R. Hamilton, Andres Villarreal, and Robert A. Hummer Department.
 Health insurance is a significant part of the Vietnamese health care system.  The percentage of people who had health insurance in 2007 was 49% and.
Measuring Equality of Opportunity in Latin America: a new agenda Washington DC January, 2009 Jaime Saavedra Poverty Reduction and Gender Group Latin America.
Land Rental Markets in the Process of Structural Transformation: Productivity and Equity Impacts in China Songqing Jin and Klaus Deininger World Bank.
Designing a Random Assignment Social Experiment In the U.K.; The Employment Retention and Advancement Demonstration (ERA)
Moving to Opportunity in Boston: early results of a randomized mobility experiment Lawrence F. Katz; Jeffrey R. Kling & Jeffrey B. Liebman Presented by.
Randomized Controlled Trials in Rural Finance: An Example from India Michael Faye and Sendhil Mullainathan Harvard University March 2007
Welfare Reform and Lone Parents Employment in the UK Paul Gregg and Susan Harkness.
1 Chronic Absence in the Early Grades: Presentation to NNIP An Applied Research Project funded by the Annie E. Casey Foundation (October 2008)
Household food insecurity among low-income Toronto families: Implications for social policy Sharon Kirkpatrick & Valerie Tarasuk Department of Nutritional.
The What and the Why of Statistics The Research Process Asking a Research Question The Role of Theory Formulating the Hypotheses –Independent & Dependent.
Financial (Dis-)Information and Disclosure: Experimental Evidence from Mexico A Xavier Giné World Bank G20 Meeting, Moscow Financial (Dis-)Information.
Social causes and solutions for mental health: towards equity in recovery Dr Kwame McKenzie Professor of Psychiatry U of T Senior Scientist CAMH.
LABOUR FORCE PARTICIPATION, EARNINGS AND INEQUALITY IN NIGERIA
HAOMING LIU JINLI ZENG KENAN ERTUNC GENETIC ABILITY AND INTERGENERATIONAL EARNINGS MOBILITY 1.
CLOSING THE GAPS – REDUCING INEQUALITIES IN OUTCOMES FOR CHILDREN AND YOUNG PEOPLE BIRMINGHAM ACHIEVEMENT GROUP SEMINAR DECEMBER 2008 JOHN HILL RESEARCH.
How Much Choice Do Seniors Want?: Survey Results on the Medicare Prescription Drug Benefit Janet Cummings Link* Thomas Rice* Yaniv Hanoch** *Department.
Today’s Schedule – 10/30 Ch. 11 & 12.2 Quiz Finish Daily Show Clip
Better Decision-Making with RCTs: A Guide for Financial Service Providers.
Randomized Evaluations: Applications Kenny Ajayi September 22, 2008 Global Poverty and Impact Evaluation.
Remittances and Human Capital Investment: Evidence from Albania Ermira Hoxha Kalaj December 2010.
J.CuiDevelopment Workshop1 Immigrate to NZ Samoan Quota Migration Lottery.
Restaurant Smoking Policies and Reported Exposure to ETS The case of Massachusetts Tandiwe Njobe National Conference on Tobacco or Health November 2002.
What is Impact Evaluation … and How Do We Use It? Deon Filmer Development Research Group, The World Bank Evidence-Based Decision-Making in Education Workshop.
A land of milk and honey with streets paved with gold: Do emigrants have over-optimistic expectations about incomes abroad? David McKenzie, World Bank.
Settling in: OECD Indicators of Immigrant Integration Jean-Christophe Dumont International Migration Division Directorate for Employment, Labour and Social.
Non-Housing Outcomes of Accessible and Affordable Housing
20th EBES Conference – Vienna
EXPERIMENTAL RESEARCH
Presentation transcript:

Why causal answers are needed and how we can get them Dr Steven Stillman Senior Fellow, Motu Adjunct Professor of Economics, Waikato NIDEA Theme Leader, New Zealands individuals, families and households Launch Symposium, November 24 th 2010 The Importance of Experimental Evidence for Social Science Research

Policymakers need to know the relative effectiveness of particular policy changes or interventions. – Using standard statistical methods, it is straightforward to examine whether there is an association between, for example, an education policy reform and student outcomes. – However, in this case, distinguishing causation from association requires strong assumptions about the underlying model of student achievement and the optimising behaviour of students, parents, teachers, and schools depending on the policy. – Without the ability to observe what would have happened to students both if the reform had and had not occurred, it is not possible to estimate the causal impact of the policy change without relying on untestable assumptions. Why are causal answers needed?

Perceived wisdom is not always correct – The Moving to Opportunity program randomly gave housing vouchers to families living in high poverty public housing projects in five US cities that encouraged them to move to wealthier neighbourhoods. This had no impact on adult economic self-sufficiency or labour market outcomes, although mental health did improve (Kling et al. 2007). – Edin et al. (2003) found that, while refugees who settled in ethnic enclaves in Sweden had worse labour market outcomes, once sorting was controlled for using a policy experiment which randomly placed individuals, living in an enclave improved labour market outcomes for less skilled immigrants. – Stillman et al. (2009) found that migration improved the mental health of Tongan migrants to New Zealand who came via randomly allocated places in the Pacific Access Category, especially for migrants in poor mental health. Why are causal answers needed?

Selection is complicated – The Pacific Island – New Zealand Migrant Survey (PINZMS) collected information on Tongan migrants to New Zealand who came via randomly allocated places in the Pacific Access Category, unsuccessful applicants to this ballot and Tongans who have not applied to the migration ballot – Comparing the characteristics of unsuccessful applicants and non-applicants allows us to examine migrant self-selection – Consistent with migration theory, we find that migrants are more educated and also have higher income relative to their education – However, migrants have lower mental health, which may explain why migration is often claimed to cause mental health problems – No evidence is found of a healthy immigrant effect when looking at physical health. Hence, any differences in physical health between migrants and non-migrants reflect the causal impact of migration on health. Why are causal answers needed?

This is where experimental design comes in. – Continuing with my initial example, by randomly assigning students, teachers or schools to two groups, one which is impacted by the reform and one which continues life under the status quo, it is now possible to estimate the impact of the reform on student outcomes without any assumptions about the underlying behavioural models. – Randomisation ensures that any observed differences in outcomes cannot be caused by other differences between the students, teachers and schools being impact by the reform and those not being impacted by the reform How can we get causal answers?

Well designed experiments can gain new insights – Kremer and Chen (2001) ran an experiment in Kenya to discourage teacher absences. Each pre-school headmaster was entrusted with monitoring the presence of the teacher and a prize was to be given to teachers with a good attendance record. The experiment seemed to indicate that having this incentive discouraged absences, but when the research team independently verified absence through unannounced visits, they found the entire effect was due to the headmasters cheating. – Giné et al. (2009) designed and tested a voluntary commitment product to help smokers quit smoking. The product offered smokers a savings account in which they deposited funds for six months, after which they took a urine test. If they passed, their money was returned; otherwise, their money was forfeited to charity. Eleven percent of smokers in the sample took up the offer which could lead to forfeiting 20% of monthly income. Smoking rate declined by percentage points for this group and still persisted in surprise tests 12 months later (see Other advantages of social experiments

Well designed experiments can test important theories – Ashraf et al. (2006) designed a commitment savings product for a bank in the Philippines. Individuals could restrict access to the funds they deposited until a given amount was achieved. This was randomly assigned and used to test whether clients had time-inconsistency in preferences. – Karlan and Zinman (2005) tested the relative importance of repayment burden versus adverse selection in causing default among high-risk borrowers. They did this by first randomly offering potential borrowers with the same observable risk a high or a low interest rate and then randomly giving a new lower offer to some who agreed to borrow at the high rate. – Bertrand et al. (2005) use the same setup to test a broader set of hypotheses, most of them coming directly from psychology. Now, the offer letters were made to vary along other dimensions that should not matter economically. For example, the lender varied the description of the offer, compared the offered interest rate to a market benchmark, added in a promotional giveaway, and manipulated the race and gender features of the lender introduced via the inclusion of a photo in the corner of the letter. Other advantages of social experiments

Thank you