Integrating IE into Project Design Felipe Alexander Dunsch June 22, 2015 ieConnect Impact Evaluation Workshop Rio de Janeiro, Brazil June 22-25, 2015

Overview 4 Group Sessions - Project teams work on their impact evaluation, facilitated by TTL & researchers

Overview 4 Group Sessions - Project teams work on their impact evaluation, facilitated by TTL & researchers -Session 1: Establish a results chain & determine the research questions for the project

Overview 4 Group Sessions - Project teams work on their impact evaluation, facilitated by TTL & researchers -Session 1: Establish a results chain & determine the research questions for the project -Session 2: Finalize research questions, identification strategy, outcome indicators, data

Overview 4 Group Sessions - Project teams work on their impact evaluation, facilitated by TTL & researchers -Session 1: Establish a results chain & determine the research questions for the project -Session 2: Finalize research questions, identification strategy, outcome indicators, data -Session 3: Finalize identification strategy, practical implementation plan (budget, staffing, timeline)

Overview 4 Group Sessions - Project teams work on their impact evaluation, facilitated by TTL & researchers -Session 1: Establish a results chain & determine the research questions for the project -Session 2: Finalize research questions, identification strategy, outcome indicators, data -Session 3: Finalize identification strategy, practical implementation plan (budget, staffing, timeline) -Session 4: Finalize design & PowerPoint for Thursday

Workshop output A short PowerPoint presentation on Thursday (4-5 slides maximum) providing a roadmap for the IE: Intervention description Evaluation question(s) Evaluation design Sample and data Timeline To cover, but not to present: Staffing, Budget

Session 1 Understanding the project Causal/results chain Identify research questions

Session 1: Understanding the project What questions: project activities/interventions Why questions: program rationale How: Work down from big picture to specifics to understand project assumptions and constraints > what is the theory of change: How do we expect to move from the intervention to impact?

Session 1: Thinking about the causal chain What is impact of high speed internet access on 1.Business growth? 2.Employment opportunities? 3.Health and education, etc…?

Session 1: Thinking about the causal chain What is impact of high speed internet access on 1.Business growth 2.Employment opportunities 3.Health and education, etc…

Session 1: Thinking about the causal chain High speed internet infrastructure What is impact of high speed internet access on 1.Business growth 2.Employment opportunities 3.Health and education, etc…

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth What is impact of high speed internet access on 1.Business growth 2.Employment opportunities 3.Health and education, etc…

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth Providers connect to the main grid

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers Businesses pay their bills

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers Businesses pay their bills How to use the internet?

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers Businesses pay their bills Reliable service How to use the internet?

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers Businesses pay their bills Reliable service How to use the internet? Cluster effects

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers Businesses pay their bills Reliable service How to use the internet? Cluster effects

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers Businesses pay their bills Reliable service How to use the internet? Cluster effects

Session 1: Thinking about the causal chain High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers Businesses pay their bills Reliable service How to use the internet? Cluster effects

Session 1: Thinking about the causal chain What if supply is intermittent?

Session 1: Thinking about the causal chain What if supply is intermittent? What if people don’t pay their bills?

Session 1: Thinking about the causal chain What if supply is intermittent? What if people don’t pay their bills? What impacts will my project have if nobody connects?

Session 1: Thinking about the causal chain What if supply is intermittent? What if people don’t pay their bills? What impacts will my project have if nobody connects? Why are people not connecting?

Session 1: Thinking about the causal chain What if supply is intermittent? What if people don’t pay their bills? What impacts will my project have if nobody connects? Why are people not connecting? How do we get people to connect?

Session 1: Thinking about the causal chain What if supply is intermittent? What if people don’t pay their bills? What impacts will my project have if nobody connects? Why are people not connecting? How do we get people to connect? Are we interested in proof of concept or program effect?

Session 1: Thinking about the causal chain What if supply is intermittent? What if people don’t pay their bills? What impacts will my project have if nobody connects? Why are people not connecting? How do we get people to connect? Are we interested in proof of concept or program effect? Addressing the last mile issue to maximize impact

Session 1: Thinking about the causal chain What if supply is intermittent? What if people don’t pay their bills? What impacts will my project have if nobody connects? Why are people not connecting? How do we get people to connect? Are we interested in proof of concept or program effect? Addressing the last mile issue to maximize impact

Session 1: Research questions High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers Businesses pay their bills Reliable service How to use the internet? Cluster effects

Session 1: Research questions High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers Businesses pay their bills Reliable service How to use the internet? Cluster effects

Session 1: Research questions High speed internet infrastructure Business growth Providers connect to the main grid Businesses make contracts with providers Businesses pay their bills Reliable service How to use the internet? Cluster effects

Session 2: Identification strategy, outcomes & data -Finalize research questions -Identification strategy -Establish valid counterfactual (control group) -Experimental or quasi-experimental? -Phase-in design? -Discontinuities? (spatial or temporal)

Session 2: Identification strategy, outcomes & data -Outcome indicators -Intermediate outcomes -Final outcomes -Measurable? -Data sources -Surveys -Administrative data -Census data

Session 2: Identification strategy, outcomes & data -Outcome indicators -Intermediate outcomes -Final outcomes -Measurable? -Data sources -Surveys -Administrative data -Census data Research questions need to be submitted after session 2! (by the facilitator)

Session 3: Implementation plan -Timeline -Budget: Sources of funding -Staffing

Session 4: Finalize PPT A short PowerPoint presentation on Thursday (4-5 slides maximum) providing a roadmap for the IE: Intervention description Evaluation question(s) Evaluation design Sample and data Timeline To cover, but not to present: Staffing, Budget Presenter should be a government representative

Have a great week!