Chapter 5 From Action to Impact

Opening Discussion Read the book case about Year Up and answer the following questions:  How important was it for Year Up to grow according to a systematic growth plan?  What if the study at the end of Phase One had not shown a positive effect on wages?

Chapter Outline Framework for venture evaluation Approaches to social venture effectiveness Outcome and impact evaluation Monetizing outcome and impact Increasing social venture impact: scaling

Chapter Outline Framework for venture evaluation Approaches to social venture effectiveness Outcome and impact evaluation Monetizing outcome and impact Increasing social venture impact: scaling

Identifying Impact and Org. Effectiveness… Is this an easy task to identify how well missions / goals have been fulfilled?  For profit organization?

Identifying Impact and Org. Effectiveness… Is this an easy task to identify how well missions / goals have been fulfilled?  Individual?

Evaluation of Venture Effectiveness Early attempts at evaluation usually considered only one measure of performance. But overall performance and effectiveness usually depends on a number of factors. Evaluation should be linked to the venture’s theory of change. This will be tied to the venture’s program logic model.

The Logic Model and Evaluation The framework for linking evaluation to the theory of change and logic model

Values-Based Evaluation Models Competing values model: based on management values

Values-Based Evaluation Models The stakeholder satisfaction model can take the values of a variety of stakeholders into account. The model gets more complex as the number and diversity of stakeholders increases. Stakeholders are also likely to have competing values. The organization can profile the nature and strength of stakeholders’ values.

Multi-Attribute Evaluation Models The Balanced Scorecard for nonprofits: mission, via the customer perspective, is key.

Multi-Attribute Evaluation Models Example of Dashboard elements (from Year Up)

Multi-Attribute Evaluation Models Example of another Dashboard

Outcome and Impact Evaluation Entails measuring what happens to recipients after programs have been delivered Becomes more difficult as time goes by due to scope growth, changes in target mkt, etc. Outcomes can include recipient’s knowledge, attitudes, skills, intentions, behavior Impacts are broad, long-term changes that occur as a result of a program Can include changes in a community’s health, income, environment, and so on

Evaluation Research Designs Designs differ in the degree to which causation (the program had the desired effect) can be inferred. Causality requires  Covariation: the behavior changed in tandem with the program.  Proper time order: the program occurred before the behavior change.  Nonspuriousness: behavior change was not caused by any other factor.

Evaluation Research Designs Experiment is the strongest design: has treatment and control groups and before and after tests. Quasi-experiment relaxes some of the requirements of the experimental design. Nonexperiment may need to relax most of the features of experiments; these could include  Before-and-after measures  Panel studies  Post-program comparisons

Evaluation Research Designs Comparison of energy use before and after program implementation : National Action Plan for Energy Efficiency (p. 4-2).

Monetizing Outcome and Impact Assigns monetary value to program outcomes or impacts. Provides a number of advantages:  Comparison of ventures  Combining value of ventures  Calculation of efficiency  Communication with stakeholders Important to note that not all benefits may be monetizable.

Monetizing Outcome and Impact Cost benefit analysis: requires comprehensive measurement of cost and program impacts  Includes primary, secondary, direct, indirect, tangible, intangible  Net benefits to participant stakeholders and society Social return on investment (SROI): calculates blended value, composed of  Enterprise value: revenue from business activities  Social purpose value: monetized value of venture for society

Monetizing Outcome and Impact Examples of problems in monetization… – Reducing poverty in NYC Why is it hard to monetize the above? What might they be able to monetize?

Scaling Social Impact Ventures usually launched in one or a limited number of locations After successful launch of venture and evaluation of effectiveness Reasons to consider expansion, or scaling, of impact:  Chance to accomplish more of the mission  Demands from beneficiaries  Pressure from funders

Scaling Social Impact Caution: poorly planned or unsupported scaling can result in strains in organization or program failure. Need to consider organizational capacity and resources to scale, factors such as  Staffing  Key stakeholders  Earnings  Market

Framework for Scaling Social Impact There are a variety of approaches to scaling.

Framework for Scaling Social Impact Scaling can be indirect or direct. Indirect methods  Are simpler and faster than direct.  Try to elicit others to bring about change.  Could lobby, raise public awareness, organize network. Direct methods  Provide services in other geographical areas.  Require resources and a transferable program model.  Could use promotion, affiliation, or branching.

Discussion Consider the points we have made about social venture effectiveness, and answer the following questions:  Are some models of effectiveness inherently better than others?  What factors might influence the choice of an effectiveness model?  How might this choice influence the type of scaling you might choose?