1. Learning and Leading for Large Scale Change Scaling Up: Lessons for Fundações Dr. Richard Kohl, Leading and Learning for Large Scale Change Sao Paolo, Brazil 28 September 2012

2. Learning and Leading for Large Scale Change Summary of the Presentation 1. Successful large scale implementation as important as new efficacious interventions 2. Successful scaling up usually not spontaneous 3. Chances for successful scaling up can be increased by pro-active strategic management 4. Requires having  a clear scaling up strategy identifying large scale delivery system and funding  understanding intervention does not exist apart from the political, social context and delivery system  Explicitly managing political process (advocacy) and implementation (system strengthening and governance ) 2

3. Learning and Leading for Large Scale Change Scaling up historically taken for granted: “Build it and they will come” Key is the intervention itself, emphasis on technical content vs. process and delivery Key is the intervention itself, emphasis on technical content vs. process and delivery Proof of effectiveness is sufficient Proof of effectiveness is sufficient Scaling up assumed to be spontaneous Scaling up assumed to be spontaneous Activities limited to Dissemination, Research Utilization, or RIP; peer-reviewed journals, conferences, dissemination workshops Activities limited to Dissemination, Research Utilization, or RIP; peer-reviewed journals, conferences, dissemination workshops slide 3

4. Learning and Leading for Large Scale Change Spontaneous Scaling Up based on information alone is empirically false Ignores ALL of new neuroscience and behavioral economics research on how decision making happens Ignores ALL of new neuroscience and behavioral economics research on how decision making happens Reinforced by foundation attitudes/incentives Reinforced by foundation attitudes/incentives – Limited resources – Institutional incentives to be innovative, leveraging, and catalytic (the reactant is?) – Strong ties (staffing) from universities; research mentality – Academic culture of publishing, “theory of change” Result is neither no scaling, policy adoption only, or scaling (quantity) without quality and impact slide 4

5. Learning and Leading for Large Scale Change 5

6. 6

7. New Approach to Scaling Up Recognizes the greater importance of politics and implementation/delivery systems capabilities and capacities i.e. & org. culture (5/80%, vs. 10/10%) Develops pro-active strategies for each issue; strategic management of scaling up process New innovations in delivery mechanisms, systems, quality; Supply, Demand and Delivery Technical support for either – growth (expansion) to create new delivery mechanisms – System strengthening of existing delivery mechanisms – Monitoring, learning and feedback of delivery at scale 7

8. Learning and Leading for Large Scale Change Two Key Concepts Capacity (Reach or Scale) – ability of an organization to delivery and implement the programme at the targeted or desired scale Capability: ability of an organization to delivery and implement the program according to the way it was designed (fidelity), with quality achieving the expected impact Delivery and Implementation Organizations have to have both Capacity and Capability to achieve the scaling up goals Slide 8

9. Learning and Leading for Large Scale Change Definitions of Scaling Up (2) 3. Scaling up Organizational Delivery and Implementation Capacity to achieve greater reach and deliver additional contents a) Expansion: creating capacity from zero, pilot organization gets bigger b) Replication: hand off from one organization to another, usually grafting onto an organization that already has large scale delivery capacity c) Collaboration: partnership between one or more organizations to leverage either reach in specific places or capacity in specific content slide 9

10. Learning and Leading for Large Scale Change Defining Successful Scaling Up Policy Adoption: integrated into policy/strategy Reach: the scale to be attained, number of people, locations, demographic, % coverage Cost and Efficiency: meets a budget/financial constraints Impact: impact on key outputs and outcomes Sustainability: political, financial and institutional Equity: reaching the hardest to reach; marginalized, poor indigenous, and rural areas slide 10

11. Learning and Leading for Large Scale Change Six Elements of Scaling Up 1.Intervention: proven or promising model/program 2.Evidence: demonstrating effectiveness, efficient, compatibility, and robustness and relevance 3.Advocacy: convincing institutions to adopt, fund/resource and implement the model at large scale 4.Modifying and Strengthening Organizations: – Creating the capacity and/or capabilities (HR, infrastructure, systems and supply) to implement at scale and – Alignment of rules/procedures, incentives and org. culture 5.Implementation at Scale: – Coordination and cooperation among multiple stakeholders – Monitoring and feedback for fidelity, quality and adaptation 6.Sustainability: political, financial and institutional slide 11

12. Learning and Leading for Large Scale Change The 8 Common Breakdowns in Scaling Up 1.Missing Key Invisible Elements of the Model 2.Lack of Generalizability 3.M&E not Designed for Scaling 4.Lack of Large Scale Delivery Capacity and Capabilities/Compatibility 5.Limited, Narrow Advocacy 6.Training ≠ Capability Building 7.No Provision for Monitoring and Feedback 8.Dropping the Baton: Resources and Responsibility for Going to Scale 12

13. Learning and Leading for Large Scale Change Missing Elements of the Model  Mistake: neglect key invisible elements  Focus only on technical contents; drugs, equipment, procedures  Ignore preparations, preconditions, processes and follow-up  Favor technical elements over tacit and intangibles  Solution:  Identify intangible, tacit elements at the pilot phase  Ensure use of qualitative methods in pilot design, process tracing  Comprehensive documentation of the model 13

14. Learning and Leading for Large Scale Change Pilot Design, OR/M&E not designed for Scaling  Mistake: measure impact/efficacy  Ignores role of social environment and organizational context in pilot phase  For scaling, not what works, but what works where and why (use Realistic Evaluation approach)  Ignores that information for policy makers not the same as information for research  Assumes whole model will go to scale as is  Solution:  Identify and test key social/org. assumptions at the pilot phase through M&E for scaling up   Design into pilot/OR anticipated variations in social environment and organizational context  Measure cost and impact and disaggregate by components  M&E for advocay Scaling Up March 2010 Chalkboard Project 14

15. Learning and Leading for Large Scale Change Lack of large scale delivery system: capacity and capability Mistake: – Large scale delivery system not identified up front and there exists no system that can deliver at scale (lack of CAPACITY) – no attempt to ensure compatibility with large scale systems (lack of CAPABILITY) Solutions: – Clearly identify goals for scale (who/where) and who the large scale implementer will be up front – Identify capabilities needed for the model – Design the model to be compatible with the implementer’s capabilities and capacities – Anticipate needed capabilities and capacities building 15

16. Learning and Leading for Large Scale Change Limited and Narrow Advocacy Mistake: – Believe impact results necessary and sufficient – Ignore political considerations of policy makers – Exclusive focus on policy champions (patrons) Solution: – Identify decision makers, stakeholders, their interests – Develop messaging & evidence to address interests – Build advocacy into pilot design and evaluation – Organize broad-based multi-stakeholder coalition across party lines and beyond usual suspects – Extend advocacy beyond adoption and strategy to programming, funding and implementation 16

17. Learning and Leading for Large Scale Change No Matter What the Problem, Training is the Solution Mistake: Ignore complete set of capabilities needed for successful implementation – aligning incentives – Ensuring adequate financial and esp. human resources, – infrastructure, supply, logistics and systems, – organizational culture; rules, regulations and procedures – Adding programs to systems w/o adding capacity – Training is short-term, not institutionalized Solution: Comprehensive capability building for entire system with dedicated MT resources 17

18. Learning and Leading for Large Scale Change slide 18 Modifying and Strengthening Organizations SYSTEMS INCENTIVES (Internal Politics) RESOURCES & INFRASTRUCTURE POLICIES/REGULATIONS ORG. CULTURE Capacity to Build Capacity Skills

19. Learning and Leading for Large Scale Change No Monitoring of Performance and Fidelity at Scale 19  Mistake: model will be implemented as designed with quality and fidelity  Key components are often dropped, especially the more tacit, intangible or process oriented  Emphasis on quantitative targets and achievements, outputs not outcomes  Tensions between dilution or distortions of the model versus necessary adaptations  Solution  Principles vs. procedures, local adaptation  Build in a monitoring system to ensure fidelity and quality, learning and adaptation

20. Learning and Leading for Large Scale Change Passing or Dropping the Baton: The Missing Link? 1.Developing a Promising Innovation 2.Generating Evidence of Efficacy (Internal Validity) 3.Generating Evidence of Effectiveness and Generalizability (External Validity) 4.Dissemination 5.Adoption into Strategies < Policies < Programs < Funding < Buy-In in District & Front-Line 6.Creating the Pre-Conditions for Large Scale Implementation: Training, Capability Building, Incentives, Org. Culture, Coordination etc. 7.Implementation at Large Scale 8.Ensuring Impact on Key Outcomes (monitoring) 9.Sustainabiility slide 20

21. Learning and Leading for Large Scale Change Summary (1) 1.Build scaling up into discovery, design and pilot testing from the very beginning – Identify processes and intangibles as part of model – Test for generalizability and robustness (social and organizational) – Disaggregated evidence on cost and effectiveness by component, able to simplify – Identify capabilities needed and cost – Identify large scale delivery system with capabilities and cost constraints up front and ensure alignment – Generate evidence for advocacy in pilot design 21

22. Learning and Leading for Large Scale Change Summary (2) 2.Advocacy – beyond dissemination and RIP, champions to broad- based advocacy (coalitions) for adoption, funding and implementation (sustainability) – Anticipate political and institutional opposition 3.Capability building beyond training to aligning entire system (health, education, etc) 4.Coordination and monitoring for fidelity and adaptation to local circumstances 5.Responsibility and resources for going to scale 22