1. Lessons Learned from Creating a Performance Culture in a School Reform Nonprofit Eric Barela, Ph.D. Partners in School Innovation 2011 American Evaluation Association Conference, Anaheim, CA

2. Partners in School Innovation SF Bay Area-based nonprofit founded in 1993 30 employees Annual operating budget of approx $3.5 million PartnersSI supports schools in San Francisco, Oakland, San Jose

3. Mission To enable public schools in high- poverty Bay Area communities— serving students of color and English Learners—to achieve educational equity through school-based reform We do this by providing coaching support to existing teachers and school leaders around effective data use.

4. Dilemma In 2008, PartnersSI was at a crossroads: –Lack of clarity around program –Culture of “private practice” with respect to program implementation –Intermittent tracking of student achievement results –Uneven staff accountability, often perceived to be subjective Solution: –Create Organizational Performance dept. –Work toward incorporating a performance culture

5. Performance Culture “An organization with a performance culture is focused on mission effectiveness (doing well what it says it does) and mission fulfillment (doing it well for as many as it reasonably— maybe even unreasonably—can).” (Morino, 2011)

6. PartnersSI Two-pronged Approach Organizational Performance Department Design and Implementation Performance measurement strategy Management roles and responsibilities Metrics, tools and data system Reporting and communication of results

7. Design and Implementation Performance Measurement Strategy –Evaluate and report on program AND organizational effectiveness –Develop and manage measurement tools/processes –Inform relevant external communities of PartnersSI work –As an org matures, performance measurement processes should evolve into evaluation processes (Forti, 2011; Price, 2011) Management Roles/Responsibilities –Dedicated Organizational Performance dept. –Defined management roles around monitoring and managing individual performance

8. Design and Implementation Metrics/Tools/Data System –Appropriate, relevant, timely data collection (Collecting the right data to measure impact in the right ways at the right time) –Data collection across all org departments Program implementation Student achievement Fundraising capacity Effectiveness of internal business, HR processes Usefulness of Organizational Performance reports

9. Design and Implementation Reporting and Communication of Results –Relevant, timely, consistent reporting –Transparency around formative and summative data However, offering an increased amount of data without systems to support data use won’t improve performance (Price, 2011).

10. PartnersSI Two-pronged Approach Organizational Performance Department Design and Implementation Performance measurement strategy Management roles and responsibilities Metrics, tools and data system Reporting and communication of results Change Management Theory of change Learning from results Culture of learning at all levels Staff engagement and buy-in

11. Consistent, high- quality delivery of PartnersSI approach Increased school capacity in leadership, professional learning, and instruction Breakthrough student achievement What we doWhat school staff do What students do Theory of Change

12. Change Management Learning from Results –Relevant, timely, consistent reporting AND analysis/interpretation assistance Culture of Learning at All Levels –Field staff –Managers –Cabinet –Board of Directors Staff engagement and buy-in –Consistency –Transparency –Relevance –Evidence of learning from results

13. Elements of Performance Culture Mission Measurement (2008) –Internal drivers for measuring performance –Leadership values performance data –Outcomes-driven strategy (“whatever it takes to achieve our objectives…”) –Continuous improvement via reflection and adjustment –20% of nonprofits operate this way, PartnersSI is striving to be in that 20%!

14. For copies of my slides or for additional information: ebarela@partnersinschools.org

15. Great leaders = Great schools Knowledge Management Systems that Promote Organizational Learning: Lessons from New Leaders Marianna Valdez, Ph.D. Director of Research and Evaluation, New Leaders Presentation to American Evaluation Association November 3, 2011

16. © New Leaders, Inc. All rights reserved. | 16 Students in schools led by New Leader Principals consistently achieve at higher levels than their peers, have higher graduation rates and are making progress in closing the achievement gap. As an organization, New Leaders has over 160 employees across 12 locations in the US. Who we are & What we do New Leaders is a national nonprofit that develops transformational school leaders and designs effective leadership policies and practices for school systems across the country. In our first decade, we trained almost 800 leaders, impacting nearly a quarter million students in high-need schools across the country.

17. © New Leaders, Inc. All rights reserved. | 17 The Need for Organizational Learning Extensive data is collected regularly Staff have a learning orientation We don’t have the right data to answer our critical questions We don’t have structures to discuss our data when we need to Learning happens in pockets Decision-making is unclear Learning is not always acted on BARRIERS THAT HOLD US BACK STRONG FOUNDATION

18. © New Leaders, Inc. All rights reserved. | 18 Benefits of Our Organizational Learning System Data and structures provide opportunities for small, intermediate, and large loop learning & action planning (Argyris, 1991) Meeting time is focused on interpretation and action planning, not only data sharing Opportunities for wide stakeholder engagement Knowledge sharing and transfer through documentation Decisions are made and actions are taken to improve program

19. © New Leaders, Inc. All rights reserved. | 19 Program Improvement Planning System Use day-to-day Feedback for improvement Problem-solve execution issues Report progress, create hypotheses, identify strategies for improvement Longer-term improvements for program design Focus on the end-to- end continuum Strategic Decisions Program Execution Leadership Team Program Review Committees: APP Residency redesign team, Emerging Leaders, Principal’s Institute, Nat’l Admissions E-team approves Strategic Decisions Community of Practice Calls Job-Embedded Learning & Quality Improvement Ongoing Data Collection Check-in Meetings National Program Team Process Check-In Meetings SKD Data Analysis Regional Calls focused on Program Review Senior Leadership Team Quarterly Program Meetings Structures and ProcessesPurposes

20. © New Leaders, Inc. All rights reserved. | 20 In general: New Leaders’ Program Improvement Planning System In progress First round of quarterly meetings being enacted now Important to have clear communication, willingness to clarify/tweak Well-received so far Other questions? Please contact: Marianna Valdez Director of Research and Evaluation mvaldez@newleaders.org 646.792.7865

21. Copyright © 2011 New Teacher Center. All Rights Reserved. Successes and Challenges in Creating a New Impact Initiative at a National Nonprofit American Evaluation Association Annual Conference Srik Gopalakrishnan, Chief Impact and Learning Officer New Teacher Center November 3, 2011

22. Copyright © 2011 New Teacher Center. All Rights Reserved. Impact Initiative at NTC Background Work done in 2010-11, and lessons learned Overall successes, challenges and skills needed

23. Copyright © 2011 New Teacher Center. All Rights Reserved. Background – New Teacher Center New Teacher Center’s mission is to improve student learning by accelerating the effectiveness of new teachers Why this matters… 40 – 50% of new teachers leave the profession in the first few years Comprehensive induction improves retention as well as classroom instructional practice Several studies have shown link between mentoring/coaching and student learning

24. Copyright © 2011 New Teacher Center. All Rights Reserved. Background – Why Impact? COHERENT IMPACT STRATEGY Improve teacher practice and student learning through use of data Tell the NTC story and communicate value to clients and funders Sharpen mission, strategy and portfolio of offerings

25. Copyright © 2011 New Teacher Center. All Rights Reserved. Pockets of data (non- standardized), mostly from sporadic internal studies Standardized and aggregated implementation data; some aggregate impact data Standardized and aggregated implementation AND impact data Until 2010 2010-11 & 2011-12 Beyond 2012 NTC Organizational Development Around Impact Evolution of NTC as a Data-driven Learning Organization

26. Copyright © 2011 New Teacher Center. All Rights Reserved. Work Done in 2010-11 What we did Built a common framework and language around impact Compiled and organized all relevant data that had been gathered to-date Implemented several pilots that advanced our ability to measure impact How we did it “Impact Summit” involving all NTC staff Cross functional team that created and implemented an “impact roadmap” Capacity-building for key program staff and leadership team

27. Copyright © 2011 New Teacher Center. All Rights Reserved. Creating a Common Framework - NTC Impact Spectrum

28. Copyright © 2011 New Teacher Center. All Rights Reserved. Lessons Learned in Year 1 What we learnedWhat we are doing A variety of measures used to assess impact Creating consistent measures across programs Program implementation and context play a major role Increasing # comprehensive programs; refining model and Theory of Action Lots of formative data; hard to aggregate Adding easily aggregatable summative tools to our toolset Programs (and NTC staff) need support on what to measure and how Adding specific consultation to programs on creating and implementing an impact plan

29. Copyright © 2011 New Teacher Center. All Rights Reserved. Lessons Learned - Dynamic Tensions Want the organization to be data-driven Want people to have “healthy skepticism” about data Some things can and should be implemented asap Funders and clients demand data that “proves” model Organizational change takes time and requires nurturing Organization needs data that “improves” model

30. Copyright © 2011 New Teacher Center. All Rights Reserved. Overall Successes and Challenges Challenges New Accountability Paradigm Resources Technology Rigor Precision Org. Learning/KM Key Measures Awareness and Appetite Successes

31. Copyright © 2011 New Teacher Center. All Rights Reserved. Skills Needed in this Role Systems- Building Change Management Problem- Solving

32. Copyright © 2011 New Teacher Center. All Rights Reserved. “Shared belief is more an outcome of a quality process than a precondition” Michael Fullan (2006) “What matters is not finding the perfect indicator, but settling upon a consistent and intelligent method of assessing your output results, and then tracking your trajectory with rigor” Jim Collins (2005) Srik Gopalakrishnan, Chief Impact and Learning Officer New Teacher Center srik@newteachercenter.org

33. Aneheim, CA FOR YOUTH DEVELOPMENT FOR HEALTHY LIVING FOR SOCIAL RESPONSIBILITY The Performance Improvement Checklist 2011 American Evaluation Association Conference Christina Krasov, Ph.D. Vice President, Performance Improvement YMCA Chicago

34. YMCA CHICAGO The Y is the nation’s leading nonprofit committed to strengthening communities in the areas of youth development, healthy living and social responsibility. The Y of Metro Chicago serves children, families and communities in Chicago and the collar counties through a network of 25 member centers, five resident camps, and hundreds of extension sites, and through the power of this association, provides equal opportunities in some of Chicago’s neediest neighborhoods. Every day our impact is felt when someone makes a healthy choice, when a mentor inspires a child, and when a community comes together for the common good. Every Y program is aimed at changing lives for the better. We are best known for swimming, water safety and fitness, but that’s just a part of what we do. Through education initiatives, character development, nutrition, obesity prevention, camping, community support and much more, we help children and families develop the skills and relationships they need to be healthy, confident, connected and secure.

35. INCREASING EXTERNAL DEMAND FROM OUR FUNDERS AND COMMUNITIES WE SERVE TO DEMONSTRATE IMPACT Increasingly, non-profits like the Y are faced with grant reporting requirements on outcome measures. Whereas before grantors would be satisfied to know the basic inputs (i.e. program enrollment, hours spent), today’s tight economy requires that they capture the outputs (i.e. academic improvement) as well.

36. NON-PROFIT MANAGEMENT PUSHING FOR CONTINUOUS LEARNING AND PROGRAM IMPROVEMENT The measure of success is not whether you have a problem to deal with, but whether it’s the same problem you had last year. J.F. Dulles time you talked about it.

37. THE “MUST DO” LIST Leverage technology for capturing and processing data in order to alleviate the human administrative burden Sample behind-the-scenes “Mission Intelligence” data architecture

38. THE “MUST DO” LIST Create easy to ready, timely and actionable reports—know your audience! Sample Y Chicago Dashboard

39. THE “MUST DO” LIST Develop data sharing agreements with third party data sources (i.e. school district, police department) where appropriate Excerpt from Data Sharing Agreement with Chicago Public Schools

40. THE “MUST DO” LIST Implement nationally recognized surveys to capture direct feedback from program participants Excerpt from Search Institute’s Developmental Assets Profile Survey

41. THE “MUST DO” LIST Triangulate outcomes and survey results with program quality assessment through validated observation tools Weikart Center for Youth Program Quality assessment framework

42. THE “MUST DO” LIST Communicate frequently and proactively Year end evaluation reports are a thing of the past

43. THE “MUST DO” LIST Hire both internal dedicated staff and external evaluators That’s more of us!

44. Contact Us Eric Barela, Partners in School Innovation ebarela@partnersinschools.org Marianna Valdez, New Leaders mvaldez@newleaders.org Srik Gopalakrishnan, New Teacher Center srik@newteachercenter.org Christina Krasov, YMCA Chicago ckrasov@ymcachicago.org Pranav Kothari, Mission Measurement pranav@missionmeasurement.com