Every Hour Counts Measurement Framework Pilot Overview for BASB Stakeholders

Every Hour Counts National coalition of citywide afterschool intermediaries that increase access to quality learning opportunities, particularly for underserved students.

EHC Measurement Framework (MF) A blueprint to help communities better collect and use data to improve system-building, program quality, and youth outcomes Contains outcomes at the system, program, and youth levels Selection of outcomes was informed by guidance and research review from the American Institutes for Research EHC MF is available for download here: http://afterschoolsystems.org/content/document/detail/4060/

EHC Measurement Framework: Levels

EHC Measurement Framework For each outcome, the Framework contains: Description of measurement activities (including frequency, level of burden, age restrictions) Suggestions for how the data can be used How data on a given outcome may be linked to other levels within the Framework Relevant evidence and literature on the value of each outcome

Pilot Study: July 2015 – August 2017 Funded by the Wallace Foundation Three cities selected to test the implementation of the EHC MF: Boston After School & Beyond Providence After School Alliance Sprockets and Youthprise in St. Paul, MN The RAND Corporation is serving as the developmental evaluator for the pilot Goals of pilot for EHC: Learn how city intermediaries are using data to inform improvement Provide MF implementation lessons to the expanded learning field Refine the MF, if needed

RAND’s Goals of Evaluation Support goals of EHC and each pilot city Gain an understanding of MF outcomes pilot cities are and are not collecting at each level What barriers and strengths exist for measuring outcomes Examine strengths and weaknesses of different measures Determine how cities are using the data to inform practices What information is useful and what could be improved?

RAND’s Goals of Evaluation Document the variation in implementation of MF across pilot cities and understand how variation may influence broader adaptation of the MF Examine how systems in MF relate to one another Provide suggested refinements to MF

BASB’s Goals for the Pilot Use the EHC MF to align and systematize our work Rethink, revise, and refine our approach to maximizing the sustainability of our initiatives, including measurement, convening and partnership-building Explore balancing different approaches to scale Increase depth with which BASB measures the quality and impact of current partners Simultaneously engage a larger share of Boston’s learning ecosystem in collective measurement and quality improvement work

BASB’s Goals for the Pilot Adopt new measures and/or tools, first within our Summer Learning Community, and then expand into our school-year initiatives Improve YouthServices.net to streamline data collection, integrate new types of data, enhance the user experience, and build out more reports for various stakeholders Improve and/or create data storage, management, and reporting systems for information that does not lend itself to being stored in YouthServices.net Improve training for program partners around importance of outcomes and using data to inform program design and quality improvement