A Year with the Quality Self-Assessment (QSA) Tool: Creating Continuous Program Quality Improvement

Agenda About the Network for Youth Success The QSA Tool The Quality Improvement Process Key People Implementation Tools to Use A Year in the Process Create a Plan

ABOUT THE NETWORK FOR YOUTH SUCCESS

formerly

Mission To strengthen the capacity and commitment of communities, programs, and professionals to increase access to high-quality programs and services beyond the traditional classroom.

Our Network 12 Regional Networks + NYC networks Partnering Organizations Accredited Programs Credentialed Staff Membership- NAA State Affiliate

Get Connected Sign up for our listservs at http://www.networkforyouthsuccess.org/join-listserv/ Follow @NYSYouthSuccess Visit www.NetworkForYouthSuccess.org “Like” us on Facebook facebook.com/NetworkforYouthSuccess

World’s Worst Youth Worker What do they SAY And What do they DO?

THE QSA TOOL

Program Self-Assessment Why Self-Assessment? Quality matters. High-quality programs create positive outcomes for youth High-quality programs operate smoothly! It helps program providers to understand overall program quality and to identify steps needed for improvement. It is a low-stakes process. It can focus on program needs and priorities while keeping areas outside the focus from slipping.

QSA Tool vs. Evaluation QSA Tool: Formal Evaluation: provides structure for dialogues around outcomes and quality. strengthens communication between stakeholders. helps build a collective vision. uses data gathered to inform practice and build capacity. is often facilitated by program staff and/or other stakeholders. Formal Evaluation: takes a more formal approach to assessing programs to determine what their outcomes are. often uses data gathered for accountability and compliance. has higher stakes and less gray areas. is often facilitated by an external observer.

What is Program Quality What is Program Quality? Ten Essential Elements of a Quality Afterschool Program Environment and Climate Administration and Organization Relationships Staffing and Professional Development Programming and Activities Linkages Between Day and Afterschool Youth Participation and Engagement Parent, Family, and Community Partnerships Program Sustainability and Growth Measuring Outcomes and Evaluation

QSA Tool and User’s Guide The User’s Guide and whole tool are available at www.networkforyouthsuccess.org/qsa/ Note that additional tools and details on the recommended process are available on the website that are not currently included in the User’s Guide.

QSA Tool Structure

Performance Levels 4: Excellent / Exceeds Standards Is prepared to help and work with others in this area. 3: Satisfactory / Meets Standards Needs help to prepare staff to work with others in this area. 2: Some Progress Made / Approaching Standard Could use additional focused assistance in this area. 1: Must Address and Improve / Standard Not Met Needs significant support in this area. Organizations are expected to strive for a Satisfactory level (3) and, over time, continue to strive for an Excellent level (4).

Let’s Take a Closer Look This is where I normally give out a copy of one of the elements and let them look through it.

Programming & Activities Provides activities that reflect the mission of the program What does it mean? What would it look like in an ideal program? What does it look like at your program?

Programming & Activities Provides activities that reflect the mission of the program Activities are selected based on materials, staff members’ interests, and space available. Activities that reflect the mission are prioritized, but some activities that do not reflect the program mission are offered. The program mission serves as the foundation for all activities selected. Activity and lesson plans include explanations of how the activity supports the program mission. The program mission serves as the foundation for all activities developed and selected. Activity and lesson plans include explanations detailing how the activity supports the program mission. Staff members are encouraged to explain the mission to youth participants and offer rationale and the connectedness of the mission to the program activities. Youth have the opportunity to give feedback on the relationship between the program mission and activities regularly throughout the year.

THE QUALITY IMPROVEMENT PROCESS

Cycle of Improvement

KEY PEOPLE

Self-Assessment Leader The self-assessment leader takes charge of the QSA process. The self-assessment leader should take the QSA first. In collaboration with other staff and stakeholders, the self-assessment leader determines: who will be engaged in the process how and when it will take place who will facilitate discussions how data and findings will be compiled and shared and helps to ensure the self-assessment process leads to planning and program improvement This role is often taken by site directors, however many others can also take on this role. This should not be your evaluator. Who will be your self-assessment leader?

The Facilitator This individual facilitates focus group discussions. This can be the self-assessment leader. Some programs choose to engage an external facilitator to lead the process and offer a neutral voice in discussions. A facilitator should engage all stakeholders and create an atmosphere that is safe, comfortable, and empowering. A good facilitator: Provides opportunities for everyone to contribute ideas and suggestions. Actively listens and connects ideas and themes as they emerge. Is prepared to answer questions about policies and procedures. Avoids personalizing what is shared in the process. Remains non–judgmental. Helps bring consensus and closure to discussions. Who will be your facilitator? The self-assessment leader may also facilitate discussions, though this role may be taken by another person as well. Often, site directors or supervisors take the lead in ensuring the organization undertakes the self-assessment process. However, staff, school personnel, volunteers, and parents can also possess the qualities needed to serve as a facilitator. In some cases, programs choose to engage an external facilitator to assist in leading the process and offering a neutral voice in discussions. The process requires a facilitator who possesses several key characteristics in order to be successful in engaging all stakeholders and creating an atmosphere that is safe, comfortable, and empowering.

IMPLEMENTATION

Breadth All 10 elements at once One element each week/month Pros: Good overall view of program quality; Covers all elements faster Cons: Can’t go as in-depth with each element; Competing priorities on action plan One element each week/month Pros: Focus given to each element Cons: Competing priorities on action plans One element until improvements are complete Pros: In-depth treatment of each element; Focused improvements made; Participants become familiar with whole process Cons: Takes longer to complete all elements

Timing Starting the Self-Assessment Consider Beginning of the Year Middle of the Year End of the Year Consider Improvement Process Staff Turnover Other Program Commitments/Priorities

Stakeholders Invite all key stakeholders External evaluator (required) Youth participants Program staff Parents, guardians, and family members School administrators Partner organizations …and anyone else involved with your program! Stakeholders should be thoughtfully included in each discussion Determine who will be involved in the self-assessment process. It is very important to involve a variety of stakeholders when using the QSA Tool, whether you are a new program in the planning stages or a mature program striving for ongoing improvement. Site directors, staff members, program participants, parents, school administrators, partner organizations, funders, and other stakeholders can all be an important part of the process. It is important to consider the aspects of the program that you plan to self-assess when selecting participants. For example, while program participants have strong knowledge of program activities, youth engagement, etc., they may not be appropriate stakeholders to involve in discussions about staffing and professional development or program growth and sustainability.

Surveying Stakeholders Paper QSA tool Available in many languages QSA survey spreadsheets Includes compilation spreadsheet for viewing results Modify a version for youth

Coming to Consensus Focus Group Discussions Facilitated discussions of survey results Come to agreement on where program is Decide what to prioritize We ask that programs come to consensus on their performance level rather than averaging scores. The discussion is one of the most important pieces for building communication and shared vision.

Developing an Action Plan

SMART Goals Specific Measurable Achievable Results-focused Time-bound Option A: To offer more project-based activities that promote creativity. Option B: To promote participant creativity by offering at least two project-based activities per club per semester by January 1st. Element 5, Indicator 4.

Creating Accountability Plan collaboratively with all who will be responsible for completing tasks Break each action plan point down into a SMART goal Assign all tasks to the appropriate person Create specific deadlines/check-in points Assign a leader to oversee improvements and check-ins Report results back to stakeholders

TOOLS TO USE

Access to the QSA Tool and User’s Guide www.networkforyouthsuccess.org/qsa/ Includes links to program quality resources and detailed definitions of the performance levels Download User’s Guide and Tool as PDFs Multiple languages available

Survey & Compilation Spreadsheets http://networkforyouthsuccess.org/qsa-survey-compilation-spreadsheets/

QSA Spreadsheets QSA Survey Site Compilation Spreadsheet Rate the program Site Compilation Spreadsheet Collect responses View ratings with charts by: Element Indicator Type of respondent Not collected by the Network for Youth Success or any state agencies

PASE Professional Development Planning Tool http://networkforyouthsuccess.org/wp-content/uploads/2016/05/QSA_Professional-Development-Planning-Tool-PASE.xls

A YEAR IN THE PROCESS

Handout- sample calendar

CREATE A PLAN

Create a Plan & Be Accountable! This is normally where I give out the planning tool

Questions? Alli Lidie Associate Executive Director [t] 646.943.8671 alli@networkforyouthsuccess.org Timothy Fowler Professional Development Director [t] 518.486.8559 timothy@networkforyouthsuccess.org