1. Outcome-Based Management Program Data Collection Requirements

2. - 2 - Data Collection Background The Outcome-Based Management (OBM) program will require improved and consistent data collection requirements to ensure that quality data is entered on a timely basis Why are we updating the data collection requirements for clubs?  The OBM program will require more numerical data to understand the clubs’ impact on our members  Better data collection may streamline work associated with program reporting (i.e., TEEN Reach, monthly reports) and grant writing Which data collection processes have updated requirements?  Club Attendance Tracking  Selected Program Attendance Tracking  Outcome Measurement Toolkit Administration  Selected Survey and Test Administration Who is responsible for meeting these updated requirements?  All staff will be held accountable for meeting these updated requirements through regular service center oversight and the annual review process You may be meeting some of these requirements, but we need to meet them all to be successful!

3. - 3 - Data Collection Requirements The updated data collection requirements encourage consistent data acquisition across clubs Data Collection Process Process DescriptionUsageUpdated Requirements Club Attendance Tracking Tracking member attendance at clubs, including entry and exit times Used with other metrics to see how club attendance impacts members  Every member must check in and out at front desk to participate in club activities  Club staff must scan members into KidTrax on a daily basis  Club staff must ensure that member entry and exit times are properly inputted into KidTrax –Batch scanning only should be used in limited circumstances Selected Program Attendance Tracking Tracking member attendance at selected programs Used with other metrics to see how program attendance impacts members  Every child must check in and out with the Program Instructor to participate in selected programs and activities  Club staff must enter program attendance within two days of program session  Batch scanning is acceptable for data entry Outcome Measurement Toolkit Administration Administering survey tool developed by BGCA on a regular basis Used with club and program attendance to analyze changes in attitudes and behaviors  Toolkit must be completed by the same members 3x per year during time period in October, January, and May  A set number of members must use toolkit over this period  All answers must be inputted into KidTrax Selected Survey and Test Administration Administering pre / post surveys and tests developed by BGCA / clubs on a regular basis Used with program attendance to analyze changes in attitudes, behaviors, and knowledge  Selected pre / post surveys and tests must be administered to all members 2x per program at the beginning and end of program  Surveys / tests must be administered each time the program is run  All answers must be inputted into KidTrax These requirements are important because operations will only review data entered in KidTrax

4. - 4 - Enforcement and Staff Impact To make sure we meet these new requirements, the service center will monitor clubs’ data on a regular basis and take appropriate actions to ensure data is entered accurately and timely Reviewed by service center on an on- going basis Review will ensure data entered within two days of program session Reviewed by service center after toolkit administration period Review will ensure that clubs are capturing appropriate % of members Selected Program Attendance Tracking Outcome Measurement Toolkit Administration Club Attendance Tracking Reviewed by service center on an on- going basis Review will ensure: –Data entered on a daily basis –Clubs inputting proper entry and exit times (i.e., not batching) These requirements are expected to be met by all staff members and will be a part of the performance review process Clubs that are not in compliance will have a letter sent to the Club Director and Board President Staff disciplinary action will be taken for non-compliance in accordance to existing policies and practices Selected Survey and Test Administration Reviewed by service center on a quarterly basis Review will ensure that survey and tests are being administered to all program participants EnforcementStaff Impact Data Collection Process

5. - 5 - Attendance Tracking Process and Details

6. - 6 - Club Attendance Tracking (Stand-Alone Clubs) – Suggested Process Enter / Exit Club Member has ID Card Member does not have ID Card Scan ID card Option 1: Retrieve backup ID card, scan in Scanner Available Scanner Not Available Scanner Available Scanner Not Available Option 2: Manual KidTrax scan Manual KidTrax scan Responsibility: Front Door Volunteer / Assigned Club Member / Staff Member Frequency: Daily Additional Data Entry Time (Staff): N/A

7. - 7 - Club Attendance Tracking (School-Based Clubs) – Suggested Process Enter / Exit Club Member has ID Card Member does not have ID Card Scan ID card Option 1: Retrieve backup ID card, scan in Scanner Available Scanner Not Available Scanner Available Scanner Not Available Option 3: Front door resource enters name and entry / exit time on sign-in sheet Front door resource enters name and entry / exit time on sign-in sheet Front door resource enters data into KidTrax w/in one day Responsibility: Front Door Volunteer / Assigned Club Member / Staff Member Frequency: Daily Additional Data Entry Time (Staff): ~10-20 min per day (depends on # of members that attend) Option 2: Manual KidTrax scan

8. - 8 - Club Attendance Tracking – Additional Details TopicDetails Backup / Manual Scans  If a member forgets his or her card and there is immediate access to Trax Attendance Manager, the front door resource should either: –Scan the member in using a backup ID card –Perform a manual scan (see Manual Scan instruction guide for further details) Sign-in Sheets  If 1) a member forgets his or her card and there is no immediate access to Trax Attendance Manager, 2) a club does not have an ID scanner, or 3) a large number of members enter the club at one time, the front door resource must record the member’s name and entry / exit times on a sign-in sheet then input that information into KidTrax within a day  All clubs should continue to use sign-in sheets for backup purposes (i.e. KidTrax down) Data Entry  Batching of sign-in sheet information is not permitted at stand-alone clubs, unless emergency situation –Clubs need to record exact entry and exit times by using ID scan or manual scan  Batching of sign-in sheet information is permitted at school-based clubs for members who remain at the site the entire time the club is open –Members who enter 30 minutes after the club opens or leave 30 minutes before the club closes must have their times recorded on the sign-in sheet and be entered into the system separately  If KidTrax is down, all clubs are allowed to batch scan entry and exit information if they have followed the existing protocol (e.g., notified the general service center) Best Practices  A volunteer, club member, or staff member should be assigned attendance data entry responsibilities –Please identify a person who is trained / familiar w/ KidTrax and able to assist others in entering data  Many clubs have created various binders to have backup scanning information for their members –Clubs can print all member names and barcodes, then keep them at the front desk for scanning –Clubs can use clear business card pages to store all backup ID cards  Need to be diligent about tracking exit times (e.g., one person should be responsible for monitoring when members exit) Expectations and Enforcement  Clubs must enter data within a day of attendance and include entry and exit times  Data from selected days will be reviewed by service center on an on-going basis  This review will look at when and how the data was entered (i.e., were the majority of members were batched)

9. - 9 - Selected Program Attendance Tracking – Suggested Process Enter Program Member puts check next to name on attendance sheet Program Instructor collects sheets and enters data into KidTrax Responsibility: Program Instructor* Frequency: Up to 2 days after program session Additional Data Entry Time (Staff): ~5-15 min per program session Exit Program Program Execution *This responsibility will be included in Program Instructor goals / performance reviews Program Instructor provides pre-populated program attendance sheet to members

10. - 10 - Selected Program Attendance Tracking – Additional Details TopicDetails Attendance Sheets  Attendance sheets (part of the program planning kit) should be used for each program session and include every member’s name who has attended the program and rows for write-in members –These attendance sheets should be used for multiple sessions and stored by the Program Director  Program Instructor is responsible for ensuring all members are accounted for on the attendance sheets  These attendance sheets should be stored for the entire school year Scanners  Handheld scanners will not be available for program attendance tracking at this time Data Entry  The Program Instructor is responsible for data entry within two days of the program session –Directors / Managers are responsible for providing appropriate KidTrax training to Program Instructors  Batching of sign-in sheet information is permitted for attendance information –See Batch Attendance instruction guide for further details Best Practices  Program Instructors should allot an extra 5-15 minutes per program session to input data  Having a dedicated computer terminal makes inputting data quicker and easier  Work w/ the Club Director / Program Director / Club Manager / Membership Clerks to ensure data is batched and entered correctly Expectations and Enforcement  Coordinators must enter data within two days of the program session  Data from selected program dates will be reviewed by service center on an on-going basis  This review will see if the majority of members were entered within the proper timeframe Participating Programs On-Going Programs  SMART Programs  Power Hour  College Bound  Keystone Club  Torch Club Quarterly Programs  SMART Moves  SMART Girls  Street SMART  Passport to Manhood  Money Matters  Career Launch  Youth for Unity  Triple Play  Skill Tech / Digital Suite

11. - 11 - Survey and Test Administration Guidelines

12. - 12 - Outcome Measurement Toolkit – Guidelines Responsibility: Club Director / Program Director / Club Manager / TEEN Reach Coordinator Frequency: 3 times per year (allot ~2 week; must be completed by October 24, January 18, and May 16) Additional Data Entry Time (Staff): ~5 mins per survey when toolkit completed on paper TopicDetails Toolkit Administration  The Outcome Measurement Toolkit is now be available in KidTrax and needs to be completed regularly  In order to measure the impact on our members, clubs are required to test the same members during at least 2 testing periods per year  Members must complete the appropriate survey toolkit for their grade level (i.e., elementary school, middle school, or high school)  Staff should administer the toolkit online for every level except Cadets, who should complete it by paper –Clubs may administer the toolkit by paper for other groups if there are resource constraints or youth are not able to use computers effectively  Toolkits administered on paper must be entered by staff members into KidTrax within the designated time period Timeline  The toolkit must be administered 3 times per year –Clubs may allot extra time to input the toolkits, but must be completed by October 24, January 18, and May 16 –Suggested administration dates: October 13-24, January 7-18, and May 5-16  Scheduling of the computer lab, staff assignments and members is the Director’s / Manager’s responsibility  Regular programming should continue during this time for all other areas TEEN Reach Participants  TEEN Reach participants are required to use the toolkit once per year, but do not need to participate during every administration period –It is recommended that coordinators administer the toolkit during the January session –All participants must use the toolkit found in KidTrax and not the BGCA website  A separate section of the toolkit containing sensitive questions must be administered by TEEN Reach coordinators to program participants –This section will be completely anonymous, but must still be entered into KidTrax

13. - 13 - Outcome Measurement Toolkit – Guidelines, Cont’d TopicDetails Best Practices  One staff member should be in charge of all computer room scheduling and works in conjunction with computer room manager to administer testing  Clubs should provide open programming to members who complete toolkit early  Incentives should be given for member participation (e.g., pizza party, field trip, Power Points) and clubs are also encouraged to sponsor raffle prizes drawn from participants (i.e., items attractive to members and their families)  See Survey Best Practices slide for additional suggestions Expectations and Enforcement  Clubs are expected to survey 70% of their projected ADA goals for that year (i.e., unduplicated members) –See OMTK Goals worksheet for target goals  Service center will review the participation rates after the testing period and report back to the clubs Other Notes  Toolkit administration will be included in the goals and performance reviews of the Club Director / Program Director / Club Manager  Aggregate results will be available in real-time for program improvement and staff discussions –Plan to spend at least two staff meetings to review results of the toolkit Responsibility: Club Director / Program Director / Club Manager / TEEN Reach Coordinator Frequency: 3 times per year (allot ~2 week; must be completed by October 24, January 18, and May 16) Additional Data Entry Time (Staff): ~5 mins per survey when toolkit completed on paper

14. - 14 - Selected Survey and Test Administration – Guidelines TopicDetails Survey / Test Administration  Pre / post surveys and tests for selected programs are available on KidTrax and must be completed by all participating members  Program Instructors may administer tests either electronically or by paper, but they are responsible for putting the information into KidTrax –Surveys and tests must be linked back to an individual member for tracking purposes Timeline  Surveys and tests must be given 2 times per program (during first and last day of instruction)  Allot 1-2 days to complete testing of members and data entry by staff Best Practices  Incentives may be given for member participation (e.g., pizza party, field trip, Power Points)  See Survey Best Practices slide for additional suggestions Expectations and Enforcement  Clubs are expected to survey all program attendees on the first and last day of instruction  Service center will review the participation rates on a quarterly basis Participating Programs  SMART Moves  SMART Girls  Street SMART Other Notes  Survey administration will be included in goals and performance reviews of the Program Instructor  Results will be available in real-time for program improvement and staff discussions  These requirements do not change any guidelines for other programs (e.g., 21 st Century) Responsibility: Program Instructor Frequency: 2 times per program (during first and last day of instruction) Additional Data Entry Time (Staff): ~5 mins per survey when survey or tests completed on paper

15. - 15 - Survey Best Practices There are a number of best practices you can use for administering surveys and tests to members  Prepare all materials prior to the scheduled survey. If using hard copies, keep in mind kids often respond better to surveys printed on colored paper  Leave enough time to set up and arrange the room appropriately prior to the members’ arrival  Introduce yourself to the members as they arrive and give a brief description of your purpose for being there and administering the survey. Sample script: –I am here to ask you to help make our clubs better. The survey your are about to take is being used to get your ideas about our club. It is not a test and there are no right or wrong answers — we want to know what your experiences and opinions are. Take your time and be sure to answer each question based on what you really think. If you don’t understand a question, please raise your hand or come find me and I can help you. The survey should take you only 8 to 10 minutes. (May differ depending on survey length).”  Request that members attempt to answer every question so their opinions and experiences can be included  Let members know you are here to help them if they have any questions  While members are completing the survey, quietly walk around the room (do not hover), answer questions and make sure things are running smoothly  When youth finish the survey, thank them for their time and provide incentives, if appropriate Source: The Colorado Trust, ” Toolkit for Evaluating Positive Youth Development”