1. The Nuts and Bolts of Enrollment Design, Implementation and Evaluation Jodi A. Ray, MA PI/Project Director Florida Covering Kids & Families USF Health The Lawton & Rhea Chiles Center Families USA Health Action 2015

2. Goal & Objectives of Evaluation GOAL: to assess the effectiveness of the activities through a meaningful, targeted evaluation of strategies and a summative assessment of overall project success. Understand why is it necessary to establish measurable goals Develop strategies and tactics designed to achieve those goals Determine if you are reaching your goals, and which strategies are successful in helping you meet them

3. Proper evaluation allows you to: Learn from your mistakes; Expand your successes; and Continuously improve your efforts Communicate success in way that stakeholders can understand and see 3 Why Evaluate My Efforts? Evaluation removes the guesswork and creates a system where evidence guides your outreach & enrollment strategies.

4. Program Evaluation Steps Standards: Utility Feasibility Propriety Accuracy Source: Monitoring PPOR Efforts. CityMatch. 2005 Engage Stakeholder s Describe the Program Focus the Evaluation Design Gather Credible Evidence Justify Conclusions Ensure Use and Share Lessons Learned

5. Objectives  Understand expectations for data collecting and reporting  Increase data accuracy and decrease human error from data reporting  Compile and analyze information collected for Events & Outreach  Compile and analyze information collected for Navigator Enrollment Assistance  Develop data management and internal reporting process

6. 12 Regional Navigator projects, covering 67 counties Each comprised of an existing local formal consortium of partners providing enrollment assistance within designated counties Includes over 150 trained Navigators Utilizes existing informal partnership networks within each regional area to create referral sites to serve targeted populations

7. Program evaluation first steps Each Consortium partner developed a strategic workplan utilizing a template provided by USF. Each navigator is required to input enrollment and outreach data daily to ensure information is always current All Consortium Leads completed data reporting training developed by the USF Evaluation Team Data is submitted via a HIPPA compliant, secure online data form USF completed and adheres to all IRB requirements for collecting consumer data All results compiled are reported to CCIIO as required Conduct monthly TA calls with all Consortium Leads to ensure that outcomes are reviewed and they have up-to-date information for proper program management Site visits are conducted to provided one-on-one TA to each region, providing additional qualitative data supplementing information already collected via online reporting. Quarterly/monthly reports on workplan progress are submitted and analyzed to determine ongoing program plan compliance.

8. Events & Outreach Google docs Entering data related to events, outreach, advertising and marketing

9. Under type of activity, select only 1. This selection will determine the next questions to appear.

10. Pay close attention to the format of the response for each item! 10/31/2014 20 Phone Calls

11. The following questions are intended to capture information about Enrollment Event and Outreach Efforts designed to inform consumers.about signing up tor health care coverage through the Federal HealthInsurance Marketplace. Information entered into this survey should be tor Project Performance Period September 16,2014 through September 15,2015.Any outreach or enrollment event information prior to this period should be entered in the Year 1linK. Please select your Consortium Partner Lead. Consortium Partner Lead Please indicate the type of Outreach/Enrollment activity tor which you are entering information. O Daily Phone Calls I 0 Education and/or Enrollment Event I O Additional Outreach/Education Efforts (Example:Flyers,Mail-outs,Newsletters,Email Blasts,Webinar, Presentation) O Advertising and Marketing Activities (Example:Newspaper/Print Media,Radio,TV,Social Media) >>

12. Get Covered Enrollment Event with USF College of Public Health 10/31/2014 Partnered with USF College of Public Health to provide education and distribute information to students about the upcoming open enrollment period. Events

13. 3 200 170 75

16. How didadvertise this event? ;zi Flyers ;zi Facebook O Twitter 0 Linkedln O Other forms of socialnetworking ;zi Organizationalwebsite O Mailout 0 Newsletter ::£1 Emailblast O Promo Board O Other (please describe): <<>>

17. The following questions are intended to capture information about Enrollment Event and Outreach Efforts designed to inform consumers.about signing up tor health care coverage through the Federal HealthInsurance Marketplace. Information entered into this survey should be tor Project Performance Period September 16,2014 through September 15,2015.Any outreach or enrollment event information prior to this period should be entered in the Year 1linK. Please select your Consortium Partner Lead. Consortium Partner Lead Please indicate the type of Outreach/Enrollment activity tor which you are entering information. O Daily Phone Calls O Education and/or Enrollment Event O Additional Outreach/Education Efforts (Example:Flyers,Mail-outs,Newsletters,Email Blasts,Webinar,Presentation) O Advertising and Marketing Activities (Example:Newspaper/Print Media,Radio,TV,Social Media) >>

18. Additional Outreach/Education Select the month of the outreach effort Indicate the type of effort and the estimated reach or date

19. Some selections have prompts for additional information. Be as detailed as possible as this information is used in both internal and external reporting!

20. The following questions are intended to capture information about Enrollment Event and Outreach Efforts designed to inform consumers.about signing up tor health care coverage through the Federal HealthInsurance Marketplace. Information entered into this survey should be tor Project Performance Period September 16,2014 through September 15,2015.Any outreach or enrollment event information prior to this period should be entered in the Year 1linK. Please select your Consortium Partner Lead. Consortium Partner Lead Please indicate the type of Outreach/Enrollment activity tor which you are entering information. O Daily Phone Calls O Education and/or Enrollment Event O Additional Outreach/Education Efforts (Example:Flyers,Mail-outs,Newsletters,Email Blasts,Webinar, Presentation) O Advertising and Marketing Activities (Example:Newspaper/Print Media,Radio,TV,Social Media) >>

21. Select the month of the advertising or marketing activity Indicate the type of effort and the estimated reach Advertising and Marketing

22. When prompted for additional information, be as detailed as possible as this information is used in both internal and external reporting!

23. Verification & eSignature Must complete this section for data to count. Click the Affirmation Box Type your full name

24. Navigator Enrollment Appointment Survey Qualtrics Entering information related to Navigator appointments

25. Enter your Navigator ID in this format Enter your Partner Name Select only one: New/Existing lead to same set of questions SHOP leads to different questions

26. Enter zip code of the consumer being assisted Enter size of entire household (all members) Enter only the number of consumers in household who are being assisted Enter the primary language of the consumer. Only check the box above if the language is other than English.

27. This should match the number in the household who are receiving Navigator services

28. Reminder : This should match the number in the household who are receiving Navigator services

29. Reminder : Check all that apply! Assistance provided during appointment

30. Outcome of appointment Reminder : Check all that apply! If there is a text box below an item, be sure to enter a number!

31. Enrollment information

32. Reminder : Consider these questions for the outcome of this individual enrollment appointment, not previous appointments.

33. Please be as specific as possible when entering the name of the health care plan and medal tier selected. ONLY record plan if enrolled in this visit.

34. When entering additional notes: Do not type IN ALL CAPS Be as detailed as possible Okay to type in English or Spanish Share positive stories as well as challenges and barriers

35. Now that you’ve met your contractual obligations how do you critique the effectiveness of your project? 35 Looking Beyond the Contract

36. Heat Map Data is transferred from Qualtrics

42. Jodi Ray Project Director (813) 974-3143 jray@health.usf.edu