Data Collection Training January 17 & 22, 2019 NM Community Survey Data Collection Training January 17 & 22, 2019

Agenda Important Dates for your Calendar Introduction to the New Mexico Community Survey Changes to the survey for 2019 Preparing to collect data Collecting data Review of documents If I knew then what I know now… Sharing stories and experiences

Important Dates for Your Calendar January 17th & 22nd - Data collection training dates February 1st - First draft of data collection protocol due February 4th – 12th – First reviews of data collection protocols Friday, February 15th - Revised data collection protocols due February 24th - Data collection may begin with approved protocol

Important Dates for Your Calendar February 24th - Online survey goes live March 5th & 7th - Data Entry Training will take place April 27th - All data collection must end May 15th - Data files (for paper surveys) are due May 31st - PIRE returns aggregated data files June 30th- Summary NMCS reports will be sent July 15th - All EOY reporting documents submitted to OSAP

Where else can NMCS information be found? The NMCS 2019 Data collection protocol will contain most of this information. All documents you need should be found online: nmprevention.org under ‘Evaluation Instruments’ and ‘New Mexico Community Survey’ ANY questions? Please ask them. Any concerns or problems? Please share them! You all make this survey process successful. We are appreciative of all ‘heads up’.

THE NEW MEXICO COMMUNITY SURVEY (NMCS) Introduction

NMCS – some history Implemented annually since 2008 to evaluate the effectiveness of locally implemented environmental strategies on intervening variables and outcomes The NMCS is conducted with adults only (18 & older).

The Data Collection Protocol seeks to balance multiple needs Seeking “randomness” Representative Sound data collection practices Sustainable Same data collection process each year Culturally appropriate Comparable in space and time Difficult to reach populations Need for community-level sample Low cost Need to consolidate resources Building community capacity

The NM Community Survey Sample: Adults 18 and older living in NM Uses a time and venue-based sampling methodology Goal is to collect comparable samples from year to year (and ideally, reasonably representative of adults in the community) Use your data collection tracking log from last year You are asked to submit last year’s tracking log with this year’s protocol.

Changes in the survey from last year Modules Core (REQUIRED) Tobacco Mental Health Community College Community Opioids Gambling Adverse Childhood Experiences Removed the two ADOR questions at the end Revised binge drinking question to reflect male (5+) and female (4+) binge drinking definitions Included college parties in where alcohol was obtained OSAP Program Manager Guidance (re: expenses)

The NM Community Survey- Core 41 questions (on paper survey) Covers the following topics: Demographics Perception of risk of getting caught Alcohol consumption & access Prescription painkiller use & access Naloxone awareness

The NM Community Survey- Seven Optional Modules Tobacco consumption & access (5 Qs) Mental health & access (8 Q’s) Community Module (9 Q’s) College Community Module (11 Q’s) Opioid Module (10 Q’s) Gambling (4 Q’s)* Adverse Childhood Events (ACEs) (11 Q’s)*

Optional Modules, Con’t. You may choose to add any/all of the additional modules to paper or on-line. NOTE: You may also add additional questions pertaining to your evaluation needs* * Within reason and with approval by the SEOW and with a statement that they are directly from the provider and not OSAP. (See protocol document) In addition, please note, that PIRE will include the extra questions to the on-line survey only if time permits the additional programming.

Community and College Community Modules PIRE recommends implementing the Community or College Community modules approximately every other year to assess community readiness.

Gambling and ACEs Modules These two modules support a grant from the National Center for Responsible Gaming to assess problem gambling and associated substance use behaviors and ACEs in New Mexico. PIRE will provide communities with an additional $500 to implement these two additional modules in your community. ($250 for each module.)

Gambling and ACEs Modules ACEs are extremely sensitive questions. Therefore, if you are collecting these modules using paper surveys, we will ask for additional precautions be taken to ensure the anonymity of the data.

Gambling and ACEs Modules Specifically, if you are implementing the Gambling and ACEs modules using paper questionnaires, we will require you to provide each respondent with a manila envelope for them to seal their responses in after completing the survey. PIRE will supply these envelopes to you. Then use the larger box to place the sealed envelopes in.

On-line Survey To ease the burden of programming, the on-line survey will include the Core, the Gambling and the ACEs modules. Additional modules may be added.

How will the data be used? By programs and communities to assess local needs and track changes over time in key prevention areas By OSAP, to track changes in ATOD use & IVs over time at a state level By BHSD, to assess need and track changes over time By state agencies to improve & increase prevention services to all New Mexicans By the SEOW (State Epi Outcomes Workgroup) to monitor and track local progress By the PPC (Statewide Prevention Policy Consortium), & PDO Advisory Board to encourage alignment of goals among state agencies By the NCRG and the RGANM, to ascertain problem gambling prevalence and correlates of problem gambling

Preparing to Collect Data

Data Collection Recruitment Approach 1: Online NMCS data collection via social media (Facebook & Instagram) recruitment. This is run by PIRE. Data will be returned to your site for inclusion in your final report. We will use the same ads as last year (no major differences by ad were found so we’ll continue to use them all). We will translate all ads in Spanish too this year. We will explore ads for Nextdoor. May do this organically instead. Local programs distribute the survey link on listservs

Some Examples of Ads

Data Collection Recruitment Approach 2: In Person This is a convenience sample, but to the extent possible, you should make it as representative of your community as possible by choosing locations and times that reflect your broad community. In person data recruitment Data collected via: Qualtrics App – Off Line QR Code On-line (going directly to the survey on a laptop or tablet) Paper and Pencil You must provide take home information for everyone taking it in person regardless of how they complete the survey.

Qualtrics App Use Qualtrics App on a tablet or iPad to collect data in same ‘convenience’ locations you use with paper and pencil With app, you can collect data when not connected to internet and upload the data later. (We actually recommend using the app off line all the time and then uploading at the end of the day.) Some users may find this more attractive than paper and pencil (particularly younger respondents) NO DATA ENTRY FOR YOU! And we can more quickly assess progress, gaps. You may need to provide an upfront incentive if not on-line and you will need to set it up for them by entering the site ID for them and other information.

QR Code By placing the QR Code to the survey on a poster where you are collecting data, you may be able to entice those with smart phones to take the survey right then or even later at their convenience. By taking it on their phones, respondents will be eligible for the gift card drawing. Consider having handouts with the survey info that people can take with them for later.

On-line: Laptops & Tablets If you are somewhere you will have a reliable internet connection, we recommend having people log into the survey directly on tablet or iPad, rather than using the Qualtrics app. This way, they are eligible for the gift card drawing. If using a laptop connected to the internet “bookmark” and use the tinyURL link to the survey directly via browser.

Paper & Pencil Paper and Pencil data collection at locations in your community identified in your local data collection protocols. Depending on the community, this may be your most effective data collection strategy and your protocol will help make this strategy as successful as possible.

In person data recruitment considerations If using tablets and/or the QR code, it should accompany your paper data collection. Never rely on just one method. Always have the backup (paper) available. When possible, if using the Qualtrics app, we recommend collecting off-line and uploading all the data later in a safe location. If using a laptop connected to the internet use the URL link to the survey directly via browser. Any theft or loss of any device with the Qualtrics app must be reported to PIRE & Coop Consulting. Rewards/incentives for survey takers

Targeting specific subpopulations Consider how you will reach subpopulations that you may need to work harder to capture in your overall sample. Many of your prevention strategies are targeting 18-25 year olds. How can you recruit this age group (especially those not in school) to insure a strong sample in your county? Are there subpopulations experiencing disparities such as Latino immigrants, elders, LGBTQ, parents, housing unstable, etc. from whom your coalitions wishes to gather data to better understand what’s happening in your community?

Adding additional questions to your survey Additional questions may be added at the end of the on-line or paper survey, with prior approval (and if PIRE has time). Additional questions, paper or app, must first be approved by the SEOW, and you need to have a back up plan for collecting if we are unable to add them at the last minute. On-line - These will be ‘turned on’ based on the county or SITE ID, entered at the beginning of the survey by the respondent. They must be prefaced appropriately, referencing the local prevention provider/grantee so that the new questions are not associated with OSAP or PIRE. You must provide that text. (See protocol document for an example.) Remember how much the additional questions burden the time to take the survey, and choose wisely.

Important Things to Know Data collection can begin as soon as you have your protocol approved, but not before. Every “community” must submit a data collection protocol and have it approved by the SEOW prior to collecting any data. Programs may partner on a single protocol (e.g., university and county programs but make it clear on the protocol that you are collaborating.) Paper survey data entry – start early!!!! Data files should be submitted to PIRE as early as possible so you can receive your complete community set.

Protocol Development

For those who have conducted the survey in previous years Use this year’s data collection protocol template, base it on last year’s data collection log. Consider any differences in dates or locations. Look at findings sheet from 2016, 2017, & 2018: How did you do with community representativeness? Are there special populations your approach hasn’t been reaching? Some new questions on protocol about managing the devices and planning to use the app. Gender, race/ethnicity, age

Planning ahead to collect data It is extremely important that you plan ahead of time where (venue) and when (time) you intend to recruit respondents. Why? You may need to get permission to recruit in many places You need to make sure your data collectors are available on those dates and can go to those locations

Selecting locations and times The locations should be diverse geographically Locations should be frequented by people reflecting your diverse community demographics (to the extent possible) Locations should be representative of your entire community Times should capture a broad sample of people attending or frequenting the location Times should be diverse- not just mornings, not just evenings, not just weekends Random & representative community sample

Completing the local data collection protocol Completion of the data collection protocol will require input from the many persons involved in the process. Completing the data collection protocol is the “planning ahead” part; it is your “blueprint” or “road map” of what you intend to do. The NM SEOW will have to review all local data collection protocols and give approval before you can begin collecting data.

Budgeting for the data collection - from OSAP New Process: Budget work page (pg.18) is now due to OSAP for review and approval Any budget modifications must also be approved by OSAP Consider how and who completes data collection and data entry least expensive options preferable (e.g. volunteers) Actual cash incentives over $1 are no longer allowed, as per OSAP Gift cards/incentives should be limited to the number of surveys expected and cannot be used for other coalition purposes

Who are you sampling? NM residents 18 and over This includes: Spanish & English speaking Parents Elderly Undocumented & non U.S. Citizens All race/ethnicities Males and females- all genders All sexual orientations Over-sample young adults!

Where will you find them? The MVD* The local library The community college, university Local employers Post office Jury pools Sporting events & Hardware stores Wal-Mart, Target, Smiths, coffee shops, restaurants and other local businesses Famers’ Markets PTA meetings Large communities with and MVD office are required to at least try to collect at the local MVD office.

Where you should NOT find them Homeless shelters Treatment facilities, clinics or AA meetings Local programming for high risk individuals Health centers or health fairs Prisons or jails Bars and other drinking venues UNLESS, you can make a very good argument for why it’s an appropriate location for your community.

The Process… Will be unique to your community and program Should play to your community strengths Should be replicable next year with respect to day, time, location, event, etc. Should be as efficient and effective as possible.

Remember… Besides helping you to collect data for monitoring & evaluation purposes, another important goal of completing the Data Collection Protocol is to build local level capacity to collect data. Use this as an opportunity to learn and teach others in the community to collect data and the value of data collection and the value of using data to make decisions. Beg, borrow & steal all good ideas from other communities. Add bullet about recording collection stats Promote your prevention program / coalition

Data Collection Protocol While waiting for feedback on your protocol, start getting site permission(s) while the protocol is being reviewed. Remember that if you make a significant change to your protocol after it has been approved, such as adding a site, you must let Martha and Liz know. Extremely important point: PIRE’s IRB requires that we supervise data collection in this way. If data collection occurs in other ways, it could inadvertently incur a violation & all data collection must stop for the entire state

Preparing for data collection

First things first… REVIEW INFORMATION IN 2019 PROTOCOL DOCUMENT Talk with PIRE folks & your local evaluator if you have any questions & concerns or run into any problems along the way Train all your data collectors really well, particularly if you’re using the app (PIRE can assist you with this.) Kim Zamarin is a great trainer and can assist with this.

What you need to have with you when collecting data & why? Mobile device with app installed (more than one if possible) Hotspot for accessing the internet if possible (having reliable internet access is important). Large box with lid (e.g., paper box or something similar) Any signs promoting the survey and the QR code Recruitment text (English & Spanish) Paper surveys (Printed front & back; English & Spanish) Take-Home Information cards (English & Spanish) Clip boards, pens, pencils, chairs, table, etc. Incentives (if you choose)* Cell phone (just in case) Extraverted, friendly data collectors 18 and older; trained and knowledgeable about the survey & project

Tips Bring a sheet to track surveys – can use the log we provide and type in the info later. Log now contains easy formulas to help you calculate time spent for future reference. Hatch marks for completed surveys (tablets/laptops) Bring name tags with “Hablo Español” for those who do. Fill in site ID (before copying!) and location on top of paper surveys to save time and if helpful.

Collecting Data Using the Paper Survey

How can you “sample” within a sample? You may find that in some locations there are way too many people to approach everyone. In those cases, after someone completes a survey, count 5 new potential respondents and ask the 6th one to participate regardless of who it is. This will help it remain somewhat random. Males are generally less likely to participate in surveys so make sure that you ask males to participate as often (or more often) as you do females.

Data Collectors Data collectors must be over 18. Given the sensitive nature of the survey, you will need to train your data collectors really well for them to feel comfortable responding to any questions and comments appropriately and effectively. Always have 2 or more data collectors at any one location for safety as well as practicality.

Recruitment Text Review this text thoroughly with your data collectors. Practice it. It does not have to be recited word for word, but your data collectors need to know what the survey is about and who will use the data and how, etc. They also need to be able to explain the consent documentation. Tape a ‘cheat sheet’ to back of clipboard. Intro Script – Initial Approach to Community Member   Hi, I’m (e.g., Ruby) from (e.g., the Roosevelt County Prevention Coalition) here in (Portales). We’re conducting a survey to find out what community members know about alcohol and drug prevention efforts and to help identify prevention needs in the community. It takes about 15-20 minutes to complete and as an incentive, we’re giving out ( ). Can you help us today? If NO – “Thanks for your time. If you see us again, please consider taking the survey as it’s very important for us to understand the needs of our community for appropriate programming.” If YES: Do you live here in New Mexico? (We ask that people know the community to participate. If they live outside your county, ask them to think about their NM county as they answer the questions). If NO – “Ok, Thanks for your time” If YES --- “Great. So are you at least 18 years old?” (Ask if person looks younger than 50) If NO ---- “I’m sorry, but you must be at least 18 or older to take the survey. Thanks!” If YES --- “Great. Well, let me tell you a little more about this survey. It asks questions about what’s happening in our community related to drinking, drinking and driving, prescription and other drug use, and mental health services. The survey helps us identify the prevention needs in our community and address those needs. It is an anonymous survey that takes about 15 minutes. We are collecting this survey face to face and via Facebook. Have you taken the survey already this year? If YES ---“That’s great, we’re asking that people take the survey only once. Thanks again so much!” IF NO - Are you still interested in participating? IF Never taken the survey, from the county and 18 OR OLDER AND interested in participating …. Script for explaining the survey and CONSENTING Person “Great! Here’s the survey. (Provide a survey on a tablet or clip board with a pencil or pen.) As I mentioned, the survey is about drug and alcohol use in our community and has some questions about mental health, gambling, childhood trauma and access to services. This first page explains your rights when taking the survey. Most importantly you should know that this survey is completely voluntary, you may skip any questions you don’t want to answer, and your responses are anonymous, so do not put your name anywhere on the survey. If you have questions as you take the survey, please feel free to ask me and I’ll do my best to answer them. Finally, if you find as you start to take the survey, that you’ve already taken it recently, please let me know.” As you answer the questions, please think about the context of your community, (e.g, Las Cruces). Please just provide one answer per question, unless it says you can answer all that apply in which case you can provide more than one response.” Thank you! 1

The Survey- Front Page For your tracking benefit: SITE ID (Don’t know it? Look here: http://www.nmprevention.org/Service-Providers.html) Location (e.g., Walmart, Library, MVD office) Interviewer initials Date (for your tracking form) For the respondent’s benefit Consent information

Debriefing: Take-home information Ask respondent if s/he have any questions that you can answer for them Remind the respondent that they should feel free to contact Martha (or Liz- esp. for Spanish speakers) if they have additional questions or concerns about the survey. If you feel that they could benefit from talking to someone about substance use and/or mental health issues, please encourage them to call the numbers provided in the take-home information. Give them the take-home information WITH LOCAL BH INFORMATION INCLUDED NMCAL Line in place of local BH!!

Tracking Log DOCUMENT ALL SITES WHERE DATA ARE COLLECTED and submit Data Collection tracking log at EOY reporting Consider tracking gender or age or other demographics periodically during data collection to monitor progress Try not to deviate from your approved protocol during data collection

Incentives You don’t have to provide a gift to participants, but highly recommended: Cash works great but you must track every bit of money that is given for auditing purposes.  OSAP new cash limit = No more than $1 cash per participant.  Water, candy bars, healthy snacks. No substance abuse, mental health, prescription drug, or alcohol related gifts, even if it promotes prevention coalition.  Consider tracking different incentives to study what works best. You may not collect any personal or identifying information on respondents If you wish to have larger incentive use non-OSAP resources such as PIRE $$ from ACEs/Gambling module, other donations from community partners e.g., coupons

An Important Word about Stipends & Budgets – from OSAP Maximum stipend $3 gift card this year $5 stipends/gift cards are inappropriate, as per SAMHSA Federal Project Officer, and should be moved away from Ask for in-kind/ donated/coupon incentives Pair with OSAP dollars only if no other options Good ballpark for stipends/incentives is $2.50, less is ideal! Focus on your elevator pitch On why data/ data collection is important Consider other options (e.g. Lottery style, raffle: donated items/cards, store coupons, etc.)

Just A few More things

Data Entry We will provide a training on data entry on March 5th and 7th. If you have a student, staff person, volunteer who will do the data entry, let us know when s/he will be available to attend. Great activity to give experience to students, etc. but must be very detail-oriented and not otherwise distracted; Not required, but newbies should attend to hear about common issues people have with data entry; importance of cleaning data; Don’t wait until the last minute to enter your data. Keeping on top of it as you go will help you know whether your sample is skewed in some way.

Data Analysis – PIRE and your LE Your LE provides PIRE all paper survey data in Excel or SPSS format. PIRE provides: A cleaned data file with all your data combined from the on-line surveys and any that are gathered in other communities but may belong in your community. Syntax for LE’s to analyze the data. A Community Specific Summary Report with results in tables and graphs. NOTE: This is new this year; it is not a complete report, and providers and the LE will need to complete the missing elements in the report (Intro & Summary) You local evaluator will run additional analyses requested. Your local evaluator will also help you to create any graphs or tables you want to share with others in your community. Every OSAP funded site must submit a completed summary report with the EOY reporting by July 15; if you have another form to present/share data with your community (i.e., a Powerpoint presentation) that will also be acceptable to submit.

Do not hesitate… … to ask, call or email, etc. Martha Waller 919-265-2631 mwaller@pire.org Liz Lilliott 575-313-7029 lilliott@pire.org Lei Zhang (data questions) 919-265-2624 lzhang@pire.org Kim Zamarin (Qualtrics and in-person training questions) 505-907-0801 KZamarin@PIRE.org

If I knew then what I know now… Sharing Experiences

Brief Intro to Qualtrics App

Installing the app on your own device The Qualtrics app is already installed in all OSAP iPads iPad screen logon is 000000 You can install the App yourself – connect to wifi Find “Qualtrics” app on your IOS or Android device Download app and to upload survey, when it enters, you will be told to enter a Username: nmcs@pire.org and a password Pire2019! Download one survey (you can toggle between English and Spanish). When survey requires updating, you will be informed to “refresh” Update!

Downloading the app from app store Search for “Qualtrics” in App store

Downloading from App store Downloading from App store. For your own device enter your own Apple ID (not Coop consulting’s)

Once the app is downloaded, then upload NMCS with 2019 information Enter Username and password below nmcs@pire.org Pire2019!

Download the survey- you will need to download the 2019 survey

Use these settings (turn off geolocation) If this notification shows up, press ‘cancel’

Troubleshooting!!!! Please READ OUR EMAILS for updates! Before beginning data collection, go through app one or more times so you are familiar with it. Let us know if you run into any problems, ASAP. Use a reliable internet source to upload trial responses (erratic or irregular internet signal can cause problems). Practice “log out”. If the app gets stuck, upload survey again. You can try downloading the app again. (PIRE and Coop Consulting have the Apple ID if you need it). Always have paper surveys as a backup.

Uploading again Make note of “device identifier” inside app settings (and consider giving it a unique name connected to your program) as PIRE can help you find surveys taken through this device nmcs@pire.org PIRE2019!

Administering the Survey using the App

Click “take survey”

When collecting data in person…. YOU ENTER THE FIRST TWO SCREENS. You will ALWAYS use the second option when collecting data with others in person. The first response option is for those taking survey without you. Show the participant your screen as you complete the first screens and then give her privacy.

SURVEYOR enters the site ID, initials and the abbreviated location SITE IDs are already assigned to OSAP funded communities. If you do not have a site ID, please ask. Site ID serves as your password so don’t lose it. Location and initials are for your site’s tracking. Note the English/Spanish toggle at top – on ALL pages of app now.

Hand the survey to participant This option leads you to more consent information

Additional consent information.

Additional consent information.

Qualifying questions These two are qualifying questions. Answer ‘no’ and you get bumped out.

You may be asked to help in these next two … This is you

You might have questions about this drag and drop question in the “mental health” module. Note: This is before dragging and dropping Changed?

This is after you have dragged all the answers

The page where you connect to the drawing The page where you connect to the drawing. Note the “HERE” at top where you click.

Important to know about the drawing Only available if online – always use up front incentive in case online is messed up for some reason. If issues with getting to incentive page, call Liz or Martha ASAP The “here” button takes you to a separate page so there is no connection between your responses and your information for the drawing. Only eligible to win during the week you completed the survey. Duplicate entries are not allowed. (We remove duplicate entries from later drawings.)

To participate in the drawing….

Ask participant to close the drawing entry screen herself If you see ‘restart’ button at bottom of last page, do not worry, responses will not be deleted. For greater security and privacy, ask participant to navigate the app, unless help is requested. Nobody can view previous pages once you complete the survey. No going back!

Upload the responses at the end of the day or when done. NOTE QUIRK IN APP! ”Uploaded responses” may not correspond to the real number of surveys!

Log Out After Uploading Surveys IMPORTANT!!!! You must make sure you log out after you upload the surveys. If you don’t log out, then you just end up loading the same surveys over and over again.

Log Out After Uploading Surveys IMPORTANT!!!! You must make sure you log out after you upload the surveys. If you don’t log out, then you just end up loading the same surveys over and over again. Don’t panic! This warning sign is okay. If you have uploaded all the surveys, go ahead and delete the responses.

Do not rely upon app’s upload screen to track surveys. Call or email Martha or Liz if you are concerned that the surveys have not uploaded. We can look it up with the name of your device. If you press ‘upload’ more than once, do not worry that the app is uploading repeated surveys. Even if you get an error message and have to download app again, un-uploaded surveys can be found and uploaded again. Always press upload to upload surveys when connected to reliable internet source. Always track the number of surveys you collect using the App – do not count on the app to track for you. Use the data collection log or other method when in the community doing data collection.

A few more things: Keep the 2-page “app form” with you when using the app. Do not write your site ID on the form or tape it to your iPad in case the iPad is stolen it has important information on it. You cannot view the responses once they are completed. Upon completion, hand participant the up-front incentive along with the Take Home Information Even if s/he hasn’t completed the survey you can give them the incentive Consider taping the take home information to your incentive so you don’t forget to give it to them. Track your completed app surveys using your data collection log or other means. When done collecting data, save time to count all surveys, paper and app, and make sure all paper surveys have proper info on them.