Diving Deeper into Survey Research in Experiential Education Survey Savvy: Diving Deeper into Survey Research in Experiential Education January 25, 2018 Webinar Moderated by Teresa DeLellis

Presenters Kimberly Plake, PhD Purdue University Patti Darbishire, PharmD

Objectives Identify rigorous survey design and validation processes that will improve the quality of survey research in experiential education. Provide examples of successful survey design components and processes related to experiential education.

Stressed Out? A Lesson in Survey Development Kimberly Plake, PhD, FAPhA Associate Professor of Pharmacy Practice Purdue College of Pharmacy

Survey Development Project Development Research Question Survey Selection Item Development

Project Development Explore idea Literature Colleagues Preliminary research Individual interviews Focus groups

Project Development Develop research question(s) and measurable study objective(s) Choose the appropriate “verb” Decide on approach: Qualitative? Quantitative? Mixed Methods? Develop and link survey items to study objective(s) Identify confounding variables

The Idea Increase in the number of students coming to my office reporting high levels of stress and poor mental outcomes. Why?

Literature: Dive Deep Is stress a concern at other institutions? How is stress conceptualized? How is stress measured? Level of stress Triggers or stressors How does stress influence outcomes? Has a similar project been done? Pharmacy? Other professions?

Research Questions What are students stress levels? What are students’ stressors? Is stress impacting students’ mental health? What are factors that influence students’ experience of stress? What coping strategies do students use to address their stress?

Identify a Theory

Do I know enough to create an instrument?

Measurement Construct Measurement Primary Stressors Validated items/scales for stressors? Secondary Stressors Psychosocial Resources Validated items/scales for how students cope with stress or the resources they use? Mental Health Outcomes Validated scales for Stress, Depression, Anxiety, Well-Being? Demographics Typical items or more? Confounding Variables What other factors influence outcomes that are not already being measures?

Measurement: Perceived Stress Scale https://das.nh.gov/wellness/Docs/Percieved%20Stress%20Scale.pdf

Do I know enough to create an instrument?

What’s next? APPROACH WHO? Individual interviews Focus groups Students? Faculty? Advisors?

Focus Group Design What are the predominant sources of stress? How do pharmacy students deal with the stress that results from the circumstances discussed? What are the primary effects of stress on pharmacy students?

Focus Group Results Develop items based on focus group results

Measurement Construct Measurement Primary Stressors Developed 107 items Secondary Stressors Psychosocial Resources Social Support Developed 24 items Coping Strategies Developed 33 items Mental Health Outcomes Validated Scales Confounding Variables Work, Finances, Academic status, Extracurricular organizations, etc.

Pilot Test: Stressors

Next Steps: Refine the Survey

References Cohen S, Kamarck T, & Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24:385-96. Fairman KA. Going to the source: A guide to using surveys in health care research. J Managed Care. 1999;5(2):150-59. Pearlin LI. The sociological study of stress. J Health Soc Behav. 1989;30(3):241-256. Pearlin LI & Bierman A. (2013). Current issues and future directions in research into the stress process. In Handbook of the sociology of mental health (pp. 325-340). Springer Netherlands. Robotham D. Stress among higher education students; towards a research agenda. High Educ. 2008;56:735-46.

Survey Research in Experiential Education Patti Darbishire, PharmD Associate Professor, Pharmacy Practice Director, Introductory Pharmacy Practice Experiences Purdue College of Pharmacy

Survey Research in Experiential Education Programs Develop preceptor training initiatives Obtain basic information from/about your constituents Administer assignments, activities, or projects Evaluate performance Assess the quality or rigor of your program Determine site metrics, e.g. capacity Assess constituents’ perceptions Many more!

Surveying Preceptors to Maximize Effectiveness ACPE ensures all EE administrators care about preceptor training and development through the Standards and Guidelines. How do we best determine our preceptors’ needs for development and training? Ask them! - But how do you best get this information while respecting their time and energy? We are all bombarded by survey research in our inbox.

Surveying Preceptors to Maximize Effectiveness Consider: Population or Sample? Suggestion: Survey carefully selected, small samples of your preceptor population about issues pertinent in their niche. Want more information? Feed your sample findings back to your preceptor population through the network you use to disseminate information to preceptors or through a discussion board. Wish list, logistical barriers, professional challenges

Surveys in the Placement Process The “Monster” we grew: 3-step, annual process to place ~450 students We asked students to: Step 1: Provide/update basic personal and professional information in their online portfolio Step 2: Complete an electronic PDF questionnaire with additional information and email back to us Step 3: Conduct a 15-minute one-on-one interview to ensure we had the full picture (took 3-4 weeks)

Survey to Collect Info through our Program Management System Present: Survey students through our PMS linked to the students’ portfolio system One data spreadsheet vs. PDFs in student files No longer consider personal circumstances as part of this process, but allow: switches selection from remaining placements Result: Freed 1 FTE for 3-4 weeks

Surveys as Student Assignments Variable nature of experiential education Charged with ensuring that students achieve the same outcomes Tasked with capturing information on an individual student level, as well as from a bird’s eye view Scholarship requirements?

Designing Assignments like Surveys Students purchase a workbook and complete active learning assignments. Preceptor signs-off on each assignment upon completion. The student turns in the workbook for review. Every workbook assignment has an associated student self-assessment and requires a preceptor’s signature upon completion. Preceptors evaluate student performance specific to each assignment at the end of the rotation. Program Management System

Designing Assignments like Surveys Use electronic questionnaires created in our EE Program Management System Responses readily retrievable at an individual student level and assignment level No matter how much variability exists between rotations, we can demonstrate outcomes were met Data as basis for scholarship and student research projects Program Management System

Surveys for Quality Assurance Survey students, preceptors, and other constituents on their opinions and perceptions of: Course/rotation content (e.g., assignments, projects, activities) Program structure and logistics (e.g., placement process, link to didactic content) Effectiveness of the experiential administrators and staff (e.g., effective communicators, responsive to needs)

Questions? Kimberly Plake, PhD kplake@purdue.edu Patricia Darbishire, PharmD darbishi@purdue.edu