1. Diving Deeper into Survey Research in Experiential Education
Survey Savvy:
Diving Deeper into Survey Research in Experiential Education
January 25, 2018 Webinar
Moderated by Teresa DeLellis

2. Presenters Kimberly Plake, PhD Purdue University
Patti Darbishire, PharmD

3. Objectives
Identify rigorous survey design and validation processes that will improve the quality of survey research in experiential education.
Provide examples of successful survey design components and processes related to experiential education.

4. Stressed Out? A Lesson in Survey Development
Kimberly Plake, PhD, FAPhA
Associate Professor of Pharmacy Practice
Purdue College of Pharmacy

5. Survey Development Project Development Research Question
Survey Selection
Item Development

6. Project Development Explore idea Literature Colleagues
Preliminary research
Individual interviews
Focus groups

7. Project Development
Develop research question(s) and measurable study objective(s)
Choose the appropriate “verb”
Decide on approach:
Qualitative? Quantitative? Mixed Methods?
Develop and link survey items to study objective(s)
Identify confounding variables

8. The Idea
Increase in the number of students coming to my office reporting high levels of stress and poor mental outcomes.
Why?

9. Literature: Dive Deep Is stress a concern at other institutions?
How is stress conceptualized?
How is stress measured?
Level of stress
Triggers or stressors
How does stress influence outcomes?
Has a similar project been done?
Pharmacy? Other professions?

10. Research Questions What are students stress levels?
What are students’ stressors?
Is stress impacting students’ mental health?
What are factors that influence students’ experience of stress?
What coping strategies do students use to address their stress?

11. Identify a Theory

12. Do I know enough to create an instrument?

13. Measurement Construct Measurement Primary Stressors
Validated items/scales for stressors?
Secondary Stressors
Psychosocial Resources
Validated items/scales for how students cope with stress or the resources they use?
Mental Health Outcomes
Validated scales for Stress, Depression, Anxiety, Well-Being?
Demographics
Typical items or more?
Confounding Variables
What other factors influence outcomes that are not already being measures?

14. Measurement: Perceived Stress Scale

15. Do I know enough to create an instrument?

16. What’s next? APPROACH WHO? Individual interviews Focus groups
Students?
Faculty?
Advisors?

17. Focus Group Design What are the predominant sources of stress?
How do pharmacy students deal with the stress that results from the circumstances discussed?
What are the primary effects of stress on pharmacy students?

18. Focus Group Results
Develop items based on focus group results

19. Measurement Construct Measurement Primary Stressors
Developed 107 items
Secondary Stressors
Psychosocial Resources
Social Support
Developed 24 items
Coping Strategies
Developed 33 items
Mental Health Outcomes
Validated Scales
Confounding Variables
Work, Finances, Academic status, Extracurricular organizations, etc.

20. Pilot Test: Stressors

21. Next Steps: Refine the Survey

22. References
Cohen S, Kamarck T, & Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24: Fairman KA. Going to the source: A guide to using surveys in health care research. J Managed Care. 1999;5(2): Pearlin LI. The sociological study of stress. J Health Soc Behav. 1989;30(3): Pearlin LI & Bierman A. (2013). Current issues and future directions in research into the stress process. In Handbook of the sociology of mental health (pp ). Springer Netherlands. Robotham D. Stress among higher education students; towards a research agenda. High Educ. 2008;56:

23. Survey Research in Experiential Education
Patti Darbishire, PharmD
Associate Professor, Pharmacy Practice
Director, Introductory Pharmacy Practice Experiences
Purdue College of Pharmacy

24. Survey Research in Experiential Education Programs
Develop preceptor training initiatives
Obtain basic information from/about your constituents
Administer assignments, activities, or projects
Evaluate performance
Assess the quality or rigor of your program
Determine site metrics, e.g. capacity
Assess constituents’ perceptions
Many more!

25. Surveying Preceptors to Maximize Effectiveness
ACPE ensures all EE administrators care about preceptor training and development through the Standards and Guidelines.
How do we best determine our preceptors’ needs for development and training?
Ask them! - But how do you best get this information while respecting their time and energy? We are all bombarded by survey research in our inbox.

26. Surveying Preceptors to Maximize Effectiveness
Consider: Population or Sample?
Suggestion: Survey carefully selected, small samples of your preceptor population about issues pertinent in their niche.
Want more information? Feed your sample findings back to your preceptor population through the network you use to disseminate information to preceptors or through a discussion board.
Wish list, logistical barriers, professional challenges

27. Surveys in the Placement Process
The “Monster” we grew:
3-step, annual process to place ~450 students
We asked students to:
Step 1: Provide/update basic personal and professional information in their online portfolio
Step 2: Complete an electronic PDF questionnaire with additional information and back to us
Step 3: Conduct a 15-minute one-on-one interview to ensure we had the full picture (took 3-4 weeks)

28. Survey to Collect Info through our Program Management System
Present:
Survey students through our PMS linked to the students’ portfolio system
One data spreadsheet vs. PDFs in student files
No longer consider personal circumstances as part of this process, but allow:
switches
selection from remaining placements
Result: Freed 1 FTE for 3-4 weeks

29. Surveys as Student Assignments
Variable nature of experiential education
Charged with ensuring that students achieve the same outcomes
Tasked with capturing information on an individual student level, as well as from a bird’s eye view
Scholarship requirements?

30. Designing Assignments like Surveys
Students purchase a workbook and complete active learning assignments.
Preceptor signs-off on each assignment upon completion.
The student turns in the workbook for review.
Every workbook assignment has an associated student self-assessment and requires a preceptor’s signature upon completion.
Preceptors evaluate student performance specific to each assignment at the end of the rotation.
Program Management System

31. Designing Assignments like Surveys
Use electronic questionnaires created in our EE Program Management System
Responses readily retrievable at an individual student level and assignment level
No matter how much variability exists between rotations, we can demonstrate outcomes were met
Data as basis for scholarship and student research projects
Program Management System

32. Surveys for Quality Assurance
Survey students, preceptors, and other constituents on their opinions and perceptions of:
Course/rotation content (e.g., assignments, projects, activities)
Program structure and logistics (e.g., placement process, link to didactic content)
Effectiveness of the experiential administrators and staff (e.g., effective communicators, responsive to needs)

33. Questions? Kimberly Plake, PhD kplake@purdue.edu
Patricia Darbishire, PharmD