1. Investigating Self-Selection Bias in Mindfulness Research
Jenna L. Shrewsbury and Kara I. Gabriel, PhD. Department of Psychology, Central Washington University
Contact:
Background
Results
What is “Meditation”?
Previous research has shown that participants may chose to participate in a study based on certain underlying personality differences (Carnahan & McFarland, 2007)
Personality differences may guide individuals to try or avoid different health-related activities.
Mindfulness meditation has demonstrated abundant health benefits, However, attitudes toward mindfulness meditation are largely unmeasured. It is unclear whether or not there are underlying personality differences between those who are interested in mindfulness meditation and those who are not.
Figure 1. Over the same time period, more participants chose to participate in the “personality” study (n=108) than in the “mindfulness “study (n=60)
X2(1, N=168)=13.71, p< Differences were not observed based upon demographic characteristics. A majority of the participants in both groups were female (N=124) and white (not Hispanic) (N=113).
Hypotheses
Figure 4. Participant responses when asked to explain what “meditation” is. Larger words indicate a higher frequency of reporting.
Participants who self-select into a “Mindfulness” study will have higher trait mindfulness scores when compared to those who self-select into a “Personality” study.
Similar to prior research (Baer, Smith, Hopkins, Krietemeyer, & Toney, 2006), participants with higher mindfulness scores will have higher levels of openness to experience and lower levels of neuroticism than participants with lower mindfulness scores.
What is “Mindfulness”?
Figure 2. Mindfulness scores did not significantly vary between conditions on any of the five facets of the FFMQ: Observing F(1,166)=.26, p=.61, Describing F(1,166)=1.30, p=.26, Non-judgment of experience F(1,166)=.07, p=.79, Non-reactivity F(1,166)=.19, p=.66, and Acting with Awareness F(1,166)=1.19, p=.28.
Method
Participants were recruited to participate either in a study about “Mindfulness” or a “Personality” study. Fliers were posted around the CWU psychology building and brief descriptions were posted on the Department’s online research participation system. All recruitment materials were identical aside from the use of the word “mindfulness” or “personality.” The description read: “This study is designed to gather information about students who are interested in a mindfulness [personality] study. You will be asked a series of questions about you, your beliefs, and your religious affiliation. You will also be asked to provide some demographic information. This study should take about 20 minutes to complete.”
All participants completed the Five Facet Mindfulness Questionnaire (Baer et al., 2006), Santa Clara Strength of Religious Faith Questionnaire (Plante & Boccaccini, 1997) and the Big Five Inventory (John & Srivastava, 1999) in counterbalanced order.
All participants were also asked to provide a brief description of what they believed “meditation” was and what they believed “mindfulness” was prior to debriefing.
Figure 5. Participant responses when asked to explain what “mindfulness” is. Larger words indicate a higher frequency of reporting.
Figure 3. Participants high in mindfulness had higher agreeableness scores F(1,166)=21.51, p<.001, higher conscientiousness scores F(1,166)=43.40, p<.001, and higher openness to experience F(1,166)=10.22, p<.001 scores and lower levels of neuroticism F(1,166)=52.53, p<.001 on the Big Five Inventory, than did participants lower in mindfulness. Extraversion did not significantly differ between participants with high and low mindfulness scores F(1,166)=3.16, p=.08.
Discussion
Results suggest that researchers conducting mindfulness studies might be negatively impacting their recruitment numbers if they specifically mention mindfulness.
However, recruiting specifically for mindfulness does appear to result in biased self-selection into such studies, allowing greater generalizability.
REFERENCES
Baer, R.A., Smith, G.T., Hopkins, J., Krietemeyer, J., & Toney, L. (2006). Using self-report assessment methods to explore facets
of mindfulness. Assessment, 13,
Carnahan, T., & McFarland, S. (2007). Revisiting the Stanford prison experiment: Could participant self-selection have led to
the cruelty? Society for Personality and Social Psychology, 33, doi: /
John, O.P., & Srivastava, S. (1999). The Big-Five trait taxonomy: History, measurement, and theoretical perspectives. In Pervin
L.A., & John, O.P. (Eds.), Handbook of personality: Theory and research, 2, New York: Guilford Press.
Plante, T.G., & Boccaccini, M. (1997). The Santa Clara Strength of Religious Faith Questionnaire. Pastoral Psychology, 45, 375-
387.