1. Faculdade de Medicina da Universidade do Porto Curso de Mestrado Integrado em Medicina 2010/2011 INTRODUÇÃO À MEDICINA II TURMA 21 REGENTE: Prof Dr. Altamiro Costa Pereira PROFESSORA E ASSISTENTE: Prof Drª. Cristina Santos e Luís Pinto http://jpn.icicom.up.pt/imagens/up/up_fmedicina.jpg

2. Biases in questionnaire construction: how much do they influence the answers given?

3. Types of errors Random Errors Affect questionnaires’ precision Related to the variability inherent to a phenomenon that is being measured [1] Systematic Errors Affect questionnaires’ validity Related to the wrong association between an exposure and a certain effect [1] Commonly known as “biases” 1.M. Delgado-Rodríguez and J Llorca, Bias, J. Epidemiol. Community Health 2004;58;635-641

4. Scenario Analysis of questionnaires’ results based on false/inaccurate information Leads To Descriptions and catalogues of common biases [2,5] To inform investigators To minimize biases Description of biases in literature 2.M. Delgado-Rodríguez and J Llorca, Bias, J. Epidemiol. Community Health 2004;58;635-641 5.Bernard C.K. Choi and Anita W.P. Pak. A Catalog of Biases in Questionnaires. Preventing Chronic Diseases. 2005; 1; 13

5. Described biases Question Design Ambiguous questions Technical jargon Uncommon words Problems with wording Leading question Leading questions Missing interval Faulty scale Sensitive questions Intrusiveness Belief vs. behavior Missing or inadequate data for intended purpose

6. Described biases Administration of questionnaire The “Don’t know”, “Undecided”, and “Neutral response” options Faking Good Respondent’s conscious reaction

7. Described biases Questionnaire Design Horizontal response format Formatting problem

8. Research Question Do these so called ‘biases’ really influence the answers given in questionnaires? http://static4.depositphotos.com/1000423/471/i/110/depositphoto s_4715431-3D-man-with-question-mark.jpg

9. Check if the way we change a normal question influences the given answers in general. Be able to sort out at what extent can each of the types of biases mentioned above manipulate the results obtained. To realize whether there is evidence that the described biases interfere in questionnaires’ results Aims

10. Methods Two versions of the same questionnaire A and B Both with biased and unbiased questions Target Population First year students from FMUP

11. Versions A and B Randomly assigned questions to versions A and B of the questionnaire Questionnaire arquitecture 19 different classes Distribution prevented them from detecting two different versions of the same questionnaire. Questionnaire Distribution Gathering the answers given Creation of a data base in SPSS Comparison between the supposedly biased and respective non-biased questions using Qui- square test and T-student test Analysing if the documented biases are indeed biases (observing p value) Questionnaire Analysis

12. Data Analysis The analysis of the answers given was only possible because we consider that the odds of obtaining a certain answer is similar in both groups of each class. Our outcome was the rates in the answers given to supposedly biased and unbiased questions, comparing the rates to see if there was any significant differences. To analyse our results we use the statistical analysis software SPSS 19.0.

13. Final Results and Discussion Final Results and Discussion

14. Question A1 vs. B1-2 Biased question (A1) : Do you lend your tooth paste or tooth brush to other people? Yes [ ] No [ ] Unbiased question (B) : 1) Do you lend your tooth paste to other people? Yes [ ] No [ ] 2) Do you lend your tooth brush to other people? Yes [ ] No [ ] AB1p No77 (77,8%)29 (30,9%) <0,0001 Yes22 (22,2%)65 (69,1) AB2p No77 (77,8%)93 (100%) <0,0001 Yes22 (22,2%)0 (0%) Ambiguous question

15. Question A2 vs. B3 Biased Question (A2): When you take a bath, do you wash your genital area? Yes [ ] No [ ] Unbiased question (B3): When you take a bath, do you wash your hair? Yes [ ] No [ ] ABp No0 (0%)1 (1%) 0,301 Yes99 (100%)92 (95,8%) Missing0 (0%)3 (3,2%) Sensitive question

16. Question A3 vs. B4 Biased question (B4): You have a shower everyday, don´t you? Yes [ ] No [ ] Unbiased question (A3): How often do you have a shower in the week: More than 6 days per week[ ] 5-6 days per week [ ] 2-4 days per week [ ] 1 day per week[ ] I never have a shower [ ] ABp (A)More than 6 days per week/(B)Yes 67 (67,7%) 82 (87,2%) 0.001 A)I never have a shower; 1, 2-4, 5-6 days per week/ (B) No 32 (32,3%) 12 (12,8%) Faking good

17. Question A4 vs. B5 Biased question (A4): Do you reckon that physical exercise is a germane aspect of your health? Yes [ ] No [ ] Unbiased question (B5): Do you consider that physical exercise is a relevant aspect of your health? Yes [ ] No [ ] AB Confidence Intervals No 7 (7,07%) [-0,00493;0,019125] 3 (3,1%) [-0,00486;0,011062] _ Yes 85 (85,8%) [0,809155; 0,908845] 91 (94,8%) [0,916201;0,979799] Statistically Significant Missing7 (7,07%) [-0,00493;0,01907] 2 (2,1%) [-0,00446;0,008656] _ Uncommon word

18. Question A5 vs. B6 Biased question (B6): Do you think it is important to practice physical exercise at least 3 times a week? Yes [ ] No [ ] Unbiased question (A5): Do you practice physical exercise at least 3 times a week? Yes [ ] No [ ] ABp No69 (69,7%)15 (16,1%) <0,0001 Yes30 (30,3%)78 (83,9%) Belief vs. behaviour

19. Question A6 vs. B7 Biased question (B7): Your eating habits are: Excellent [ ] Very good [ ] Good [ ] Fair [ ] Somehow deficient [ ] Poor [ ] Very poor [ ] Unbiased question (A6): Your eating habits are: Excellent [ ] Very good [ ] Good [ ] Fair [ ] Somehow deficient [ ] Poor [ ] Very poor [ ] ABp Very Poor2 (2%)0 (0%) 0,463 Poor2 (2%)2 (2,1%) Somehow deficient 5 (5,1%)2 (2,1%) Fair14 (14,1%)20 (21,3%) Good55 (55,6%)56 (59,6%) Very Good17 (17,2%)11 (11,7%) Excellent4 (4%)3 (3,2%) Horizontal response format

20. Question A7 vs. B8 Biased question (A7): Do you agree that McDonalds is responsible for the increase of hypertension rates? Agree [ ] Neutral opinion [ ] Disagree [ ] Unbiased question (B8): Do you agree that McDonalds is responsible for the increase of hypertension rates? Agree [ ] Disagree [ ] AB Confidence Intervals Disagree 12 (12,1%) [-0,00356;0,027759] 33 (35,9%) [0,008297;0,060503] _ Agree 49 (49,5%) [0,018434;0,080566] 59 (64,1%) [0,027091;0,095909] _ “Neutral response” options

21. Question A8 vs. B9 Biased question (B9): How many days per week do you dedicate to study anatomy? 2 days [ ] 5 days [ ] 7days [ ] Unbiased question (A8): How many days per week do you dedicate to study anatomy? 1 day [ ] 2 days [ ] 3 days [ ] 4 days [ ] 5 days [ ] 6 days [ ] 7 days [ ] Mean Standard deviation p A 4,111,435 0,462 B 3,951,649 Faulty scale

22. Question A9 vs. B10 Biased question (B10): Do you arrange activities with your friends, such as cinema, every week? Yes [ ] No [ ] Unbiased question (A9): Do you arrange activities with your friends every week? Yes [ ] No [ ] ABp No30 (30,9%)25 (36,6%) 0,509 Yes67 (69,1%)69 (73,4%) Leading question

23. Question A10 vs. B11 Biased question (A10): Did you exercise your Wernicke’s area while responding to this questionnaire? Yes [ ] No [ ] Unbiased question (B11): Did you understand what you read in this questionnaire? Yes [ ] No [ ] AB Confidence Intervals Yes 53 (53,5%) [0,463564;0,606436] 93 (96,9%) [0,944177;0,993823] Statistically Significant No 20 (20,2%) [0,144497;0,259503] 1 (1%) [-0,00353;0,005527] Statistically Significant Missing 26 (26,3%) [0,003381;0,049219] 2 (2,1%) [-0,00446;0,008656] _ Technical jargon

24. Conclusion Summing up, there are a lot of biases described in literature. Through this study, it has been concluded that some of them might work and others not. We cannot, however, extrapolate these results to the general population. http://us.cdn2.123rf.com/168nwm/texelart/texelart1011/texelart101100 030/8220784-3d-human-by-the-symbol-of-victory-with-his-fingers.jpg

25. References (Presentation) 1. M. Delgado-Rodríguez and J Llorca, Bias, J. Epidemiol. Community Health 2004;58;635-641 2. Bernard C.K. Choi and Anita W.P. Pak. A Catalog of Biases in Questionnaires. Preventing Chronic Diseases. 2005; 1; 13 3. http://www.statpac.com/surveys/undecided-category.htm

26. Afonso Quelhas André Santos Bárbara Araújo Carolina Silva Catarina Marques Cláudia Oliveira Laura Gonçalves Luís Sousa Marta Sanhá Rita Costa Tânia Costa