1. TAUCHI – Tampere Unit for Computer-Human Interaction ERIT 2015: Data analysis and interpretation (1 & 2) Hanna Venesvirta Tampere Unit for Computer-Human Interaction

2. TAUCHI – Tampere Unit for Computer-Human Interaction Aims See which analysis are used for the course projects and how

3. TAUCHI – Tampere Unit for Computer-Human Interaction Overview of comparing samples Our aim is simple: we wish to find out, if the means of our collected data samples are separated enough to conclude that the means are likely to be different

4. TAUCHI – Tampere Unit for Computer-Human Interaction Overview of comparing samples Null hypothesis: no difference Alternative hypoth.: there is difference Aim to reject the null hypoth. Result is ”statistically significant” when there is only little likehood that the null hypothesis is true – p-value < 0.05

5. TAUCHI – Tampere Unit for Computer-Human Interaction Analysis of Variance (ANOVA) Is used to find out differences between means from more than two sample means Two sample designs: t-tests Can be used for testing the effects of more than one independent variable (IV) at one time 2-way / 3-way / etc.-way designs

6. TAUCHI – Tampere Unit for Computer-Human Interaction Repeated measures ANOVA Is used if we have measured all the participants under all the different levels of the (different) IV(s) Standard ANOVA cannot be used as the data is correlated

7. TAUCHI – Tampere Unit for Computer-Human Interaction Analysis example step-by-step Experimental task: select an object as fast as possible Depend variable: selection time (ms) One independent variable: diameter of an object With three levels: diameter either 25, 30, or 40 mm All the participants made the same task -> one-way within subjects design

8. TAUCHI – Tampere Unit for Computer-Human Interaction Note! The values per participant per level of IV are averages of several tasks - usually one exact task is repeated several time during the trial.

9. TAUCHI – Tampere Unit for Computer-Human Interaction Note! The values per participant per level of IV are averages of several tasks - usually one exact task is repeated several times during the trial...Thus, these means are averages of averages.

10. TAUCHI – Tampere Unit for Computer-Human Interaction Visualize your data! (on this case: means) Good for initial inspection of the possible difference Excellent for showing a summary of the results to the reader

11. TAUCHI – Tampere Unit for Computer-Human Interaction Visualize your data! Column graphs are good when presenting means The one below shows only the means of different levels of the IV

12. TAUCHI – Tampere Unit for Computer-Human Interaction Visualize your data! This one shows also the deviation of the data sample Here: Standard Error of the Mean (S.E.M.) If you add error bars to the graphs, see, e.g., https://www.youtube.com/watch?v=AfAG61UWsWA

13. TAUCHI – Tampere Unit for Computer-Human Interaction The means differ! …significantly?

14. TAUCHI – Tampere Unit for Computer-Human Interaction Data to SPSS? 1) Select “variable view” –tab from the bottom left corner 2) Add descriptive variable names 3) Select “data view” –tab from the bottom left corner 4) Add your data by, e.g., copy-and-paste from, e.g., excel NOTE! Only the numbers – you defined the column headings already on points no. 1-2

15. TAUCHI – Tampere Unit for Computer-Human Interaction Parametric tests – One way repeated measures ANOVA and Paired samples t-test

16. TAUCHI – Tampere Unit for Computer-Human Interaction

19. From the output, find table called “tests of within-subjects effects” – this is where ANOVA result is

20. TAUCHI – Tampere Unit for Computer-Human Interaction …but which row to read? ???

21. TAUCHI – Tampere Unit for Computer-Human Interaction Go back up and find table called ”mauchly’s test of sphericity” Tests, if the data looks like this: …or, more like this:

22. TAUCHI – Tampere Unit for Computer-Human Interaction If the result from this test is significant.. …the variances of the data are not equal, that is, the sphericity cannot be assumed

23. TAUCHI – Tampere Unit for Computer-Human Interaction Back to the result table… - thus we read the second row As we cannot assume sphericity, we cannot read the first row from the result table

24. TAUCHI – Tampere Unit for Computer-Human Interaction..also the with Greenhouse-Geisser corrected degrees of freedom the significance value is less than 0.05

25. TAUCHI – Tampere Unit for Computer-Human Interaction NOTE: If it happens that the result from the mauchly’s table is not significant we can assume sphericity, and thus we can use the result from the first row

26. TAUCHI – Tampere Unit for Computer-Human Interaction Thus there is a difference, but where? -> we shall find out by running pairwise comparisons with paired samples t-tests NOTE: pairwise comparisons are not to run if the ANOVA shows non-sign. result Comparing 25 mm to 30 mm, 25 mm to 40 mm, and 30 mm to 40 mm -> 3 comparisons Multiple comparisons – remember to adjust the p-value in order to avoid Type I error! Bonferroni correction: original p / number of comparisons Here: 0.05/3 = ~0.017

27. TAUCHI – Tampere Unit for Computer-Human Interaction

29. Paired Samples Test Paired DifferencestdfSig. (2-tailed) MeanStd. DeviationStd. Error Mean 95% Confidence Interval of the Difference LowerUpper Pair 1 Diameter25mm - Diameter30mm 534,25047932330 8200 1188,6194789491 00000 265,78339543105 6930 - 22,040560576190 714 1090,5415192228 07000 2,01019,059 Pair 2 Diameter25mm - Diameter40mm 634,76911027568 9700 942,43528288740 0500 210,73493569304 7280 193,69582076596 8620 1075,8423997854 10600 3,01219,007 Pair 3 Diameter30mm - Diameter40mm 100,51863095238 1600 456,37710097444 7500 102,04902211531 5080 - 113,07242706381 1220 314,10968896857 4230,98519,337 From the output, find table called ”paired samples test” – here are the results This one is smaller than the adjusted p (0.007 < 0.017), thus the significant difference is between this comparison… …and you can check the direction of the difference from, e.g., the graph you made.

30. TAUCHI – Tampere Unit for Computer-Human Interaction Reporting ANOVA result following is needed: (fixed) degrees of freedom (here: ~1.2 and ~22.4) F-value (here: ~5.6) p-value (here p < 0.05) ANOVA reporting

31. TAUCHI – Tampere Unit for Computer-Human Interaction Tests of Within-Subjects Effects Measure: MEASURE_1 SourceType III Sum of Squares dfMean SquareFSig. size Sphericity Assumed 4656395,95522328197,9775,567,008 Greenhouse-Geisser 4656395,9551,1803946581,0215,567,023 Huynh-Feldt 4656395,9551,2123842071,2655,567,022 Lower-bound 4656395,9551,0004656395,9555,567,029 Error(size) Sphericity Assumed 15892110,38138418213,431 Greenhouse-Geisser 15892110,38122,417708923,041 Huynh-Feldt 15892110,38123,027690149,988 Lower-bound 15892110,38119,000836426,862 “One-way within subjects ANOVA with object diameter size as a factor revealed a statistically significant effect of the object diameter size, F(1.2, 22.4) = 5.6, p < 0.05.” ANOVA reporting

32. TAUCHI – Tampere Unit for Computer-Human Interaction For reporting the results from pairwise comparisons (with paired sample t-tests) following is needed: Degrees of freedom (here: 19) t-value (here: ~3.0) p-value (here: p < 0.01) Reporting pairwise comparisons

33. TAUCHI – Tampere Unit for Computer-Human Interaction Paired Samples Test Paired DifferencestdfSig. (2-tailed) MeanStd. DeviationStd. Error Mean 95% Confidence Interval of the Difference LowerUpper Pair 1 Diameter25mm - Diameter30mm 534,25047932330 8200 1188,6194789491 00000 265,78339543105 6930 - 22,040560576190 714 1090,5415192228 07000 2,01019,059 Pair 2 Diameter25mm - Diameter40mm 634,76911027568 9700 942,43528288740 0500 210,73493569304 7280 193,69582076596 8620 1075,8423997854 10600 3,01219,007 Pair 3 Diameter30mm - Diameter40mm 100,51863095238 1600 456,37710097444 7500 102,04902211531 5080 - 113,07242706381 1220 314,10968896857 4230,98519,337 Reporting pairwise comparisons “Post hoc pairwise comparisons for the object diameter size showed that the participants pointed significantly faster the 40 mm diameter object than the 25 mm diameter object, t(19) = 3.0, p < 0.01. Other pairwise comparisons were not statistically significant.”

34. TAUCHI – Tampere Unit for Computer-Human Interaction Take the data from following slide and Create a data matrix excel & SPSS Visualize data Column graph is recommended Run analysis with SPSS Write down the results// Sent graph & the written-down-results to Hanna via mail as.pdf We shall take a look on this next week Task: parametric analysis (1)

35. TAUCHI – Tampere Unit for Computer-Human Interaction Errors P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17 P18 P19 P20 25 mm 25500 250 12,5500012,587,525 37,512,50 30 mm 012,5002512,537,5025012,5 507512,5 2512,5 0 40 mm 0250012,50250012,500250000 12,5 Experimental task: select an object as accurately as possible Depend variable: errors One independent variable: diameter of an object With three levels: diameter either 25, 30, or 40 mm Task: parametric analysis (2)

36. TAUCHI – Tampere Unit for Computer-Human Interaction Non-parametric analysis – Friedman’s test and Wilcoxon Repeated Measures Signed-Rank test

37. TAUCHI – Tampere Unit for Computer-Human Interaction These analysis do not make any assumptions about the probability distribution of the data Before the analysis, the data is transformed to ranks (by the statistical SW) Usually a non-parametric test has an equivalent parametric test Non-parametric analysis

38. TAUCHI – Tampere Unit for Computer-Human Interaction One-way repeated measures ANOVA Equivalent tests Friedman’s rank test for k- correlated samples Matched pairs t-test Wilcoxon Repeated Measures Signed- Rank test Parametric tests Non-parametric tests

39. TAUCHI – Tampere Unit for Computer-Human Interaction Friedman’s rank test for k-correlated samples

40. TAUCHI – Tampere Unit for Computer-Human Interaction

42. Test Statistics a N 20 Chi-Square 10,900 df 2 Asymp. Sig.,004 a. Friedman Test From the output, find ”test statistics”

43. TAUCHI – Tampere Unit for Computer-Human Interaction Again, a difference; find out where -> pairwise comparisons, this time done with non-parametric paired samples test Comparing 25 mm to 30 mm, 25 mm to 40 mm, and 30 mm to 40 mm -> 3 comparisons Multiple comparisons – remember to adjust the p-value in order to avoid Type I error! Bonferroni correction: original p / number of comparisons Here: 0.05/3 = ~0.017

44. TAUCHI – Tampere Unit for Computer-Human Interaction Wilcoxon Repeated Measures Signed-Rank test

45. TAUCHI – Tampere Unit for Computer-Human Interaction

47. Test Statistics a Diameter30mm - Diameter25mm Diameter40mm - Diameter25mm Diameter40mm - Diameter30mm Z -2,165 b -3,472 b -,261 b Asymp. Sig. (2-tailed),030,001,794 a. Wilcoxon Signed Ranks Test b. Based on positive ranks. From the output, find ”test statistics” Again, his one is smaller than the adjusted p (0.007 < 0.017), thus the significant difference is here again.

48. TAUCHI – Tampere Unit for Computer-Human Interaction Test Statistics a N 20 Chi-Square 10,900 df 2 Asymp. Sig.,004 a. Friedman Test Reporting Friedman’s test “Friedman's test showed that there was statistically significant effect of object diameter, Χ²(2) = 10.9, p < 0.01.”

49. TAUCHI – Tampere Unit for Computer-Human Interaction Reporting Wilcoxon test Test Statistics a Diameter30mm - Diameter25mm Diameter40mm - Diameter25mm Diameter40mm - Diameter30mm Z -2,165 b -3,472 b -,261 b Asymp. Sig. (2-tailed),030,001,794 a. Wilcoxon Signed Ranks Test b. Based on positive ranks. Note: double click the table (twice) and you will see more accurate p-value; on this case p = 0.000517, thus it is significant in 0.01 level as 0.01 / 3 = 0.0033. “Post hoc pairwise comparisons with Wilcoxon signed-rank tests showed that the selection time was significantly faster when the object diameter was 40 mm than when the diameter was 25 mm, Z = -3.47, p < 0.01. Other pairwise comparisons were not statistically significant.”

50. TAUCHI – Tampere Unit for Computer-Human Interaction Check your previous exercise and compare it to Hanna’s answer Answer is in course web-side/schedule Try to fix if different Run non-parametric analysis to the same error data in SPSS Are the results different in any way? Mail possible fix of the 1 st task and the written-down-results of the 2 nd to Hanna If the results differ, note it, and how 1) Check the previous task 2) Make non-parametric analysis