1. Quantitative Methods for Researchers Paul Cairns paul.cairns@york.ac.uk

2. Your objectives  Pretty general! – Refresher or introduction to experiments – Specific stats questiosn  Your research areas? Quant for RS, Paul Cairns

3. My objectives  Experimental Style – Exploratory or severe test  Validity  Testing as modelling – Distributions – Choice of tests Quant for RS, Paul Cairns

4. Context  HCI examples  Widely applicable principles Quant for RS, Paul Cairns

5. Why do we do experiments? Quant for RS, Paul Cairns

6. Philosophy of experiments  Test theories  Isolate phenomena  Severely test Quant for RS, Paul Cairns

7. Some consequences  Intrinsic value  Big is not always better  Narrow focus is essential Quant for RS, Paul Cairns

8. Experimental argument  Belief:X causes Y – A reason for looking  Try: change X and measure Y  Analyse carefully  Produce evidence Quant for RS, Paul Cairns

9. Statistical experiments  Natural variation – People, environment, stochastic  Systematic vs chance differences  No certainty Quant for RS, Paul Cairns

10. Exploratory experiment  Controlled environment  Unknown focus – Unknown phenomenon?  Reason for looking  Weak evidence Quant for RS, Paul Cairns

11. Devising an experiment  Research question (disposable)  One sentence  May use jargon  Answer is “yes/no” but probably “maybe”  Question suggests how to answer it Quant for RS, Paul Cairns

12. Devise a research question In groups of two or three, each have a go at a research question. Take turns to explain and be criticised. Be happy to be wrong/stupid. RQs are disposable. Quant for RS, Paul Cairns

13. Experimental Design  Addresses question  Design => Data => Interpretation  Validity Quant for RS, Paul Cairns

14. Working example  Does making things harder to read reduce transcription errors? Quant for RS, Paul Cairns

15. Variables  Independent variable (IV, X) – Experimental conditions  Dependent variable (DV, Y) – quantitative  Confounding variables Quant for RS, Paul Cairns

16. Validity  Construct  Internal  External  Ecological Quant for RS, Paul Cairns

17. Construct validity  What are you measuring?  What are you manipulating? Quant for RS, Paul Cairns

18. Internal validity  Is X causing Y?  Is X the only thing causing Y? – Confounding variables  Experimental drift Quant for RS, Paul Cairns

19. External validity  Generalisability  People  Places  Systems  Interactions Quant for RS, Paul Cairns

20. Ecological validity  Relevance to out there? Quant for RS, Paul Cairns

21. Critique the paper  Challenges to validity Quant for RS, Paul Cairns

22. Components of a Method  What you are trying to find out  What the variables are  Who/what you examined  What you used to do it  How you did it Quant for RS, Paul Cairns

23. Components of a Method  Research question  Design  Participants/test sets  Materials  Procedure Quant for RS, Paul Cairns

24. Homework  Does making game playing more sociable improve the player experience?  Devise an experiment Quant for RS, Paul Cairns

25. Reading  Hacking, Representing and Intervening  Cairns, Encyclopedia of HCI, 2 nd edn, chap 34  Cairns, Cox, Research Methods for HCI: chaps 1, 6  Harris, Designing and reporting experiments in psychology, 3rd edn Quant for RS, Paul Cairns