Quantitative Methods for Researchers Paul Cairns

Your objectives  Pretty general! – Refresher or introduction to experiments – Specific stats questiosn  Your research areas? Quant for RS, Paul Cairns

My objectives  Experimental Style – Exploratory or severe test  Validity  Testing as modelling – Distributions – Choice of tests Quant for RS, Paul Cairns

Context  HCI examples  Widely applicable principles Quant for RS, Paul Cairns

Why do we do experiments? Quant for RS, Paul Cairns

Philosophy of experiments  Test theories  Isolate phenomena  Severely test Quant for RS, Paul Cairns

Some consequences  Intrinsic value  Big is not always better  Narrow focus is essential Quant for RS, Paul Cairns

Experimental argument  Belief:X causes Y – A reason for looking  Try: change X and measure Y  Analyse carefully  Produce evidence Quant for RS, Paul Cairns

Statistical experiments  Natural variation – People, environment, stochastic  Systematic vs chance differences  No certainty Quant for RS, Paul Cairns

Exploratory experiment  Controlled environment  Unknown focus – Unknown phenomenon?  Reason for looking  Weak evidence Quant for RS, Paul Cairns

Devising an experiment  Research question (disposable)  One sentence  May use jargon  Answer is “yes/no” but probably “maybe”  Question suggests how to answer it Quant for RS, Paul Cairns

Devise a research question In groups of two or three, each have a go at a research question. Take turns to explain and be criticised. Be happy to be wrong/stupid. RQs are disposable. Quant for RS, Paul Cairns

Experimental Design  Addresses question  Design => Data => Interpretation  Validity Quant for RS, Paul Cairns

Working example  Does making things harder to read reduce transcription errors? Quant for RS, Paul Cairns

Variables  Independent variable (IV, X) – Experimental conditions  Dependent variable (DV, Y) – quantitative  Confounding variables Quant for RS, Paul Cairns

Validity  Construct  Internal  External  Ecological Quant for RS, Paul Cairns

Construct validity  What are you measuring?  What are you manipulating? Quant for RS, Paul Cairns

Internal validity  Is X causing Y?  Is X the only thing causing Y? – Confounding variables  Experimental drift Quant for RS, Paul Cairns

External validity  Generalisability  People  Places  Systems  Interactions Quant for RS, Paul Cairns

Ecological validity  Relevance to out there? Quant for RS, Paul Cairns

Critique the paper  Challenges to validity Quant for RS, Paul Cairns

Components of a Method  What you are trying to find out  What the variables are  Who/what you examined  What you used to do it  How you did it Quant for RS, Paul Cairns

Components of a Method  Research question  Design  Participants/test sets  Materials  Procedure Quant for RS, Paul Cairns

Homework  Does making game playing more sociable improve the player experience?  Devise an experiment Quant for RS, Paul Cairns

Reading  Hacking, Representing and Intervening  Cairns, Encyclopedia of HCI, 2 nd edn, chap 34  Cairns, Cox, Research Methods for HCI: chaps 1, 6  Harris, Designing and reporting experiments in psychology, 3rd edn Quant for RS, Paul Cairns