1. Chapter 11: Experiments Introduction 11.1 Experiment goal
11.2 Reaching the goal
11.3 Experiment design
Experimental and control groups
Single group
Blind experiments
11.4 Conclusion

2. Introduction
Do an experiment to confirm some theory or to find out what would happen if …
Want to make generalisations: Conclusions should not be limited to specific cases tested.

3. 11.1 Experiment goal To see if you can find something interesting
Play around with data and see what is there to learn.
To test a theory
Conduct a limited experiment to see if the theory holds for some specific cases
To prove a theory
To disprove it is only necessary to find a single example for which the theory does not holds.
To prove, need: Control over all factors; Representative sampling

4. 11.2 Reaching the goal
See fig. 11.1, p 70.
Five examples

5. 11.3 Experiment design Split participants randomly into two groups
(e.g. placebo in medical research)
Participants should not know in which group they fall
Participants should not learn from experience, i.e. the same participant may not test system A and thereafter system B.

6. 11.3.1 Experimental and control groups
Two groups are treated in exactly the same way.
Observe effects & process data
Fundamental assumption: Two groups are identical in all other respects.
E.g. difference in computer literacy and/or previous computer experience

7. Single group
The same group used first for control experiment and then for experiment itself.
No question that the groups may differ, but the control case may provide experience.
Especially fruitful if you test hardware and and/or software and not humans.

8. 11.3.3 Blind experiments E.g. placebo in medical experiments
Not as easy in IT research
Double-blind refers to case where even the observer does not know which group has been exposed to which treatment
Hardly ever conducted in IT research

9. 11.4 Conclusion Major concerns
Observations are caused by experimental inputs and nothing else (internal validity)
Results observed can be generalised (external validity)
Compile a protocol (what, when, how, why, where) activities are to be performed