1. Research Methology IV - BNV45211 Chapter 11 Experiments

2. Research Methology IV - BNV45212 Experiments useful When we can make general statements after conducting the experiment When we can make general statements after conducting the experiment

3. Research Methology IV - BNV45213 Why experiments? To see if something interesting happens To see if something interesting happens “test’ some theory “test’ some theory Prove a theory Prove a theory

4. Research Methology IV - BNV45214 See if something interesting unstructured unstructured playing around playing around See what there is to learn See what there is to learn

5. Research Methology IV - BNV45215 “Testing” a theory Conduct a limited experiment Conduct a limited experiment See whether theory holds for some specific cases See whether theory holds for some specific cases Could have as its objective Could have as its objective ‘Feel’ if theory correct ‘Feel’ if theory correct Refine the theory Refine the theory Justify a full-scale experiment Justify a full-scale experiment

6. Research Methology IV - BNV45216 Proving a theory Must remove all doubt Must remove all doubt Differ from problem to problem Differ from problem to problem Chemist Chemist Psychologist Psychologist

7. Research Methology IV - BNV45217 Why different? Chemist Chemist Has complete control over experiment Has complete control over experiment Psychologist Psychologist Many factors over which they have no control Many factors over which they have no control Average out over sample Average out over sample But choice of sample important But choice of sample important

8. Research Methology IV - BNV45218 Experiments Compare two (or more) cases with one another Compare two (or more) cases with one another E.g. current to new E.g. current to new E.g. impact of software bugs on system performance / user productivity E.g. impact of software bugs on system performance / user productivity Keep all factors the same except the one you are interested in. Keep all factors the same except the one you are interested in. Variables Variables Experimental variables Experimental variables

9. Research Methology IV - BNV45219 Reaching the goal Statistical sound experiment design Statistical sound experiment design Common sense and knowledge from literature study Common sense and knowledge from literature study Play devil’s advocate Play devil’s advocate Think of criticisms Think of criticisms Limit / broaden objectives of experiment to address critisism Limit / broaden objectives of experiment to address critisism “test” then prove “test” then prove See examples on pp 71 See examples on pp 71

10. Research Methology IV - BNV452110 Experimental design Human involvement - problems Remove the human factor from design Remove the human factor from design Hawthorne effect (1920 Western Electric, Chicago) Hawthorne effect (1920 Western Electric, Chicago) Basic requirement: Basic requirement: Random split into groups Random split into groups Placebo Placebo Subjects should not know which group Subjects should not know which group Maybe even researcher need not know Maybe even researcher need not know

11. Research Methology IV - BNV452111 Can everything be countered? NO Cannot guarantee total objectivity NO Cannot guarantee total objectivity Point out and explain limitations of your experiments Point out and explain limitations of your experiments

12. Research Methology IV - BNV452112 More problems People learning from the experiment People learning from the experiment Word processor A and B Word processor A and B Two groups (but identical in all respects?) Two groups (but identical in all respects?)

13. Research Methology IV - BNV452113 Experimental and control groups Independent two group design Assume that they are identical in all respects (except the one being tested) Assume that they are identical in all respects (except the one being tested) May need to “prove” equivalence of groups May need to “prove” equivalence of groups Test the groups before the time Test the groups before the time Useful to counter “learning effect” Useful to counter “learning effect”

14. Research Methology IV - BNV452114 Other techniques Single group Single group If learning does not play a role If learning does not play a role Non-human subjects Non-human subjects Blind experiments Blind experiments placebo placebo Double blind Double blind

15. Be Aware Of psychological effects that knowledge of the experiment may have on the participants’ perceptions Research Methology IV - BNV452115

16. A fool … is a man who never tried an experiment in this life ERASMUS DARWIN [The mice have] been experimenting on you, I’m afraid. DOUGLAS ADAMS Research Methology IV - BNV452116