1. Reminder for next week
CUELT Conference

2. EXPERIMENTAL RESEARCH

3. A very common way of research design
A good design to establish cause and effect relationships among variables
It is the type of research that
directly attempts to influence a particular variable
tests hypotheses about cause-and-effect relationships

4. Major Characteristics of Experimental Research
The researcher looks at the effect(s) of at least one independent variable (experimental or treatment variable) on one or more dependent variables (criterion or outcome variable)
The researcher manipulates the independent variable, which is not the case in other research designs
Aim of the researcher is to see whether the treatment/manipulation has made a difference.

5. Some independent variables:
Methods of instruction
Types of assignment
Learning materials
Types of questions asked by teachers…
Some dependent variables:
Achievement
Interest
Motivation
Attitudes…

6. Comparison of Groups
Experimental research involves two groups of subjects
Experimental group
Control / comparison group
Experimental group receives the treatment; control group receives no treatment / comparison group receives a different treatment
Comparison group rather than control group in educational research

7. Manipulation of the Independent Variable
The researcher actively manipulates the independent variable: s/he decides the nature of the treatment, to whom and to what extent it will be applied, and when, where and how it will be done
Variables that can be manipulated: teaching method, learning activities, materials, etc
Variables that cannot be manipulated: gender, ethnicity, age, etc

8. Ways of establishing independent variables
A) one form of the variable versus another
E.g. comparing the inquiry method vs the lecture method in teaching listening
B) presence versus absence of a particular form
E.g. comparing the use of videos vs no videos in teaching speaking
C) varying degrees of the same form
Comparing the effects of different amount of homework on students’ motivation

9. Group Designs in Experimental Research: WEAK Experimental Designs
The One-Shot Case Study:
A single group is exposed to a treatment and a dependent variable is observed to assess the effect of the treatment
Why weak?
Absence of control: the researcher does not know whether the result is due to the treatment
Not possible to compare so the researcher doesn’t know whether the treatment had any effect at all X O
treatment
Observation
(post-test)

10. O X observation (pre-test) treatment (post-test)
The One-Group Pretest-Posttest Design:
A single group is observed before and after treatment
Better than one-shot case study (we know whether change occurred), but it is still weak. Why? O X
observation
(pre-test)
treatment
(post-test)

11. X O treatment Observation (post-test)
The Static-Group Comparison Design
Two already existing groups (non-equivalent) are used
Although it provides better results, it might still have problems. Why? X O
treatment
Observation
(post-test)

12. True Experimental Designs
The Randomized Posttest-Only Control Group Design
Involves two groups which are formed by random assignment: one receives the treatment; the other does not
treatment group R X O
random
treatment
Observation
(post-test)
control/
comparison group C
comparison

13. The Randomized Pretest-Posttest Control Group
The difference is in using both pre and post test for both groups
This design helps the researcher see if the random assignment actually succeeded (pretest).
This is particularly important
if the number in each group is small
If the amount of change over time is to be assessed
treatment group R O X
random
observation
(pre-test)
treatment
(post-test)
control/
comparison group C
control/ comparison

14. The Randomized Solomon Four-Group Design
Used to eliminate the possible effect of a pretest
The first group is like pretest-posttest control group; the second is posttest only control group
Best control of threats
Weakness: requires a large sample (subjects are assigned to four groups); requires a lot of energy and effort
Treatment group R O X
random
Observation
(pre-test)
treatment
observation
Control group C
control

15. QUASI-EXPERIMENTAL DESIGN
It is like experimental design but it doesn’t use random assignment.
It uses other techniques to control threats
There are many techniques used in this design but the most commonly used one in social research is nonequivalent groups design

16. O X observation (pre-test) treatment (post-test) C control/ comparison
Nonequivalent groups design
It is like pretest-posttest randomized experiment but does not use random assignment.
Instead similar groups are used as the treatment and the control groups
E.g. you can pick up two comparable classrooms or schools. O X
observation
(pre-test)
treatment
(post-test) C
control/
comparison
(post-test

17. CONCLUSION
In any type of experimental design, we create two groups that are “equivalent” to each other.
One group gets the treatment; the other does not.
In all other respects the groups are treated the same.
If we observe differences in the outcome between these groups, then we believe that the difference is due to the treatment.
However, in all these designs, we are setting up an artificial situation, so we have to consider all the situation.
If the situation is right, then an experiment can be a very strong design to use.

18. Internal Validity Possible threats to internal validity
A) Selection Bias (subject characteristics)
Might be differences in groups (age, gender, intelligence…)
B) Loss of Subjects (mortality)
Losing some of the subjects (e.g. some subjects may not return the questionnaires)

19. C) Location
Place in which data are collected or an implementation is conducted may affect the study
D) Instrumentation
instrument decay (e.g. essay scoring causes fatigue and can be scored differently)
data collector characteristics (age, gender, experience of data collector)
data collector bias (e.g. if the collector knows the hypotheses, s/he might unconsciously distract the data)

20. E) Testing
Subjects can remember the test if the same is used in pre and post test at a short interval
F) History
personal experiences during the course of the study
G) Maturation
E.g. young learners will change due to maturation

21. H) Attitude of Subjects
The way subjects view the study (e.g. experimental group might perform better because they know they are having a special treatment)
I) Regression
e.g. if you include all good students in one group, they will improve no matter what you do
J) Implementation
The researcher can give the experimental group an advantage