1. EXPERIMENTAL RESEARCH

2. A very common way of research design
A good design to establish cause and effect relationships among variables
It is the type of research that
directly attempts to influence a particular variable
tests hypotheses about cause-and-effect relationships

3. Major Characteristics of Experimental Research
The researcher looks at the effect(s) of at least one independent variable (experimental or treatment variable) on one or more dependent variables (criterion or outcome variable)
The researcher manipulates the independent variable, which is not the case in other research designs
Aim of the researcher is to see whether the treatment/manipulation has made a difference.

4. Some independent variables:
Methods of instruction
Types of assignment
Learning materials
Types of questions asked by teachers…
Some dependent variables:
Achievement
Interest
Motivation
Attitudes…

5. Comparison of Groups
Experimental research involves two groups of subjects
Experimental group
Control / comparison group
Experimental group receives the treatment; control group receives no treatment / comparison group receives a different treatment
Comparison group rather than control group in educational research

6. Manipulation of the Independent Variable
The researcher actively manipulates the independent variable: s/he decides the nature of the treatment, to whom and to what extent it will be applied, and when, where and how it will be done
Variables that can be manipulated: teaching method, learning activities, materials, etc
Variables that cannot be manipulated: gender, ethnicity, age, etc

7. Ways of establishing independent variables
A) one form of the variable versus another
E.g. comparing the inquiry method vs the lecture method in teaching listening
B) presence versus absence of a particular form
E.g. comparing the use of videos vs no videos in teaching speaking
C) varying degrees of the same form
Comparing the effects of different amount of homework on students’ motivation

8. Hypotheses
A hypothesis is a prediction of the possible outcomes of a study.
E.g.
RQ: Will students who are taught English by a teacher of the same gender like the subject more than students taught by a teacher of a different gender:
Hypothesis: Students taught English by a teacher of the same gender will like the subject more than students taught English by a teacher of a different gender.

9. Null hypotheses
A hypothesis is a prediction of the possible outcomes of a study.
It states our expectations in a positive sense.
There will be a difference between the groups.
A null hypothesis states our expectations in a negative sense.
There will be no difference between the groups

10. Directional vs Non-directional Hypotheses
A directional hypothesis shows a specific direction (higher, lower, more…) that a researcher expects to emerge in a relationship.
Second-graders like school less than first-graders but more than third-graders.
A non-directional hypothesis does not make a specific prediction about the direction of the outcome of the study.
First-, second-, and third-graders will feel differently toward school.

11. Examples Non Directional Null Hypothesis Directional Null Hypothesis
There is no difference between two groups on variable x.
There is no difference among three or more groups on variable x.
There is no relationship between variable x and variable y.
Directional Null Hypothesis
Group A will not have a higher mean score than Group B.
Group A will not have a higher mean score than Group B and Group C.
There is no positive relationship between variable x and variable y.

12. Now, turn the following research question into hypothesis:
RQ: How do teachers feel about special classes for the educationally handicapped?
Hypothesis: Teachers in XYZ School District believe that students attending special classes for the educationally handicapped will be stigmatized. OR
Teachers in XYZ School District believe that special classes for the educationally handicapped will help such students improve their academic skills.

13. Advantages of Stating Hypotheses
It makes us think more deeply and specifically about the possible outcomes of the study.
It involves a philosophy of science as it enables the researcher to make specific predictions based on prior evidence or theoretical argument.
It helps us see if we are investigating a relationship.

14. Disadvantages of Stating Hypotheses
It may lead to a bias on the part of the researcher: when the researcher states a hypothesis, s/he may manipulate the data to obtain the desired outcome.
It may sometimes be unnecessary/inappropriate, such as in descriptive surveys, ethnographic studies, etc.
Focusing on the hypotheses may prevent the researcher from noticing other phenomena which might be important to study.

15. Group Designs in Experimental Research: WEAK Experimental Designs
The One-Shot Case Study:
A single group is exposed to a treatment and a dependent variable is observed to assess the effect of the treatment
Why weak?
Absence of control: the researcher does not know whether the result is due to the treatment
Not possible to compare so the researcher doesn’t know whether the treatment had any effect at all X O
treatment
observation

16. O X observation (pre-test) treatment (post-test)
The One-Group Pretest-Posttest Design:
A single group is observed before and after treatment
Better than one-shot case study (we know whether change occurred), but it is still weak. Why? O X
observation
(pre-test)
treatment
(post-test)

17. X O treatment observation The Static-Group Comparison Design
Two already existing groups (non-equivalent) are used
Although it provides better results, it might still have problems. Why? X O
treatment
observation

18. True Experimental Designs
The Randomized Posttest-Only Control Group Design
Involves two groups which are formed by random assignment: one receives the treatment; the other does not
treatment group R X O
random
treatment
observation
control/
comparison group C
comparison

19. The Randomized Pretest-Posttest Control Group
The difference is in using both pre and post test for both groups
This design helps the researcher see if the random assignment actually succeeded (pretest).
This is particularly important
if the number in each group is small
If the amount of change over time is to be assessed
treatment group R O X
random
observation
(pre-test)
treatment
(post-test)
control/
comparison group C
control/ comparison

20. The Randomized Solomon Four-Group Design
Used to eliminate the possible effect of a pretest
The first group is like pretest-posttest control group; the second is posttest only control group
Best control of threats
Weakness: requires a large sample (subjects are assigned to four groups); requires a lot of energy and effort
Treatment group R O X
random
observation
treatment
Control group C
control

21. QUASI-EXPERIMENTAL DESIGN
It is like experimental design but it doesn’t use random assignment.
It uses other techniques to control threats
There are many techniques used in this design but the most commonly used one in social research is nonequivalent groups design

22. O X observation (pre-test) treatment (post-test) C control/ comparison
Nonequivalent groups design
It is like pretest-posttest randomized experiment but does not use random assignment.
Instead similar groups are used as the treatment and the control groups
E.g. you can pick up two comparable classrooms or schools. O X
observation
(pre-test)
treatment
(post-test) C
control/
comparison
(post-test

23. CONCLUSION
In any type of experimental design, we create two groups that are “equivalent” to each other.
One group gets the treatment; the other does not.
In all other respects the groups are treated the same.
If we observe differences in the outcome between these groups, then we believe that the difference is due to the treatment.
However, in all these designs, we are setting up an artificial situation, so we have to consider all the situation.
If the situation is right, then an experiment can be a very strong design to use.

24. Internal Validity Possible threats to internal validity
A) Selection Bias (subject characteristics)
Might be differences in groups (age, gender, intelligence…)
B) Loss of Subjects (mortality)
Losing some of the subjects (e.g. some subjects may not return the questionnaires)
C) Location
Place in which data are collected or an implementation is conducted may affect the study

25. D) Instrumentation E) Testing
instrument decay (e.g. essay scoring causes fatigue and can be scored differently)
data collector characteristics (age, gender, experience of data collector)
data collector bias (e.g. if the collector knows the hypotheses, s/he might unconsciously distract the data)
E) Testing
Subjects can remember the test if the same is used in pre and post test at a short interval

26. H) Attitude of Subjects
F) History
personal experiences during the course of the study
G) Maturation
E.g. young learners will change due to maturation
H) Attitude of Subjects
The way subjects view the study (e.g. experimental group might perform better because they know they are having a special treatment)

27. I) Regression J) Implementation
e.g. if you include all good students in one group, they will improve no matter what you do
J) Implementation
The researcher can give the experimental group an advantage