EXPERIMENTAL RESEARCH

A very common way of research design A good design to establish cause and effect relationships among variables It is the type of research that directly attempts to influence a particular variable tests hypotheses about cause-and-effect relationships

Major Characteristics of Experimental Research The researcher looks at the effect(s) of at least one independent variable (experimental or treatment variable) on one or more dependent variables (criterion or outcome variable) The researcher manipulates the independent variable, which is not the case in other research designs Aim of the researcher is to see whether the treatment/manipulation has made a difference.

Some independent variables: Methods of instruction Types of assignment Learning materials Types of questions asked by teachers… Some dependent variables: Achievement Interest Motivation Attitudes…

Comparison of Groups Experimental research involves two groups of subjects Experimental group Control / comparison group Experimental group receives the treatment; control group receives no treatment / comparison group receives a different treatment Comparison group rather than control group in educational research

Manipulation of the Independent Variable The researcher actively manipulates the independent variable: s/he decides the nature of the treatment, to whom and to what extent it will be applied, and when, where and how it will be done Variables that can be manipulated: teaching method, learning activities, materials, etc Variables that cannot be manipulated: gender, ethnicity, age, etc

Ways of establishing independent variables A) one form of the variable versus another E.g. comparing the inquiry method vs the lecture method in teaching listening B) presence versus absence of a particular form E.g. comparing the use of videos vs no videos in teaching speaking C) varying degrees of the same form Comparing the effects of different amount of homework on students’ motivation

Hypotheses A hypothesis is a prediction of the possible outcomes of a study. E.g. RQ: Will students who are taught English by a teacher of the same gender like the subject more than students taught by a teacher of a different gender: Hypothesis: Students taught English by a teacher of the same gender will like the subject more than students taught English by a teacher of a different gender.

Null hypotheses A hypothesis is a prediction of the possible outcomes of a study. It states our expectations in a positive sense. There will be a difference between the groups. A null hypothesis states our expectations in a negative sense. There will be no difference between the groups

Directional vs Non-directional Hypotheses A directional hypothesis shows a specific direction (higher, lower, more…) that a researcher expects to emerge in a relationship. Second-graders like school less than first-graders but more than third-graders. A non-directional hypothesis does not make a specific prediction about the direction of the outcome of the study. First-, second-, and third-graders will feel differently toward school.

Examples Non Directional Null Hypothesis Directional Null Hypothesis There is no difference between two groups on variable x. There is no difference among three or more groups on variable x. There is no relationship between variable x and variable y. Directional Null Hypothesis Group A will not have a higher mean score than Group B. Group A will not have a higher mean score than Group B and Group C. There is no positive relationship between variable x and variable y.

Now, turn the following research question into hypothesis: RQ: How do teachers feel about special classes for the educationally handicapped? Hypothesis: Teachers in XYZ School District believe that students attending special classes for the educationally handicapped will be stigmatized. OR Teachers in XYZ School District believe that special classes for the educationally handicapped will help such students improve their academic skills.

Advantages of Stating Hypotheses It makes us think more deeply and specifically about the possible outcomes of the study. It involves a philosophy of science as it enables the researcher to make specific predictions based on prior evidence or theoretical argument. It helps us see if we are investigating a relationship.

Disadvantages of Stating Hypotheses It may lead to a bias on the part of the researcher: when the researcher states a hypothesis, s/he may manipulate the data to obtain the desired outcome. It may sometimes be unnecessary/inappropriate, such as in descriptive surveys, ethnographic studies, etc. Focusing on the hypotheses may prevent the researcher from noticing other phenomena which might be important to study.

Group Designs in Experimental Research: WEAK Experimental Designs The One-Shot Case Study: A single group is exposed to a treatment and a dependent variable is observed to assess the effect of the treatment Why weak? Absence of control: the researcher does not know whether the result is due to the treatment Not possible to compare so the researcher doesn’t know whether the treatment had any effect at all X O treatment observation

The One-Group Pretest-Posttest Design: A single group is observed before and after treatment Better than one-shot case study (we know whether change occurred), but it is still weak. Why? O X observation (pre-test) treatment (post-test)

The Static-Group Comparison Design Two already existing groups (non-equivalent) are used Although it provides better results, it might still have problems. Why? X O treatment observation  

True Experimental Designs The Randomized Posttest-Only Control Group Design Involves two groups which are formed by random assignment: one receives the treatment; the other does not treatment group   R X O random treatment observation control/ comparison group C comparison

The Randomized Pretest-Posttest Control Group The difference is in using both pre and post test for both groups This design helps the researcher see if the random assignment actually succeeded (pretest). This is particularly important if the number in each group is small If the amount of change over time is to be assessed treatment group   R O X random observation (pre-test) treatment (post-test) control/ comparison group C control/ comparison

The Randomized Solomon Four-Group Design Used to eliminate the possible effect of a pretest The first group is like pretest-posttest control group; the second is posttest only control group Best control of threats Weakness: requires a large sample (subjects are assigned to four groups); requires a lot of energy and effort Treatment group   R O X random observation treatment Control group C control

QUASI-EXPERIMENTAL DESIGN It is like experimental design but it doesn’t use random assignment. It uses other techniques to control threats There are many techniques used in this design but the most commonly used one in social research is nonequivalent groups design

O X observation (pre-test) treatment (post-test) C control/ comparison Nonequivalent groups design It is like pretest-posttest randomized experiment but does not use random assignment. Instead similar groups are used as the treatment and the control groups E.g. you can pick up two comparable classrooms or schools. O X observation (pre-test) treatment (post-test) C control/ comparison (post-test

CONCLUSION In any type of experimental design, we create two groups that are “equivalent” to each other. One group gets the treatment; the other does not. In all other respects the groups are treated the same. If we observe differences in the outcome between these groups, then we believe that the difference is due to the treatment. However, in all these designs, we are setting up an artificial situation, so we have to consider all the situation. If the situation is right, then an experiment can be a very strong design to use.

Internal Validity Possible threats to internal validity A) Selection Bias (subject characteristics) Might be differences in groups (age, gender, intelligence…) B) Loss of Subjects (mortality) Losing some of the subjects (e.g. some subjects may not return the questionnaires) C) Location Place in which data are collected or an implementation is conducted may affect the study

D) Instrumentation E) Testing instrument decay (e.g. essay scoring causes fatigue and can be scored differently) data collector characteristics (age, gender, experience of data collector) data collector bias (e.g. if the collector knows the hypotheses, s/he might unconsciously distract the data) E) Testing Subjects can remember the test if the same is used in pre and post test at a short interval

H) Attitude of Subjects F) History personal experiences during the course of the study G) Maturation E.g. young learners will change due to maturation H) Attitude of Subjects The way subjects view the study (e.g. experimental group might perform better because they know they are having a special treatment)

I) Regression J) Implementation e.g. if you include all good students in one group, they will improve no matter what you do J) Implementation The researcher can give the experimental group an advantage

EXERCISES

1. a) How is the research designed? Exercise 1. A researcher uses his morning and afternoon history classes to test a new instructional strategy. In the morning class the students are given a pretest to assess their knowledge of WWI and then they are given the opportunity to learn about this topic using a computerized program. In the afternoon class the students are also given a pretest; they learn about WWI through discussion. At the end of six weeks, a final exam is given to both classes. 1. a) How is the research designed? Quasi-experimental (non-equivalent pre-test/post- test) Gr. A (NR) O X1 O Gr. B (NR) O X2 O

A researcher uses morning and afternoon history classes to test a new instructional strategy. In the morning class the students are given a pretest to assess their knowledge of WWI and then they are given the opportunity to learn about this topic using a program. In the afternoon class the studecomputerized nts are also given a pretest; they learn about WWI through discussion. At the end of six weeks, a final exam is given to both classes. 1. b) How can you analyze the data? t-test (comparison of the means in pre- and post-tests for both groups)

How can we present this data? Table 1 Students’performance results   N mean s.d. t Pre-test Experimental 31 7.23 .88 3.45* Control 38 7.10 .81 Post-test 9.78 .96 3.88* 7.88 .89 * p< .05

Exercise 2. A social studies teacher is interested in determining the effects of a new textbook series, which has a multicultural thematic approach. The teacher will teach a unit on colonization in America. After the unit, a multicultural attitude scale will be administered to students. 2. a) How is the research designed? one-shot case study

A social studies teacher is interested in determining the effects of a new textbook series, which has a multicultural thematic approach. The teacher will teach a unit on colonization in America. After the unit, a multicultural attitude scale will be administered to students. 2. b) How can you analyze the data? Descriptive statistics

Exercise 3 A school administration wants to field-test a Methodology curriculum in the ELT course that is offered in the summer. Two classes are to be offered during the summer session. Students in both classes will be given a pretest to measure their knowledge of Methodology. One class will study Methodology using traditional textbook materials with a mostly lecture-style of delivery. Another class, an experimental class, will study Methodology using films and contemporary materials along with presentations by guest speakers. The final exam for the course, a parallel form of the pretest, will be given to both groups. 3. a) How is the research designed? Quasi-experimental (pre-test/post-test nonequivalent comparison groups)

A school administration wants to field-test a Methodology curriculum in the ELT course that is offered in the summer. Two classes are to be offered during the summer session. Students in both classes will be given a pretest to measure their knowledge of Methodology. One class will study Methodology using traditional textbook materials with a mostly lecture-style of delivery. Another class, an experimental class, will study Methodology using films and contemporary materials along with presentations by guest speakers. The final exam for the course, a parallel form of the pretest, will be given to both groups. 3. b) How can you analyze the data? t-test between and within groups

For the following exercises, work in pairs and describe the research design answering following questions: 1. How will the subjects be assigned to groups? 2. Identify relevant variables 3. Describe the treatment design 4. Identify the way the data will be analyzed.

Exercise 4 With the current emphasis on parental involvement in children’s academic endeavors, a researcher is interested in assessing the confidence level of parents who will read orally to children in schools. He also wants to know if there is a difference in the confidence level of parents of different education levels. From a list of volunteers who are interested in reading orally to students, parents will be randomly selected and assigned to one of two conditions—training or no training. Following training of parents in the experimental condition, all parent volunteers will complete a confidence scale. Describe this study using a true experimental posttest-only control group design.

Random assignment for two groups: give a number to each and draw out a number for Group A and then Group B until you finish all the numbers. Indep variable: training parents Dep variable: parents’ confidence level Group A will get training whereas Group B will not. Parents in both groups will read orally to children and then they will be given a confidence scale. T-test for the mean scores obtained from this scale for each group will be found out to see the effect of training on confidence level.

Please read a few article which used experimental research method!!