1. Experimental design

2. Experimental design Outlines What is an experiment?
Controlling the experiment
Design categories (Experimental, Quasi-experimental and Ex Post Facto)
Comparison
Meta analysis
Examples

3. Experimental design
Experiment Ordered investigation that attempts to prove or disapprove a hypothesis
Questions 
Hypothesis 
Experiment 
Analysis 
Conclusion
Scientific Method Components

4. Experimental design

5. Experimental design So .. The hypothesized relationship is
Independent variable -> dependent variable

6. Experimental design
An independent variable is one that the researcher studies as a possible cause of something else.
Dependent variable is a variable that is potentially influenced by the independent variable. 
Manipulation: Researcher manipulates the independent variable ( treatment – intervention)

7. Experimental design So .. The hypothesized relationship is
Independent variable -> dependent variable

8. Experimental design So .. The hypothesized relationship is
Independent variable -> dependent variable
Certain Drug impact on cancer

9. Experimental design
A researchers can convincingly identify cause-and-effect relationships by experimental design
Many possible factors that might cause or influence a particular condition or phenomena.
The researcher attempts to control for all influential factors except the study factor.

10. Controlling for confounding variables
1- keep something constant
2- Include a control group
3- Randomly assign people to group
4- Assess equivalence before the treatment with one or more pretests .
5- Expose participants to all experimental conditions
6- statistically control for confounding variables.

11. Internal Validity
Regarding cause-effect or causal relationships, internal Validity is the approximate truth about inferences.
It is relevant in studies that try to establish a causal relationship.
The key question in internal validity is whether observed changes can be attributed to your program or intervention (i.e., the cause) and not to other possible causes.
Did the treatment cause the outcome to occur? Or were there other confounding factors that caused the outcome?
shorted

12. External Validity
Are the findings unique to just participants we studied or could they apply to other groups?
Refers to the extent to which the results of an experiment can be generalized across populations, time and settings.
shorted

13. Single Group Threats
Imagine that we are studying the effects of an education program in mathematics for first grade students on a measure of math performance such as a standardized math achievement test.
In the post-only design, we would give the first graders the program and then give a math achievement posttest.
Consider what would happen if you observe a certain level of posttest math achievement or a change or gain from pretest to posttest.
You want to conclude that the outcome is due to your math program. How could you be wrong?

14. Single Group Threats
History Threat: It's not your math program that caused the outcome, it's something else, some historical event that occurred. 
Maturation Threat:  The children would have had the exact same outcome even if they had never had your special math training program. All you are doing is measuring normal maturation or growth in understanding that occurs as part of growing up. 
Testing Threat: This threat only occurs in the pre-post design. What if taking the pretest made some of the children more aware of that kind of math problem -- it "primed" them for the program so that when you began the math training they were ready for it in a way that they wouldn't have been without the pretest.

15. Single Group Threads
Instrumentation Threat: Like the testing threat, this one only operates in the pretest-posttest situation. What if the change from pretest to posttest is due not to your math program but rather to a change in the test that was used? 
Mortality Threat:  It means that people are "dying" with respect to your study.
Regression Threat: Also known as a "regression artifact" or "regression to the mean" is a statistical phenomenon that occurs whenever you have a nonrandom sample from a population and two measures that are imperfectly correlated.

16. Design Categories Pre-experimental designs True Experimental designs
Pre-experimental designs
True Experimental designs
Quasi-experimental designs
Ex post facto designs
Factorial designs

17. Design Categories
Some are true experimental designs as such they allow us to identify cause-and-effect 
Some give alternative explanations of an observed change.

18. Design Categories
All of the designs have one thing in common : clearly identify independent and dependent variable.

19. Design Categories
We illustrate the designs using tables that have this general format "Table"
Group Time ->
Group 1
Group 2

20. Pre- Experimental Design
The cells have one of four notations:
Tx: Indicates that a treatment is presented
Obs: Indicates that observation is made
: Indicates that nothing occurs during a particular time period.
Exp: Indicates a previous experience

21. Pre- Experimental Design: One-shot Experimental Case Study
The most primitive type of experiment that might be termed "research“
The design has low internal validity 
1- the characteristics or the behavior observed after the treatment existed before the treatment as well.
2- influenced by other factors 
3- a single measurement or observation doesn't guarantee that situation has change or not. 
Group 1 Tx
Obs

22. Pre- Experimental Design: One-shot Experimental Case Study
Example ..
The design will be something like this :
Exposure to cold + Damp ground (TX)-> Child has a cold (Obs)
One-shot experimental case study is simple to carry out, its results are meaningless.

23. Pre- Experimental Design: One- Group Pretest-posttest Design
We know that a change has taken a place. But we have not ruled out other possible explanation for the change. 
Group 1
Obs Tx

24. Pre- Experimental Design: Static Group Comparison
Involves both an experimental group and a control group.
No attempt is made to check wither they are similar or not before the treatment so no way to know if the treatment actually causes any differences between the two groups.  
Group 1 Tx
Obs
Group 2

25. Types of Design
If random assignment is used, we call the design a randomized experiment or true experiment.
If random assignment is not used, then we have to ask a second question: Does the design use either multiple groups or multiple waves of measurement?
If the answer is yes, we would label it a quasi-experimental design. If no, we would call it a non-experimental design. 

26. True Experimental Design
True experimental design is regarded as the most accurate form of experimental research, in that it tries to prove or disprove a hypothesis mathematically, with statistical analysis.
For an experiment to be classed as a true experimental design, it must fit all of the following criteria:
The sample groups must be assigned randomly.
There must be a viable control group.
Only one variable can be manipulated and tested. The tested subjects must be randomly assigned to either control or experimental groups.

27. True Experimental Design: Pretest-Posttest Control Group Design
The pretest-posttest equivalent groups design provides for both a control group and a measure of change but also adds a pretest.
It is important that the two groups be treated in a similar manner.
Random Assignment
Group 1
Obs Tx
Group 2

28. True Experimental Design: The Solomon Four-Group Design
The Solomon Four-Group Design is designed to deal with a potential testing thread.
 This design has four groups.
 Two of the groups receive the treatment and two do not. 
Random Assignment
Group 1
Obs Tx
Group 2 -
Group 3
Group 4
clip

29. Quasi-Experimental Designs
Sometimes, randomness is either impossible or impractical. In those situations use quasi-experimental design.
However, has features that can eliminate many threats to internal validity.
Used frequently in evaluation because:
Often randomization is impossible or difficult
Ethical/legal prohibitions against randomization
No viable control group available
Inadequate resources to conduct randomization
clip

30. Quasi-Experimental Designs: Nonrandomized Control Group Pretest-Posttest Design
To show that two groups are equivalent with respect to the dependent variable prior to the treatment, thus eliminating initial group differences as an explanation for post-treatment differences.
Differs from experimental designs because test and control groups are not totally equivalent; equivalence on the pretest ensures equivalence only for variables that have specifically been measured.
clip
Group 1
Obs Tx
Group 2 -

31. Quasi-Experimental Designs: Nonrandomized Control Group Pretest-Posttest Design
Threats to validity:
Partly controls for history threat(external event would affect both groups, provided groups, provided groups are similar), maturation, instrument threats and instrumentation threats.
However, even if groups are statistically very similar, if the intervention is given to volunteers they may behave differently than control group(due to self-selection).
Regression threat.
clip

32. Quasi-Experimental Designs: Simple time-series experiment
To show that, for a single group change occurs during a lengthy period only after the treatment has been administered.
Provides a stronger alternative to “One group pretest-posttest design; external validity can be increased by repeating the experiment in different places under different condition.
Group 1
Obs Tx
clip

33. Quasi-Experimental Designs: Simple time-series experiment
Threats to validity:
Extension of pretest-posttest design but reduces maturation, testing, regression threats
No control group so no selection threat.
History threat is controlled partially.
if measurement changes around time of program instrumentation threat may be present
clip

34. Quasi-Experimental Designs: Control group, time-series design
Bolstering the internal validity of the preceding design with the addition of a control group.
Involves conducting parallel series of observations for experiment and control groups.
Group 1
Obs Tx
Group 2 -
clip

35. Quasi-Experimental Designs: Reversal, time-series design
Showing , in a single group or individual, that a treatment consistently leads to a particular effect.
Is an on-again, off-again design in which the experimental treatment is sometimes present, sometimes absent.
Group 1 Tx
Obs -
clip

36. Quasi-Experimental Designs: Alternating treatments design
Showing , in a single group or individual, that different treatments have different effects.
Involves sequentially administrating different treatments at different times and comparing their effects against the possible consequent of non-treatment.
Group1
Txa
Obs -
Txb
clip

37. Quasi-Experimental Designs: Multiple-baseline design
Showing, the effect of a treatment by initiating at different times for different groups or individuals, or perhaps in different setting for a single individual.
Involves tracking two or more groups or individuals over time, or tracking a single individual in two or more settings, for a lengthy period of time, as well as initiating the treatment at different times for different groups, individuals, or settings.
clip
Group1 -
Obs Tx
Group2

38. Quasi-Experimental Designs: Alternating treatments design
Showing , the effect of a treatment by initiating it at different times for
Involves sequentially administrating different treatments at different times and comparing their effects against the possible consequent of non-treatment.
Group1
Txa
Obs -
Txb
clip

39. Ex Post Facto Designs Sometimes we can’t manipulate some variables
Impossible, e.g.: Characteristics
Unethical, e.g.: Virus
Ex post facto = after the fact
Already happened in the past

40. Ex Post Facto: Simple Design
Possible effect of an experience/condition that occurred in the past
May show difference but not conclusive
Group Time ->
Group 1
Exp
Obs
Group 2 -

41. Factorial Designs 2+ independent variables
Simultaneously or sequential
Time ->

42. Factorial Design: Two-factor experimental design
Group Time ->
Random Assignment
Group 1
Tx1
Tx2
Obs
Group 2 -
Group 3
Group 4

43. Factorial Design: Two-factor experimental design
Group Time ->
Random Assignment
Group 1
Tx1
Tx2
Obs
Group 2 -
Group 3
Group 4

44. Factorial Design: Two-factor experimental design
Group Time ->
Random Assignment
Group 1
Tx1
Tx2
Obs
Group 2 -
Group 3
Group 4

45. Factorial Design: Two-factor experimental design
Group Time ->
Random Assignment
Group 1
Tx1
Tx2
Obs
Group 2 -
Group 3
Group 4

46. Factorial Design: Two-factor experimental design
Group Time ->
Random Assignment
Group 1
Tx1
Tx2
Obs
Group 2 -
Group 3
Group 4

47. Factorial Design: Two-factor experimental design
Group Time ->
Random Assignment
Group 1
Tx1
Tx2
Obs
Group 2 -
Group 3
Group 4

48. Factorial Design: Two-factor experimental design
Group Time ->
Random Assignment
Group 1
Tx1
Tx2
Obs
Group 2 -
Group 3
Group 4

49. Factorial Design: Two-factor experimental design
Group Time ->
Random Assignment
Group 1
Tx1
Tx2
Obs
Group 2 -
Group 3
Group 4

50. Factorial Design: Two-factor experimental design
Example: Clustering using K-means
Tx1: A different method to choose initial centroids
Tx2: A different formula to calculate distance

51. Factorial Design: Combined experimental and ex post facto design
Group Time ->
Group1
Expa
Random Assignment
Group 1a
Txa
Obs
Group 1b
Txb
Group2
Expb
Group 2a
Group 2b

52. Factorial Design: Combined experimental and ex post facto design
Example: Clustering using K-means
Expa, Expa: 2 different kinds of datasets
Txa, Txb: 2 different formulas to calculate distance

53. Comparison

54. Meta-analyses Replication Quantitative analysis of the analyses
Primarily statistical

55. Examples & Discussion