1. Chapter 13!
One Brick At A Time!

2. 4 Principles of Experimental Design
Control
Randomization
Replication
Blocking

3. Control
We want to be able to control everything when we do an experiment.
If a factor is not something we are testing, we should remove its effect by equalizing everyone in that group.
So, for example, if we think the part of the country makes a difference, we can just make sure everyone in our experiment is from the same part of the country, so the effect of that factor is removed.

4. Randomization
If we have not controlled a source of variation, then we want to randomize it to remove the effect.
By randomizing, the effect is scattered.
If you do not randomize, then you make the experiment invalid for use with the statistical methods we learn in this class.

5. Replication There are two aspects of replication.
One is that you replicate your experiment enough when you run it.
One is that you or other people could replicate your experiment by creating the same conditions.

6. Replication – Part 1
We need to run our experiment on a large enough sample.
The word replication is used because we are replicating the experiment procedure for each subject/experimental unit.
We need to make sure that there are multiple subjects in each and every category for our experiment.
“Category” will make more sense later.

7. Replication – Part 2
If we controlled a factor but we want to prove our relationship is more general, then we can run our experiment again.
We would want to still control the same variable, but assign it to a different value.
Such as a new part of the country.
If other scientists cannot duplicate our experiment, it will get no respect.

8. Blocking
Sometimes we want to separate our data into groups before we randomize.
This is basically the same as strata from chapter 12.
We will randomize from within certain groups in order to make sure that certain factors are not underrepresented or overrepresented.

9. Blocking Common forms of blocking are gender, income, and age.
Blocking variables will refer to each value for the blocking variable as a block.
For example, male and female are each blocks of the blocking variable gender.
These blocks are sometimes also called levels.

10. Factors
When we do an experiment, every variable we are using as an explanatory variable is called a factor.
Simple experiments use a single factor, and the methods we learn in this class will apply for experiments which use a single factor.

11. Treatments We will have the factors broken down into categories.
For medicines we will use dosage levels.
Sometimes it will be a simple yes and no or present and absent.
Income might go by brackets.
These levels are called treatments.

12. Levels and Categories Blocks and Treatments are both levels.
Blocks are levels of blocking variables.
Treatments are levels of factors.
A category will be a specific combination of blocks and treatments.
For example if we use dosage as a factor and block by gender, then “Males who take a High Dosage” would be a single category.
Each category needs multiple people from the sample.

13. Visual Example Open your books to page 305.
Each of the groups are a category and need a minimum of 2 experimental units.

14. 4 Principles United
The four principles seem, at first, to be contradictions of each other.
They possibly even are.
Each principle is used, however, to address a different source of error or bias that could impact our experiment.
In any given situation, there is a specific principle of experimental design that applies.

15. How Experiments Begin The basis of an experiment is curiosity.
There is something we want to discover.
Then we make a plan for discovering what we want to know.
In the experiments we look at in this class, we will be identifying just one explanatory variable.
Remember: This variable is called a factor.

16. How Experiments Begin
The factor will have different levels, and each level is called a treatment.
We will also have one or more response variables that we will measure.
The response variables will not be split into levels, but will instead just have data gathered for them.

17. How Experiments Begin
Our response variable is impacted by many things, not just the factor we have chosen.
Remember: These other variables are called lurking variables.
The lurking variables are going to come in three basic varieties.

18. Lurking Variables
Sometimes we will recognize ahead of time that a specific lurking variable exists.
The experimental design principle of control is how we deal with these.
By making every subject of the experiment just like the other subjects when it comes to these lurking variables, we can ignore the effect these variables have.

19. Lurking Variables
Sometimes we will identify a lurking variable, but focusing on just one group might make it less valid.
For example, if gender was a lurking variable, it might hurt the validity of our study if we only include males or females.
So, we rig things so that each group definitely gets some representation in our study.

20. Lurking Variables
When we force these groups to be represented, this is considered to be blocking.
Blocking removes a certain element of randomness, but replaces it with an element of fairness.
This only works for lurking variables we can measure ahead of time.

21. Lurking Variables
Sometimes we cannot measure a lurking variable ahead of time and there are also lurking variables we were not even aware of.
That is when we randomize.
Once we have done our blocking and our controlling, we will randomly assign subjects in each block to their treatment groups.

22. Lurking Variables This scatters the effect of these lurking variables.
In order to make sure it does a good enough job of scattering the variables, we have to make sure we have enough subjects to randomize.
This is the first part of replication – having enough people so our randomness works.

23. Lurking Variables How many is enough?
Every treatment group needs at least two subjects…in each block.
So if you have three treatments, then each block you made needs at least 6 subjects.
3 treatments x 2 subjects each = 6 subjects
So the first part of replication is there to make sure that our randomizing can do its job.

24. Lurking Variables There is another side of replication as well.
It is the ability to re-run our experiment using new values for all of the lurking variables we controlled.
In other words we will control the same variables, but we will control them with new values.
This replication makes sure that our controlling process was valid.

25. 4 Principles United!
When we do an experiment, we have a factor of interest.
Other influences are either controlled or blocked if we can help it, and randomized to cover anything we could not or did not.
We replicate with a large enough sample so the random part works and we replicate the experiment more than once to validate our controls.

26. Intermission
We will now seriously have a 2 minute intermission.

27. Significance There are two kinds of significance we will look at.
The first one is statistical significance.
This means that the methods of statistics strongly support that there is a difference between two groups.
Remember: We cannot prove there is a difference, but can only support it with evidence.

28. Significance There is another version of significance.
It is the one you already know. (I hope.)
If something is significant, it has meaning.
In other words, practical significance refers to the difference being a meaningful difference.
Usually we will focus on how much of a difference there is.

29. Significance So consider this example…
We run an observational study to find out how the math grades for redheads compare to those of other hair colors.
After reviewing thousands of students, using valid procedures, our evidence suggests that there is a difference.
Specifically, the average for redheads is higher than the overall average.

30. Significance
That would be statistically significant, because our statistical methods did strongly support a difference.
Now, let’s say this difference was a higher average by 1%.
In other words, redheads get 1% higher than other hair colors in math, on average.
Is a 1% advantage high enough to matter?

31. A Second Control
Since two forms of replication and two versions of significance were not already confusing enough, we will introduce a second version of control.
The first version of control was that we made everyone the same for variables other than what we were looking at.
This was used to help control lurking variables.

32. A Second Control
In an experiment, we might have what is known as a control treatment and a control group.
This version of control refers to a treatment where we did nothing.
If we were handing out medicine, they get no medicine.
If we were giving plants different amounts of fertilizer, these plants get no fertilizer.

33. A Third Control?
Sometimes instead of doing nothing, a control treatment needs to find clever ways to do nothing.
The placebo, or sugar pill, is an example.
Not giving any medication is one control group, and the placebo group is another control group.
This is still the second kind of control, but is just an additional way to do it.

34. Assignments We will go into more details tomorrow.
Ch. 12: Choose 2 problems from 1 through 10, and also do problems 11 and 12.
Due Friday
1st Ch. 13 Quiz Friday
Ch. 13: Design an experiment using the following conditions:
The experiment must be business related, but it can totally be a made-up business.
The experiment must involve testing a new product in comparison to an existing product or it must be testing conditions that affect the productivity of workers.
We will go into more details tomorrow.
Second Ch. 13 Quiz on Wednesday after break.

35. Quiz Bulletpoints (After Break)
Given an experimental design prompt, be able to determine a sensible response variable.
Given an experimental design prompt, be able to list treatments for the suggested factor.
Given an experimental design prompt, be able to determine three sensible lurking variables to control.
Given an experimental design prompt, be able to defend whether or not a particular variable would be better for control or blocking.
Given an experimental design prompt, be able to determine one sensible blocking variable other than gender, age, or income level.

36. Quiz Bulletpoints (This Week)
Know the best way to randomize a sample, if you were taking the AP Test.
Know the difference between a nonresponse bias, a response bias, and undercoverage bias.
Know the four principles of experimental design, especially control and blocking.
Know the difference between statistical significance and practical significance.
Know the difference between levels, factors, and treatments.