1. Sampling Design Corresponds to Chapter 2 in Triola
Unit 1.2
Sampling Design
Corresponds to Chapter 2 in Triola

2. Overview of Section 1.2 Observational Studies vs. Experiments
Issues with Designing an Observational Study
Representativeness
Timing the Study
Issues with Designing Good Experiments
Randomization
Control
Replication
Blinding

3. The basics of creating a study
Observational Study vs. an Experiment
An observational study is designed to observe and measure specific characteristics- no treatment
Treatment- An activity meant to make some effective change in the attribute being measured
Experiment- A treatment is applied and then the experimenter proceeds to observe its effects

4. Issues in Designing Observational Studies

5. Sampling Error
A non- sampling error- the difference between a sample result and the true population result
Sampling Error- This occurs when data are incorrectly collected. For example, if we collected a sample
Non-sampling error is inevitable and the law of large numbers allows us to minimize problems from non-sampling error. Sampling Error is bad research design and we want to eliminate sampling error as much as possible.

6. Sampling Bias
Sampling Bias is a kind of non-sampling error that can invalidate the results of your study.
A bias is skewing of the data towards one conclusion or another that is not a result of the actual views or tendencies of a population.
We want to avoid sampling bias as much as possible so that our results can be robust.

7. Representativeness
When conducting an observational study, the single most important factor in creating the study is the representativeness of the sample.
By representativeness I mean, is your sample selected in such a way as to insure that all of the people that you want to know something about have a good chance of being selected for the study.
We can either make sure that you get a representative sample by how you select your sample, or you can use randomness to select your sample.

8. Random Samples
Random Sample- Members from the population are selected in such a way that each individual member in the population has an equal chance of being selected.
Simple Random Sample- N subjects is selected in such a way that every possible sample of the same size n has the same chance of being chosen.
We are going to be discussing samples a lot and the simple random sample is one of the primary requirements for us to be able to run our statistical tests.
We want to make sure that we can select any sample of a particular size for our study because the mathematics that we are going to use allows us to draw conclusions about any sample of a particular size.

9. Voluntary Sample
Voluntary Sample- Individuals select themselves for participation in the study or treatment
In general, a voluntary sample has all sorts of problems because the persons in the sample self-select. For example, if you conducted a study on an individual’s political opinions through a voluntary sample, you would probably only get individuals who had strong political beliefs. The sample would most likely not be representative.

10. Convenience Sampling
A convenience sample is one where the researcher measures particular attributes from a sample that is readily at hand.
By readily at hand that means the people that are most easily available for testing.
A convenience sample is hardly ever done for scientific purposes because it is easily subjected to sampling bias. For example, if you were to stand outside of the student store and ask students their opinions about increasing to help with schools, your sample might be skewed towards being in favor because the students have a vested interest in more money for schools.

11. Probability Sample
Sometimes it is necessary to stack the deck in order to make sure that you have certain members of the population represented in your study.
A probability sample involves selecting members from a population in such a way that each member of the population has a known(but not necessarily the same) chance of being selected.
For example, weighting women in a study so that there is a greater likelihood that they will be chosen in order to insure their representativeness
What I mean by representation is that we are actually getting information from that segment of the population.

12. Systematic Sampling
Systematic Sampling- Selecting every kth (such as every 50th) member of the population
Systematic Sampling is a way of getting randomization if the individuals in whatever group that you are studying have sufficiently similar characteristics and are somewhat evenly distributed in your population.
Systematic sampling is not an effective way of getting randomization if you have individuals with similar characteristics who are bunched together.
For example, suppose that you wanted to know the buying habits of people in Orange County. if you selected every 50th person in the Santa Ana phonebook this would not be a good way of sampling. Since Santa Ana is 95% Latino and of generally lower socioeconomic status, you would most likely only be getting the views of Latinos or people with lower incomes. Hence your sample would not be representative.

13. Stratified Sampling
Stratified Sampling- Subdivide the population into at least two different subgroups (or strata) so that subjects in the same subgroup have similar characteristics and then the sample is drawn from the subgroup.
A common way of doing stratified sampling would be to divide a group of individuals by ethnicity. Then we would select an equal amount of people from each ethnicity to sample. This would insure that we have representativeness of all of the ethnicities in our study. Because I have tested all ethnicities, I can factor out ethnicity as a possible lurking variable.

14. Cluster Sampling
Cluster sampling- divide the population area into sections ( or clusters), then randomly select some of those clusters and then choose all the members from those clusters.
For example, suppose that I wanted to study the effects of statistics workshops on student grades. I could separate all of the individuals who are taking Math 219 by the class that they are in. These are the clusters. Then I could select 5 classes to do my measurements in. This could insure that I get a good cross-section of instructors and hence remove that issue as a lurking variable.

15. Timing of a Study
One way to eliminate sampling error in an observational study is to choose a timing that fits with the particulars of your study.

16. Timing of a study
Cross-Sectional Study- Data are observed, measured, and collected at one point in time
Retrospective- Collected from the past by going back in time (through examination of previously collected data)
Prospective Study (also called a longitudinal or cohort study)- Collected throughout a set period of time- generally demonstrates change or persistence

17. Designing experiments
1.2b

18. Basic Vocabulary
Treatment or Experimental group- Those individuals selected for your study who will have the particular variable that you are concerned with changed. For example, they are given the drug that you are looking to test.
Control group- These are the individuals who will not be given the treatment that you are looking to test.
Placebo- A pill or intervention that is meant to have the members of the control group think that they are given the treatment. Literally, a placebo is a sugar pill that is used to make a sick person think that they are getting medicine.
The Placebo Effect- When a person who is given a placebo shows improvement in the variables under observation because they think that they have been given a treatment.

19. Bias
Scientists are human beings and would like for their treatments to show effects. However, the community of science needs to have robust outcomes in order to build knowledge. Hence, processes are put into place by the scientific community in order to make sure that there is no systematic bias. A bias is a skewing of the data towards a particular outcome due to variables that are not the subject of the study. A systematic bias is a bias in which the scientist, either knowingly or unknowingly, skews the data by how they construct the experiment.

20. Ways of Preventing Bias in Experimental Studies

21. Randomization
Subjects are assigned to different groups through a process of random selection.
Suppose that we wanted to conduct an experiment where we tested the effect of a new drug on treating the common cold. We could randomly assign members to the treatment group and the experimental group and that would help us to insure that there was no systematic bias in our samples.

22. Randomized Block Design
Selecting subjects that have a similar characteristic and then sampling from each of those blocks.
For example, a study on political opinions is conducted in which the study’s participants are grouped according to their political parties. Then, a random sample of equal size is selected from each of the groups and then tested.

23. Completely Randomized design
Selecting members of the population through a process of random selection.
For example, a random number generator picks student ID numbers from SAC students and assigns those persons to either treatment or placebo group.
If the sample is of sufficient size then this is the most common way of designing an experiment.

24. Rigorously Controlled Design
Making sure each of the members of a sample have a similar member of the control for all of the important characteristics involved in the study.
For example, if a 40-year old overweight man who smokes is selected for the treatment group of a health study, then a second overweight man who smokes will be chosen for the placebo group.

25. Matched Pair Design
Compare two treatment groups where the subjects are matched in such a way that sets of two subjects are identical in some way.
For example, a set of twins are chosen for a toothpaste study. One twin is placed into the treatment group and the second twin is placed into the placebo group.
A more common version of this is a study in which a participant is tested at the beginning of the study and then that same characteristic is tested at the end of treatment time or at some point during the treatment time.

26. Replication The repetition of an experiment on more than one subject.
The law of small numbers- Samples that are too small will have too much variation to accurately draw inferences about the population.

27. Blinding
The subject does not know that they are a part of the treatment or the control group.
The Placebo Effect- The fact that individuals who know that they are taking a medicine may improve because of their belief that the medicine is effective.

28. Problems in Experimental and Observational Design
Confounding occurs in an experiment when you are not able to distinguish amongst the effects of different factors.
A factor is a possible variable that could have an effect upon an outcome but is not accounted for in the model or in our conclusions.
When we draw assumptions about a result and do not take into account possible confounders, we are committing an error. We may apply cause to a situation where cause is not warranted

29. Confounding Variables
Example: In the early 50’s, health researchers concluded that excessive amounts of saturated fat was responsible for heart disease. This conclusion was based upon the fact that as Americans ate more saturated fat, there was also a commiserate increase in heart disease. However, the researchers did not account for the fact that processed sugar consumption also rose with saturated fat consumption. The amount of sugar consumption is called a confounder.

30. Review of Section 1.2 Observational Studies vs. Experiments
Issues with Designing an Observational Study
Representativeness
Timing the Study
Issues with Designing Good Experiments
Randomization
Control
Replication
Blinding