Statistics 100 Lecture Set 4

Lecture Set 4 Chapter 5 and 6 … please read Read Chapter 7 … you are responsible for all of this chapter Some suggested problems: –Chapter 4: 4.9, 4.17, 4.21, 4.29 –Chapter 5: 5.3, 5.7, 5.11, 5.15, 5.17, 5.25 –Chapter 6: 6.5, 6.9, 6.11, 6.15, 6.21 –Chapter 7: 7.7, 7.9, 7.13, 7.21

Generally, we have been discussing observational studies These methods look at the current population and try and extract information No attempt is made to change anything or influence the response Note: a sample survey is a type of observational study Observational studies

Observational studies often subject to the presence of lurking variables Observational studies

A quick glance at the weekly ice cream sales in the summer months shows that when sales are high, deaths in recreational boating accidents is also high. Similarly, when sales are low, deaths in recreational boating accidents is also low What does this tell us? Example

Designed experiments: we modify the factors of interest and see the response By manipulating the explanatory variables, we are trying to establish a cause-and-effect relationship between a set of variables and the response of interest Experiments

Response variable: Explanatory variable: Treatment: Some definitions

A study deliberately exposes rats to cigarette smoke to see whether they are more prone to develop lung tumours than unexposed rats Response variable: Explanatory variable: Treatment: Example

Lurking variables are not only found in observational studies This can occur in experiments as well, if you are not careful A properly designed experiment should attempt to account for potential lurking variables Experiments

How should treatments be assigned to subjects? –Self-Selection: subjects choose own treatment –Systematic: assign to subjects based on some characteristic –Random: Use a randomizing device to assign them Experiments

What is the goal the treatment assignment? What do we want our groups to be like when we are done with assigning the treatments? So what do we want to achieve by the assignment? Experiments

Goal is to learn how treatment impacts the response Want treatment to be only real difference between groups Need groups to be as similar as possible No lurking variables favouring one group over another Note: It is called confounding when effects of two variables cannot be distinguished from each other Experiments

Self Selection –Example: exercise vs. medicine for arthritis –Patient decides upon the treatment Back to treatment assignment

Systematic Assignment –Assign to subjects based on some characteristic (say men in one group women in the other) Back to treatment assignment

Random Assignment –Use external device to determine who gets each treatment Back to treatment assignment

So, we have to be careful about how we assign subjects to treatments What about the treatments Often there is one main thing of interest –A new drug –Smoking –Online learning Can we run an experiment with just one treatment? Experiments

One treatment experiments, in addition to being less informative, often suffer from a certain type of confounding called Experiments

The aim of an experiment is to uncover the effect of an explanatory variable on the response variable Randomized Comparative Experiments

Need to have at least two treatments in a comparative experiment A Control is a treatment representing something standard. –Give a patient a placebo (if doing nothing is OK) –Use current best practice (cancer studies) Gives you something to compare the “new” treatment against Randomized Comparative Experiments

Researcher Bias –If you invented a new thing, don’t you hope it works? –If someone pays you to try something for them, don’t you want their thing to work? –Blinding prevents these biases –Double blind studies are those where both the researcher and the subject do not know which treatment is assigned to each subject… done in an to eliminate subjective bias from both subjects and the researchers Randomized Comparative Experiments

Experiments are designed as carefully as surveys Use individuals who represent the population Randomize them to treatments Control against confounding by lurking variables Analyze treatment comparisons using good statistics Use enough subjects to reduce the impact of chance variation

Experiments Basic principles:

Experiments How to interpret results? –Why was the study performed?

Statistical significance Statistical significance:

Statistical significance Suppose that you read a newspaper report that said: – a study at SFU found that there was a statistically significant difference between the obesity rates of breast fed infants compared to those who are not breast fed What does this mean?

Statistical significance We use statistical methods to determine when an observed effect is statistically significant Major area for statistical practice and research

Similar to a jury trial Start with the assumption that all treatments impact the response in the same way (there is no difference) Collect data and compute a statistic that can compare the difference in treatments Assess the size of the effect Reject or do not that the hypothesis that there is no difference in the treatment responses Not guilty Guilty Collect evidence and present to jury Weigh evidence Decide if guilty or not guilty

Statistical significance If you reject the hypothesis that the effects are all the same, the treatments are said to be statistically significant Do you ever PROVE that they are the same or different? Notice you reject or do not reject …

What to do about observational studies? Does smoking cause cancer? The USA Surgeon General (The 1964 Report on Smoking and Health) warned about the link between lung cancer and cigarette smoking Why did it take years until tobacco companies finally stopped battling this claim?

What to do about observational studies? Establishing causation from observational studies

Chapter 6: Experiments in the real world Suppose that we could do an experiment studying smoking in humans. How would we conduct the study?

Chapter 6: Experiments in the real world Experiments require a lot of careful planning and knowledge Goal is to select subjects and assign treatments to minimize bias and variability of comparisons… and to estimate the effects of interest as efficiently as possible

Chapter 6: Experiments in the real world How could we form similar groups of individuals?

Chapter 6: Experiments in the real world Problem: Variability in experimental materials –Subjects are variable Not all have the same potential for response –Conditions are variable Educational outcomes from schools Different regions in a clinical trial –Treatments have characteristics other than what is being studied Macs versus PCs

Chapter 6: Experiments in the real world Try to avoid the variability in subjects

Chapter 6: Experiments in the real world Try to avoid the variability in conditions

Quick Example: Are These the Same? Experiment 1: 10 boys were selected at random Each boy was given a pair of shoes 5 boys received a pair of shoes with the old sole (Sole A) and 5 boys received shoes with the the new sole (sole B) Each boy wears the shoes for 1 month and the amount of wear is measured Experiment 2 10 boys were selected at random Each boy was given a pair of shoes Each pair had 1 shoe with the old sole and 1 shoe with the new sole For each pair of shoes, the sole type was randomly assigned to the right or left foot Each boy wears the shoes for 1 month and the amount of wear is measured

Data

Plot of Responses by Treatment

Some types of experiments Completely randomized experiments:

Example Suppose have 4 different diets which we want to compare (A,B,C and D) Interested in how the diets affect the weight gain in rats undergoing chemotherapy Have 16 rats available for the experiment – 4 for each diet How should we use randomization to assign rats to treatments Note: The rats arrive and are placed in a pen until the experiment begins, at which time they will be placed into cages

Example How about this?

Example

Example

Randomized block designs A block is a group of experimental subjects that are know beforehand to be similar in some way expected to impact the response In a randomized block design, the treatments are randomly assigned to units with the block

Example Suppose have 4 different diets which we want to compare (A,B,C and D) Interested in how the diets affect the weight gain in rats undergoing chemotherapy Have 16 rats available for the experiment – 4 for each diet How should we use randomization to assign rats to treatments Suppose cages are arranged on 4 shelves…4 cages/shelf Note: The rats arrive and are placed in a pen until the experiment begins, at which time they will be placed into cages

Example How about this?

Experiments in the real world Problem: Placebo Effect –Pretending to cure often helps (jury is out on this) –Giving fake pills –Using machine that doesn’t do anything If a real treatment shows positive effect, how do you know that the treatment is helpful beyond this?

Experiments in the real world Way to deal with this is not to tell the patient that they are getting the placebo Called blinding Can interpret the experiment in terms of the comparison with the placebo When a placebo is not possible, use …

Experiments in the real world Another problem: Investigator Bias –Investigators may have a stake in results –Drug companies do their own clinical trials –Clinicians is hired by drug company –You really, really, really want to be right Person who originates experiment often wants a certain answer Solution:

Experiments in the real world Yet Another Problem: Refusers, non-adherers and dropouts –People often refuse to participate in a study What does this do to the study? –People in the study may not properly follow instructions What does this do to the study? –People leave the study, especially longer-term trials What does this do to the study?