SP 2015 CP PROBABILITY & STATISTICS Observational Studies vs. Experiments Chapter 11

Observational Study vs. Experiment Observational Study Observe only, no manipulation of factors is used Experiment Factors are manipulated to create treatments and randomly assign subjects to the different treatments

Observational Studies Simply observing regular occurrences Researchers do not assign choices Can be RETROSPECTIVE or PROSPECTIVE

Observational Studies Useful for: Discovering trends and potential relationships Used in public health and marketing Observational Studies DO NOT demonstrate cause-and-effect relationships

Retrospective Study Not based on random samples Typically used to estimate:  Differences between groups  Associations between variables Typically focus on small segment of entire population

Retrospective Study - STEPS Steps: Select Subjects Determine their previous conditions/behaviors  Exam historic information  Pull records from data bases/sources (prior grades, classes, etc...)  Ask subjects questions in order to gather knowledge of past events

Retrospective Study - EXAMPLE Example: Identify people with a disease Look at their history, heritage to determine things which may be related to their condition.

Retrospective Study - WARNING Data collected through asking subjects to recall past events tends to have errors!!!!!! What did you eat last week, exactly?

Retrospective Study – WARNING 2 Lurking Variables

Prospective Study Observe subjects over time Identifying subjects in advance Collecting data as events unfold No treatments/conditions are intentionally controlled

Example

Experiments Manipulate factors to create treatments Can show cause-and-effect relationships Requires random assignment of subjects to treatments Subjects are also referred to as Participants/ Experimental Units Compares results from different treatment groups

Experiments - BASICS Requires a random assignment of subjects to treatments Study the relationship between two or more variables Must have at least one of each: Explanatory variable (Factor) to manipulate & Response variable to measure

Experimenter - ROLE Actively and deliberately manipulates factors to control the details of the possible treatments RANDOMLY assigns the subjects to the treatments Observes the response variable Compares responses for different groups of subjects who have been treated differently.

Experiments - WHO Humans are commonly called subjects or participants Other individuals (rats, days, petri dishes of bacteria) are commonly referred to by the more generic term experimental units.

Experiments - WHAT Factor – Variable whose levels are manipulated by the experimenter Levels - Specific values chosen for a factor ex. In a sleep study, we might assign participants to sleep for 4, 6, or 8 hours Treatment – Process applied to a group of subjects. Treatments are the different levels of the factor.

Example - Continued Treatments: Typical portions of 2 dog foods (1 from original company, 1 we know is safe) Response: Veterinarian’s assessment of test animals health

Experimental Design - 3 Principles Control Randomize Replicate

Experimental Design - 3 Principles - CONTROL We control sources of variation other than the factors we are testing by making conditions as similar as possible for all treatment groups. For human subjects, we try to treat them alike. Controlling sources of variation makes it easier to detect any differences caused by the treatments.

Control - Methods Control treatment - baseline measurement Control group - experimental units to whom the control treatment is applied Placebo, Blinding

Blinding People are so good at picking up subtle cues about treatments that it’s important to keep anyone who could affect the outcome or the measurement of the response from knowing which subjects have been assigned to which treatments. 2 groups:  People who can influence the results (interviewer, subjects, test administrator)  People who evaluate the results (researcher, doctors, judges)

Blinding Single Blinding – when every individual in 1 of the groups is blinded Double Blinding – when every individual in both groups is blinded Recall: 2 groups  People who can influence the results  People who evaluate the results

Placebo & Placebo Effect Placebo - A “fake” treatment that looks just like the treatments being tested. Placebos are the best way to blind subjects from knowing whether they are receiving the treatment or not. Placebo effect - when subjects treated with a placebo improve. It’s not unusual for 20% or more of subjects given a placebo to report reduction in pain, improved movement, or greater alertness, or even to demonstrate improved health or performance. Placebo controls are so effective that you should use them as an essential tool for blinding whenever possible.

Blocking We use blocking when there are pre-existing differences between groups of experimental units. Randomization is introduced when we randomly assign treatments within each block.

Experimental Design - 3 Principles - RANDOMIZE Allows us to equalize the effects of unknown or uncontrollable sources of variation. Note: Randomization cannot eliminate the effects of these sources, but it should spread them out fairly equally across the treatment levels

Experimental Design - 3 Principles - Replicate 2 Types of Replication Type 1 - Within the experiment Apply each treatment to several subjects Type 2 – Additional Experiment Replication of an entire experiment with different subjects

Example - Continued In the experiment comparing animal food, how could you implement the principles of control, randomization, and replication? Control the portion sizes Reduce possible variability from other factors: (Standardize other aspects of their environments—housing the dogs in similar pens and ensuring that each got the same amount of water, exercise, play, and sleep time.) Restrict the experiment to a single breed and age Assign dogs to the two feed treatments randomly. To try and equalize traits, pre-existing conditions, and other unknown influences Replicate by assigning more than one dog to each treatment to allow for variability among individual dogs. If time and funding, possibly replicate the entire experiment using, for example, a different breed of dog.

Diagram of Experimental Procedure

Your Turn! – CW1 On the following slide, you will be asked to design an experiment. Follow these steps: State what you want to know (in general) Specify the response variable Specify the factors and treatments/levels Specify the Experimental Units Design the experiment (Control, Replicate, Random) Create a diagram to show the process How will you display results Reflect – are the observations made, significant?

Your Turn! – CW1