1. C1, L3-4, S1 Types of Studies and Study Design

2. C1, L3-4, S2 Research classifications Observational vs. Experimental Observational – researcher collects info on attributes or measurements of interest, but does not influence results. Experimental – researcher deliberately influences events and investigates the effects of the intervention, e.g. clinical trials and laboratory experiments. We often use these when we are interested in studying the effect of a treatment on individuals or experimental units.

3. C1, L3-4, S3 Experiments & Observational Studies We conduct an experiment when it is (ethically, physically etc) possible for the experimenter to determine which experimental units receive which treatment.

4. C1, L3-4, S4 Experiments & Observational Studies Experiment Terminology Experimental Unit Treatment Response patient drug cholesterol patient pre-surgery antibiotic infection mouse radiation mortality

5. C1, L3-4, S5 Experiments & Observational Studies In an observational study, we compare the units that happen to have received each of the treatments.

6. C1, L3-4, S6 e.g. You cannot set up a control (non-smoking) group and treatment (smoking) group. Observational Study patientsmokinglung cancer RNunitjob stress hospital ICU staffing level ICU mortality Experiments & Observational Studies UnitTreatmentResponse

7. C1, L3-4, S7 Experiments & Observational Studies Note: Only a well-designed and well-executed experiment can reliably establish causation. An observational study is useful for identifying possible causes of effects, but it cannot reliably establish causation.

8. C1, L3-4, S8 1. Completely Randomized Design The treatments are allocated entirely by chance to the experimental units.

9. C1, L3-4, S9 1. Completely Randomized Design Example: Which of two varieties of tomatoes (A & B) yield a greater quantity of market quality fruit? Factors that may affect yield: different soil fertility levels exposure to wind/sun soil pH levels soil water content etc.

10. C1, L3-4, S10 Divide the field into plots and randomly allocate the tomato varieties (treatments) to each plot (unit). 8 plots – 4 get variety A (A) (B) 1. Completely Randomized Design What if the field sloped upward from left to right? UPHILL (B) (A) Randomly assign A & B varieties in each strip of similar elevation.

11. C1, L3-4, S11 1. Completely Randomized Design Note: Randomization is an attempt to make the treatment groups as similar as possible — we can only expect to achieve this when there is a large number of experimental units to choose from.

12. C1, L3-4, S12 2. Blocking Group (block) experimental units by some known factor and then randomize within each block in an attempt to balance out the unknown factors. Use: blocking for known factors (e.g. slope of field in previous example) and randomization for unknown factors to try to “balance things out”.

13. C1, L3-4, S13 2. Blocking Example 2: Multi-Center Clinical Trial Suppose a Mayo clinical trial comparing two chemotherapy regimens in treatment of patients with colon cancer will be conducted using cancer patients in Scottsdale, AZ and Rochester, MN.

14. C1, L3-4, S14 Scottsdale Rochester 2. Blocking How should we allocate treatments to the 12 patients? 7 (B) 2 (A) 3 (A) 5 (A) 6 (A) 1 (A) 2 (B) 3 (A) 4 (B) 8 (B) 4 (B) 1 (B) Randomly assign treatments to 4 the patients from Scottsdale and then to the 8 Rochester patients.

15. C1, L3-4, S15 2. Blocking Example 3: Comparing Three Pain Relievers for Headache Sufferers How could blocking be used to increase precision of a designed experiment to control to compare the pain relievers? What are some other design issues?

16. C1, L3-4, S16 Example 4: Comparing 17 Different Leg Wraps on Used on Race Horses 17 “boots” tested, each boot is tested n = 5 times. Why? Because of the time constraints all boots were not tested on the same day. 8 tested 1 st day, 5 tested 2 nd day, 4 tested 3 rd day. Leg was placed in freezer and thawed before the 2 nd and 3 rd days of testing.

17. C1, L3-4, S17 Horse Leg Wraps (cont’d) What problems do you foresee with this experimental design? Discussion Question 1 What actually happened? What are the implications of these results? Discussion Question 2 Forces readings obtained from cadaver leg when no boot or wrap was used.

18. C1, L3-4, S18 Horse Leg Wraps (cont’d) FINAL BOOT COMPARISONS

19. C1, L3-4, S19 Horse Legs Wraps (cont’d) What should have been done? Discussion Question 3

20. C1, L3-4, S20 3. People as Experimental Units Example: Cholesterol Drug Study – Suppose we wish to determine whether a drug will help lower the cholesterol level of patients who take it. How should we design our study? Discussion Question 4

21. C1, L3-4, S21 Polio Vaccine Example

22. C1, L3-4, S22 Polio Vaccine Example Dr. Jonas Salk, vaccine pioneer 1914-95 Iron Lung

23. C1, L3-4, S23 The Salk Vaccine Field Trial 1954 Public Health Service organized an experiment to test the effectiveness of Salk’s vaccine. Need for experiment: –Polio, an epidemic disease with cases varying considerably from year to year. A drop in polio after vaccination could mean either: Vaccine effective No epidemic that year

24. C1, L3-4, S24 The Salk Vaccine Field Trial Subjects: 2 million, Grades 1, 2, and 3 500,000 were vaccinated –(Treatment Group) 1 million deliberately not vaccinated –(Control Group) 500,000 not vaccinated - parental permission denied

25. C1, L3-4, S25 The Salk Vaccine Field Trial NFIP Design Treatment Group: Grade 2 Control Group: Grades 1 and 3 + No Permission Flaws ? Polio contagious, spreading through contact. i.e. incidence could be greater in Grade 2 (bias against vaccine), or vice-versa. Control group included children without parental permission (usually children from lower income families) whereas Treatment group could not (bias against the vaccine).

26. C1, L3-4, S26 The Salk Vaccine Field Trial Double-Blinded Randomized Controlled Experimental Design Control group only chosen from those with parental permission for vaccination Random assignment to treatment or control group Use of placebo (control group given injection of salted water) Diagnosticians not told which group the subject came from (polio can be difficult to diagnose) i.e., a double-blind randomized controlled experiment

27. C1, L3-4, S27 (NFIP rate) (25) Grade 2 (54) Grade 1/3 (44) Grade 2 The Salk Vaccine Field Trial The double-blind randomized controlled experiment (and NFIP) results Size of group Rate per 100,000 Treatment200,00028 Control200,00071 No consent350,00046

28. C1, L3-4, S28 3. People as Experimental Units control group: –Receive no treatment or an existing treatment blinding: –Subjects don’t know which treatment they receive double blind: –Subjects and administers / diagnosticians are blinded placebo: –Inert dummy treatment

29. C1, L3-4, S29 3. People as Experimental Units placebo effect: –A common response in humans when they believe they have been treated. –Approximately 35% of people respond positively to dummy treatments - the placebo effect

30. C1, L3-4, S30 Observational Studies There are two major types of observational studies: prospectiveand retrospective studies

31. C1, L3-4, S31 Observational Studies 1. Prospective Studies –(looking forward) –Choose samples now, measure variables and follow up in the future. –E.g., choose a group of smokers and non-smokers now and observe their health in the future.

32. C1, L3-4, S32 Observational Studies –Looks back at the past. –E.g., a case-control study Separate samples for cases and controls (non-cases). Look back into the past and compare histories. E.g. choose two groups: lung cancer patients and non-lung cancer patients. Compare their smoking histories. 2. Retrospective Studies –(looking back)

33. C1, L3-4, S33 Observational Studies Important Note: 1. Observational studies should use some form of random sampling to obtain representative samples. 2.Observational studies cannot reliably establish causation.

34. C1, L3-4, S34 Controlling for various factors A prospective study was carried out over 11 years on a group of smokers and non- smokers showed that there were 7 lung cancer deaths per 100,000 in the non- smoker sample, but 166 lung cancer deaths per 100,000 in the smoker sample. This still does not show smoking causes lung cancer because it could be that smokers smoke because of stress and that this stress causes lung cancer.

35. C1, L3-4, S35 Controlling for various factors To control for this factor we might divide our samples into different stress categories. We then compare smokers and non-smokers who are in the same stress category. This is called controlling for a confounding factor.

36. C1, L3-4, S36 Example 1 “Home births give babies a good chance” NZ Herald, 1990 –An Australian report was stated to have said that babies are twice as likely to die during or soon after a hospital delivery than those from a home birth. –The report was based upon simple random samples of home births and hospital births. Q: Does this mean hospitals are dangerous places to have babies in Australia? Why or why not? Discussion Question 5

37. C1, L3-4, S37 Example 2 “Lead Exposure Linked to Bad Teeth in Children” ~ USA Today The study involved 24,901 children ages 2 and older. It showed that the greater the child’s exposure to lead, the more decayed or missing teeth. Q: Does this show lead exposure causes tooth decay in children? Why or why not? Discussion Question 6

38. C1, L3-4, S38 Example 2 ~ cont’d “Lead Exposure Linked to Bad Teeth in Children” ~ USA Today Researcher: “We controlled for income level, the proportion of diet due to carbohydrates, calcium in the diet and the number of days since the last dental visit.”

39. C1, L3-4, S39 Limitations on Scope of Inference

40. C1, L3-4, S40 Discussion Question 7 – Determine Whether Age at 1 st Pregnancy is a Risk Factor for Cervical Cancer How might we proceed?

41. C1, L3-4, S41 Discussion Question 8 – Determine what job related factors Mayo nurses are most dissatisfied with. How might we proceed?

42. C1, L3-4, S42 Discussion Question 9 – Determine if a new pre-operative antibiotic reduces the risk of infection for patients undergoing knee replacement. How might we proceed?

43. C1, L3-4, S43 Surveys and Polls (and the errors inherent in them)

44. C1, L3-4, S44 Nonsampling Errors Sampling/Chance/ Random Errors Selection bias Interviewer effects Non-response bias Behavioural considerations Self selection Transfer findings Question effects Survey-format effects Sampling

45. C1, L3-4, S45 Sources of Nonsampling Errors Selection bias Population sampled is not exactly the population of interest. e.g. KARE 11 poll, telephone interviews population sample

46. C1, L3-4, S46 Sources of Nonsampling Errors Non-response bias People who have been targeted to be surveyed do not respond. Non-respondents tend to behave differently to respondents with respect to the question being asked.

47. C1, L3-4, S47 1936 U.S. Election Country struggling to recover from the Great Depression 9 million unemployed 1929-1933 real income dropped by 1/3

48. C1, L3-4, S48 1936 U.S. Election Candidates: –Albert Landon (Republican) “The spenders must go!” – Franklin D Roosevelt (Democrat) Deficit financing - “Balance the budget of the people before balancing the budget of the Nation”

49. C1, L3-4, S49 1936 U.S. Election Roosevelt’s percentage –Digest prediction of the election result –Gallup’s prediction of the Digest prediction –Gallup’s prediction of the election result –Actual election result 43% 44% 56% 62% Digest sent out 10 million questionnaires to people on club membership lists, telephone directories etc. – received 2.4 million responses Gallup Poll used another sample of 50,000 Gallup used a random sample of 3,000 from the Digest lists to predict Digest outcome

50. C1, L3-4, S50 Sources of Nonsampling Errors Self-selection bias People decide themselves whether to be surveyed or not. Much behavioural research can only use volunteers.

51. C1, L3-4, S51 Sources of Nonsampling Errors

52. C1, L3-4, S52 Sources of Nonsampling Errors This poll is not scientific and reflects the opinions of only those Internet users who have chosen to participate. The results cannot be assumed to represent the opinions of Internet users in general, nor the public as a whole. The QuickVote sponsor is not responsible for poll content, functionality or the opinions expressed therein.

53. C1, L3-4, S53 Sources of Nonsampling Errors Question effects Subtle variations in wording can have an effect on responses. Eg “Should euthanasia be legal?” vs “Should voluntary euthanasia be legal?”

54. C1, L3-4, S54 New York Times/CBS News Poll (8/18/80) “Do you think there should be an amendment to the constitution prohibiting abortions?” Yes 29%No62% Later the same people were asked: “Do you think there should be an amendment to the constitution protecting the life of the unborn child?” Yes50%No39%

55. C1, L3-4, S55 Sources of Nonsampling Errors Interviewer effects Different interviewers asking the same question can obtain different results. Eg sex, race, religion of the interviewer

56. C1, L3-4, S56 Interviewer Effects in Racial Questions In 1968, one year after a major racial disturbance in Detroit, a sample of black residents were asked: “Do you personally feel that you trust most white people, some white people or none at all?” White interviewer: 35% answered “most” Black interviewer: 7% answered “most”

57. C1, L3-4, S57 Sources of Nonsampling Errors Behavioural considerations People tend to answer questions in a way they consider to be socially desirable. e.g. pregnant women being asked about their drinking habits

58. C1, L3-4, S58 Behavioural Considerations in Election Official vote counts show that 86.5 million people voted in the 1980 U.S. presidential elections. A census bureau survey of 64,000 households some weeks later estimated 93.1 million people voted.

59. C1, L3-4, S59 Sources of Nonsampling Errors Transferring findings Taking the data from one population and transferring the results to another. e.g. Twin Cities opinions may not be a good indication of opinions in Winona. Twin Cities sample Winona

60. C1, L3-4, S60 Sources of Nonsampling Errors Survey-format effects Eg question order, survey layout, interviewed by phone or in- person or mail.

61. C1, L3-4, S61 Nonsampling Errors Sampling/Chance/ Random Errors Selection bias Interviewer effects Non-response bias Behavioural considerations Self selection Transfer findings Question effects Survey-format effects Sampling

62. C1, L3-4, S62 Survey Errors Sampling/Chance/ Random Errors Nonsampling Errors

63. C1, L3-4, S63 Sampling / Chance / Random Errors errors caused by the act of taking a sample have the potential to be bigger in smaller samples than in larger ones possible to determine how large they can be unavoidable (price of sampling)

64. C1, L3-4, S64 Nonsampling Errors can be much larger than sampling errors are always present can be virtually impossible to correct for after the completion of survey virtually impossible to determine how badly they will affect the result must try to minimize in design of survey (use a pilot survey etc.)

65. C1, L3-4, S65 Surveys / Polls A pilot survey is a small survey that is carried out before the main survey and is often used to identify any problems with the survey design (such as potential sources of non-sampling errors).

66. C1, L3-4, S66 Surveys / Polls A report on a sample survey/poll should include: –target population (population of interest) –sample selection method –the sample size and the margin of error –the date of the survey –the exact question(s)