1. Foodborne Disease Outbreak Investigation Team Training
Module 6 – Epidemiologic Investigation
Estimated time: 90 minutes
Foodborne Disease Outbreak Investigation Team Training

2. At the end of this module, you will be able to
Compare a case series, a cohort study, and a case-control study.
Interpret the measure of association for a cohort and a case-control study.
Explain what is meant by the term “statistically significant.”
Identify potential problem areas in the conduct of an epidemiologic study which might impact the findings.

3. Epidemiologic Activities
Perform descriptive epidemiology Conduct hypothesis-generating interviews Undertake analytic studies Case series Cohort studies Case-control studies
Epidemiologic investigators collect, analyze, and interpret exposure information from cases in an outbreak (and comparison groups, where appropriate) to determine the causative agent, persons at risk, modes of transmission, and the vehicle of the outbreak.
We already spent time in Module 3 talking about the “descriptive epidemiology” and “hypothesis-generating interviews,” traditionally considered to be components of the epidemiologic investigation. In this module we will talk about the epidemiologic studies used to test hypotheses about an outbreak (i.e., case series, case-control studies, and cohort studies) and interpretation of the results from these studies.
Like with the module on the Environmental Health Investigation, epidemiologists participating in this course will already know the material to be covered. It is the other team members (environmental health investigators and laboratory investigators) who will benefit by this brief overview which will help them better understand the efforts undertaken by the epidemiologist and be better able to interpret (and use) the results from epidemiologic studies.
We will start with the case series …

4. Examination of a series of cases associated with an outbreak
Case Series
Examination of a series of cases associated with an outbreak
Collection of detailed information on foods eaten by cases (and other exposures)
Common foods (or other exposures) among cases suggestive of source of outbreak
Ate food
Population
at risk
Cases
A case series is the examination of a collection (or series) of cases from an outbreak. In a case series, detailed information on exposures that occurred during the incubation period of the disease (including foods eaten) is collected from cases. Commonalities among cases are then used to suggest possible sources of the outbreak.
Only cases are examined. Because no comparison group is available in a case series to put findings into perspective, investigators must be careful in making inferences from a case series. But occasionally, what is learned from just cases is so unusual that inferences about the outbreak can be made and control measures implemented. (Use of comparison groups such as the results from the FoodNet Atlas for Exposures can help.)
[NOTE: A case series is essentially the same thing as hypothesis-generating interviews (covered in Module 3). However the intent is slightly different. With hypothesis generation, you are collecting information with the intent of developing hypotheses to be tested in other studies. With a case series, you are hoping to find the answer to the outbreak (i.e., the exposure transmitting the illness.)]
Did not eat food

5. Multistate Outbreak of Salmonella Enteriditis
Using shopper card information, determined that 7 of 9 cases bought Turkish pine nuts from chain store in week before illness Background rate: <1% of all shoppers bought Turkish pine nuts at store in previous six months Lab testing identified outbreak strain of S. Enteritidis in pine nuts/pesto from store Store and producer voluntarily recalled pine nuts
Here is an example of a case series used to identify the source of an outbreak.
A multistate outbreak of Salmonella Enteritidis was detected through PulseNet. CDC collaborated with public health and agriculture officials in New York and other states and the U.S. Food and Drug Administration (FDA) to investigate.
Early in the investigation, information was collected from cases on foods eaten prior to becoming ill. A review of shopper card records from 7 of 9 cases identified that ill persons had purchased the same type of Turkish pine nuts from bulk bins at different locations of a chain grocery stores before becoming ill. The store reported that less than 1% of all shoppers bought Turkish pine nuts at the chain in the previous 6 months.
Subsequent laboratory testing conducted by public health laboratories in several states identified the outbreak strain of Salmonella Enteritidis in 14 samples of Turkish pine nuts or pesto containing Turkish pine nuts. 
The chain grocery store recalled approximately 5,000 lbs. of Turkish pine nuts sold in the bulk foods department in New York, Pennsylvania, New Jersey, Virginia, and Maryland. The repacker and producer recalled approximately 3,800 lbs. of pine nuts and four lots of the bulk Turkish pine nuts, totaling more than 21,000 pounds.
Conclusions about the source of this outbreak were essentially based on a case series only, although shopper card information did provide background rates of exposure.

6. The Need for a Comparison Group
Commonalities among cases:
Reflective of population at large?
A chance happening?
Related to some unknown factor that is the true source of the outbreak? (confounders)
Comparison (control) group needed to put findings into perspective
Although FDA took action based on the S. Enteritidis case series in New York, case series usually are only suggestive of a source, but not conclusive. Investigators are left wondering: Are the commonalities among cases unique or reflective of the population at large (common things are common)? Are the findings just a chance happening (a large proportion of cases just happened to have these exposures)? Or is there some other unknown factor related to the common factor?
To address these questions, a controlled study (also called analytic study) needs to be undertaken. A controlled study includes a comparison group. A controlled study allows investigators to quantify relationships between a suspected exposure and a disease, test suspicions about the cause of an outbreak, and explore the role of chance in the findings.
Two types of controlled studies are commonly used in outbreak investigations: cohort and case-control studies. Let’s start with talking about cohort studies.

7. Well-defined group in which outbreak occurs
Cohort Study
Well-defined group in which outbreak occurs
Compare attack rates among people who ate and did not eat certain food(s)
Higher attack rates among people eating a food (compared to those not eating it) suggest the food might be associated with illness
Illness
Ate food
No illness
Well defined group
In a cohort study, subjects are enrolled into the study because they were a member of a well-defined group in which an outbreak occurred (e.g., were a guest at a wedding banquet or were the employee of a hospital). It is assumed that members of the group will include both people exposed to the (unknown) source of the outbreak and people not exposed.
In the analysis of a cohort study, people enrolled in the study are categorized as to whether they ate particular foods or not. The epidemiologist then determines the proportion of people becoming ill in each group.
If the occurrence of disease is higher among people who ate a particular food than people who did not eat the food, the food is considered likely to be associated with the disease in some way.
This is somewhat intuitive; however, an example will help you understand this better.
Illness
Did not eat food
No illness

8. Outbreak of Salmonellosis at Hospital X
Over 200 cases of salmonellosis occurred among nurses (a well-defined group) at Hospital X after Nurse’s Appreciation Day Luncheon
Cohort study of nurses at hospital
736 nurses and nursing students responded
195 (34%) of 571 persons who attended luncheon became ill
8 (5%) of 165 persons who did not attend luncheon became ill
On May 14, administrators at Hospital X became aware of a large number of absences among nursing personnel due to a febrile gastrointestinal illness that included diarrhea, nausea, and vomiting. Salmonella was identified in stool cultures from several ill nurses. The illnesses occurred after “Nurses Appreciation Day” luncheons held at the hospital on May 9-10.
A cohort study was undertaken among nursing staff to determine the source of the outbreak. (“Nursing staff” is the cohort or well-defined group being enrolled in this study.) A total of 736 nurses and nursing students provided information. Of the 571 individuals attending the luncheon who responded in the study, 195 (34%) became ill. Of the 165 not attending the luncheon, 8 (5%) became ill.
Just eyeballing the data, illness appears to be more common among persons attending the luncheon than those not attending the luncheon. So the luncheon seems very suspicious. Epidemiologists will quantify the relationship between attending the luncheon and becoming ill using a “measure of association.”

9. Measure of association for a cohort study
Relative Risk (RR)
Measure of association for a cohort study
Compares proportion of people who ate the food who became ill with the proportion of people did not eat the food who became ill
Answers: “How much more likely is it for people who ate the food to become ill than people not eating the food?”
relative risk
attack rate among exposed
attack rate among unexposed =
The relative risk is the measure of association for a cohort study. The relative risk is the ratio of the disease attack rate among people who ate a particular food and the attack rate among people who did not eat the food.
The attack rate is the incidence of disease in a group (i.e., the number of people in the group who became ill divided by the total number of people in the group).
A relative risk answers the question “How much more likely is it for people who ate the food to become ill than people not eating the food?”

10. Relative Risk INTERPRETATION
Close to 1.0 = risk of disease is similar among people eating and not eating the food  food not associated with illness Greater than 1.0 = risk of disease is higher among people eating the food than people not eating the food  food could be risk factor Less than 1.0 = risk of disease is lower among people eating the food than people not eating the food  food could be “protective factor” Magnitude reflects strength of association between eating food and illness.
The relative risk tells us how much more likely (or less likely) it is for people who ate a particular food to develop a disease compared to people not eating that food. A relative risk of:
Close to 1.0 means the risk of disease is similar among people eating and not eating the food. Eating the food is not associated with the outbreak.
Greater than 1.0 means that the risk of disease is higher among people eating the food than people not eating the food. Eating the food could be a risk factor for the illness.
Less than 1.0 means that the risk of disease is lower among people eating the food than people not eating the food. Eating the food could be “protective” against the illness.
Note, “protective” is in quotes. “Protective” can in some instances mean that the factor actually prevents the person from developing the illness. Such might be the case for use of a particular antibiotic or vaccination against a particular infectious disease. In foodborne outbreaks, protective factors tend to reflect that persons who ate a particular food were less likely to eat the food that was the cause of the outbreak. For example, if an outbreak of salmonellosis occurs at a Thanksgiving dinner due to contaminated turkey, ham served at the same meal might appear protective because people who ate the ham were less likely to also eat the turkey and become ill. So it doesn’t really protect the person from developing the illness but prevents their exposure to the food that did.
The magnitude of the relative risk is call the “strength of association”. The further away a relative risk is from 1.0, the more likely that the relationship between eating the food and illness is causal. For example, a relative risk of 1.2 is above 1.0, but it is not impressively so. A relative risk of 10 is a much stronger association.

11. Outbreak of Salmonellosis at Hospital X
Returning to the outbreak of salmonellosis:
195 (34%) of 571 attending luncheon became ill
8 (5%) of 165 not attending luncheon became ill
relative risk
attack rate (attended)
attack rate (did not attend)
34% % = = =
6.8
A relative risk of 6.8 means that people who attended the luncheon were almost 7 times more likely to become ill than those who did not attend. Attending the luncheon might be a risk factor for salmonellosis in this outbreak.
So, back to the example of salmonellosis at Hospital X.
Among the 571 persons who went to the luncheon (that is “exposed persons”), 195 became ill. So the attack rate is 195 divided by 571 which equals 34%.
Among the 165 persons who did not go to the luncheon (that is “unexposed persons”), 8 became ill. So the attack rate for unexposed persons is 8 divided by 165 or 5%.
The relative risk is the attack rate among exposed persons divided by the attack rate among unexposed persons which is 34% divided by 5% which equals 6.8.
Now what does that 6.8 mean? Persons attending luncheon were almost seven times more likely to become ill than those not attending luncheon. Attending the luncheon could be a risk factor for becoming ill. The magnitude of the relative risk is substantial.
Note to instructor: You might ask students, “What about the illness among persons who did not attend the luncheon?” “What about people who went to the luncheon and did not become ill?”

12. 
Question
Looking only at the nurses who attended the Nurse’s Appreciation Day Luncheon
14 (18%) of 78 eating tuna salad became ill
172 (40%) of 431 not eating tuna salad became ill
Relative risk = 0.45
What does this relative risk mean?
Answer: Persons eating tuna salad sandwiches were half as likely to develop illness as those not eating tuna salad sandwiches. Eating tuna salad could be “protective” in this outbreak.
Okay, to test your understanding of a relative risk:
On further investigation of the outbreak of salmonellosis at Hospital X, investigators looked at persons eating (and not eating) different foods at the luncheon. The luncheon included a variety of sandwiches prepared in the kitchen of a private home. Sandwiches included tuna salad, cheese, turkey, and roast beef. Here are the findings for tuna salad sandwich:
14 (18%) of 78 people eating tuna salad sandwiches became ill
172 (40%) of 431 people not eating tuna salad sandwiches became ill
The relative risk is 0.45.
What does that mean?
ANSWER: Persons eating tuna salad sandwiches were half as likely to develop illness as those not eating tuna salad sandwiches. The relative risk is less than Eating tuna salad sandwiches could be “protective” in this outbreak.
Now do you think tuna salad sandwiches have some ingredient that keeps people from getting sick? Maybe, but a more likely explanation is that people who ate tuna salad sandwiches didn’t eat something else that was the source of the outbreak which was the case in this outbreak. The relative risk for eating only turkey sandwiches was 4.1 and the relative risk for eating only roast beef sandwiches was 2.8. The turkey and roast beef sandwiches were made the evening before the luncheon and were transported to the hospital without further refrigeration. The meat slicer used to slice both turkey and roast beef, was wiped down between slicing the turkey and the roast beef but was not cleaned with soap and water until after the roast beef was sliced, so both turkey and roast beef sandwiches became contaminated. And since they were not refrigerated, it is likely that the Salmonella present multiplied.
Note to instructor: Students interested in calculating relative risks should talk with the instructor after class. Do not spend any more time on it.

13. Cases (people with illness) and controls (people with no illness)
Case-Control Study
Cases (people with illness) and controls (people with no illness)
Compare foods eaten by cases and controls
Foods more commonly eaten by cases than controls might be associated with illness
Ate food
Cases
Did not eat food
Population
at risk
We will now move on to the second type of controlled (or analytic) study (that is a study with a comparison group): the case-control study.
In a case-control study, subjects are enrolled based on whether they have (or had) the disease associated with the outbreak or not. (In these studies, persons with the disease of interest are called “cases” or “case-patients” and persons without the disease are called “controls”).
Prior exposures, such as eating a particular food item, are compared between cases and controls to see if there is a relationship between the disease and the exposure. (Notice how this differs from cohort studies. In a cohort study we look at people who ate a food or did not eat a food and determine if they became ill or not. With a case-control study, we look at people who were ill or not and look back to see if they ate the food or not.)
Exposures substantially more common among cases than controls are considered likely to be associated with the outbreak in some way. This is somewhat intuitive. An example will help bring this point home.
Ate food
Controls
Did not eat food

14. Outbreak of Botulism in Vancouver, B.C.
36 cases of botulism among patrons of Restaurant X
Case-control study undertaken
20 (91%) of 22 cases ate beef dip sandwich
3 (14%) of 22 controls ate beef dip sandwich
Two sisters and their mother from Vancouver, British Columbia, developed signs and symptoms suggestive of botulism. After these cases were publicized, 34 additional cases of botulism were identified in the area. All case-patients had eaten at a single, family-styled restaurant.
A case-control study was undertaken to determine the source of the outbreak at the restaurant. Cases were persons who had eaten at the Vancouver restaurant who had neurologic signs and symptoms suggestive of botulism. Controls were persons who ate at the restaurant with case-patients but developed no gastrointestinal or neurologic symptoms in the following 2 weeks. Twenty-two case-patients and 22 controls were interviewed. It was determined that 20 (91%) of 22 case-patients but only 3 (14%) of 22 controls ate a beef dip sandwich at the restaurant.
Just eyeballing these numbers, it seems that eating beef dip sandwiches was somehow related to becoming ill. (You don’t have to be an epidemiologist to see that those beef tip sandwiches are very suspicious.)
Now, epidemiologists typically don’t just “eyeball” the numbers. They use the numbers to quantify the relationship between an exposure and becoming ill. This number is called the “measure of association”.

15. Measure of association for a case-control study
Odds Ratio (OR)
Measure of association for a case-control study
Compares odds of cases having eaten a certain food to odds of controls having eaten the food
Answers: “How much higher is the odds of eating the food among cases than controls?”
odds of eating food among cases
odds of eating food among controls
odds ratio =
For a case-control study, the measure of association is an odds ratio. The odds ratios is the single best estimate for quantifying the relationship between an exposure (such as eating a particular food) and a disease for a case-control study.
The odds ratio compares the odds of eating the food among cases to the odds of eating the food among controls.
The odds ratio answers the question “How much higher is the odds of eating the food among cases than controls?”

16. Odds Ratio INTERPRETATION
Close to 1.0 = odds of eating food is similar among cases and controls  no association between food and illness Greater than 1.0 = odds of eating food among cases is higher than among controls  food could be risk factor Less than 1.0 = odds of eating food among cases is lower than among controls  food could be “protective factor” Magnitude reflects strength of association between illness and eating the food.
An odds ratio tells you how much higher (or lower) the odds of eating a food is among cases compared with the odds of eating the food among controls. An odds ratio of:
1.0 (or close to 1.0) means that the odds of eating the food among cases is the same as the odds of eating the food among controls. (Dividing a number by itself will be 1.0.) Eating the food is not associated with the disease.
Greater than 1.0 means that the odds of eating the food among cases is greater than the odds of eating the food among controls. Eating the food may be a risk factor for the disease.
Less than 1.0 means that the odds of eating the food among cases is lower than the odds of eating the food among controls. Well people are more likely to have eaten the food. Eating the food may be “protective” against the disease.
Now you see how we have put “protective” in quotes. That is because you cannot take this label too literally. “Protective” can, in some instances, mean that the factor actually prevents the person from developing the illness. Such might be the case for use of a particular antibiotic or vaccination against a particular infectious disease. In foodborne outbreaks, however, protective factors tend to reflect that persons who ate a particular food were less likely to eat the food that was the cause of the outbreak. As you can see, the “protective factor” doesn’t really protect the person from developing the illness but prevents their exposure to the food that did.
Finally, the magnitude of the odds ratio is called the “strength of association”. The further away an odds ratio is from 1.0, the more likely that the relationship between illness and an exposure is causal. An odds ratio of 1.2 is above 1.0, but it is not impressively so. An odds ratio of 10 suggests a stronger association.

17. Outbreak of Botulism in Vancouver, B.C.
Returning to the outbreak of botulism:
20 of 22 cases ate beef dip sandwich (2 didn’t)
3 of 22 controls ate beef dip sandwich (19 didn’t)
odds of eating food (cases) /2
odds of eating food (controls) /19
odds ratio = =
odds ratio = 63
An odds ratio of 63 means the odds that cases ate the beef dip sandwich was 63 times higher than the odds among controls. Eating the beef dip sandwich might be a risk factor for botulism in this outbreak.
So, going back to the botulism cases in Vancouver, British Columbia:
The odds of eating the beef dip sandwich among cases is 20/2. The odds of eating beef dip sandwiches among control was 3/19. The odds ratio is 20/2 divided by 3/19 or 63.
Now what does an odds ratio of 63 mean? The odds of eating beef dip sandwiches was 63 times higher among cases than controls. Because the odds of eating the food is higher among cases than controls (greater than 1.0), eating the food could be a risk factor for the outbreak. The magnitude of the odds ratio suggests a strong association.
Note to instructors: Students might ask about the cases who did not eat the beef dip sandwiches yet became ill and controls who ate the beef dip sandwiches but did not become ill. It is possible that cases forgot that they ate the beef dip sandwich or ate a different food that was cross-contaminated by the beef dip sandwich. It is possible that controls misremembered what they ate or that contamination of the beef dip sandwiches was not uniform and they ate a portion that was not contaminated.

18. 
Question
Outbreak of cyclosporiasis in New Jersey not associated with particular event/establishment
Case-control study undertaken
21 (70%) of 30 cases ate raspberries
4 (7%) of 60 controls ate raspberries
Odds ratio = 32.7
What does this odds ratio mean?
Answer: Odds of eating raspberries is 33 times higher among cases than controls. Eating rasp-berries could be a risk factor for cyclosporiasis. The association between raspberries and illness is strong.
Let’s see if you understand this …
Cyclosporiasis is a parasitic disease caused by the microorganism Cyclospora cayetanensis. Cyclospora infects the small bowel and usually causes watery diarrhea, bloating, increased gas, stomach cramps, nausea, loss of appetite, and profound weight loss. Cyclosporiasis is transmitted in food or water.
In 1996, a number of outbreaks of cyclosporiasis were occurring across the United States. In late June, the New Jersey Department of Health and Senior Services (NJDHSS) undertook a case-control study to examine an association between cyclosporiasis and eating raspberries. The cases did not come from one particular setting or event but were spread across the state. (Sporadic is the term often used to describe cases that do not appear to be related to a particular event or exposure.)
In the case-control study, 21 (70%) of 30 cases reported eating raspberries in the week before onset of illness whereas 4 (7%) of 60 controls ate raspberries. If you calculate the odds ratio you get What does this mean?
ANSWER: The odds of eating raspberries among cases is 33 times higher among cases than controls. 33 is greater than Eating raspberries could be a risk factor for cyclosporiasis in this outbreak.
Note to instructor: Students interested in calculating the odds ratio should talk with the instructor after class. Do not spend any more time on it.

19. When to Do Which Type of Study?
Case series – when the number of cases is small (less than five) and no controls are available Cohort study – when investigators can easily identify the population at risk (i.e., outbreak has occurred in a well-defined group) and the population at risk can be enumerated Case-control study – when the population at risk (i.e., people potentially exposed to source of outbreak) is unknown or cannot be enumerated or the illness is rare
So the three common types of epidemiologic studies used in foodborne disease outbreaks are case series, case-controls studies, and cohort studies. When do you use each?
The study type depends largely on the circumstances of the particular outbreak and the resources available. The following guidance will be helpful:
Case series – when the number of cases is small (less than five) and no controls are available. The majority of outbreaks reported to local jurisdictions will result in case series studies.
Cohort study – when investigators can easily identify the population at risk (i.e., the outbreak has occurred in a well-defined group) and the population can be enumerated. Examples of when you might use a cohort study is when an outbreak occurs among students at a school, residents of a particular nursing home, or the people attending a certain event.
Case-control study – when the population at risk (i.e., people potentially exposed to the source of the outbreak) is unknown or cannot be enumerated or the illness is rare. An example of when you might use a case-control study is a multijurisdictional outbreak of a particular disease in which cases are distributed across many different states with no obvious association.
Because the study approach and conduct will differ with each study type, it is important that the study type be selected before an investigation is begun.

20. Developed disease or not
Summary
Case Series
Cohort
Study
Case-control
Enroll
People with disease
Analysis
Ate food or not
Measure of association
None
When to use
Small number of cases
Population at risk
Easily identified
Can be enumerated
Population at risk unknown
Rare disease
People in a well-defined group who ate and did not eat certain foods
People with and without disease
Developed disease or not
Ate food or not
Relative risk
Odds ratio
To conclude this discussion of different epidemiologic study types, let’s review some of the key points about a case series, cohort study, and case-control study.

21. Role of Chance
Note to instructor: This is just a transition slide to indicate topics. Move to next slide before beginning lecture.

22. Things do just happen by coincidence!
Role of Chance
Things do just happen by coincidence!
Odds ratios and relative risks are estimates
Observed results could be due to chance alone
Role of chance explored through
p-value
Confidence interval (CI)
Once the odds ratio or relative risk has been determined for a food item and a disease, tests of statistical significance must be used to determine the likelihood (probability) of finding an odds ratio or relative risk as high as the one observed due to chance alone (that is if there was no association between the food item and disease).
Two commonly used tests to explore the role of chance are the p-value and the confidence interval.

23. Probability that findings due to chance alone
p-value
Probability that findings due to chance alone
Ranges from 0 to 1 (0% to 100%)
Closer to 1.0 (100%)  high probability findings due to chance
Closer to 0.0 (0%)  low probability findings due to chance
Example:
p-value = 0.02
Interpretation: finding occurred by chance in 100 times
The p-value is the probability of finding an association between an exposure (e.g., eating a certain food) and a disease as strong as the one observed, if the exposure is not actually related to disease. Stated another way, the p-value is the likelihood that the finding occurred simply through chance.
Probabilities range from 0.0 (0%) (zero probability the finding is due to chance alone) to 1.0 (100%).
A p-value close to 1.0 (100%) indicates a high probability that the finding is due to chance alone. (You would be likely to observe such an association due to chance alone).
A p-value close to 0.0 (0%) indicates a low probability that the finding is due to chance. (You would be unlikely to observe such an association if exposure is not related to disease).
So, for example, if we find that the measure of association between a particular food and becoming ill has a p-value of 0.02, that means there is a 2 in 100 probability that the observed measure of association is due to chance alone. A 2 in 100 chance is fairly low.

24. p-value AND STATISTICAL SIGNIFICANCE
If p-value smaller than predetermined value  considered “statistically significant”
Cut-off for statistical significance set by investigator (usually 0.05 meaning the finding could have occurred by chance alone 5 in 100 times)
Example:
If cut-off for statistical significance is 0.05
p-value = 0.02  statistically significant
If the p-value for a particular measure of association is equal to or smaller than a certain predetermined cutoff, the association has a low probability of being due to chance. The association is then said to be “statistically significant”.
If the p-value is higher than that cut-off value, the association has a higher probability of resulting from chance. It is said to be “not statistically significant”.
For example, using the p-value from the previous slide: If we set the cut-off for statistical significance to be 0.05, then 0.02 is smaller than the cut-off value and the association is said to be statistically significant.
The p-value for statistical significance is determined by investigators prior to execution of the study. It is commonly set at 0.05 meaning a 5 in 100 chance that the findings are due to chance alone.
[FOR MORE ADVANCED STUDENTS ONLY: The predetermined cut-off is selected by the investigator before the study and is based on the acceptable level of concluding that there is an association between an exposure and a disease when in truth there is no association. If the disease under investigation is very serious and investigators do not want to miss a possible risk factor, they might set the cut-off value to be higher. If a large number of exposures is being examined, meaning by chance alone one of those exposures will have an elevated measure of association, investigators might set the cut-off value for statistical significance to be lower.]

25. Confidence Intervals (CI)
Confidence Intervals (CI)
Measure of association single best estimate
Confidence interval (CI)
Range of values for the measure of association that are consistent with study findings
Has specified probability (e.g., 95%) of including “true value” for the measure of association
Example:
odds ratio = % CI = 4.0 – 6.1
Another indicator of the role that chance plays in an epidemiologic study is the confidence interval (abbreviated CI).
As you remember the measure of association (odds ratio or relative risk) is the best single estimate for quantifying the relationship between exposure (eating a certain food) and disease. But it is an estimate.
The confidence interval is the range of values for a particular measure of association consistent with the study findings. (It is the range of plausible values.) It is described using the lower and upper bound of the range of values. (NOTE: The measure of association should lie somewhere within the lower and upper bounds. If it doesn’t, there is an error.)
The confidence interval is constructed so that the range has a specified probability of including the “true value” of the measure of association (which, of course, is unknown). The 95% confidence interval is the most commonly used confidence interval and has a 95% probability of including the true value of the measure of association.
For example, if the odds ratio for a case-control study is The best single estimate for quantifying the relationship between illness and eating the food is 5.2, but a confidence interval of suggests that study results are consistent with an odds ratio anywhere between 4.0 and 6.1 and that there is a 95% probability that the true value for the odds ratio lies in that range. [ ]
4.0
6.1
5.2

26. Confidence Intervals (CI) (CONT’D)
Confidence Intervals (CI) (CONT’D)
Measure of association of 1 means unlikely association.
Therefore, if CI includes 1.0  not statistically significant
Example: 95% CI = 0.8 – 4.2 [ ]
0.8
1.0
4.2
1.0 [ ]
1.8
4.2
If CI does not include 1.0  statistically significant
Example: 95% CI = 1.8 – 4.2
Since a confidence interval reflects the range of values consistent with the data in a study, one can use the confidence interval to determine the role of chance in finding a particular association, like a test of statistical significance.
But first, a review. If an odds ratio (or relative risk) for eating a particular food is 1.0 (or close to one), what does that mean? That means the odds of eating the food among cases and controls is similar (or for a relative risk the attack rate among people who ate and did not eat that food is similar). It means that eating that particular food item is unlikely to be associated with the outbreak and is not statistically significant.
Since a confidence interval is the plausible range of values for an odds ratio (or relative risk), if a confidence interval includes 1.0, it means that the odds ratio (or relative risk) could plausibly be If the odds ratio or relative risk could be 1.0, that means that there is no association between eating the food and becoming ill. We say that the finding is not statistically significant.
On the other hand, if the confidence interval does not include 1.0, we are fairly certain that the odds ratio (or relative risk) is unlikely to be 1.0 which is consistent with there being an association between eating the food and illness. We say the finding is statistically significant.

27. Question
An outbreak of Salmonella Typhi in Tajikistan Case-control study undertaken Exposures in 30 days before illness for cases (or before interview for controls) Results analyzed using a p-value of 0.05 as the cut-off for statistical significance
In February 1997, an outbreak of typhoid fever was detected in Dushanbe, the capital of Tajikistan (population approximately 600,000). Although typhoid fever was endemic in this area, more than 2,000 cases had been reported during a 2-week period from January 29−February 11, compared with approximately 75 cases each week during the previous month
Because previous typhoid fever outbreaks had been associated with foods and beverages sold by street vendors, the city government prohibited such sales. However, considerable debate remained about the source of the outbreak and appropriate control measures.
A case-control study was undertaken. A variety of exposures in the 30 days before illness (or before interview) were explored with cases and controls. Results were analyzed using a p-value of 0.05 as the cut-off for statistical significance.

28. Statistically significant
Question
Which odds ratios are statistically significant?
Exposure
Odds ratio
p-value
95% CI
Statistically significant
Eating street vendor food
1.5
0.3
0.9−5.6
Eating apples
0.2
0.03
0.04−0.9
Drinking un- treated water
9.6
0.0005
2.7−34
Odds ratio for eating food from street vendor > 1.0 suggesting could be risk factor. But p-value > 0.05 and confidence interval includes Finding is likely to be due to chance. No
Odds ratio for eating apples less than 1.0 suggesting possible protective factor. p-value < 0.05 and confidence interval does not include Finding is unlikely to be due to chance.
Yes
Yes
Odds ratio for drinking untreated tap water > 1.0 suggesting possible risk factor. p-value < 0.05 and confidence interval does not includes Finding unlikely to be due to chance.
What was the likely source of the outbreaks? Was it eating food from street vendors?

29. Statistical Significance
Means chance is an unlikely (though not impossible) explanation for observed association Does not mean cause and effect or indicate “public health significance” Is affected by size of study (the more subjects included in a study, the smaller the p-value will be regardless of the measure of association)
One must be careful in interpreting statistical significance and drawing conclusions. “Statistical significance” indicates only that chance is an unlikely (though not an impossible) explanation for an association. An association with a small p-value (below the cut-off for statistical significance [e.g., 0.05]) could still be due to chance. It is just unlikely.
Statistically speaking, “nonsignificance” indicates that chance may play a large role in the observed association. A true association may exist between eating the food and becoming ill even with a large p-value, but one has to be careful, because chance is a likely explanation for the observed association.
Statistical significance does not by itself indicate a cause-effect relationship. An observed association may indeed represent a causal relationship, but it may still be due to chance or another factor related to both the exposure and the diseases (i.e., confounders), or other issues surrounding study design and execution.
Furthermore, statistical significance is impacted by study size. The more subjects included in the study, the more likely that the measure of association (even if only slightly elevated above 1.0) will be found to be statistically significant. Conversely, “nonsignificance” may reflect no association between eating a food and illness or it could reflect a study size too small to detect a true association.

30. No Statistically Significant Findings?
Too few study subjects Did not ask about food or other exposure that led to outbreak Multiple contaminated food items Everyone ate the contaminated food Problems with study
Sometimes, no statistically significant association is found in a controlled (analytic) epidemiologic study. What does that mean?
If a study is too small, you might not find an association even though one exists.
It could mean that you did not ask about the exposure that led to the outbreak.
If multiple foods from an event or establishment are contaminated (not an unlikely scenario), some cases might have become ill because of exposure to one food and other cases might have become ill because of exposure to a different food. As you can imagine, this will influence the odds ratios and relative risks.
If everyone ate the contaminated item, it will be impossible to identify the item through an analytic study which compare those who ate the item with those who did not.
If the study had other problems
Let’s focus a bit on this last area …

31. Potential Study Problem Areas
Investigator “beliefs” about the cause of the outbreak (investigator bias)
Study participation (selection bias)
Accuracy of information on development of illness or foods eaten (information bias)
Quality of study (investigator error)
Many things can affect whether the results from an epidemiologic study reflect the truth.
Investigator beliefs – Sometimes investigators come into an investigation with unfounded, predetermined beliefs about the source of an outbreak and (consciously or unconsciously) guide investigation activities and interpretation of results to support those unfounded beliefs. This is sometimes referred to as investigator bias (and is why clinical trials are done in a “blinded” fashion, that is the investigators [and the patients] do not know what treatment the patient received when they determine the outcome of treatment.) Investigators must keep an open mind about the source of an outbreak and undertake studies based on sound hypothesis generation (not unfounded beliefs).
Study participation – If the persons enrolled in a study are not representative of all of the people who could have participated, study results might not represent the truth. This is sometimes called selection bias. Selection bias is more likely to influence study results if participation is poor. Good study participation by potential study subjects is critical.
Accuracy of information on exposure to the food in question and development of illness – The ability of epidemiologic studies to pinpoint the cause of an outbreak rests on the comparison of ill people to well people and exposed people to unexposed people. If study subjects are not categorized appropriately, the study results might not represent the truth. This is sometimes called information bias. “Illness” and “not being ill” must be carefully defined and consistently applied to all subjects. Exposure information should be collected as soon as possible after recognition of the outbreak in a systematic manner without biasing subjects.
Quality of study – If errors are made in study design or conduct and/or data entry or analysis, study results might not represent the truth. This is sometimes referred to as investigator error. Knowledgeable and experienced investigators need to be involved in designing studies. Study protocols need to be carried out as designed. Information collected from study subjects needs to be recorded and entered accurately, and data need to be analyzed and interpreted carefully.
Be an astute consumer of statistical information. Consider the quality of the study and results carefully!

32. Work in groups of two or three.
SMALL GROUP Exercise
Work in groups of two or three.
Read the brief description of a study that was undertaken following at outbreak associated with an office potluck.
Circle any red flags that make you concerned about the study conduct or its findings
Do you agree with the investigator about the cause of the outbreak?
FULL-SIZED HANDOUT: Identifying Problem Areas in an Epidemiologic Study (at end of module)
Be prepared to share with class.
Time: 10 minutes

33. SMALL GROUP Exercise
A holiday potluck luncheon was held on December 22 at the headquarters of a private business. Submarine sandwiches were purchased from a local deli. Staff members and their spouses were invited and asked to bring a side dish. Over 120 persons attended the luncheon. About 40 side dishes were brought by attendees. Staff and spouses socialized, ate, and drank most of the afternoon. The office was then closed for the holidays.
This slide is just setting the scene.
Red flag:
Conga dances and people thinking they can dance after too much partying!

34. SMALL GROUP EXERCISE (cont’d)
The office reopened on January 5. At a managers’ meeting a week later (January 12), several managers reported that they or their spouses had become ill following the potluck. Symptoms included predominantly nausea with some vomiting. None had fever. None sought medical care. The illnesses lasted less than a day. Several managers thought the illness was due to potato salad brought by the boss’ wife.
Red flags:
Over three weeks since potluck.
Illness is rather vague and doesn’t seem to have been too severe. Could these symptoms have just resulted from too much partying?
Discussion amongst managers that the potato salad was the problem might bias staff into thinking that potato salad was the problem.

35. SMALL GROUP EXERCISE (cont’d)
An intern working with the company, who had taken a course in epidemiology in college, volunteered to do a cohort study. The intern sent an to all persons invited to the party asking: Did you get sick after the office holiday potluck held on December 22? Did you eat the potato salad?
By January 20, responses had been received from 30 people. The intern analyzed the results using Epi-Info.
Red flags:
What experience has the intern had analyzing data? Data analysis software makes it very easy for investigators to enter data and run with the analysis when they might have little understanding of what they are doing.
The intern does not collect detailed information on the illness (e.g., signs and symptoms) and just asks if the person “got sick”. He doesn’t ask when they got sick either, just whether they were sick after the potluck.
The intern asks only about the potato salad and does not examine any other exposures.
The information is collected a fairly long time (almost a month) after the potluck. Will people remember what they ate? Will people be able to identify the potato salad? Was there more than one potato salad?
Thirty people out of 120 attendees is not a very good response rate. They may not represent all persons attending the potluck.

36. Question (cont’d)
The intern reported his findings to the boss: Fourteen people who attended the potluck said they had been ill. 15 said they had not been ill. Ten of the 14 ill were managers or their spouses. The 10 ill managers or spouses said they all had eaten potato salad made by the boss’ wife.
The intern shared these calculations:
10 (64%) of 14 ill people ate potato salad.
4 (25%) of 16 well people ate potato salad.
Red flags:
Some of the numbers do not match up. The report says 15 of the 30 respondents did not get sick. The calculations below suggest 16 people did not get sick.
A large portion of the “ill persons” who participated in the study were involved in the discussion at the managers meeting about the potato salad being the cause of the illness. Is it possible that hearing that discussion influenced whether they remembered they ate the potato salad or not?
The intern said he would conduct a cohort study but he calculates the odds ratio. The measure of association for a cohort study is a relative risk.
10 divided by 14 is 71%. (That’s for you epidemiologists out there!)
The odds ratio (best single estimate for the measure of association) does not fall within the confidence interval. (Something is wrong!)
odds ratio=10
p-value = % CI =

37. Question (cont’d)
The intern reported to the boss that the outbreak was caused by his wife’s potato salad and that he was not surprised because it tasted terrible. The intern was reassigned to the file storage room in the basement.
Red flags:
Just because an association is “statistically significant” based on the p-value that does not necessarily mean the relationship is causal.

38. ALWAYS SCRUTINIZE STUDY RESULTS!

39. Quick Quiz
These “quick quizzes” are meant to be a rapid self-assessment so that participants can see if they understood the most important concepts in this module.
The instructor will read the question aloud.
Think about the answer for a moment.
When asked, please indicate your answer by raising the appropriate color-coded card.
If you do not get an answer right, please make note of it and revisit the module during the break or over lunch (or talk to the instructor) to see if you can determine the correct answer. This is meant to be quick. We will not discuss the answers at any length.
[NOTE: Instructors should make sure everyone is aware of the correct answer and briefly explain why it is the answer, but should not go into great detail. The detailed explanations with each question are provided to you to make sure you agree with the answer. If students have questions they should ask the instructor during the break or over lunch.]

40. A case series includes a comparison group. True False
Quick Quiz
A case series includes a comparison group.
True
False
ANSWER: B. False.
A case series includes only cases. It does not include a comparison group. Case-control and cohort studies include a comparison group that allows investigators to quantify relationships between a suspected exposure and a disease, test suspicions about the cause of an outbreak, and explore the role of chance in the findings.

41. What is the measure of association for a case- control study?
Quick Quiz
What is the measure of association for a case- control study?
Odds ratio
Relative risk
p-value
Confidence interval
ANSWER: A. Odds ratio
The odds ratio is the measure of association for a case-control study.
The relative risk is the measure of association for a cohort study.
The p-value and the confidence interval indicate the role that chance plays in the study findings. (The confidence interval also reflects the precision of the measure of association.)

42. Apple cider is the cause of the outbreak.
Quick Quiz
In a cohort study, the relative risk for drinking apple cider is Which interpretation is correct?
Apple cider is the cause of the outbreak.
People who drank apple cider were almost 5 times more likely to become ill than those who did not.
Apple cider is protective.
The association between apple cider and illness is statistically significant.
ANSWER: B. People who drank apple cider were almost 5 times more likely to become ill than those who did not.
A cohort study compares the attack rate among people exposed to a particular food with the attack rate of people not exposed to the food. It tells us how much more likely it is for people exposed to the food to become ill than those not exposure to the food.
A relative risk greater than 1.0 indicates that an exposure might be a risk factor for illness. (A relative risk less than 1.0 indicates the exposure might be “protective”.)
The relative risk itself does not indicate statistical significance.

43. Quick Quiz
Which of the following is a true statement about the p-value?
The p-value cut-off for statistical significance is always 0.05.
The p-value indicates the public health significance of an association between a food and an illness.
The p-value is not affected by study size.
A p-value of 0.05 means that there is a 5 in 100 probability that the observed association between the food and illness is due to chance alone.
ANSWER: D. A p-value of 0.05 means that there is a 5 in 100 probability that the association between the food and illness that was observed in the epidemiologic study is due to chance alone.
The p-value indicates the role that chance plays in the study findings. It is the probability of finding an association between an exposure and a disease (as strong as the one observed) due to chance alone.
The p-value cut-off for “statistical significance” is set by the investigator is often the cut-off selected, but other p-values are used in certain studies.
By itself, the p-value does not indicate a cause-effect relationship. An observed association may indeed represent a causal relationship, but it may still be due to chance or another factor related to both the exposure and the diseases (i.e., confounders), or other issues surrounding the study design and execution.
The p-value is impacted by study size. The more subjects included in the study, the more likely that the measure of association (even if only slightly elevated above 1.0) will be found to be statistically significant. For this and other reasons, the p-value does not reflect public health significance.

44. Quick Quiz
Only epidemiologists should interpret results from epidemiologic studies.
True
False
ANSWER: B. False
All outbreak investigation team members should be able to interpret measures of association and p-values. They should also routinely scrutinize (and question) the results of epidemiologic studies.

45. Relative Risk (Optional)
Relative Risk (Optional)
Ill
Well
TOTAL
Ate food a b
a + b
Did not eat food c d
c + d
(two-by-two table)
attack rate(ate food) = a/(a + b)
attack rate(did not eat food) = c/(c+d)
attack rate among those eating food
attack rate among those not eating food
relative risk =
NOTE: It is good for students to appreciate where the numbers come from in calculating relative risks, but they will not be expected to calculate them. We will leave the actual calculation of relative risks to our epidemiology friends and their computers. A number of programs are available to support epidemiologists in these analyses including Epi Info, SAS, SPSS, and other tools.
To calculate the relative risk, epidemiologists usually layout the numbers from a cohort study in a two-by-two table. It is called a two-by-two table because there are two categories for exposure (i.e., ate food item and did not eat food item) and two categories for disease (i.e., ill and well).
Note: This two by two table looks a lot like the one used in the case-control study, but you approach it the other way around looking at people who ate the food and people who did not eat the food.
In the two-by-two table, “a” represents the number of persons eating the food who became ill. “b” represents the number of persons eating the food who did not become ill. “a + b” represents the total number of persons eating the food. [Advance slide.] The attack rate among persons eating the food is “a/(a+b)”.
“c” represents the number of persons who did not eat the food who became ill. “d” represents the number of persons who did not eat the food who became ill. “c + d” represents the total number of persons who did not eat the food. [Advance slide.] The attack rate among persons who did not eat the food is “c/(c+d)”.
[Advance slide.]
The relative risk is then the ratio of the attack rates among persons eating the food and persons not eating the food or “exposed/unexposed.”
a/(a+b)
c/(c+d) =
relative risk
For more information, see Appendix on Calculating Measures of Association

46.  Odds Ratio (Optional) Case Control Ate food a b Did not eat food c d
TOTAL
a + c
b + d
(two-by-two table)
odds of eating food (cases)
odds of eating food (controls)
odds ratio = =
a/c
b/d
NOTE: It is good for students to appreciate where the numbers come from in calculating odds ratios, but they will not be expected to calculate them. We will leave the actual calculation of odds ratios to our epidemiology friends and their computers. A number of programs are available to support epidemiologists in these analysis including Epi Info, SAS, and SPSS.
To calculate the odds ratio, epidemiologists usually lay out the numbers from a case-control study in a two-by-two table. It is called a two-by-two table because there are two categories for exposure (i.e., ate food item and did not eat food item) and two categories for disease (i.e., case and control).
In the two-by-two table, “a” represents the number of cases who ate the food of interest. “c” equals the number of cases who did not eat the food. “b” equals the number of controls who ate the food. And “d” equals the number of controls who did not eat the food.
[Advance slide.]
The odds of eating food among cases is the number of cases who ate the food divided by the number of cases who did not eat the food (“a” divided by “c”).
The odds of eating food among controls is the number of controls who ate the food divided by the number of controls who did not eat the food (“b” divided by “d”).
The odds ratio is the odds of eating the food among cases divided by the odds of eating the food among controls.
If you do the math, the odds ratio is the cross product ratio: a times d divided by b times c.
Some remember this by thinking “Expected/Unexpected”. You expect cases to have eaten the food (cell A) and you expected controls not to have eaten the food (cell d). You don’t expect cases not to have eaten the food (cell B) or the controls to have eaten the food (cell D).
odds ratio =
a x d
b x c
(cross product)
For more information, see Appendix on Calculating Measures of Association