1. Medical Research: Validity, Bias, Confounds, Probability, P-value, and Power
Kimberly Pendell

2. Validity
Validity implies that the measurement reliably measures what it intends to measure.
Validity relates to the magnitude of bias.
Example of non-valid measure: eye-color as measure of the likelihood of having a stroke
Example of valid measure: diagnosis of atherosclerosis as measure of the likelihood of having a stroke

3. Bias
Bias is the systematic tendency to produce an outcome that differs from the underlying truth.
Bias is different from random error. Bias leads to systematic deviations (the error has direction). Random error is inherent in all measurement and is by chance--the average value of a measurement of a sample population is subject to variation from the true value for the entire population.

4. Some different types of bias
Selection bias
Interviewer bias
Recall bias
Detection bias
Verification bias
Publication bias
Selection: An error in choosing the individuals or groups to take part in a study. Ideally, the subjects in a study should be very similar to one another and to the larger population from which they are drawn (for example, all individuals with the same disease or condition). If there are important differences, the results of the study may not be valid.
Interviewer: Greater probing by an interviewer in one of two groups being compared
Recall: patients who experience an adverse outcome have a greater likelihood of recalling an exposure than patients who do not have an adverse outcome, regardless of the true extent of exposure
Detection (or surveillance): the tendency to look more carefully for an outcome in one of two groups being compared
Verification: results of diagnostic test influence whether patients are assigned to treatment group
Publication: occurs when the publication of research depends on the direction of the study results and whether they are statistically significant.

5. Bias and Research Designs
Ex. research design
Ex. of potential bias
Ex. of potential solution
RCT
Selection bias
Randomization
Case-control study
Recall bias
Blind subjects to research hypothesis
Meta-analysis
Publication bias
Look for unpublished data

6. Confounds
A confound is a variable that is distributed differently in the study group and the control group and that affects the outcome being assessed. (Riegelman, 2005)
Confounding variables can appear through bias or random chance.
Find example!!

7. Probability
Probability is the quantitative estimate of the likelihood of condition existing (as in a diagnosis) or of subsequent events (such as in a treatment study). (Guyatt, Rennie, 2002)

8. P-value
P-value is the probability of an outcome occurring by chance. The p-value establishes statistical significance.
In medical research the standard p-value is p<.05

9. Power
Power is the ability of a research study to demonstrate statistical significance.
Researchers determine the size of study population needed for a set amount of power before beginning their study.
If the study has statistically significant results, the study has enough power.
In medical research the standard for power is 80%.

10. Readings:
Guyatt, G., Rennie, D. (Eds.). (2002). Users’ guide to the medical literature: a manual for evidence-based practice. USA: American Medical Association.
Katzer, J., Cook, K.H., Crouch, W.W. (1998). Evaluating information: a guide for users of social science research. Massachusetts: McGraw Hill.
Riegelman, R.K. (2005). Studying a study & testing a test: how to read the medical evidence. Philadelphia, PA: Lippincott Williams & Wilkins.