Validity Generalization Hein Stigum Presentation, data and programs at: http://folk.uio.no/heins/ Sep-18 H.S.

Precision and validity Measures of populations precision - random error - statistics validity - systematic error – epidemiology Lack of validity = measures are biased type of bias direction of the bias True value Estimate Precision Bias Random error: study samples, p-values and confidence intervals Systematic error: largely observational studies, little control Freq measures (incidence and prevalence) and association measures (RR, OR, RD) The measures are biased (not the study): type, direction Example CC: prevalence is biased, OR is correct Sep-18 H.S.

Type of bias Selection bias Information bias Confounding Are those who answer different? Information bias Do they tell the truth? Confounding Is the association a cause? def. of main types distortion of estimate ... Ex: If you send out a questionairre to study illiteracy, subject who cannot write will not answer, and we have a serious selection bias. Information bias ... Ex: If you want to study the effect of passiv smoking on astma, the exposure to passiv smoking is difficult to measure,... Recall events in the past Confounding… ecological study of obesity and diabetes by contry, many other factors that differ. Comparison bias. Also in individ bases designs, example later Classification not alway clear Sep-18 H.S.

Selection bias Sep-18 Sep-18 H.S. H.S. 4

Sources of selection bias Selective response sexual survey Self selection Nevada atom test and leukemia Loss to follow up connected to disease air pollution and astma Healthy worker effect aluminium workers and lung disease Selective response sexual survey, never sex: no response. Show next 2 slides All designs Cohort Self selection: Nevada atom test and leukemia, soldiers exposed , 20% self selected, 80% of the exposed sample traced by investigators, cases counted, 4 times as much leukemia among self selected. Internet surveys is another example Loss to follow up not random: air pollution and asthma, worst cases move to less polluted areas, lost , lower incidence, less RR In general: if loss is connected to disease, get bias Selective survival. Show slide +3 after this Study effect of work exposures: heat and dust i Norw aluminium and disease, less disease among heavy exposed worker. Disease self-selceted out, healthy worker effect. Cross, CC with prevalent cases Case-control, Cases: Ex: HTLV-and leukemia/neurological disorder (ATL, TSP), death after 20-30 y disease. If select prevalent cases, most severe case may have died, selective survival Controls: Controls from a hospital (without the study disease), get biases results. Controls should be from the same population as the cases. Sep-18 H.S.

Selection bias Population Sample Respons Responders Non-responders If non-respone is random, OK If non-respone is connected to outcome, get bias (Go back) Outcome Sep-18 H.S.

Selective survival Disease Death Sep-18 H.S. Exposure at time 0. Start study at time 10 If we do our study CC on prevalent cases, get only the the two lower ones Use incident cases instead Go back 3 slides Sep-18 H.S.

Information bias Sep-18 Sep-18 H.S. H.S. 8

Non-differential misclassification True smoking 10% of smokers report no smoking Non-differential: RR Sep-18 H.S.

Sources of information bias Not true males report more partners than females Not blinded passive smoking and astma Selective recall alcohol in pregnancy and malformations Wrong info sex surveys partners: recall or wrong information Not blinded Cohort: asthma unclear, doctors not blinded to exposure may lead to differential misclass CC: passiv smoking unclear, experimenter not blinded to disease may lead to differential misclass Selective recall CC: recall: mothers with malformed babies remember more alcohol use, differential misclass information bias depends on data source: registry, doctors diagnosis, laboratory test, self report interview or questionnaire A bias in the information may lead to a misclassification of disease status or exposure status or both. 2 types of measures: Freq Ex: if we measure prev of a disease and we know that doctors do not find all cases, then some cases are misclassified, we say that our prev measure is biased (toward lower value) Unclear diagnosis, asthma,. For assoc. we may have .. an important consideration is connected to exp or dis. Assoc.Ex: asthma and air pollution, assume no association, doctors know who are exposed, then the exposed may be examined more carefully than unexposed. This would be a differential miscl. Therefore doctors should be “blinded” to exposure status. Sep-18 H.S.

CONFOUNDER DEFINITION Sep-18 Sep-18 H.S. H.S. 11

Confounding Ideal: Practice: Comparison bias: Same subjects are both exposed and unexposed at the same time, (counterfactual) Practice: As equal as possible Comparison bias: Confounding Exposed Unexposed General definition As equal as possible, except for exposure Now focus on one spesific confounder Sep-18 H.S.

Associations E and D associated E causes D E and D have common cause Both Overall E-D association = spurious effect from C + causal E-D effect E D E D C 3 ways to be associated (show 2 here): Smoking causes lung cancer Yellow fingers and lung cancer are associated from common cause smoking Remove spurious effect by stratify, adjust or condition on C E D C Sep-18 Sep-18 H.S. H.S. 13 13

Classic confounder C is the confounder RRED is biased RRED|C is unbiased RRED =0.8 positive bias Adjust for age: RRED|C=0.5 is unbiased E D C birth defects vitamins age + - true biased 1  Sep-18 Sep-18 H.S. H.S. 14

Confounding: Downs syndrom by parity % Confounding, ex Downs (mongoloism), reduced mental capacity, birth registry, prev=12/10 000=0.1% Parity=number of previous births Sep-18 H.S.

Downs by parity and mothers age % Stratify on mothers age No effect of parity, large effect og age Sep-18 H.S.

Downs syndrom, logistic regression Crude Adjusted Mod 1 unadjusted (crude) (rare disease: 1/1000 ) Mod 2 adjusted Same adjustment in linear models Confounding, example of systematic error->bias Example of estimation (rather than testing) Sep-18 H.S.

Generalization Sep-18 Sep-18 H.S. H.S. 18

Generalization Do the result apply outside the sample? Statistical generalization Smoking among males, generalize to females? Representative sample Biological generalization Drug effect on males, generalize to females? Information from outside the study Animal studies, generalize to humans? Homogenous sample Convinced ourselves that the results are valid, do they apply outside the sample. Generalize from males to females: Prevalence of smoking No Association smoking-early sexual debut Probably not Drug effect Yes, if information on metabolic pathway Stronger results if homogenous sample Do hamsters need to be representative? Two conflicting ways: representative will not be homogenous Sep-18 H.S.