Decision theory and Bayesian statistics. Tests and problem solving Petter Mostad

Overview Statistical desicion theory Bayesian theory and research in health economics Review of tests we have learned about From problem to statistical test

Statistical decision theory Statistics in this course often focus on estimating parameters and testing hypotheses. The real issue is often how to choose between actions, so that the outcome is likely to be as good as possible, in situations with uncertainty In such situations, the interpretation of probability as describing uncertain knowledge (i.e., Bayesian probability) is central.

Decision theory: Setup The unknown future is classified into H possible states: s 1, s 2, …, s H. We can choose one of K actions: a 1, a 2, …, a K. For each combination of action i and state j, we get a ”payoff” (or opposite: ”loss”) M ij. To get the (simple) theory to work, all ”payoffs” must be measured on the same (monetary) scale. We would like to choose an action so to maximize the payoff. Each state s i has an associated probability p i.

Desicion theory: Concepts If action a 1 never can give a worse payoff, but may give a better payoff, than action a 2, then a 1 dominates a 2. a 2 is then inadmissible The maximin criterion The minimax regret criterion The expected monetary value criterion

Example No birdflu outbreak Small birdflu outbreak Birdflu pandemic No extra precautions Some extra precautions Vaccination of whole pop states actions

Decision trees Contains node (square junction) for each choice of action Contains node (circular junction) for each selection of states Generally contains several layers of choices and outcomes Can be used to illustrate decision theoretic computations Computations go from bottom to top of tree

Updating probabilities by aquired information To improve the predictions about the true states of the future, new information may be aquired, and used to update the probabilities, using Bayes theorem. If the resulting posterior probabilities give a different optimal action than the prior probabilities, then the value of that particular information equals the change in the expected monetary value But what is the expected value of new information, before we get it?

Example: Birdflu Prior probabilities: P(none)=95%, P(some)=4.5%, P(pandemic)=0.5%. Assume the probabilities are based on whether the virus has a low or high mutation rate. A scientific study can update the probabilities of the virus mutation rate. As a result, the probabilities for no birdflu, some birdflu, or a pandemic, are updated to posterior probabilities: We might get, for example:

Expected value of perfect information If we know the true (or future) state of nature, it is easy to choose optimal action, it will give a certain payoff For each state, find the difference between this payoff and the payoff under the action found using the expected value criterion The expectation of this difference, under the prior probabilities, is the expected value of perfect information

Expected value of sample information What is the expected value of obtaining updated probabilities using a sample? –Find the probability for each possible sample –For each possible sample, find the posterior probabilities for the states, the optimal action, and the difference in payoff compared to original optimal action –Find the expectation of this difference, using the probabilities of obtaining the different samples.

Utility When all outcomes are measured in monetary value, computations like those above are easy to implement and use Central problem: Translating all ”values” to the same scale In health economics: How do we translate different health outcomes, and different costs, to same scale? General concept: Utility Utility may be non-linear function of money value

Risk and (health) insurance When utility is rising slower than monetary value, we talk about risk aversion When utility is rising faster than monetary value, we talk about risk preference If you buy any insurance policy, you should expect to lose money in the long run But the negative utility of, say, an accident, more than outweigh the small negative utility of a policy payment.

Desicion theory and Bayesian theory in health economics research As health economics is often about making optimal desicions under uncertainty, decision theory is increasingly used. The central problem is to translate both costs and health results to the same scale: –All health results are translated into ”quality adjusted life years” –The ”price” for one ”quality adjusted life year” is a parameter called ”willingness to pay”.

Curves for probability of cost effectiveness given willingness to pay One widely used way of presenting a cost-effectiveness analysis is through the Cost- Effectiveness Acceptability Curve (CEAC) Introduced by van Hout et al (1994). For each value of the threshold willingness to pay λ, the CEAC plots the probability that one treatment is more cost-effective than another.

Review of tests Below is a listing of most of the statistical tests encountered in Newbold. It gives a grouping of the tests by application area For details, consult the book or previous notes!

One group of normally distributed observations Goal of test:Test statistic:Distribution: Testing mean of normal distribution, variance known standard normal: Testing mean of normal distribution, variance unknown t-fordelingen, n-1 frihetsgrader: Testing variance of normal population Chi-kvadrat, n-1 frihetsgrader

Comparing two groups of observations: matched pairs Assuming normal distributions, unknown variance: Compare means Sign test: Compare only which observations are largest S = the number of pairs with positive difference. Large samples (n>20): Wilcoxon signed rank test: Compare ranks and signs of differences T=min(T +,T - ); T + / T - are sum of positive/negative ranks Wilcoxon signed rank statistic (D 1, …, D n differences) Large samples:

Comparing two groups of observations: unmatched data Diff. between pop. means: Known variances Standard normal Diff. between pop. means: Unknown but equal variances Diff. between pop. means: Unknown and unequal variances Testing equality of variances for two normal populations Assuming identical translated distributions: test equal means: Mann Whitney U test Based on sum of ranks of obs. from one group; all obs. ranked together Standard normal (n>10) see book for d.f.

Comparing more than two groups of data One-way ANOVA: Testing if all groups are equal (norm.) Kruskal-Wallis test: Testing if all groups are equal Based on sums of ranks for each group; all obs. ranked together Two-way ANOVA: Testing if all groups are equal, when you also have blocking Two-way ANOVA with interaction: Testing if groups and blocking variable interact

Studying population proportions Test of population proportion in one group (large samples) Standard normal Comparing the population proportions in two groups (large samples) Standard normal (p 0 common estimate)

Regression tests Test of regression slope: Is it ? Test on partial regression coefficient: Is it ? Test on sets of partial regression coefficients: Can they all be set to zero (i.e., removed)?

Model tests Contingency table test: Test if there is an association between the two attributes in a contingency table Goodness-of-fit test: Counts in K categories, compared to expected counts, under H 0 Tests for normality: Bowman-Shelton Kolmogorov-Smirnov

Tests for correlation Test for zero population correlation (normal distribution) Test for zero correlation (nonparametric): Spearman rank correlation Compute ranks of x- values, and of y- values, and compute correlation of these ranks Special distribution

Tests for autocorrelation The Durbin-Watson test (based on normal assumption) testing for autocorrelation in regression data Special distribution The runs test (nonparametric), testing for randomness in time Counting the number of ”runs” above and below the median in the time series Special distribution, or standard normal for large samples

From problem to choice of method Example: You have the grades of a class of studends from this years statistics course, and from last years statistics course. How to analyze? You have measured the blood pressure, working habits, eating habits, and exercise level for 200 middleaged men. How to analyze?

From problem to choice of method Example: You have asked 100 married women how long they have been married, and how happy they are (on a specific scale) with their marriage. How to analyze? Example: You have data for how satisfied (on some scale) 50 patients are with their primary health care, from each of 5 regions of Norway. How to analyze?