Decision Analysis Lecture 4

Decision Analysis Lecture 4
Tony Cox My Course web site:

Agenda Problem set 3 solutions Assignment 4: Bayesian inference
Simulation-optimization for Joe’s pills Assignment 4: Bayesian inference Introduction to Netica for Bayesian inference Wrap-up on decision trees Binomial distribution

Homework #4 (Due by 4:00 PM, February 14)
Problems Machines Fair coin Readings Required: Clinical vs. statistical predictions, Recommended: Important probability distributions Binomial Recommended: Binomial distribution in R

Assignment 3, Problem 1 (ungraded)
A fair coin is tossed once. Draw the risk profile (cumulative distribution function) for the number of heads. Purpose: Be able to draw, interpret risk profiles Practice! You do not have to turn this in, but we will go over the solution next class Helpful background on discrete CDFs:

Assignment 3, Solution 1 A fair coin is tossed once. Draw the risk profile (cumulative distribution function) for the number of heads. Solution:

Assignment 3, Problem 2 Joe’s medicine
Joe takes pills to reduce his risk of heart attack Pharmacist can prescribe for him either 1 pill per day at full strength, or 2 pills per day, each at half strength The probability that Joe forget to take any given pill on any occasion is p. Its value is uncertain. Here is how pills affect daily heart attack risk: If he takes full strength pill, multiply his risk by 0.5 (it is cut in half) If he takes 1 half-strength pill, multiply risk by 0.7 If he takes both half-strength pills, multiply his risk by 0.5 If he takes no pill, multiply risk by 1 What should the pharmacist prescribe? Please submit answer as two ranges (intervals) of p values for which the best choice is (A) Prescribe 1 full strength pill; (B) Prescribe 2 half-strength pills

Binomial distribution with parameters p and N = 2
The probability that Joe takes 0 pills is p2 Pr(takes 2 pills) = (1- p)2 Pr(takes 1 pill) = p(1 - p) + (1 - p)p = 2p(1-p)

Assignment 3, Solution 2 Joe’s medicine
The probability that Joe forgets to take any pill is p. Here is how pills affect daily heart attack risk: If he takes full strength pill, multiply his risk by 0.5 (it is cut in half) If he takes 1 half-strength pill, multiply risk by 0.7 If he takes both half-strength pills, multiply his risk by 0.5 If he takes no pill, multiply risk by 1 What should the pharmacist prescribe? 1 full pill per day reduces Joe’s risk by p*1 + (1-p)*0.5 = p 2 half pills per day reduce risk by (p2)*1 + 2p(1-p)*0.7 + (1-p)2*0.5 = p p – 1.4p2 + (1 -2p + p2)*0.5 = 0.1p p + 0.5 2 pills are better than 1 (i.e., have lower risk for Joe) if p > 0.1p p  0.5p > 0.1p p  0.1p > 0.1p2  p > p2 But p > p2 for all 0 < p < 1. If p = 0 or 1, they are equally good. Otherwise, 2 pills are always better than 1, for all p in (0, 1).

Graphical solution 2 pills deterministically dominates 1 pill over the whole (infinite) set of states in (0, 1). If EU(a) lines crossed, then we would have to assess probabilities for the values of p. Simulation-optimization: Could solve using decision tree and simulation of EU(a), given model {u(c), Pr(s), and Pr(c | a, s)}. (Here, state s is p.) For each a: Draw s from Pr(s) Draw c from Pr(c | a, s) Evaluate u(c) Repeat and average u(c) values Select a with greatest mean(u(c))

Assignment 3, Problem 3 Certainty Equivalent calculation
If you buy a raffle ticket for $2.00 and win, you will get $19.00; else, you will receive nothing from the ticket. The probability of winning is 1/3 Your utility function for final wealth x is u(x) = log(x) Your initial wealth (before deciding whether to buy the ticket) is $10 What is your certainty equivalent (selling price) for the opportunity to buy this raffle ticket? Please submit one number Should you buy it? Please answer Yes or No

Assignment 3, Solution 3 Certainty Equivalent calculation
If you buy a raffle ticket for $2 and win, you will get $19.00; else, you will receive nothing from the ticket. The probability of winning is 1/3 X = random variable for final wealth if you buy ticket = = $27 with probability 1/3, else = $8. Your utility function for final wealth x is u(x) = log(x) Your initial wealth is $10 Let CE = CE(X) = CE of final wealth if you buy ticket u(CE) = EU(X) = (1/3)*log( ) + (2/3)*log(10 - 2) = CE = exp(2.4849) = $12. So, deciding to buy the ticket increases your CE(wealth) from $10 to $12. This transaction is worth $2 to you.

Assignment 3, Solution 3 Certainty Equivalent calculation
u(CE) = EU(X) = (1/3)*log( ) + (2/3)*log(10 - 2) = CE = exp(2.4849) = $12. So, ticket increases your CE(wealth) from $10 to $12 and is worth $2 to you Note: EMV(X) = (1/3)*( ) + (2/3)*(10 - 2) = $14.33, so your risk premium is $ $12.00 =$2.33 Note: Suppose initial wealth is 1000: Then CE(final wealth) is exp((1/3)*log( ) + (2/3)*log( )) = (compared to EMV = (1/3)*17 + (2/3)*(-2) = ). Risk premium is $0.04.

Assignment 4,Problem 1: Fair Coin Problem (due 2-14-17)
A box contains two coins: (a) A fair coin; and (b) A coin with a head on each side. One coin is selected at random (we don’t know which) and tossed once. It comes up heads. Q1: What is the probability that the coin is the fair coin? Q2: If the same coin is tossed again and shows heads again, then what is the new (posterior) probability that it is the fair coin? Solve manually and/or using Netica.

Assignment 4, Problem 2: Defective Items (due 2-14-17)
Machines 1, 2, and 3 produced (20%, 30%, 50%) of items in a large batch, respectively. The defect rates for items produced by these machines are (1%, 2%, 3%), respectively. A randomly sampled item is found to be defective. What is the probability that it was produced by Machine 2? Exercise: (a) Solve using Netica (b) Solve manually answer (a single number,) to

Introduction to Bayesian inference with Netica®

Example: HIV screening
Pr(s) = 0.01 = fraction of population with HIV s = has HIV, s′ = does not have HIV y = test is positive Pr(test positive | HIV) = 0.99 Pr(test positive | no HIV) = 0.02 Find: Pr(HIV | test positive) = Pr(s | y) Subjective probability estimates?

Solution via Bayesian Network (BN) Solver
DAG model: “True state  Observation” DAG = “directed acyclic graph”: Nodes and arrows, no cycles allowed Store “marginal probabilities” at input nodes (having output arrows only) Store “conditional probability tables” at all other nodes. Make observations Enter query Solver calculates conditional probabilities

Solution in Netica Step 1: Build model, compile network

Solution in Netica Step 1: Build model, compile network
Step 2: Condition on observation (right-click, choose “Enter findings”), view conditional probabilities

Wrap-up on Netica introduction
User just needs to enter model and observations (“findings”) Netica uses Bayesian Network algorithms to update all probabilities (conditioning them on findings) We will learn to do this manually for small problems Algorithms and software are essential for large, complex inference problems

Review and wrap-up on decision trees and probabilities

Decision tree ingredients
Three types of nodes Choice nodes (squares) Chance nodes (circles) Terminal nodes / value nodes Arcs show how decisions and chance events can unfold over time Uncertainties are resolved as time passes and choices are made

Solving decision trees
“Backward induction” “Stochastic dynamic programming” “Average out and roll back”  implicitly, tree determines Pr(c | a) Procedure: Start at tips of tree, work backward Compute expected value at each chance node “Averaging out” Choose maximum expected value at each choice node

Obtaining Pr(s) from Decision trees http://www. eogogics
Decision 1: Develop or Do Not Develop Development Successful + Development Unsuccessful (70% X $172,000) + (30% x (- $500,000)) $120,400 + (-$150,000)

What happened to act a and state s. http://www. eogogics
Decision 1: Develop or Do Not Develop Development Successful + Development Unsuccessful (70% X $172,000) + (30% x (- $500,000)) $120,400 + (-$150,000)

What are the 3 possible acts in this tree?

What are the 3 possible acts in this tree? (a) Don’t develop; (b) Develop, then rebuild if successful; (c) Develop, then new line if successful.

Optimize decisions! What are the 3 possible acts in this tree? (a) Don’t develop; (b) Develop, then rebuild if successful; (c) Develop, then new line if successful.

Key points Solving decision trees (with decisions) requires embedded optimization Make future decisions optimally, given the information available when they are made Event trees = decision trees with no decisions Can be solved, to find outcome probabilities, by forward Monte-Carlo simulation, or by multiplication and addition In general, sequential decision-making cannot be modeled well using event trees. Must include (optimal choice | information)

What happened to state s. http://www. eogogics
What are the 4 possible states?

What happened to state s. http://www. eogogics
What are the 4 possible states? C1 can succeed or not; C2 can be high or low demand

Acts and states cause consequences http://www. eogogics

Key theoretical insight
A complex decision model can be viewed as a (possibly large) simple Pr(c | a) model. s = selection of branch at each chance node a = selection of branch at each choice node c = outcome at terminal node for (a, s) Pr(c | a) = sPr(c | a, s)*Pr(s) Other complex decision models can also be interpreted as c(a, s), Pr(c | a, s), or Pr(c |s) models s = system state & information signal a = decision rule (information  act) c may include changes in s and in possible a.

Real decision trees can quickly become “bushy messes” (Raiffa, 1968) with many duplicated sub-trees

Influence Diagrams help to avoid large trees http://en. wikipedia
Often much more compact than decision trees

Limitations of decision trees
Combinatorial explosion Example: Searching for a prize in one of N boxes or locations involves building a tree of depth N! = N(N – 1)…*2*1. Infinite trees Continuous variables When to stop growing a tree? How to evaluate utilities and probabilities?

Optimization formulations of decision problems
Example: Prize is in location j with prior probability p(j), j = 1, 2, …, N It costs c(j) to inspect location j What search strategy minimizes expected cost of finding prize? What is a strategy? Order in which to inspect How many are there? N!

With two locations, 1 and 2 Strategy 1: Inspect 1, then 2 if needed:
Expected cost: c1 + (1 – p1)c2 = c1 + c2 – p1c2 Strategy 2: Inspect 2, then 1 if needed: Expected cost: c2 + (1 – p2)c1 = c1 + c2 – p2c1 Strategy 1 has lower expected cost if: p1c2 > p2c1, or p1/c1 > p2/c2 So, look first at location with highest success probability per unit cost

With N locations Optimal decision rule: Always inspect next the (as-yet uninspected) location with the greatest success probability-to-cost ratio Example of an “index policy,” “Gittins index” If M players take turns, competing to find prize, each should still use this rule. A decision table or tree can be unwieldy even for such simple optimization problems

Other optimization formulations
maxa A EU(a) Typically, a is a vector, A is the feasible set More generally, a is a strategy/policy/decision rule, A is the choice set of feasible strategies In previous example, A = set of permutations s.t. EU(a) = ∑cPr(c | a)u(c) Pr(c | a) = ∑sPr(c | a, s)p(s) g(a) ≤ 0 (feasible set, A)

Advanced decision tree analysis
Game trees Different decision-makers Monte Carlo tree search (MCTS) in games with risk and uncertainty Generating trees Apply rules to expand and evaluate nodes Learning trees from data Sequential testing

Summary on decision trees
Decision trees show sequences of choices, chance nodes, observations, and final consequences. Mix observations, acts, optimization, causality Good for very small problems; less good for medium-sized problems; unwieldy for large problems  use IDs instead Can view decision trees and other decision models as simple c(a, s) models But need good optimization solvers!

Road map: Filling in the normal form matrix
Assessing probabilities Eliciting well-calibrated probabilities Deriving probabilities from models Estimating probabilities from data Assessing utilities Utility elicitation Single-attribute utility theory Multi-attribute utility theory

Binomial probability model

Some useful probability models
Uniform = unif Binomial (n trials, 2 outcomes) = binom Poisson (“rare events” law) = pois Exponential (waiting time) = exp Normal (sums, random errors) = norm Beta (proportions) = beta p = distribution function (cdf) r = random sample (simulation) q = quantile, d = density

Binomial model pbinom(x, n, p)
Two outcomes on each of n independent trials, “success” and “failure” Probability of success = p for each trial independently Expected number of successes in n trials with success probability p = ? Probability of no more than x successes in n trials with success probability p = pbinom(x, n, p)

pbinom(x, n, p) includes probability of x succeses
Expected number of successes in n trials with success probability p = np Probability of no more than x successes in n trials with success probability p = pbinom(x, n, p) Note that pbinom is for less than or equal to x successes in n trials

Binomial model pbinom(x, n, p)
2 outcomes on each of n independent trials P(success) = p for each trial independently E(successes in n trials) = np = mean Pr(x successes in n trials) = nC xpx(1 – p)n-x = dbinom(x,n,p) nC x = “n choose x” = number of combinations of n things taken x at a time = n(n-1)…(n – x + 1)/x! Example: Pr(1 or 2 heads in 4 tosses of a fair coin) = ?

Binomial model pbinom(x,n,p), dbinom(x,n,p)
Pr(1 or 2 heads in 4 tosses of a fair coin) = Pr(1 head) + Pr(2 heads) = 4C 1p1(1 – p)3 + 4C 2p2(1 – p)2 = (4 + 6)*0.54 = 10/16 = 5/8 = 0.625 = dbinom(1,4,0.5)+ dbinom(2,4,0.5) = pbinom(2,4,0.5)- pbinom(0,4,0.5)

Example of binomial model pbinom(x, n, p)
Susan goes skiing each weekend if the weather is good n = 12 weekends in ski season Probability of good weather = 0.65 for each weekend independently What is the probability that she will ski for 8 or more weekends? (Use pbinom) Find her expected number of ski weekends

Do it!

Example of binomial model pbinom(x, n = 12, p = 0.65)
Expected number of weekends she skis is np = ? Probability of skiing for 8 or more weekends = 1 – Pr(no more than 7 ski trips in 12 weekends, with p = 0.65 for each) = ?

Example of binomial model pbinom(x, n = 12, p = 0.65)
Expected number of weekends she skis is np = 12*0.65 = 7.8 Probability of skiing for 8 or more weekends = 1 – Pr(no more than 7 ski trips in 12 weekends, with p = 0.65 for each) = 1- pbinom(7, 12, 0.65) > 1- pbinom(7, 12, 0.65) [1]

Optional practice problems on binomial calculations: Do using R
Ten percent of computer parts produced by a certain supplier are defective. What is the probability that a sample of 10 parts contains more than 3 defective ones? On the average, two tornadoes hit major U.S. metropolitan areas every year. What is the probability that more than five tornadoes occur in major U.S. metropolitan areas next year? A lab network consisting of 20 computers was attacked by a computer virus. This virus enters each computer with probability 0.4, independently of other computers. a) Find the probability that the virus enters at least 10 computers. b) A computer manager checks the lab computers, one after another, to see if they were infected by the virus. What is the probability that she has to test at least 6 computers to find the first infected one? Check answers at any questions on R solutions to

Plotting a binomial distribution (probability density function)
> x = c(0:12) > y = dbinom(x, 12, 0.65); plot(x,y) Probability distribution for number of ski weekends This “probability density” or “probability mass” function lets us calculate expected utility of season pass if its utility is determined by the number of ski weekends.

Plotting a binomial distribution
> barplot(dbinom(x, 12, 0.65))

Risk profile (CDF) for binomial
R: x <- c(0:12) R: y <- pbinom(x, 12, 0.65) G: plot(x, y)

Using the binomial model to calculate probabilities
A company will remain solvent if at least 3 of its 8 markets are profitable. The probability that each market is profitable is 25%. What is the probability that the company remains solvent?

Using the binomial model to calculate probabilities
A company will remain solvent if at least 3 of its 8 markets are profitable. The probability that each market is profitable is 25%. What is the probability that the company remains solvent? Pr(no more than 5 failure) = pbinom(5, 8, 0.75) [1]

Bayesian analysis and probability basics

How to get needed probabilities?
Derive from other probabilities and models; condition on data Bayes’ rule, decomposition and logic, event trees, fault trees, probability theory & models Monte Carlo simulation models Make them up (subjective probabilities), ask others (elicitation) Calibration, biases (e.g., over-confidence) Estimate them from data

Decision Analysis Lecture 4

Similar presentations

Presentation on theme: "Decision Analysis Lecture 4"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Decision Analysis Lecture 4

Similar presentations

Presentation on theme: "Decision Analysis Lecture 4"— Presentation transcript:

Similar presentations

About project

Feedback