Decision Analysis Lecture 7 Tony Cox My e-mail: tcoxdenver@aol.com Course web site: http://cox-associates.com/DA/

Agenda Project ideas Comments on problem set 6 (due March 7) Family out: Typos in Charniak Car buying: Hint – calculate EMV for each act. Decision psychology (cont.) Heuristics and biases Probabilistic expert systems (Netica) Wason selection task, Simpson’s Paradox Other probability models and practice problems Discrete: geometric, Poisson Continuous: exponential, beta, normal

Goals for applied projects Extract good problems from available knowledge and data “Good” = high value of analysis = large improvement in decisions, results, etc. Apply DA techniques to produce valuable answers and insights Unexpected directions are ok! Present results so that value is clear If possible, discuss impact and/or next steps Research projects: Algorithm research, software project, etc.

General ideas for projects Read and report on a book or a collection of about 3-5 technical papers in an area where DA methods are extensively used Marketing, pricing, A/B testing; Location, operations, inventory and logistics; Bidding Policy analysis Research and report on algorithms behind Netica, simulation-optimization, or other analytics software Create a piece of software for advanced analytics Apply statistical consulting methods and software to a real-world problem from work or life. Propose your own novel research or application

Some project ideas Optimizing buyer discounts given history Psychology of self-control and will power Report on Silver’s The Signal and the Noise; report on Poundstone’s Priceless: The Myth of Fair Value Excel simulation-optimization (SO) solver for multi-item inventory control decisions SO for call center intent-based routing to reduce call handling times NFL decision analytics; Improving 4th down decisions Providing more valuable information to help change customer/patient behaviors

Charniak – two typos Figure 2, last entry with value 0.3 should be not-fo and not-bp. “I can calculate the conditional probability of family-out given these pieces of evidence. (For this case, it is .5.)” Netica model says “0.5” is wrong. It should be 0.448

Car-buying: Structuring the problem and solution Not trivial to formulate well in Netica Multiple decision nodes (what test to choose, if any, and what to do after each outcome) and possibility of an option disappearing complicate the BN diagram Easier approach may be to develop an EMV formula for each possible (undominated) act Calculate EMV(a) directly from the problem data Use Netica only as a Bayes Rule calculator

Defining acts Act = one or more if-then rules determining what to do in all cases: If I see Y, then I will do X Examples of acts: Buy old Buy new Buy old if AAA test says good and car is available, else buy new Buy old if garage test says good, else buy new

Defining states A state resolves relevant uncertainties Car is good or bad AAA test says good if good AAA test says bad if bad Car remains available if AAA test Garage says good if good Garage test says bad if bad Only some state components are relevant to each decision. Decisions may depend on states and their probabilities via observations and Bayes’ Rule

Decision psychology heuristics and biases (cont.)

Seeking and interpreting data for probabilities: Simpson’s Paradox, Wason selection task

Using data to guide decisions Evaluate how well interventions, policies, and decisions have worked in creating significant improvements in outcomes Compare treatments or interventions Which has the greatest effect? How big is it? How sure can we be? Decide: Use data to help guide risky choices, adaptively optimize choices

Choosing a kidney stone treatment Which kidney stone treatment should you choose? How confident should you be? Are available data sufficient to support a confident decision? Data/evidence: Treatment A Treatment B 78% (273/350) success rate 83% (289/350) success rate

Which treatment should you recommend? Treatment A Treatment B Small stones Group 1 93% (81/87) Group 2 87% (234/270) Large stones Group 3 73% (192/263) Group 4 69% (55/80) Both 78% (273/350) 83% (289/350) https://en.wikipedia.org/wiki/Simpson's_paradox

Detecting discrimination You are hired to investigate discrimination in admissions at a state university What valid statistical conclusions can you draw from the following data?

Discrimination data revisited Data from 6 largest departments https://en.wikipedia.org/wiki/Simpson's_paradox

Simpson’s Paradox - discussion Q: Which relations should be used, aggregate or sub-population-specific? A: Could be either, depending on causality

Example: Designing data collection to test a hypothesis Vowels: A, E, I, O, U Even numbers: 0, 2, 4, 6, 8 http://images.slideplayer.com/16/5138662/slides/slide_43.jpg

Variation: Testing a more concrete hypothesis www.researchgate.net/publication/258179748_Dual-Process_Theories_of_Higher_Cognition

Wrap-up on exercises Continual open-mindedness about uncertainties + technical skills to reduce them are essential for using data to learn how to act more effectively. A combination of humility (willingness to learn from data) and skepticism (insistence on learning from data) about what is assumed or considered known boosts performance in forecasting, decision-making, diagnosis, research, and management