Download presentation
Presentation is loading. Please wait.
Published byDuane Webb Modified over 9 years ago
1
1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour
2
2 Key Question Given a set of choices (product offerings), which one is optimal? u For each choice s Probability of success (sale) s Revenue realized given a success u Revenue Management perspective s Which of the choices will yield the greatest revenue Elicits the best customer response
3
3 Deterministic world Customer response to each choice is known and constant u Plug in the known success rate and revenue for each choice and optimize
4
4 Random World Customer response is random with known parameter values u For example, success rate is 0.5 or realized revenue is Poisson(7) How to handle random response u Risk neutral – Expected Response u Risk averse – Expected Utility u Minimax (Minimizes the worst possible outcome) u Maximin (Maximizes the best possible outcome)
5
5 Random World Typically don’t know precisely the demand parameters u Estimate the demand s Parametric estimators of unknown parameters s Robust estimators of unknown parameters s Non-parametrics Sample quantiles, Empirical CDF, …
6
6 Real World Random response u In either the success rate or the realized revenue Uncertainty in the demand distribution u Based on earlier observations (expert opinion, similar processes, etc.) u The Bayesian approach quantifies this uncertainty in a directly
7
7 Effect of Uncertainty What effect does the uncertainty have with regard to making optimal decisions? u Assuming risk neutrality, should decisions be based only on expected revenue? u Is there potential value in exploring choices that do not yield the best expected revenue as of now? s Quantify value in obtaining additional information, i.e. reducing uncertainty in the demand response
8
8 Simple Example Choose either to place a $1 bet on a flip of a coin coming out heads or keep your $1 You estimate that the probability of a head is 0.4 Should you play? u Expected Revenue s Don’t play: 0 s Do play: 1(p) – 1(1-p) = 2p – 1 = 2(0.4) –1 = -0.2
9
9 Simple Example Solution Depends
10
10 Scenario 1 There is no uncertainty u The probability of a head is exactly 0.4 u Assuming that you’re risk neutral, you should not play
11
11 Scenario 2 You know that the coin is either a 2-headed coin or it is a 2-tailed coin u Pr[2-headed] = 0.4 u Pr[2-tailed] = 0.6 u If you play once s Observe head 2-headed Continue to play and make $1 every time s Observe tail 2-tailed Stop after losing $1
12
12 Scenario 2 (cont.) If you could play up to 10 times u Expected Revenue s Don’t play – 0 s Do play Observe the first flip »If it’s heads then play the next 9 times »If it’s tails then quit 0.4(10) - 0.6(1) = 3.4
13
13 Scenario 3 You estimate that the probability of head is 0.4 plus or minus 0.2 u Playing once no longer yields perfect information s Playing any finite number of times will not yield perfect information u There is a possibility of long-term positive revenue from playing s Head probabilities greater than 0.5 are plausible u Should you play at all? s If so, under what circumstances should you stop playing?
14
14 Scenario 3 (Bayesian Perspective) You estimate that the probability of head is 0.4 with uncertainty of 0.1 u Prior mean = 0.4 u Prior standard deviation = 0.1 u Beta prior: = 9.2, = 13.8
15
15 Scenario 3 (cont.) Let X be the number of heads in any set of n flips of the coin and assume X is binomial distributed (iid Bernoulli trials) Updated uncertainty in the head probability after the n trials u Beta u * = 9.2 + X, * = 13.8 + n – X u Expectation: */( *+ *) u Variance: * */[( *+ *)( *+ *)( *+ *+1)] u Never know for sure
16
16 Key Points Short term expected revenue is not the only thing to consider There is also value in obtaining additional information about the demand distribution because this information can be used to improve long-term revenue performance u How do we calculate the value of this information in terms of future revenue? u How much experimentation is optimal?
17
17 One Approach Evaluate any possible choice at every time u Completely reevaluate after every observation s Due to the prior uncertainty, lose independence because observations share the same unknown parameters s Enumerate all possible combinations of observations and strategies along with their associated probabilities and search for the best u Computationally intensive u Complications with continuous variables
18
18 More Restrictive Approach Define long-term in terms of a number of observations u For example, 10 flips of the coin Allow for a single experiment with a single conclusion u For example, 3 flips of the coin s If 3 heads continue to play over the last 7 s Otherwise stop u Calculate the expected net return from an experiment s Expected Cost – Expected Revenue from additional observations
19
19 Expected Cost of Experiment Based on prior distribution on the unknown parameters and the conditional distribution of the customer response (given the parameters) u Calculate the marginal distribution for the data u Calculate the marginal expectation over the trials u Subtract the baseline revenue
20
20 Expected Revenue from Trial Expected revenue over the rest of the long-term period given the decision made after the trial u For each choice s Win implies the highest posterior expectation of revenue s Pr[win] s E[return|win] s For the rest of the period (n-m) Expected revenue given win is (n-m)E[return|win] u The expected revenue is then the weighted sum s Pr[Win](n-m)E[return|win] u Subtract the baseline revenue
21
21 Scenario 3 (revisited) Set n = 10 Investigate an experiment with m trials Expected cost of the experiment u.2m Assume we observed m 1 heads u Updated uncertainty s Beta( = 9.2 + m 1, = 13.8 + m – m 1 )
22
22 Scenario 3 (cont.) Continue to play if the new expectation on the probability of a head is greater than 0.5 u * = 9.2 + m 1 > 13.8 + m – m 1 = * Expected return on the experiment u Pr[ * > * ](10-m){2E[ */( *+ *)| * > * ]-1]} + (1- Pr[ * > * ])(10-m)0 u Note s Non-negative s If you let n and Pr[ * > * ] > 0, then the expected return for any finite experiment
23
23 Scenario 3 (Details) * > * u 9.2 + m 1 > 13.8 + m – m 1 u 2m 1 > 4.6 + 0.5m u m 1 > 2.3 + 0.5m m 1 | p ~ Bin(m,p) m 1 ~BetaBinomial(m,9.2,13.8) */( *+ *) = (9.2 + m 1 ) / (23+m) u E[ */( *+ *)| * > * ] = (9.2 +E[m 1 | m 1 > 2.3 + 0.5m])/(23+m)
24
24 Scenario 3 (Cont.) If m = 5 u m 1 > 2.3 + 0.5m = 4.8 s m 1 = 5 s * = 9.2 + 5 =14.2 s * = 13.8 + 0 = 13.8 u Expected Return s Pr[ * > * ](10-m){2E[ */( *+ *)| * > * ]-1]} s Pr[m 1 = 5 ](10-5){2(14.2/28)-1} = 0.0175(5)0.0143 = 0.0012
25
25 Scenario 3 (Cont.) Value of the 5 flip experiment u 0.0012 – 0.2(5) = -0.9988 Value of each possible experiment
26
26 Scenario 4 Equivalent to Scenario 3 except u = 0.092, = 0.138 u Uncertainty (prior standard deviation) = 1 Value of Experiment
27
27 General Rules of Thumb Larger values of n will lead to longer experiments u The longer the “long-term,” the greater the return on experiments More uncertainty (larger prior standard deviations) will lead to more experimentation u The less you know, the greater the return on experiments
28
28 Extensions to the Simple Example Method There are multiple choices u Each with a current prior distribution for the success probability The revenue achieved for a success may be random for each choice u The parameters of the distribution are not known
29
29 Example Two choices u Choice 1 and Choice 2 s The success probabilities are unknown for both choices s The realized revenue from a success is random Unknown mean and variance u Long term horizon (n) is 50 u Experiment s m 1 trials for choice 1 s m 2 trials for choice 2
30
30 Choice 1: Current Opinion Success Rate (Beta distributed) u Expectation: 0.50 u Uncertainty: 0.15 Mean revenue on a success (Normal given the variance) u Expectation: 8.00 u Uncertainty: 0.41 Variance in revenue on a success (Inverted Gamma) u Expectation: 2.07 u Uncertainty: 0.05
31
31 Choice 2: Current Opinion Success Rate (Beta distributed) u Expectation: 0.45 u Uncertainty: 0.28 Mean revenue on a success (Normal given the variance) u Expectation: 7.00 u Uncertainty: 2.11 Variance in revenue on a success (Inverted Gamma) u Expectation: 2.11 u Uncertainty: 0.08
32
32 Results Search for the best experiment in terms of net revenue over the 50 observations
33
33 Results The best trial consists of a sample of 6 observations u 4 with choice 1 u 2 with choice 2 After the trial the choice with the best posterior mean is used for the remaining 44
34
34 Future Work Develop efficient optimization routine u Search methods (concave envelope, gradients) Extend the types of experimentation u Multiple experiments Further investigate the asymptotics
35
35
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.