2. Aims
Research aim
to provide internally consistent, practically applicable, methodology of dynamic decision making (DM)
Talk aims
to provide DM-based justification of Bayesian learning
to point to open problems worth of research effort

3. Design of optimal DM strategy
Optimal strategy RT should reach the best behavior Q = (dT, T) using available information while respecting given restrictions
unseen t
System ST is a world part, St: world state, at yt, t
innovations yt
data
dt= ( yt, at )
actions at
Strategy RT generates actions by rules Rt: prior, dt at
information, complexity, range
restrictions

4. Behaviors’ & strategies’ ordering
Aim orders strategies:
R is better than R iff behavior Q = ( dT, T ) of S-R is closer to the aim than behavior Q = (dT , T) of S-R
Order of behaviors via loss Z: Q fully ordered set 
R better than R  Z(Q)  Z(Q)
But Z is unsuitable for design if dependence of Q on R is unknown !

5. Loss as uncertainty function
Uncertainty =  U
Split Q = (knowingly R-given part,U) 
{R} U 
Z(U) = Z(Q) 
unseen T
complexity
modeling errors
vague quantification
neglected influences
For U  , prior ordering on {R} is needed
E: Z• fully ordered set  R better than R  EZ  EZ
…what E’s are good?

6. Requirements on E
The best R implied by the chosen E must not be a priori bad
The E should be as objective as possible 
- it has to be good on designer chosen subsets of {R}
- it has to serve for any (reasonable) designer chosen loss
The best R must not depend on designer attitude to uncertainty
The best R has to minimize loss when uncertainty is empty

7. Consequences of requirements        
Z(U)  U U  
EZ=EZ +EZ  
Neutral
(Z) = Z
…E mathematical expectation of utility function (Z) of loss Z!

8. E Bayesian calculus R: D a a min E[Z] = E[min E[Z|a,D]] Basic DM lemma
V(dT ) = E[ Z(dT, T) | dT, MT]
V(dt-1) = min E[V(dt) |at, dt-1, Mt-1],
Design of DDM
t =T, T-1, …,1
Mt model of St at
Optimal RT
randomized
supp f (at | dt-1,Mt-1) = Arg min E[V(dt) | at, dt-1, Mt-1]
f (T| dT ,MT) … Bayesian filtering
Needed pdfs
f (yt|at, dt-1 ,Mt-1) … Bayesian prediction
Required models Mt of St implied by the need to filter & predict !

9. Mt: observation & time evolution
NC DM
f(at, t-1 | dt-1) = f(at | dt-1, t-1 ) f(t-1 | dt-1)
Observation model
f(yt | at, dt-1, t)
relates seen to unseen
Evolution model
f(t+1| at+1, dt, t)
models unseen
Predictive pdf
f (yt| at, dt-1) =  f(yt | at, dt-1, t) f(t| at, dt-1)dt
data update
f(t | dt) 
f(yt | at, dt-1, t) f(t | at, dt-1)
Filtering
time update
f(t+1| at+1,dt) =  f(t+1| at+1,dt,t) f(t| dt) dt
Prior pdf f(1 | d0 ) = f(1) = belief in possible values of 1
… why f(1) > 0 when MT  ST for any 1?

10. Bayesian paradigm & reality
World part
generating dT, T
Prior pdf = belief that 1
is the best projection ST
the nearest model
… unknown as ST unknown
posterior pdf = belief in 1
corrected by data dT
Model set
indexed by T
… any practical consequence ?

11. Projection consequences
The World model is a subjective thing
 DM is free to see the World … at own responsibility
Bayes’ rule learns the best projection  minimizes entropy rate
 entropy rate is an inherent Bayesian discrepancy measure
Quality of the best projection depends heavily on the model set
 careful modeling of the World pays back
Projection error cannot be measured
… a range of methodologies ignores this !
… any chance to get information about better models ?

12. Model comparison f(dT|M)= f(dT,T) dT f(dT |M)= f(dT,T) dT
Model set M
indexed by T
Model set M
indexed by T
The best M*  {M M} uncertain
Point estimation
Model combination
f(M* | dT)  f(dT |M*) f(M*)
f(dT) = M* f(dT|M*) f(M*)
preserves complexity
avoids unnecessary DM

13. Lesson from model comparison
Values of predictive pdf serve for model comparison
Compound hypothesis testing is straightforward if
alternative model sets are specified (modeling!)
Unnecessary DM should be avoided (valid generally!)
No new techniques are needed
… just modeling !

14. Open problems
Is it useful to exploit that often strategies R  R with U  U ?
… probably yes
Has to be E additive on losses with uncommon support ?
… probably reasonable, I see no alternative
Is the notion of the loss, ordering a posteriori behaviors, needed ?
… probably unnecessary and just matter of explanation: worth
trying in justifying fully probabilistic design of DM strategies
Does Bayesian scheme lead to quantum mechanical effects ?
… probably yes via modeling measurement as generally
non-commutating projections of physical (societal) quantities

15. Open problems
Is it a priori possible to point to drawbacks of non-Bayesian DM ?
… conceptually probably yes, but job was not done even
for important classes of methodologies …
Is DM possible without an explicit modeling ?
… definitely yes at the cost of final quality & reliability …
Is there a systematic and feasible way of generating alternative Ms ?
… probably no, but guides fixing good practice (!?) are needed
How to translate partial, domain specific, knowledge pieces into
model and prior pdf ?
… I do not know a sufficiently general way but partial
results make me optimistic

16. Open problems
Is there possibility to make modeling fully algorithmic ?
… probably no, but guides fixing good practice (!?) are needed
How to cope with the fact that filtering (estimation) takes its
outputs from the feasible class where prior pdf is naturally chosen
… methodological unification of Bayesian paradigm with
approximation theory is probably only systematic but hard way;
otherwise, partial ad hoc solutions have to be elaborated
Is it possible to formulate (and finally solve) DM design under
complexity restrictions
… I do not know so it is time to stop here !