1. Microscopic Traffic Modeling by Optimal Control and Differential Games
Using Differential Game Theory to derive a behavior-based Microscopic Flow Model
Prof. Dr. Ir. S. P. Hoogendoorn
December 9, 2018
Faculty of Civil Engineering and Geosciences

2. Main contributions
Developing a generic microscopic theory of driving behavior based on subjective predicted utility maximization / disutility minimization
Using theory of differential games to derive mathematical microscopic model, including:
Longitudinal driving task (free driving and car-following)
Lateral driving task (lane changing)
Show model characteristics by simple case example
December 9, 2018

3. Principle of disutility minimization
Several authors have proposed describing driver task execution as on subjective utility optimization problem, where the (dis-) utility reflects objectives such as:
Maximize safety and minimize risks
Maximize travel efficiency
Minimize lane-changing maneuvers
Maximize smoothness and comfort
Minimize stress, inconvenience, fuel consumption, etc.
The importance of each of these objectives will vary among individuals, given capabilities of the drivers and the possibilities of their vehicles
December 9, 2018

4. Psychological foundations?
Considered driving tasks take place on operational level and tactical level (Michon,1985)
Both are based on immediate driver environment
Decisions are to be made in seconds or milliseconds
Experienced has skilled drivers in making user-optimal decisions subconsciously
Description of drivers are ‘optimal controllers’ due to a.o. Minderhoud, Weverinke and to Hogema
December 9, 2018

5. Behavioral assumptions
Drivers control actions stem from minimizing generalized predicted costs reflecting their objectives
Predicted costs are determined by a.o.:
Not driving at the free driving speed, in the desired lane
Driving too close (or too far) to leader (or follower!)
Acceleration and braking (fuel consumption and smoothness)
Lane changing effort
Drivers (re-) consider their control decisions at time instants tk
Drivers may anticipate on control actions of other drivers
Drivers have limited prediction capabilities and make errors
Operational control objectives may change over time (adaptation)
December 9, 2018

6. Drivers as optimal controllers
Traffic system
Observation
k:=k+1
State estimation
Predicted utility / cost
Prediction
Candidate control
December 9, 2018

7. State and control definition
In this model, state z(t) summarizes system state, including:
Longitudinal positions xj(t) of vehicles j
Speeds vj(t) of vehicles j
Lateral positions yj(t) of vehicles j
Controls u(t) of driver are:
Acceleration ai(t)
Lane change decision Di(t) (=-1,0,+1)
Driver i makes assumptions regarding controls of other drivers j  i (acceleration and lane change decisions)
Driver state prediction model
December 9, 2018

8. Control objective
Driving objectives can be formalized into objective function J
Cost discount factor   0 describes importance of future cost
Running cost L(x(s),u(s)) denotes the contribution of driver situation at time s  tk to the total predicted cost J
Driver aims to find optimal control:
December 9, 2018

9. Control objective
Example specification of running cost L assuming linear relation:
Smoothness factor:
Quickness factor:
Distance to leader(s) factor:
Easy to include other cost components, including those reflecting ‘satisficing’ driver strategies
December 9, 2018

10. Derivation optimal control law
Two well-known (and strongly related) techniques exist:
Dynamic programming
Pontryagin’s minimum principle
We use the latter to derive the optimal acceleration (assuming no lane changes) and the former to determine whether lane change is beneficial
Both lean on so-called Hamilton function H:
where  are the co-states or marginal costs of the state z
December 9, 2018

11. Optimal control law for single lane
Stationary conditions used to determine optimal acceleration
So-called co-state equations allow for determination of co-states
Some relatively simple mathematics then yield the optimal acceleration for driver i:
December 9, 2018

12. Intermezzo: walker models
Same approach was used to derive walker model NOMAD
Resulting model is similar to the social forces model of Helbing and is the basis of the NOMAD pedestrian simulation software
Software will be available soon on the TU Delft pedestrian website (
December 9, 2018

13. December 9, 2018

14. Optimal control for lane changing
Dynamic programming entails solving so-called Hamilton-Jacobi-Bellman equation for infinite horizon discounted cost problem
Let W(z) denote minimum value of objective function (so-called value function), i.e. upon applying optimal control law u*, W satisfies the HJB equation:
Relation between costates and value function:
allows us to determine W(z) from H:
December 9, 2018

15. Derivation of optimal control law
Predicted cost can be computed per lane to see if a lane change is beneficial or not
Lane change occurs when predicted cost of current lane is larger that predicted cost on target lane + switching cost :
Lane changing depends on:
Expected acceleration / deceleration on target lane
Expected changes in the proximity costs
December 9, 2018

16. So, what is so generic?
Many known car-following models can be formalized as an optimal control model
E.g. model of Bexelius (multi-anticipatory using two leaders) can be found using:
For these models, lane-changing criteria can be derived which are consistent with this model
Note: strong relation with the MOBIL model of Kesting, Treiber and Helbing, but lane changing criteria are different
December 9, 2018

17. Example application Three lane motorway
On each lane, platoon leader with speed 16, 24 and 32 m/s
Example application shows plausible behavior of drivers modelled according to control formulation
In particular:
Considered driver chooses to change lanes because expected costs are less on target lane than on current lane
December 9, 2018

18. Example application
December 9, 2018

19. Model calibration and benchmarking
Calibration of model can be achieved either by considering macroscopic properties or by using microscopic trajectory data (such as those collected from the helicopter)
See
December 9, 2018

20. Model extensions
Several ‘simple model’ extensions can be easily included, in particular to describe driving behavior near discontinuities (merges, diverges and weaving areas):
Anticipated reaction of drivers to control action of driver i
Cooperative driving (common objective function)
Inter-driver variability (differences between drivers) and intra-driver variability (adaptive driving behavior, e.g. by slowly changing cost weights allowing prediction of capacity funnel, capacity drop, etc.)
Empirical investigations show importance of these!
Inclusion of explicit reaction times, errors in observing, state estimation, predicting, decision making, etc.
December 9, 2018

21. December 9, 2018