Announcements  Project 3 will be posted ASAP: hunting ghosts with sonar

2

3

CS 188: Artificial Intelligence Dynamic Bayes Nets and Particle Filters Instructor: Stuart Russell University of California, Berkeley

Today  Dynamic Bayes nets  Relationship to HMMs  Application: ICU monitoring  Particle filtering  And maybe some cool demos

Dynamic Bayes Nets

Dynamic Bayes Nets (DBNs)  We want to track multiple variables over time, using multiple sources of evidence  Idea: Repeat a fixed Bayes net structure at each time  Variables from time t can condition on those from t-1 G1aG1a E1aE1a E1bE1b G1bG1b G2aG2a E2aE2a E2bE2b G2bG2b t =1t =2 G3aG3a E3aE3a E3bE3b G3bG3b t =3

DBNs and HMMs  Every HMM is a single-variable DBN  Every discrete DBN is an HMM  HMM state is Cartesian product of DBN state variables  Sparse dependencies => exponentially fewer parameters  E.g., 20 state variables, 3 parents each; DBN has 20 x 2 3 = 160 parameters, HMM has 2 20 x 2 20 =~ parameters

Exact Inference in DBNs  Variable elimination applies to dynamic Bayes nets  Offline: “unroll” the network for T time steps, then eliminate variables to find P(X T |e 1:T )  Online: eliminate all variables from the previous time step; store factors for current time only  Problem: largest factor contains all variables for current time (plus a few more) t =1t =2t =3 G1aG1a E1aE1a E1bE1b G1bG1b G2aG2a E2aE2a E2bE2b G2bG2b G3aG3a E3aE3a E3bE3b G3bG3b G3bG3b [Demo: pacman sonar ghost DBN model (L15D6)]

Video of Demo Pacman Sonar Ghost DBN Model

Application: ICU monitoring  State: variables describing physiological state of patient  Evidence: values obtained from monitoring devices  Transition model: physiological dynamics, sensor dynamics  Query variables: pathophysiological conditions (a.k.a. bad things) 11

Toy DBN: heart rate monitoring parameter variable state variable sensor variable sensor state variable

Vasculature Medullary cardiovascular center Cardiac parasympathetic output Cardiac sympathetic output Card. M2 Card. β1 Card. β2 Vasc. α1 Vasc. α2 Vasc. β2 Heart rate Cardiac contrac- tility Venous tone Arterio- lar tone Cardiac preload Capillary pressure Cardiac stroke volume Cardiac output Vascular resistance Mean arterial blood pressure Barorecep- tor discharge Brain Neurotransmitters Heart Blood flow

Vasculature Medullary cardiovascular center Cardiac parasympathetic output Cardiac sympathetic output Card. M2 Card. β1 Card. β2 Vasc. α1 Vasc. α2 Vasc. β2 Cardiac contrac- tility Venous tone Arterio- lar tone Cardiac preload Capillary pressure Barorecep- tor discharge Brain Neurotransmitters Heart Blood flow Heart rate Cardiac stroke volume Cardiac output Vascular resistance Mean arterial blood pressure

Vasculature Medullary cardiovascular center Cardiac parasympathetic output Cardiac sympathetic output Card. M2 Card. β1 Card. β2 Vasc. α1 Vasc. α2 Vasc. β2 Heart rate Cardiac contrac- tility Venous tone Arterio- lar tone Cardiac preload Capillary pressure Cardiac stroke volume Cardiac output Vascular resistance Mean arterial blood pressure Barorecep- tor discharge MAP sensor model Heart rate sensor model Central venous pressure sensor model Brain Neurotransmitters Heart Blood flow

Vasculature Medullary cardiovascular center Cardiac parasympathetic output Cardiac sympathetic output Card. M2 Card. β1 Card. β2 Vasc. α1 Vasc. α2 Vasc. β2 Heart rate Cardiac contrac- tility Venous tone Arterio- lar tone Cardiac preload Capillary pressure Cardiac stroke volume Cardiac output Vascular resistance Mean arterial blood pressure Barorecep- tor discharge ABP sensor model Heart rate sensor model Central venous pressure sensor model Brain Neurotransmitters Heart Blood flow

Medullary cardiovascular center Cardiac parasympathetic output Cardiac sympathetic output Card. M2 Card. β1 Card. β2 Vasc. α1 Vasc. α2 Vasc. β2 Heart rate Cardiac contrac- tility Venous tone Blood [volume] Arterio- lar tone Cardiac preload Capillary pressure Cardiac stroke volume Cardiac output Vascular resistance Mean arterial blood pressure Barorecep- tor discharge Intracranial physiology Tissues- NOS [perfusion] MAP sensor model GI/Liver [perfusion] Heart rate sensor model Central venous pressure sensor model Blood [transu- dation] Setpoint inputs from ANS, CNS, intracranial, blood Pulm. [intra- thoracic press.] Brain Neurotransmitters Heart Blood flow Vasculature

Medullary cardiovascular center Cardiac parasympathetic output Cardiac sympathetic output Card. M2 Card. β1 Card. β2 Vasc. α1 Vasc. α2 Vasc. β2 Heart rate Cardiac contrac- tility Venous tone Arterio- lar tone Cardiac preload Capillary pressure Cardiac stroke volume Cardiac output Vascular resistance Mean arterial blood pressure Barorecep- tor discharge Intracranial physiology Tissues- NOS [perfusion] MAP sensor model GI/Liver [perfusion] Heart rate sensor model Central venous pressure sensor model Blood [transu- dation] Setpoint inputs from ANS, CNS, intracranial, blood Pulm. [intra- thoracic press.] Brain Neurotransmitters Heart Blood flow Vasculature Blood [volume]

Medullary cardiovascular center Cardiac parasympathetic output Cardiac sympathetic output Card. M2 Card. β1 Card. β2 Vasc. α1 Vasc. α2 Vasc. β2 Heart rate Cardiac contrac- tility Venous tone Blood [volume] Arterio- lar tone Cardiac preload Capillary pressure Cardiac stroke volume Cardiac output Vascular resistance Mean arterial blood pressure Barorecep- tor discharge Intracranial physiology Tissues- NOS [perfusion] MAP sensor model GI/Liver [perfusion] Heart rate sensor model Central venous pressure sensor model PK [conc. of phenyl- ephrine] Blood [transu- dation] Setpoint inputs from ANS, CNS, intracranial, blood Pulm. [intra- thoracic press.] Brain Neurotransmitters Heart Blood flow Vasculature

Medullary cardiovascular center Cardiac parasympathetic output Cardiac sympathetic output Card. M2 Card. β1 Card. β2 Vasc. α1 Vasc. α2 Vasc. β2 Heart rate Cardiac contrac- tility Venous tone Blood [volume] Arterio- lar tone Cardiac preload Capillary pressure Cardiac stroke volume Cardiac output Vascular resistance Mean arterial blood pressure Barorecep- tor discharge Intracranial physiology Tissues- NOS [perfusion] MAP sensor model GI/Liver [perfusion] Heart rate sensor model Central venous pressure sensor model PK [conc. of phenylephrine] Blood [transu- dation] Setpoint inputs from ANS, CNS, intracranial, blood Pulm. [intra- thoracic press.] Brain Neurotransmitters Heart Blood flow Vasculature

Medullary cardiovascular center Cardiac parasympathetic output Cardiac sympathetic output Card. M2 Card. β1 Card. β2 Vasc. α1 Vasc. α2 Vasc. β2 Heart rate Cardiac contrac- tility Venous tone Blood [volume] Arterio- lar tone Cardiac preload Capillary pressure Cardiac stroke volume Cardiac output Vascular resistance Mean arterial blood pressure Barorecep- tor discharge Intracranial physiology Tissues- NOS [perfusion] MAP sensor model GI/Liver [perfusion] Heart rate sensor model Central venous pressure sensor model PK [conc. of phenyl- ephrine] Blood [transu- dation] Setpoint inputs from ANS, CNS, intracranial, blood Pulm. [intra- thoracic press.] Medullary cardiovascular center Cardiac parasympathetic output Cardiac sympathetic output Card. M2 Card. β1 Card. β2 Vasc. α1 Vasc. α2 Vasc. β2 Heart rate Cardiac contrac- tility Venous tone Blood [volume] Arterio- lar tone Cardiac preload Capillary pressure Cardiac stroke volume Cardiac output Vascular resistance Mean arterial blood pressure Barorecep- tor discharge Intracranial physiology Tissues- NOS [perfusion] MAP sensor model GI/Liver [perfusion] Heart rate sensor model Central venous pressure sensor model PK [conc. of phenyl- ephrine] Blood [transu- dation] Setpoint inputs from ANS, CNS, intracranial, blood Pulm. [intra- thoracic press.] Brain Neurotransmitters Heart Blood flow Vasculature

Human physiology v0.1

Other people have done a lot of work! 10% of Guyton 1972

ICU data: 22 variables, 1min ave

27

ICP closeup

1s diastolic/systolic/mean ABP

Blood pressure measurement

ALARM

Particle Filtering

We need a new algorithm!  When |X| is more than 10 6 or so (e.g., 3 ghosts in a 10x20 world), exact inference becomes infeasible  Likelihood weighting fails completely – number of samples needed grows exponentially with T X1X1 X0X0 X2X2 X3X3 E1E1 E2E2 E3E3

We need a new idea!  The problem: sample state trajectories go off into low-probability regions, ignoring the evidence; too few “reasonable” samples  Solution: kill the bad ones, make more of the good ones  This way the population of samples stays in the high-probability region  This is called resampling or survival of the fittest t=2t=7

Particle Filtering  Represent belief state by a set of samples  Samples are called particles  Time per step is linear in the number of samples  But: number needed may be large  This is how robot localization works in practice

Representation: Particles  Our representation of P(X) is now a list of N particles (samples)  Generally, N << |X|  Storing map from X to counts would defeat the point  P(x) approximated by number of particles with value x  So, many x may have P(x) = 0!  More particles, more accuracy  Usually we want a low-dimensional marginal  E.g., “Where is ghost 1?” rather than “Are ghosts 1,2,3 in {2,6], [5,6], and [8,11]?”  For now, all particles have a weight of 1 Particles: (3,3) (2,3) (3,3) (3,2) (3,3) (3,2) (1,2) (3,3) (2,3)

Particle Filtering: Propagate forward  A particle in state x t is moved by sampling its next position directly from the transition model:  x t+1 ~ P(X t+1 | x t )  Here, most samples move clockwise, but some move in another direction or stay in place  This captures the passage of time  If enough samples, close to exact values before and after (consistent) Particles: (3,3) (2,3) (3,3) (3,2) (3,3) (3,2) (1,2) (3,3) (2,3) Particles: (3,2) (2,3) (3,2) (3,1) (3,3) (3,2) (1,3) (2,3) (3,2) (2,2)

 Slightly trickier:  Don’t sample observation, fix it  Similar to likelihood weighting, weight samples based on the evidence  W = P(e t | x t )  Normalize the weights: particles that fit the data better get higher weights, others get lower weights Particle Filtering: Observe Particles: (3,2) w=.9 (2,3) w=.2 (3,2) w=.9 (3,1) w=.4 (3,3) w=.4 (3,2) w=.9 (1,3) w=.1 (2,3) w=.2 (3,2) w=.9 (2,2) w=.4 Particles: (3,2) (2,3) (3,2) (3,1) (3,3) (3,2) (1,3) (2,3) (3,2) (2,2)

Particle Filtering: Resample  Rather than tracking weighted samples, we resample  N times, we choose from our weighted sample distribution (i.e., draw with replacement)  Now the update is complete for this time step, continue with the next one Particles: (3,2) w=.9 (2,3) w=.2 (3,2) w=.9 (3,1) w=.4 (3,3) w=.4 (3,2) w=.9 (1,3) w=.1 (2,3) w=.2 (3,2) w=.9 (2,2) w=.4 (New) Particles: (3,2) (2,2) (3,2) (2,3) (3,3) (3,2) (1,3) (2,3) (3,2)

Summary: Particle Filtering  Particles: track samples of states rather than an explicit distribution Particles: (3,3) (2,3) (3,3) (3,2) (3,3) (3,2) (1,2) (3,3) (2,3) Propagate forwardWeightResample Particles: (3,2) (2,3) (3,2) (3,1) (3,3) (3,2) (1,3) (2,3) (3,2) (2,2) Particles: (3,2) w=.9 (2,3) w=.2 (3,2) w=.9 (3,1) w=.4 (3,3) w=.4 (3,2) w=.9 (1,3) w=.1 (2,3) w=.2 (3,2) w=.9 (2,2) w=.4 (New) Particles: (3,2) (2,2) (3,2) (2,3) (3,3) (3,2) (1,3) (2,3) (3,2) [Demos: ghostbusters particle filtering (L15D3,4,5)] Consistency: see proof in AIMA Ch. 15

Particle filtering on umbrella model 46

Robot Localization  In robot localization:  We know the map, but not the robot’s position  Observations may be vectors of range finder readings  State space and readings are typically continuous so we cannot usually represent or compute an exact posterior  Particle filtering is a main technique

Particle Filter Localization (Sonar) [Video: global-sonar-uw-annotated.avi]

Robot Mapping  SLAM: Simultaneous Localization And Mapping  We do not know the map or our location  State consists of position AND map!  Main techniques: Kalman filtering (Gaussian HMMs) and particle methods DP-SLAM, Ron Parr [Demo: PARTICLES-SLAM-mapping1-new.avi]

Particle Filter SLAM – Video 1 [Demo: PARTICLES-SLAM-mapping1-new.avi]

Particle Filter SLAM – Video 2 [Demo: PARTICLES-SLAM-fastslam.avi]