Bayes Nets & HMMs Tamara Berg CS 560 Artificial Intelligence Many slides throughout the course adapted from Svetlana Lazebnik, Dan Klein, Stuart Russell, Andrew Moore, Percy Liang, Luke Zettlemoyer, Rob Pless, Killian Weinberger, Deva Ramanan 1

Announcements HW3 due Oct 29, 11:59pm Today: Talk by Vijay Kumar (“Aerial Robot Swarms”), 4pm in SN011

Reminder: HW3

Example BN – Naïve Bayes C – Class F - Features We only specify (parameters): prior over class labels how each feature depends on the class

Naïve Bayes Representation Goal: estimate likelihoods P(message | spam) and P(message | ham) and priors P(spam) and P(ham) Likelihood: bag of words representation –The message is a sequence of words (w 1, …, w n ) –The order of the words in the message is not important –Each word is conditionally independent of the others given message class (spam or not spam) –Thus, the problem is reduced to estimating marginal likelihoods of individual words P(w i | spam) and P(w i | ham) along with the priors P(spam), P(ham)

General MAP rule for Naïve Bayes: Classify the message as spam if Decision rule likelihood priorposterior

Classify the message as spam if Decision rule Example “The quick brown fox jumped over the lazy dog” How would you decide what class to predict? Why is this called a bag of words representation?

Slide from Dan Klein

Percentage of documents in training set labeled as spam/ham Slide from Dan Klein

In the documents labeled as spam, occurrence percentage of each word (e.g. # times “the” occurred/# total words). Slide from Dan Klein

In the documents labeled as ham, occurrence percentage of each word (e.g. # times “the” occurred/# total words). Slide from Dan Klein

Parameter estimation Model parameters: feature likelihoods P(word | spam) and P(word | ham) and priors P(spam) and P(ham) –How do we obtain the values of these parameters? –Need training set of labeled samples from both classes P(word | spam) = # of word occurrences in spam messages total # of words in spam messages P(word | ham) = # of word occurrences in ham messages total # of words in ham messages

Parameter estimation Parameter estimate: Parameter smoothing: dealing with words that were never seen or seen too few times –Laplacian smoothing: pretend you have seen every vocabulary word m more times than you actually did P(word | spam) = # of word occurrences in spam messages total # of words in spam messages P(word | spam) = # of word occurrences in spam messages + m total # of words in spam messages + m*V (V: total number of unique words)

Summary of model and parameters Naïve Bayes model: Model parameters: P(spam) P(¬spam) P(w 1 | spam) P(w 2 | spam) … P(w n | spam) P(w 1 | ¬spam) P(w 2 | ¬spam) … P(w n | ¬spam) Feature likelihoods given spam prior Feature likelihoods given ¬ spam

Classification The class that maximizes:

Classification In practice –Multiplying lots of small probabilities can result in floating point underflow –Since log(xy) = log(x) + log(y), we can sum log probabilities instead of multiplying probabilities. –Since log is a monotonic function, the class with the highest score does not change. –So, what we usually compute in practice is:

Naïve Bayes on images

Visual Categorization with Bags of Keypoints Gabriella Csurka, Christopher R. Dance, Lixin Fan, Jutta Willamowski, Cédric Bray

Bag of words for images  Represent images as a “bag of words”

Bag-of-word models for images Csurka et al. (2004), Willamowski et al. (2005), Grauman & Darrell (2005), Sivic et al. (2003, 2005)

Bag-of-word models for images 1.Extract image features

Bag-of-word models for images 1.Extract image features

2.Learn “visual vocabulary” Bag-of-word models for images

1.Extract image features 2.Learn “visual vocabulary” 3.Map image features to visual words Bag-of-word models for images

Method Steps: –Detect and describe image patches. –Assign patch descriptors to a set of visual words. –Apply naïve Bayes, treating the images as a bag of visual words. –Predict which category the image belongs to as the MAP estimate.

Results

Next: Adding time! 29

Reasoning over Time Often, we want to reason about a sequence of observations –Speech recognition –Robot localization –Medical monitoring –DNA sequence recognition Need to introduce time into our models Basic approach: hidden Markov models (HMMs) More general: dynamic Bayes’ nets 30

Reasoning over time Basic idea: –Agent maintains a belief state representing probabilities over possible states of the world –From the belief state and a transition model, the agent can predict how the world might evolve in the next time step –From observations and an emission model, the agent can update the belief state

Markov Model 32

Markov Models A Markov model is a chain-structured BN –Value of X at a given time is called the state –As a BN: …. P(X t |X t-1 )….. –Parameters: initial probabilities and transition probabilities or dynamics (specify how the state evolves over time) X2X2 X1X1 X3X3 X4X4

Conditional Independence Basic conditional independence: –Past and future independent of the present –Each time step only depends on the previous –This is called the (first order) Markov property Note that the chain is just a (growing) BN –We can always use generic BN reasoning on it if we truncate the chain at a fixed length X2X2 X1X1 X3X3 X4X4 34

Example: Markov Chain Weather: –States: X = {rain, sun} –Transitions: –Initial distribution: 1.0 sun –What’s the probability distribution after one step? rainsun X2X2 X1X1 X3X3 X4X4

Mini-Forward Algorithm Question: What’s P(X) on some day t? sun rain sun rain sun rain sun rain Forward simulation 36

Example From initial observation of sun From initial observation of rain P(X 1 )P(X 2 )P(X 3 )P(X  ) P(X 1 )P(X 2 )P(X 3 )P(X  ) 37

Stationary Distributions If we simulate the chain long enough: –What happens? –Uncertainty accumulates –Eventually, we have no idea what the state is! Stationary distributions: –For most chains, the distribution we end up in is independent of the initial distribution –Called the stationary distribution of the chain –Usually, can only predict a short time out

Hidden Markov Model 39

Hidden Markov Models Markov chains not so useful for most agents –Eventually you don’t know anything anymore –Need observations to update your beliefs Hidden Markov models (HMMs) –Underlying Markov chain over states S –You observe outputs (effects) at each time step –As a Bayes’ net: X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3 E4E4 E5E5

Example An HMM is defined by: –Initial distribution: –Transitions: –Emissions:

Conditional Independence HMMs have two important independence properties: –Markov hidden process, future depends on past via the present –Current observation independent of all else given current state Quiz: does this mean that observations are independent given no evidence? –[No, correlated by the hidden state] X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3 E4E4 E5E5

Real HMM Examples Speech recognition HMMs: –Observations are acoustic signals (continuous valued) –States are specific positions in specific words (so, tens of thousands) Machine translation HMMs: –Observations are words (tens of thousands) –States are translation options Robot tracking: –Observations are range readings (continuous) –States are positions on a map (continuous)

Filtering / Monitoring Filtering, or monitoring, is the task of tracking the distribution B(X) (the belief state) over time We start with B(X) in an initial setting, usually uniform As time passes, or we get observations, we update B(X) The Kalman filter was invented in the 60’s and first implemented as a method of trajectory estimation for the Apollo program

Example: Robot Localization t=0 Sensor model: never more than 1 mistake Motion model: may not execute action with small prob. 10 Prob Example from Michael Pfeiffer

Example: Robot Localization t=1 10 Prob

Example: Robot Localization t=2 10 Prob

Example: Robot Localization t=3 10 Prob

Example: Robot Localization t=4 10 Prob

Example: Robot Localization t=5 10 Prob

Filtering Filtering – update belief state given all evidence (observations) to date Recursive estimation - given the result of filtering up to time t, the agent computes the result for t+1 from the new evidence: Filtering consists of two parts: –Projecting the current belief state distribution forward from t to t+1 –Updating the belief using the new evidence

Forward Algorithm Transition model Current belief state Emission model New belief state Normalizing constant used to make probabilities sum to 1.

Decoding: Finding the most likely sequence Query: most likely seq: X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3 E4E4 E5E5 53 [video]

Speech Recognition with HMMs

Particle Filtering

Sometimes |X| is too big to use exact inference –|X| may be too big to even store B(X) –E.g. X is continuous –|X| 2 may be too big to do updates Solution: approximate inference –Track samples of X, not all values –Samples are called particles –Time per step is linear in the number of samples –But: number needed may be large –In memory: list of particles, not states This is how robot localization works in practice

Representation: Particles Our representation of P(X) is now a list of N particles (samples) P(x) approximated by number of particles with value x Initially, all particles have a weight of 1 64 Particles: (3,3) (2,3) (3,3) (3,2) (3,3) (3,2) (2,1) (3,3) (2,1)

Particle Filtering: Elapse Time Each particle is moved by sampling its next position based on the transition model This captures the passage of time

Particle Filtering: Observe Based on observations: –Similar to likelihood weighting, we down weight our samples based on the evidence –Note that, the probabilities won’t sum to one, since most have been down weighted

Particle Filtering: Resample Rather than tracking weighted samples, we resample N times, we choose from our weighted sample distribution (i.e. draw with replacement) This is equivalent to renormalizing the distribution Now the update is complete for this time step, continue with the next one Old Particles: (3,3) w=0.1 (2,1) w=0.9 (3,1) w=0.4 (3,2) w=0.3 (2,2) w=0.4 (1,1) w=0.4 (3,1) w=0.4 (2,1) w=0.9 (3,2) w=0.3 New Particles: (2,1) w=1 (3,2) w=1 (2,2) w=1 (2,1) w=1 (1,1) w=1 (3,1) w=1 (2,1) w=1 (1,1) w=1

Robot Localization In robot localization: –We know the map, but not the robot’s position –Observations may be vectors of range finder readings –State space and readings are typically continuous (works basically like a very fine grid) and so we cannot store B(X) –Particle filtering is a main technique [Demos,Fox]DemosFox

SLAM SLAM = Simultaneous Localization And Mapping –We do not know the map or our location –Our belief state is over maps and positions! –Main techniques: Kalman filtering (Gaussian HMMs) and particle methods DP-SLAM, Ron Parr

Particle Filters for Localization and Video Tracking

Exact Inference 71

The Forward Algorithm We are given evidence at each time and want to know We can derive the following updates We can normalize as we go if we want to have P(x|e) at each time step, or just once at the end…

Online Belief Updates Every time step, we start with current P(X | evidence) We update for time: We update for evidence: The forward algorithm does both at once (and doesn’t normalize) Problem: space is |X| and time is |X| 2 per time step X2X2 X1X1 X2X2 E2E2

Best Explanation Queries Query: most likely seq: X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3 E4E4 E5E5 74

State Path Trellis State trellis: graph of states and transitions over time Each arc represents some transition Each arc has weight Each path is a sequence of states The product of weights on a path is the seq’s probability Can think of the Forward algorithm as computing sums of all paths in this graph sun rain sun rain sun rain sun rain 75

Viterbi Algorithm sun rain sun rain sun rain sun rain 76

Example 77

Summary Filtering Push Belief through time Incorporate observations Inference: –Forward Algorithm (/ Variable Elimination) –Particle Filter (/ Sampling) Algorithms: –Forward Algorithm (Belief given evidence) –Viterbi Algorithm (Most likely sequence) 78

Inference Recap: Simple Cases E1E1 X1X1 X2X2 X1X1

Passage of Time Assume we have current belief P(X | evidence to date) Then, after one time step passes: Or, compactly: Basic idea: beliefs get “pushed” through the transitions –With the “B” notation, we have to be careful about what time step t the belief is about, and what evidence it includes X2X2 X1X1

Observation Assume we have current belief P(X | previous evidence): Then: Or: Basic idea: beliefs reweighted by likelihood of evidence Unlike passage of time, we have to renormalize E1E1 X1X1

Example HMM

The Forward Algorithm We are given evidence at each time and want to know We can derive the following updates We can normalize as we go if we want to have P(x|e) at each time step, or just once at the end…

Online Belief Updates Every time step, we start with current P(X | evidence) We update for time: We update for evidence: The forward algorithm does both at once Problem: space is |X| and time is |X| 2 per time step X2X2 X1X1 X2X2 E2E2

Exact Inference 85

Just like Variable Elimination 86 X2X2 e1e1 X1X1 X3X3 X4X4 ¬e 2 e3e3 e4e4 XEP (E|X) rainumbrella0.9 rainno umbrella0.1 sunumbrella0.2 sunno umbrella0.8 X1X1 P( X 1 ) rain0.1 sun0.9 XP (e|X) rain0.9 sun0.2 evidence X1X1 P( X 1,e 1 ) rain0.09 sun0.18 Emission probability:

Just like Variable Elimination 87 X2X2 X 1,e 1 X3X3 X4X4 ¬e2¬e2 e3e3 e4e4 X1X1 P( X 1,e 1 ) rain0.09 sun0.18 XtXt X t+1 P (X t+1 |X t ) rain 0.9 rainsun0.1 sunrain0.2 sun 0.8 Transition probability: X1X1 X2X2 P (X 2,X 1,e 1 ) rain rainsun0.009 sunrain0.036 sun 0.144

Just like Variable Elimination 88 X 1,X 2,e 1 X3X3 X4X4 ¬e2¬e2 e3e3 e4e4 Marginalize out X 1... X1X1 X2X2 P (X 2,X 1, e 1 ) rain rainsun0.009 sunrain0.036 sun X2X2 P( X 2,e 1 ) rain0.117 sun0.153

Just like Variable Elimination 89 X 2,e 1 X3X3 X4X4 ¬e2¬e2 e3e3 e4e4 X2X2 P( X 2,e 1 ) rain0.117 sun0.153 XEP (E|X) rainumbrella0.9 rainno umbrella0.1 sunumbrella0.2 sunno umbrella0.8 Emission probability:

Just like Variable Elimination 90 X 2,e 1, ¬e 2 X3X3 X4X4 e3e3 e4e4

Approximate Inference 91

HMMs: MLE Queries HMMs defined by –States X –Observations E –Initial distr: –Transitions: –Emissions: Query: most likely explanation: X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3 E4E4 E5E5 92