HMM and CRF Lin Xuming

Catalog Review and Continue: HMM CRF

HMM

HMM——three problems

HMM——problem 1

HMM——problem 1

HMM——problem 1

HMM——problem 1

HMM——problem 1

HMM——problem 2

HMM——problem 2 A simple example

HMM——problem 3 When we know the state sequences and the observation sequences

HMM——problem 3

HMM——problem 3 When we know the observation sequences and we need to build models to fit into these observed sequences

HMM——problem 3

HMM——problem 3

HMM——scaling In order to avoid underflow caused by multiple products of probabilities

HMM——scaling In order to avoid underflow caused by multiple products of probabilities

HMM——scaling In order to avoid underflow caused by multiple products of probabilities

HMM——scaling In order to avoid underflow caused by multiple products of probabilities

HMM——example Gaussian HMM of stock data

CRF——starting with ME Conditional entropy Objective function Feature function

CRF——starting with ME

CRF——starting with ME

CRF——starting with ME

CRF——starting with ME The first part

CRF——starting with ME The second part

CRF——starting with ME Complete derivation

CRF——starting with ME Complete derivation

CRF——starting with ME Graphical model

CRF——starting with ME Graphical model of NB(left)

CRF——Linear-chain CRFs (undirected) graphical model of LC-CRFs(left)

CRF——Linear-chain CRFs

CRF——Linear-chain CRFs

CRF——Linear-chain CRFs How to build a LC-CRFs

Linear-chain CRFs——training Log-likelihood function

Linear-chain CRFs——training The parameter \sigma^2 models the trade-of between fitting exactly the observed feature frequencies and the squared norm of the weight vector

Linear-chain CRFs——training Part A

Linear-chain CRFs——training Part B

Linear-chain CRFs——training Part C

Linear-chain CRFs——training Total formula Easy to calculate(empirical distribution) Not quite easy to calculate (The forward-backward algorithm)

Linear-chain CRFs——training Update params

Linear-chain CRFs——inference

Linear-chain CRFs——inference A simple example

Linear-chain CRFs——example Let’s use CoNLL 2002 data to build a NER system