1. HMM and CRF
Lin Xuming

2. Catalog
Review and Continue: HMM
CRF

3. HMM

4. HMM——three problems

5. HMM——problem 1

6. HMM——problem 1

7. HMM——problem 1

8. HMM——problem 1

9. HMM——problem 1

10. HMM——problem 2

11. HMM——problem 2
A simple example

12. HMM——problem 3
When we know the state sequences and the observation sequences

13. HMM——problem 3

14. HMM——problem 3
When we know the observation sequences and we need to build models to fit into these observed sequences

15. HMM——problem 3

16. HMM——problem 3

17. HMM——scaling
In order to avoid underflow caused by multiple products of probabilities

18. HMM——scaling
In order to avoid underflow caused by multiple products of probabilities

19. HMM——scaling
In order to avoid underflow caused by multiple products of probabilities

20. HMM——scaling
In order to avoid underflow caused by multiple products of probabilities

21. HMM——example
Gaussian HMM of stock data

22. CRF——starting with ME Conditional entropy Objective function
Feature function

23. CRF——starting with ME

24. CRF——starting with ME

25. CRF——starting with ME

26. CRF——starting with ME
The first part

27. CRF——starting with ME
The second part

28. CRF——starting with ME
Complete derivation

29. CRF——starting with ME
Complete derivation

30. CRF——starting with ME
Graphical model

31. CRF——starting with ME
Graphical model of NB(left)

32. CRF——Linear-chain CRFs
(undirected) graphical model of LC-CRFs(left)

33. CRF——Linear-chain CRFs

34. CRF——Linear-chain CRFs

35. CRF——Linear-chain CRFs
How to build a LC-CRFs

36. Linear-chain CRFs——training
Log-likelihood function

37. Linear-chain CRFs——training
The parameter \sigma^2 models the trade-of between fitting exactly the observed feature frequencies and the squared norm of the weight vector

38. Linear-chain CRFs——training
Part A

39. Linear-chain CRFs——training
Part B

40. Linear-chain CRFs——training
Part C

41. Linear-chain CRFs——training
Total formula
Easy to calculate(empirical distribution)
Not quite easy to calculate
(The forward-backward algorithm)

42. Linear-chain CRFs——training
Update params

43. Linear-chain CRFs——inference

44. Linear-chain CRFs——inference
A simple example

45. Linear-chain CRFs——example
Let’s use CoNLL 2002 data to build a NER system