1. 11-15-2010 Mostly pilfered from Pedro’s slides
Poon and Domingos
Mostly pilfered from Pedro’s slides

2. Motivation The real world is complex and uncertain
Logic handles complexity
Probability handles uncertainty

3. MLN’s Bold Agenda: “Unify Logical and Statistical AI”
Field
Logical approach
Statistical approach
Knowledge representation
First-order logic
Graphical models
Automated reasoning
Satisfiability testing
Markov chain Monte Carlo
Machine learning
Inductive logic programming
Neural networks
Planning
Classical planning
Markov decision processes
Natural language
processing
Definite clause grammars
Prob. context-free grammars

4. Markov Networks: [Review]
Undirected graphical models
Smoking
Cancer
Asthma
Cough
Potential functions defined over cliques
Smoking
Cancer
Ф(S,C)
False
4.5
True
2.7

5. Markov Networks Smoking Cancer Asthma Cough
Undirected graphical models
Smoking
Cancer
Asthma
Cough
Log-linear model:
Weight of Feature i
Feature i

6. Inference in Markov Networks
Goal: Compute marginals & conditionals of
Exact inference is #P-complete
Conditioning on Markov blanket is easy:
Gibbs sampling exploits this

7. MCMC: Gibbs Sampling state ← random truth assignment
for i ← 1 to num-samples do
for each variable x
sample x according to P(x|neighbors(x))
state ← state with new value of x
P(F) ← fraction of states in which F is true

8. Overview Motivation Foundational areas Putting the pieces together
Probabilistic inference [Markov nets]
Statistical learning
Logical inference
Inductive logic programming
Putting the pieces together
MLNs
Applications

9. Learning Markov Networks
Learning parameters (weights)
Generatively
Discriminatively
Learning structure (features)
In this tutorial: Assume complete data (If not: EM versions of algorithms)

10. Generative Weight Learning
Maximize likelihood or posterior probability
Numerical optimization (gradient or 2nd order)
No local maxima
Requires inference at each step (slow!)
[Like CRF learning but for P(x) not P(y|x) – WC]
No. of times feature i is true in data
Expected no. times feature i is true according to model

11. Pseudo-Likelihood
Likelihood of each variable given its neighbors in the data
Does not require inference at each step
Consistent estimator
Widely used in vision, spatial statistics, etc.
But PL parameters may not work well for long inference chains
[Like MEMM learning – WC]

12. Discriminative Weight Learning
Maximize conditional likelihood of query (y) given evidence (x)
Approximate expected counts by counts in MAP state of y given x
[Like CRF learning – WC]
No. of true groundings of clause i in data
Expected no. true groundings according to model

13. Overview Motivation Foundational areas Putting the pieces together
Probabilistic inference [Markov nets]
Statistical learning
Logical inference
Inductive logic programming
Putting the pieces together
MLNs
Applications

14. First-Order Logic
Constants, variables, functions, predicates E.g.: Anna, x, MotherOf(x), Friends(x, y)
Literal: Predicate or its negation
Clause: Disjunction of literals
Grounding: Replace all variables by constants E.g.: Friends (Anna, Bob)
World (model, interpretation): Assignment of truth values to all ground predicates

15. Inference in First-Order Logic
“Traditionally” done by theorem proving (e.g.: Prolog)
Propositionalization followed by model checking turns out to be faster (often a lot)
[This was popularized by Kautz & Selman – WC]
Propositionalization: Create all ground atoms and clauses
Model checking: Satisfiability testing
Two main approaches:
Backtracking (e.g.: DPLL)
Stochastic local search (e.g.: WalkSAT)

16. Satisfiability
Input: Set of clauses (Convert KB to conjunctive normal form (CNF))
Output: Truth assignment that satisfies all clauses, or failure
The paradigmatic NP-complete problem
Solution: Search
Key point: Most [random] SAT problems are actually easy
Hard region: Narrow range of #Clauses / #Variables

17. Backtracking Assign truth values by depth-first search
Assigning a variable deletes false literals and satisfied clauses
Empty set of clauses: Success
Empty clause: Failure
Additional improvements:
Unit propagation (unit clause forces truth value)
Pure literals (same truth value everywhere)

18. Stochastic Local Search
Uses complete assignments instead of partial
Start with random state
Flip variables in unsatisfied clauses
Hill-climbing: Minimize # unsatisfied clauses
Avoid local minima: Random flips
Multiple restarts

19. The WalkSAT Algorithm for i ← 1 to max-tries do
solution = random truth assignment
for j ← 1 to max-flips do
if all clauses satisfied then
return solution
c ← random unsatisfied clause
with probability p
flip a random variable in c
else
flip variable in c that maximizes
number of satisfied clauses
return failure

20. Overview Motivation Foundational areas Putting the pieces together
Probabilistic inference [Markov nets]
Statistical learning
Logical inference
Inductive logic programming
Putting the pieces together
MLNs
Applications

21. Markov Logic Logical language: First-order logic
Probabilistic language: Markov networks
Syntax: First-order formulas with weights
Semantics: Templates for Markov net features
Learning:
Parameters: Generative or discriminative
Structure: ILP with arbitrary clauses and MAP score
Inference:
MAP: Weighted satisfiability
Marginal: MCMC with moves proposed by SAT solver
Partial grounding + Lazy inference

22. Markov Logic: Intuition
A logical KB is a set of hard constraints on the set of possible worlds
Let’s make them soft constraints: When a world violates a formula, It becomes less probable, not impossible
Give each formula a weight (Higher weight  Stronger constraint)

23. Markov Logic: Definition
A Markov Logic Network (MLN) is a set of pairs (F, w) where
F is a formula in first-order logic
w is a real number
Together with a set of constants, it defines a Markov network with
One node for each grounding of each predicate in the MLN
One feature for each grounding of each formula F in the MLN, with the corresponding weight w

24. Example: Friends & Smokers

25. Example: Friends & Smokers

26. Example: Friends & Smokers

27. Example: Friends & Smokers
Two constants: Anna (A) and Bob (B)

28. Example: Friends & Smokers
Two constants: Anna (A) and Bob (B)
Smokes(A)
Smokes(B)
Cancer(A)
Cancer(B)

29. Example: Friends & Smokers
Two constants: Anna (A) and Bob (B)
Friends(A,B)
Friends(A,A)
Smokes(A)
Smokes(B)
Friends(B,B)
Cancer(A)
Cancer(B)
Friends(B,A)

30. Example: Friends & Smokers
Two constants: Anna (A) and Bob (B)
Friends(A,B)
Friends(A,A)
Smokes(A)
Smokes(B)
Friends(B,B)
Cancer(A)
Cancer(B)
Friends(B,A)

31. Example: Friends & Smokers
Two constants: Anna (A) and Bob (B)
Friends(A,B)
Friends(A,A)
Smokes(A)
Smokes(B)
Friends(B,B)
Cancer(A)
Cancer(B)
Friends(B,A)

32. Markov Logic Networks MLN is template for ground Markov nets
Probability of a world x:
Typed variables and constants greatly reduce size of ground Markov net
Functions, existential quantifiers, etc.
Infinite and continuous domains
Weight of formula i
No. of true groundings of formula i in x

33. Relation to Statistical Models
Special cases:
Markov networks
Markov random fields
Bayesian networks
Log-linear models
Exponential models
Max. entropy models
Gibbs distributions
Boltzmann machines
Logistic regression
Hidden Markov models
Conditional random fields
Obtained by making all predicates zero-arity
Markov logic allows objects to be interdependent (non-i.i.d.)

34. Relation to First-Order Logic
Infinite weights  First-order logic
Satisfiable KB, positive weights  Satisfying assignments = Modes of distribution
Markov logic allows contradictions between formulas

35. <aside> Is this a good thing?
Pros:
Drawing connections between problems
Specifying/visualizing/prototyping new methods …
Cons:
Robust, general tools are very hard to build for computationally hard tasks
This is especially true when learning is involved
</aside> -wc

36. Learning Data is a relational database
Closed world assumption (if not: EM)
Learning parameters (weights)
Learning structure (formulas)

37. Weight Learning Parameter tying: Groundings of same clause
Generative learning: Pseudo-likelihood
Discriminative learning: Cond. Likelihood
[still just like CRFs - all we need is to do inference efficiently. Here they use a Collins-like method. WC]
No. of times clause i is true in data
Expected no. times clause i is true according to MLN

38. Problem 1… Memory Explosion
Problem: If there are n constants and the highest clause arity is c, the ground network requires O(n ) memory
Solution: Exploit sparseness; ground clauses lazily → LazySAT algorithm [Singla & Domingos, 2006] c

39. Ground Network Construction
queue ← query nodes
repeat
node ← front(queue)
remove node from queue
add node to network
if node not in evidence then
add neighbors(node) to queue
until queue = Ø

40. Problem 2: MAP/MPE Inference
Problem: Find most likely state of world given evidence
Query
Evidence

41. MAP/MPE Inference
Problem: Find most likely state of world given evidence

42. MAP/MPE Inference
Problem: Find most likely state of world given evidence

43. MAP/MPE Inference
Problem: Find most likely state of world given evidence
This is just the weighted MaxSAT problem
Use weighted SAT solver (e.g., MaxWalkSAT [Kautz et al., 1997] )
Potentially faster than logical inference (!)

44. The MaxWalkSAT Algorithm
for i ← 1 to max-tries do
solution = random truth assignment
for j ← 1 to max-flips do
if ∑ weights(sat. clauses) > threshold then
return solution
c ← random unsatisfied clause
with probability p
flip a random variable in c
else
flip variable in c that maximizes
∑ weights(sat. clauses)
return failure, best solution found

45. Ended about here on 11/15

46. MCMC is Insufficient for Logic
Problem: Deterministic dependencies break MCMC Near-deterministic ones make it very slow
Solution: Combine MCMC and WalkSAT → MC-SAT algorithm [Poon & Domingos, 2006]

47. Current Best Solution: The MC-SAT Algorithm
[Basic idea: figure out how to sample from SAT assignments, and use this as a subroutine for MCMC - WC]
MC-SAT = MCMC + SAT
MCMC: Slice sampling with an auxiliary variable
for each clause
SAT: Wraps around SampleSAT (a uniform sampler)
to sample from highly non-uniform distributions
Sound: Satisfies ergodicity & detailed balance
Efficient: Orders of magnitude faster than
Gibbs and other MCMC algorithms
In this talk, I will talk about a new inf alg, MC-SAT, that we developed to meet this challenge.
MC-SAT combines ideas in MCMC and SAT.
From the MCMC perspective, MC-SAT is a slice sampling method using an aux var for each clause.
From the SAT perspective, MC-SAT wraps around SampleSAT, a uniform SAT sampler, to sample from highly non-uniform dist.
MC-SAT is sound and our experiments show that it is much faster than existing MCMC methods like Gibbs and ST.
MC-SAT accepts problems defined in ML, which is a very general language that subsumes finite FOL and MN. So MC-SAT is applicable to any S.R.L. problems. 48

48. The MC-SAT Algorithm
X ( 0 )  A random solution satisfying all hard clauses
for k  1 to num_samples
M  Ø
forall Ci satisfied by X ( k – 1 )
With prob. 1 – exp ( – wi ) add Ci to M
endfor
X ( k )  A uniformly random solution satisfying M
So! we arrive at the MC-SAT alg, simple and elegant.
We init w. a random sol that sat all hard clauses.
In each sampling, we first choose a random subset M of the sat clauses, subj to each prb of 1-e^-w_i.
Then, the next sample is drawn uniformly from solution to M. 49

49. The MC-SAT Algorithm Sound: Satisfies ergodicity and detailed balance
Assuming we have a perfect uniform sampler
Of course, we don’t (since the problem is #P-hard)
But there is an approximate uniform sampler [Wei et al. 2004]
SampleSAT = WalkSAT + Simulated annealing
Trade off uniformity vs. efficiency
by tuning the prob. of WS steps vs. SA steps
It is easy to prove that MC-SAT satisfies ergodicity and DB, assuming we have a perfect uniform sampler.
In practice, there is no efficient uniform sampler. Otherwise prb inf won’t be #P-complete.
So we use an approx sampler, the samplesat alg, developed by Wei et al
SS combines WS and SA. WS …; whereas SA …
SS works by, at each step, w. certain prob, running a WS step, and otherwise running an SA step.
And it achieves efficiency close to WS, and uniformity close to SA.
This prb can be used to trade off …
Now the question is whether MC-SAT will work w. the imperfect uniformity provided by SS. In exp, we find out that the answer is yes. 56

50. Overview Motivation Foundational areas Putting the pieces together
Probabilistic inference [Markov nets]
Statistical learning
Logical inference
Inductive logic programming
Putting the pieces together
MLNs
Applications

51. Applications Basics Logistic regression Hypertext classification
Information retrieval
Entity resolution
Hidden Markov models
Information extraction
Statistical parsing
Semantic processing
Bayesian networks
Relational models
Robot mapping
Planning and MDPs
Practical tips

52. Uniform Distribn.: Empty MLN
Example: Unbiased coin flips
Type: flip = { 1, … , 20 }
Predicate: Heads(flip)

53. Binomial Distribn.: Unit Clause
Example: Biased coin flips
Type: flip = { 1, … , 20 }
Predicate: Heads(flip)
Formula: Heads(f)
Weight: Log odds of heads:
By default, MLN includes unit clauses for all predicates
(captures marginal distributions, etc.)

54. Multinomial Distribution
Example: Throwing die
Types: throw = { 1, … , 20 }
face = { 1, … , 6 }
Predicate: Outcome(throw,face)
Formulas: Outcome(t,f) ^ f != f’ => !Outcome(t,f’).
Exist f Outcome(t,f).
Too cumbersome!

55. Multinomial Distrib.: ! Notation
Example: Throwing die
Types: throw = { 1, … , 20 }
face = { 1, … , 6 }
Predicate: Outcome(throw,face!)
Formulas:
Semantics: Arguments without “!” determine arguments with “!”.
Also makes inference more efficient (triggers blocking).

56. Multinomial Distrib.: + Notation
Example: Throwing biased die
Types: throw = { 1, … , 20 }
face = { 1, … , 6 }
Predicate: Outcome(throw,face!)
Formulas: Outcome(t,+f)
Semantics: Learn weight for each grounding of args with “+”.

57. Logistic Regression Logistic regression: Type: obj = { 1, ... , n }
Query predicate: C(obj)
Evidence predicates: Fi(obj)
Formulas: a C(x)
bi Fi(x) ^ C(x)
Resulting distribution:
Therefore:
Alternative form: Fi(x) => C(x)

58. Text Classification page = { 1, … , n } word = { … } topic = { … }
Topic(page,topic!)
HasWord(page,word)
!Topic(p,t) [bias term, since topics are sparse? –WC]
HasWord(p,+w) => Topic(p,+t)

59. Entity Resolution Problem: Given database, find duplicate records
HasToken(token,field,record)
SameField(field,record,record)
SameRecord(record,record)
HasToken(+t,+f,r) ^ HasToken(+t,+f,r’)
=> SameField(f,r,r’)
SameField(f,r,r’) => SameRecord(r,r’)
SameRecord(r,r’) ^ SameRecord(r’,r”)
=> SameRecord(r,r”)

60. Entity Resolution Can also resolve fields:
HasToken(token,field,record)
SameField(field,record,record)
SameRecord(record,record)
HasToken(+t,+f,r) ^ HasToken(+t,+f,r’)
=> SameField(f,r,r’)
SameField(f,r,r’) <=> SameRecord(r,r’)
SameRecord(r,r’) ^ SameRecord(r’,r”)
=> SameRecord(r,r”)
SameField(f,r,r’) ^ SameField(f,r’,r”)
=> SameField(f,r,r”)
P. Singla & P. Domingos, “Entity Resolution with
Markov Logic”, in Proc. ICDM-2006.

62. Hidden Markov Models obs = { Obs1, … , ObsN } state = { St1, … , StM }
time = { 0, … , T }
State(state!,time)
Obs(obs!,time)
State(+s,0)
State(+s,t) => State(+s',t+1)
Obs(+o,t) => State(+s,t)

63. Information Extraction (simplified)
Problem: Extract database from text or semi-structured sources
Example: Extract database of publications from citation list(s) (the “CiteSeer problem”)
Two steps:
Segmentation: Use HMM to assign tokens to fields
Entity resolution: Use logistic regression and transitivity

64. Motivation for joint extraction and matching

65. Information Extraction (simplified)
Token(token, position, citation)
InField(position, field, citation)
SameField(field, citation, citation)
SameCit(citation, citation)
Token(+t,i,c) => InField(i,+f,c)
InField(i,+f,c) <=> InField(i+1,+f,c)
f != f’ => (!InField(i,+f,c) v !InField(i,+f’,c))
Token(+t,i,c) ^ InField(i,+f,c) ^ Token(+t,i’,c’)
^ InField(i’,+f,c’) => SameField(+f,c,c’)
SameField(+f,c,c’) <=> SameCit(c,c’)
SameField(f,c,c’) ^ SameField(f,c’,c”) => SameField(f,c,c”)
SameCit(c,c’) ^ SameCit(c’,c”) => SameCit(c,c”)

66. Information Extraction (simplified)
Token(token, position, citation)
InField(position, field, citation)
SameField(field, citation, citation)
SameCit(citation, citation)
Token(+t,i,c) => InField(i,+f,c)
InField(i,+f,c) ^ !Token(“.”,i,c) <=> InField(i+1,+f,c)
f != f’ => (!InField(i,+f,c) v !InField(i,+f’,c))
Token(+t,i,c) ^ InField(i,+f,c) ^ Token(+t,i’,c’)
^ InField(i’,+f,c’) => SameField(+f,c,c’)
SameField(+f,c,c’) <=> SameCit(c,c’)
SameField(f,c,c’) ^ SameField(f,c’,c”) => SameField(f,c,c”)
SameCit(c,c’) ^ SameCit(c’,c”) => SameCit(c,c”)
More: H. Poon & P. Domingos, “Joint Inference in Information
Extraction”, in Proc. AAAI-2007.

67. Information Extraction (less simplified)
Token(token, position, citation)
InField(position, field, citation)
SameField(field, citation, citation)
SameCit(citation, citation)
Token(+t,i,c) => InField(i,+f,c)
InField(i,+f,c) ^ !Token(“.”,i,c) <=> InField(i+1,+f,c)
f != f’ => (!InField(i,+f,c) v !InField(i,+f’,c))
Token(+t,i,c) => InField(i,+f,c)
~Token("aardvark",i,c) v InField(i,”author”,c) …
~Token("zymurgy",i,c) v InField(i,"author",c)
~Token("zymurgy",i,c) v InField(i,"venue",c)

68. Information Extraction (less simplified)
Token(token, position, citation)
InField(position, field, citation)
SameField(field, citation, citation)
SameCit(citation, citation)
Token(+t,i,c) => InField(i,+f,c)
InField(i,+f,c) ^ !Token(“.”,i,c) <=> InField(i+1,+f,c)
f != f’ => (!InField(i,+f,c) v !InField(i,+f’,c))
Token(+t,i,c) => InField(i,+f,c)
InField(i,+f,c) => InField(i+1,+f,c)
InField(i,+f,c) ^ !HasPunct(c,i) => InField(i+1,+f,c)
InField(i,+f,c) ^ !HasComma(c,i) => InField(i+1,+f,c)
=> InField(1,”author”,c)
=> InField(2,”author”,c)
=> InField(midpointOfC, "title", c)

69. Information Extraction (less simplified)
Token(token, position, citation)
InField(position, field, citation)
SameField(field, citation, citation)
SameCit(citation, citation)
Token(+t,i,c) => InField(i,+f,c)
f != f’ => (!InField(i,+f,c) v !InField(i,+f’,c))
Token(+t,i,c) => InField(i,+f,c)
InField(i,+f,c) => InField(i+1,+f,c)
InField(i,+f,c) ^ !HasPunct(c,i) => InField(i+1,+f,c)
InField(i,+f,c) ^ !HasComma(c,i) => InField(i+1,+f,c)
=> InField(1,”author”,c)
=> InField(2,”author”,c)
=> InField(midpointOfC, "title", c)

70. Information Extraction (less simplified)
Token(+t,i,c) => InField(i,+f,c)
f != f’ => (!InField(i,+f,c) v !InField(i,+f’,c))
Token(+t,i,c) => InField(i,+f,c)
InField(i,+f,c) => InField(i+1,+f,c)
InField(i,+f,c) ^ !HasPunct(c,i) => InField(i+1,+f,c)
InField(i,+f,c) ^ !HasComma(c,i) => InField(i+1,+f,c)
=> InField(1,”author”,c)
=> InField(2,”author”,c)
InField(midpointOfC, "title", c)
Initial(t) => InField(t,i,"author") v InField(t,i,"venue")
i<j ^ VenueWord(t) ^ Token(t,i,c) ^ Token(t’,j,c) =>
!InField(t’,j,”author”) v !InField(t’,j,”title”)
Initial(t) => InField(t,”venue”) v InField(t,”author”)
i<j ^ Token(t,i,c) ^ LastInitial(j,c) =>
!InField(i,”title”) ^ !InField(i,”venue”)
i<j<k ^ Token(t,i,c) ^ Initial(j,c) ^ Initial(k,c) & !InField(j,”venue”) => !InField(k,”venue”) v [!InField(i,”author”) v !InField(i,”title”)]

71. Information Extraction (less simplified)
SimilarTitle(c,i,j,c’,i’,j’): true if
c[i..j] and c’[i’…j’] are both “titlelike”
i.e., no punctuation, doesn’t violate rules above
c[i..j] and c’[i’…j’] are “similar”
i.e. start with same trigram and end with same token
SimilarVenue(c,c’): true if c and c’ don’t contain conflicting venue keywords (e.g., journal vs proceedings)

72. Information Extraction (less simplified)
SimilarTitle(c,i,j,c’,i’,j’): …
SimilarVenue(c,c’): …
JointInferenceCandidate(c,i,c’):
trigram starting at i in c also appears in c’
and trigram is a possible title
and punct before trigram in c’ but not c

73. Information Extraction (less simplified)
SimilarTitle(c,i,j,c’,i’,j’): …
SimilarVenue(c,c’): …
JointInferenceCandidate(c,i,c’):
trigram starting at i in c also appears in c’
and trigram is a possible title
and punct before trigram in c’ but not c

74. Information Extraction (less simplified)
SimilarTitle(c,i,j,c’,i’,j’): …
SimilarVenue(c,c’): …
JointInferenceCandidate(c,i,c’):
InField(i,+f,c) ^ !HasPunct(c,i) => InField(i+1,+f,c)
InField(i,+f,c) ^ ~HasPunct(c,i) ^
(!exists c’:JointInferenceCandidate(c,i,c’)) =>
InField(i+1,+f,c)
Jnt-Seg

75. Information Extraction (less simplified)
SimilarTitle(c,i,j,c’,i’,j’): …
SimilarVenue(c,c’): …
JointInferenceCandidate(c,i,c’):
InField(i,+f,c) ^ ~HasPunct(c,i) ^
(!exists c’:JointInferenceCandidate(c,i,c’) ^ SameCitation(c,c’) ) =>
InField(i+1,+f,c)
Jnt-Seg-ER

76. Results

77. Results
Fraction of clusters correctly constructed using transitive closure of pairwise decisions
Cora F-S: F1
Cora TFIDF: max F1

78. William’s summary
MLNs are a compact, elegant way of describing a Markov network
Standard learning methods work
Network may be very very large
Inference may be expensive
Doesn’t eliminate feature engineering
E.g., complicated “feature” predicates
Experimental results for joint matching/NER are not that strong overall
Cascading segmentation and then matching improves segmentation, probably not matching
But it needs to be carefully restricted (efficiency?)
And it’s no better than a “pseudo” joint model using plausible matches as features