1. Markov Logic Networks for NLP CSCI-GA.2591
NYU
Markov Logic Networks for NLP CSCI-GA.2591
Ralph Grishman

2. Markov Logic Networks
There is strong interest in combining symbolic and statistical reasoning
The general area is termed “Statistical Relational Learning”
One of the most widely used representations is Markov Logic Networks
early software support (Alchemy)
continuing software development (Tuffy)
lots of publications (Domingos)

3. Markov Logic
In a traditional knowledge base, consisting of a set of formulas, all these formulas must be true in each possible world
In Markov logic, each formula is assigned a weight
a formula which is violated makes the world less likely but still possible

4. Markov Logic Network Consists of:
a set of pairs {<Fi, wi>} of a formula and its weight
a set of constants C = {c1, … , cn}
assume that C is closed under all functions
Network has a node for each possible grounding of each formula
two nodes are connected if their formulas share a ground atom

5. Probability of a state x
where
F = number of formulas Fi
ni(x) = number of true groundings of Fi in x

6. An Example (from last week)
Four axioms:
friends of friends are friends
people without friends smoke
people who smoke get cancer
if two people are friends, either both smoke or neither does
Predicates:
are friends: F(x, y)
smokes: S(x)
gets cancer: C (x)

7. … adding weights
Clearly these formulas are not always true: not everyone who smokes gets cancer
Adding weights to these formulas allows us to capture formulas which are generally true
the weights reflect the strength of the coupling of the two nodes, but are not probabilities (may be more than 1)

8. An Example … with weights
Four axioms:
friends of friends are friends
people without friends smoke
people who smoke get cancer
if two people are friends, either both smoke or neither does
Predicates:
are friends: F(x, y)
smokes: S(x)
gets cancer: C (x)

9. Harnessing the Power
This seems intuitively a more powerful representation than FOL or MRFs separately
can it be made efficient enough?
in view of the brute-force approach:
we deal with quantified formulas by grounding them
producing very large sets of formulas

10. Inference First task: finding most likely state (grounding)
subproblem: SAT (satisfiability task for propositional logic): deciding whether a propositional logic formula can be satisfied, and give a state which satisfies it
ex: x  y satisfied if x=false and y = true
x & !x not satisfiable
How about (a + b + c) (a’ + b’ + c) (a + b’ + c’) (a’ + b + c’)?
intensively studied problem [Davis-Putnam procedure, 1962]
NP hard

11. WalkSAT Random walk to search for satisfiability Repeat:
select a clause not satisfied by current assignment to variables (if none, done)
choose a variable at random and flip it
Generally prefer flipping a variable which minimizes the number of clauses which become unsatisfied (but sometimes make a pure random choice c

12. MaxWalkSAT
Generalization of WalkSAT to handle weighted propositions

13. Reducing memory requirements
problem: a domain may be too large to fully instantiate
address through lazy instantiation: LazyWalkSAT
observe that
most propositions are false
most clauses are true
so just keep true propositions and false clauses in memory

14. More SATs SampleSAT combines WalkSAT with simulated annealing
MC-SAT in Alchemy extends SampleSAT

15. Learning Scaled conjugant gradient or similar
Maximize likelihood of data base
Can be quite slow – must do inference at cycle
Use pseudo-likekihood (dependent only on Markov blanket) to speed up
Optimizing more difficult because of very different constraints
Deterministic and near-deterministic dependencies

16. Alchemy Implementation of Markov Logic Networks
From Univ. of Washington

17. Installation / Access Distribution is for Linux systems
General site:
I have been running Alchemy 2.0
Distribution is for Linux systems
User’s manual (section 2)
Suggestions for building for Mac or Windows at: (may not be sufficient)
Installed on CIMS servers at ~grishman/alchemy/alchemy2

18. MLN Files (highlights)
3 components
Declaration of types and constants
Declaration of predicates
List of formulas and their weights

19. Data Base File Typically large file of ground atoms
Training and test sets

20. Basic operations Learn weights Infer
learnwts -g –i f.mln –t f.train.db –o f.out.mln
Infer
infer –i f.out.mln –e f.test.db –q … -r results

21. Warm-Up Assignment
The ACE event inventory includes Attack, Injure, and Die events, which are correlated … intuitively, attacks cause the attacked party both injury and death.
Using Alchemy,
Represent the predicates by attack(x,y), injured(y), and die(y)
Write the formulas to expess the relations between these predicates
Prepare small training and test data bases
Learn weights for these formulas
Infer the probability of the ground atoms
Very briefly, explain the inferred results.
Due: one week (Nov. 20)

22. Looking ahead
Hopefully it is a small step from this to doing joint inference for trigger identification … combining these rules with P(event-type | trigger-word)