1 Tree-edit CRFs for RTE Mengqiu Wang and Chris Manning

2 Tree-edit CRFs for RTE Extension to McCallum et al. UAI2005 work on CRFs for finite-state String Edit Distance Key attractions: Models the transformation of dependency parse trees (thus directly models syntax), unlike McCallum et al. ’05, which only models word strings Discriminatively trained

3 TE-CRFs model in details First of all, let’s look at the correspondence between alignment (with constraints) and edit operations

Bush NNP person met VBD French JJ location president NN Jacques Chirac NNP person who WP qword leader NN is VB the DT France NNP location Q:A: $ root $ root subjobj detof root subjwith nmod substitute delete insert Fancy substitute

5 TE-CRFs model in details Each valid tree edit operation sequence that transforms one tree into the other corresponds to an alignment. A tree edit operation sequence is models as a transition sequence among a set of states in a FSM S1 S2 S3 D, S, I D, E, I D, S, I substitute deletesubstitute insert substitute S1 S2 S1 S3S1 S2 S3 S2 S1S2 S1S3 S2 S1 S3 S2 … … … … … … …

6 FSM This is for one edit operation sequence substitute deletesubstitute insert substitute S1 S2 S1 S3S1 S2 S3 S2 S1S2 S1S3 S2 S1 S3 S2 … … … … … … … delete substitute insert substitute S1 S2 S1 S3S1 … … … … … … … substitute deletesubstitute insert S1 S2 S1 S3S1 … … … … … … … substitute deletesubstitute insert substitute S1 S2 S1 S3S1 … … … … … … … There are many other valid edit sequences

7 FSM cont. S1 S2 S3 D, S, I Start Stop ε ε S1 S2 S3 D, S, I Positive State Set Negative State Set ε ε

8 FSM transitions S3 S2 S1 S3 S2 Start S2 S3 S1 S2 S1 S2 S1 S3 … …… … S2 … … … … … … … Stop S3 S2 S1 S3 S2 S3 S1 S2 S1 S2 S1 S3 … …… … S2 … … … … … … … Positive State Set Negative State Set

9 What is the semantic interpretation of the FSM states? At this moment since all the states in the FSM are all fully-connected, it’s unclear what they mean. We fix the number of states to 3, and experiments shows that setting it to 1 or 6 hurts performance. We are running new experiments with more meaningfully designed FSM topologies, e.g., each states deterministically corresponds to a particular edit operation.

10 Parameterization S1 S2 substitute positive or negative positive and negative

11 Training using EM E-step M-step Using L-BFGS Jensen’s Inequality

12 Features for RTE Substitution Same -- Word/WordWithNE/Lemma/NETag/Verb/Noun/Adj/Adv/Other Sub/MisSub -- Punct/Stopword/ModalWord Antonym/Hypernym/Synonym/Nombank/Country Different – NE/Pos Unrelated words Delete Stopword/Punct/NE/Other/Polarity/Quantifier/Likelihood/Condition al/If Insert Stopword/Punct/NE/Other/Polarity/Quantifier/Likelihood/Condition al/If Tree RootAligned/RootAlignedSameWord Parent,Child,DepRel triple match/mismatch Date/Time/Numerical DateMismatch, hasNumDetMismatch, normalizedFormMismatch

13 Tree-edit CRFs for Textual Entailment Preliminary results Trained on RTE2 dev, tested on RTE2 test. model taken after 50 EM iterations acc:0.6275, map: SUM, acc=0.675 QA, acc=0.64 IR, acc=0.615 IE, acc=0.58

14 Work in progress Implementing a unordered tree-edit algorithm, which would allow swapping of sub-trees Use Stanford Parser dependency structure. Need to getting rid of cycles in CollapsedDependencyGraph (almost there, only have a few self-loops now). Experiment with deterministic topologies More features!! Training a separate model for each sub-task (is task information given at test time?)