1. Approximate Factoring for A* Search Aria Haghighi, John DeNero, and Dan Klein Computer Science Division University of California Berkeley

2. Inference for NLP Tasks A* Search

3. Inference as Search y a1a1 a2a2 a3a3 Partial Hypothesis a2a2

4. VP S NP Bitext Parsing as Search translation is hard, la traducción es dificil Weighted Synchronous Grammar Parsing O(n 6 ) Modified CKY over bi-spans (X[i,j],X’[i’,j’]) Source Target VP S NP SS’

5. A* Search Completion ScoreScore So Far y

6. A* Search  Heuristic Design  Tight small  Admissible  Efficient to compute This way hypothesis! A* Heuristic Man Optimal Result

7. A* Example: Bitext Search Viterbi Inside Score Cost So Far Bi-Span

8. A* Bitext Search Viterbi Outside Score Completion Score O(n6)O(n6) Ideal Heuristic

9. Of Stately Projections ¼ SS’ S S VP S NP S S’ S VP S NP VP’ S’ NP’VP’ S’ NP’

10. A* Bitext Search Suppose, Then, VP S NP S S’ VP S NP S VP’ S’ NP’

11. Projection Heuristic O(n3)O(n3) O(n3)O(n3) O(n6)O(n6) Klein and Manning [2003]

12. When models don’t factorize

13. Pointwise Admissibility y c( a ) x ¼s(y)¼s(y) Ás(a)Ás(a) ¼s(x)¼s(x) ¼t(y)¼t(y) Át(a)Át(a) ¼t(x)¼t(x)

14. When models don’t factorize Admissibility ¼s(y)¼s(y) ¼t(y)¼t(y) y

15. Finding Factored Costs Pointwise Gap How to find Á s and Á t ?

16. Finding Factored Costs Small gaps

17. Finding Factored Costs Pointwise Admissibility

18. Finding Factored Costs

19. Bitext Experiments Synchronous Tree-to-Tree Transducer  Trained on 40k sentences of English-Spanish Europarl [Galley et. al, 2004]  Rare words replaced with POS tags  Tested on 1,200 sent. max length 5-15 Optimization Problem  Solved only once per grammar  206K Variables  160K Constraints  29 minutes

20. Bitext Experiments

22. Zhang and Gildea (2006)

23. Bitext Experiments Zhang and Gildea (2006)

24. Lexicalized Parsing NP- (translation,NN) S- (is,VBZ) VP-(is,VBZ) (is,VBZ) (translation, NN) NP S VP Klein and Manning [2003]

25. Lexicalized Parsing

26. Too many constraints to efficiently solve! Over 64e 13 possible lexicalized rules

27. Lexicalized Parsing

31. Lexicalized Model Experiments Standard Setup  Train on section 2-21 of the treebank  Test on section 23 (length · 40) Models Tested  Factored model [Klein and Manning, 2003]  Non-Factored Model

32. Lexicalized Parsing Factored Model [Klein and Manning, 2003]

33. Lexicalized Parsing Non-Factored Model

34. Conclusions  General technique for generating A* estimates  Can explicitly control admissibility tightness trade-off  Future Work: Explore different objectives and applications

35. Thanks http://nlp.cs.berkeley.edu