1. Prototype-Driven Grammar Induction Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley

2. Grammar Induction DT NN VBD DT NN IN NN The screen was a sea of red

3. First Attempt DT NN VBD DT NN IN NN The screen was a sea of red

4. Central Questions How do we specify what we want to learn? How do we fix observed errors? What’s an NP? That’s not quite it!

5. Experimental Set-up Binary Grammar { X 1, X 2, … X n } plus POS tags Data WSJ-10 [7k sentences] Evaluate on Labeled F 1 Grammar Upper Bound: 86.1 XjXj X i XkXk

6. Experiment Roadmap Unconstrained Induction Need bracket constraint! Gold Bracket Induction Prototypes and Similarity CCM Bracket Induction PhrasePrototype NPDT NN PPIN DT NN

7. Unconstrained PCFG Induction Learn PCFG with EM Inside-Outside Algorithm Lari & Young [93] Results 0 i j n  (Inside)  (Outside)

8. Gold Brackets Periera & Schables [93] Result Constrained PCFG Induction

9. Encoding Knowledge What’s an NP? Semi-Supervised Learning

10. Encoding Knowledge What’s an NP? For instance, DT NN JJ NNS NNP Prototype Learning

11. Add Prototypes Manually constructed Grammar Induction Experiments

12. How to use prototypes? ? ? ? ? DT The NN koala VBD sat IN in DT the NN tree ¦¦ ? NP PP VP S

13. How to use prototypes? NP PP VP DT The NN koala VBD sat IN in DT the NN tree ¦ ¦ S JJ hungry ?NP

14. Distributional Similarity Context Distribution  (DT JJ NN) = { ¦ __ VBD : 0.3, VBD __ ¦ : 0.2, IN __ VBD: 0.1, ….. } Similarity  (DT NN)  (DT JJ NN)  (JJ NNS)  (NNP NNP) NP

15. Distributional Similarity Prototype Approximation  (NP) ¼ Uniform (  (DT NN),  (JJ NNS),  (NNP NNP) ) Prototype Similarity Feature span(DT JJ NN) emits proto=NP span(MD NNS) emits proto=NONE

16. Prototype CFG + Model NP PP VP DT The NN koala VBD sat IN in DT the NN tree ¦ ¦ S JJ hungry NP P (DT NP | NP) P (proto=NP | NP)

17. Prototype CFG+ Induction Experimental Set-Up BLIPP corpus Gold Brackets Results

18. Summary So Far Bracket constraint and prototypes give good performance!

19. Constituent-Context Model

20. Product Model Different Aspects of Syntax CCM : Yield and Context properties CFG : Hierarchical properties Intersected EM [Klein 2005] Encourages mass on trees compatible with CCM and CFG

21. Grammar Induction Experiments Intersected CFG and CCM No prototypes Results

22. Grammar Induction Experiments Intersected CFG + and CCM Add Prototypes Results

23. Reacting to Errors Possessive NPs Our Tree Correct Tree

24. Reacting to Errors Add Prototype: NP-POS NN POS New Analysis

25. Error Analysis Modal VPs Our TreeCorrect Tree

26. Reacting to Errors Add Prototype: VP-INF VB NN New Analysis

27. Fixing Errors Supplement Prototypes NP-POS and VP-INF Results

28. Results Summary

29. Conclusion Prototype-Driven Learning Flexible Weakly Supervised Framework Merged distributional clustering techniques with supervised structured models

30. Thank You! http://www.cs.berkeley.edu/~aria42

31. Unconstrained PCFG Induction Binary Grammar { X 1, X 2, … X n } Learn PCFG with EM Inside-Outside Algorithm Lari & Young [93] 0 i j n  (Inside)  (Outside) V XjXj X i XkXk N XkXk XjXj V N