1. LEXICALIZATION AND CATEGORIAL GRAMMARS: ARAVIND K. JOSHI A STORY BAR-HILLEL MIGHT HAVE LIKED UNIVERSITY OF PENNSYLVANIA PHILADELPHIA, PA 19104 USA June 1995

2. lex-cg, Israel, June 95:2 Outline u Introduction u Lexicalization –Weak Lexicalization and Strong Lexicalization u Strong lexicalization and Lexicalized Tree- Adjoining Grammars (LTAGs) u Strong Lexicalization and Categorial Grammars (CG) –Basic partial proof trees u Inference rules from proof trees to proof trees –Formal characterization of the inference rules –Relevance to parsing u Summary

3. lex-cg, Israel, June 95:3 Introduction Equivalence of categorial grammars and context-free grammars (Bar-Hillel, Gaifman and Shamir 1960) Fate of grammars in the 60’s that were shown to be equivalent or conjectured to be equivalent to CFGs Non-transformational or minimally transformational grammars of the 70’s, 80’s and 90’s - GPSG, LFG, HPSG, various types of CGs, LTAG and others - GB, Minimalist Theory - LTAGs are, in a sense, transformational, reminiscent of ‘generalized transformations’ in the earliest formulation of transformational grammars

4. lex-cg, Israel, June 95:4 Introduction Bar-Hillel et al. 1960 suggested that CFGs (and by implication grammars equivalent to CFGs) can be used for the so-called ‘kernel’ sentences of Chomsky Categorial Grammars with partial proof trees CG (PPT), the system presented here, can be thought of as related to this suggestion of Bar-Hillel et al. 1960 This relationship and Bar-Hillel’s strong interest in comparative studies of formal grammars are the basis for the second half of the title -- A story Bar-Hillel might have liked

5. lex-cg, Israel, June 95:5 Related work u Proof trees, Morrill et al. 1990 u Description trees, Vijayshanker 1993 u HPSG compilation into LTAG trees, Kasper et al. 1992/1995

6. lex-cg, Israel, June 95:6 Lexicalization u A grammar G is a lexicalized grammar if it consists of – a finite set of structures (strings, trees, dags, for example), each structure being associated with a lexical item, called its anchor – a finite set of operations for composing these structures u A grammar G strongly lexicalizes another grammar G’ if G is a lexicalized grammar and the structural descriptions (trees, for example) of G and G’ are exactly the same

7. lex-cg, Israel, June 95:7 Lexicalized grammars Context-free grammar (CFG) CFG, G S  NP VP VP  V NP VP  VP ADV NP  Harry NP  peanuts V  likes ADV  passionately (Non-lexical) (Lexical)S NPVP Harry VPADV V NP passionately likespeanuts

8. lex-cg, Israel, June 95:8 CFGs can weakly lexicalize CFGs but not strongly Greibach Normal Form (GNF) CFG rules are of the form A  a B 1 B 2... B n A  a This lexicalization gives the same set of strings but not the same set of trees, i.e., the same set of structural descriptions. Hence, it is a weak lexicalization. Converting a CFG to a categorial grammar (CG) gives only weak lexicalization and not necessarily a strong lexicalization. Ajdukiewicz and Bar-Hillel Categorial Grammars CG(AB) weakly lexicalize CFGs but not strongly.

9. lex-cg, Israel, June 95:9 Strong lexicalization of CFGs Same set of strings and same set of trees or structural descriptions. Tree substitution grammars Increased domain of locality Substitution as the combining operation

10. lex-cg, Israel, June 95:10 CG(AB) cannot strongly lexicalize CFGs CFG, G: S  SS S  a CG(AB), G’ a: S a: S/S G’ weakly lexicalizes G but not strongly. Not all trees of G are proof trees of G’ (assuming appropriate relabeling of nodes ). Note: Adding function composition helps in this example but, in general, it will not help.

11. lex-cg, Israel, June 95:11 Strong lexicalization -- Tree substitution grammars CFG, G S  NP VP VP  V NP NP  Harry NP  peanuts V  likes TSG, G’ a 1 S NP  VP V NP  likes 22 NP Harry  3 NP peanuts

12. lex-cg, Israel, June 95:12 TSGs cannot strongly lexicalize CGFs Formal insufficiency of TSG G: S  SS (non-lexical) S  a (lexical) TSG: G ’ :  1 : S SS S a 2:2: S SS S a 3:3: S a

13. lex-cg, Israel, June 95:13 TSGs cannot lexicalize CFGs TSG: G’:  1 : S SS S a 2:2: S SS S a 3:3: S a t: S S S SS SS S S a a a a a G’ can generate all strings of G but not all trees of G. TSGs cannot strongly lexicalize CFGs. Thus substitution alone is not enough.

14. lex-cg, Israel, June 95:14 TSGs are also linguistically inadequate Linguistic inadequacy of TSG G: S  NP VP VP  VP ADV VP  V NP NP  Harry/ peanuts V  likes ADV  passionately G’:  1 : S NP  VP V NP  likes  2 : NP Harry  3 : NP peanuts  4 : VP VP  ADV passionately G’ is inadequate. It cannot achieve recursion on VP.

15. lex-cg, Israel, June 95:15 Linguistic inadequacy of TSGs G’’:  1 : S NP  VP V NP  likes  2 : NP Harry  3 : NP peanuts  4 : VP VP  ADV passionately 5:5: S NP  VP VP  ADV passionately 6:6: VP V NP  likes Even when a CFG can be lexicalized by substitution alone, the lexical anchors may not be linguistically appropriate.

16. lex-cg, Israel, June 95:16 TSGs with substitution and adjoining -- LTAGs TSG: G’:  1 : S S*S a 2:2: S S a 3:3: a S G: S  SS S  a Adjoining  2 to  3 at the S node, the root node and then adjoining  1 to the S node of  2, the left daughter of the root node, we have . :: S SS SS a a a LTAGs strongly lexicalize CFGs. Adjoining is crucial for lexicalization.

17. lex-cg, Israel, June 95:17 Adjunction permits appropriate choice of lexical anchors G3:G3:  1 : S NP*VP V NP  likes  2 : NP Harry  3 : NP peanuts  4 : VP VP* ADV passionately A tree rooted in S and anchored in ‘passionately’ is not needed. Lexical anchors as functors.

18. lex-cg, Israel, June 95:18 Adjoining :: X :: X X*  ’: X X* Summary of lexicalization LTAGs strongly lexicalize CFGs. Adjoining and, therefore, LTAGs arise out of lexicalization of CFGs.

19. lex-cg, Israel, June 95:19 Lexicalized Tree-Adjoining Grammars (LTAGs) Finite set of elementary trees anchored on lexical items Elementary trees Initial trees Auxiliary trees Operations Substitution Adjoining Derivation Derivation tree -- How elementary trees are put together. Derived tree

20. lex-cg, Israel, June 95:20 Properties of LTAGs Localization of dependencies Syntactic locality Agreement Subcategorization Filler-gap Word order Local scrambling Long distance scrambling-- movement across clauses Word clusters (flexible idioms) -- non-compositionality Function -- argument

21. lex-cg, Israel, June 95:21 Properties of LTAGs Extended domain of locality (EDL) Factoring recursion from the domain of dependencies (FRD) All interesting properties of LTAG follow from EDL and FRD Mathematical - Computational: mild context-sensitivity, polynomial parsability, semi-linearity, etc. Linguistic

22. lex-cg, Israel, June 95:22 Strong lexicalization: EDL, FRD lex-cg-June 95 CFGLTAG CG (AB) CG (PPT) Weak equivalence? Weak equivalence Strong Lex-EDL, FRD CG (AB), although weakly equivalent to CFG, do not lexicalize CFG. CG (AB) has function application only. In analogy to LTAG, we work with larger structures, Partial Proof Trees (PPT) and inference rules from proof trees to proof trees. CG (PPT) has properties (linguistic and mathematical) similar to LTAG. Strong Lex-EDL, FRD

23. lex-cg, Israel, June 95:23 Strong lexicalization: EDL, FRD likes (NP\S)/NP [NP] [NP] (NP\S) S Main idea Each lexical item is associated with one or more (basic) partial proof trees (BPPT) obtained by unfolding arguments. B(PPT) is the (finite) set of BPPTs -- the set of basic types. Informal description of the inference rule -- linking

24. lex-cg, Israel, June 95:24 How is B(PPT), finite set of basic partial proof trees, constructed? Unfold arguments of the type associated with a lexical item in a CG (AB) by introducing assumptions. No unfolding past an argument which is not an argument of the lexical item. If a trace assumption is introduced while unfolding then it must be locally discharged, i.e., within the basic PPT which is being constructed. While unfolding we can interpolate, say, from X to Y where X is a conclusion node and Y is an assumption node.

25. lex-cg, Israel, June 95:25 Unfolding arguments the NP/N [N] NP man N apples NP likes (NP\S)/NP [NP] [NP] (NP\S) S the man likes the apples Linking conclusion nodes to assumption nodes

26. lex-cg, Israel, June 95:26 How is B(PPT), finite set of basic partial proof trees, constructed? Unfold arguments of the type associated with a lexical item in a CG (AB) by introducing assumptions. No unfolding past an argument which is not an argument of the lexical item. If a trace assumption is introduced while unfolding then it must be locally discharged, i.e., within the basic PPT which is being constructed. While unfolding we can interpolate, say, from X to Y where X is a conclusion node and Y is an assumption node.

27. lex-cg, Israel, June 95:27 No unfolding past a non-argument passionately [(NP\S)] (NP\S)\ (NP*\S) (NP*\S) The subject NP marked by * is not an argument of ‘passionately’. This a property of the lexical item and thus it can be marked on the type assigned to the lexical item by CG (AB). No unfolding past an argument marked by *. Thus unfolded arguments are only those which are the arguments of the lexical item.

28. lex-cg, Israel, June 95:28 Stretching and linking -- First informal inference rule A proof tree can be stretched at any node. u v w X Y A proof tree to be stretched at the node X.

29. lex-cg, Israel, June 95:29 Stretching a proof tree at node X u v w X Y X Y [X] X is the conclusion from v Y is the conclusion from u [X] w i.e., from u, assumption X and w Linking X to [X] we have the original proof tree.

30. lex-cg, Israel, June 95:30 Stretching and linking -- an example likes (NP\S)/NP [NP] [NP] (NP\S) S Stretching at the indicated node

31. lex-cg, Israel, June 95:31 Stretching and linking -- an example likes (NP\S)/NP [NP] [NP] (NP\S) S (NP\S]

32. lex-cg, Israel, June 95:32 Stretching and linking -- an example likes (NP\S)/NP [NP] [NP] (NP\S) S (NP\S)] passionately [(NP\S)] (NP\S)\ (NP*\S) (NP*\S) Linking conclusion nodes to assumption nodes and assuming that appropriate proof trees are linked to the two NP assumption nodes, we have John likes apples passionately

33. lex-cg, Israel, June 95:33 Unfold arguments of the type associated with a lexical item in a CG (AB) by introducing assumptions. No unfolding past an argument which is not an argument of the lexical item. If a trace assumption is introduced while unfolding then it must be locally discharged, i.e., within the basic PPT which is being constructed. While unfolding we can interpolate, say, from X to Y where X is a conclusion node and Y is an assumption node. How is B(PPT), finite set of basic partial proof trees, constructed?

34. lex-cg, Israel, June 95:34 Introduction and discharge of trace assumption likes e (NP\S)/NP [NP] [NP] (NP\S) S [NP] S Trace assumption Local discharge of the trace assumption. The appropriate directionality by convention. Apples Mary likes

35. lex-cg, Israel, June 95:35 An example using a PPT with trace assumption, stretching and linking likes e (NP\S)/NP [NP] [NP] (NP\S) S [NP] S [S] thinks (NP\S)/S[S][NP] (NP\S) S Apples John thinks Mary likes John NP apples NP Mary NP

36. lex-cg, Israel, June 95:36 An example of a PPT with trace assumption, stretching and linking calls e (NP\S)/NP [NP] [NP] (NP\S) S [NP] S [NP\S] John NP Mary NP (NP*\S)\(NP\S)[ (NP\S) ] (NP*\S) everyday Note: In a natural deduction type CG, a permutation operator is needed for this the system. John Mary calls everyday (NP\S)

37. lex-cg, Israel, June 95:37 Basic PPT for object relative clause meets e (NP\S)/NP [NP] [NP] (NP\S) S (N\N) Trace assumption Local discharge of the trace assumption. The appropriate directionality by convention. wh (N\N)/(NP\S)[N] N who Bill meets

38. lex-cg, Israel, June 95:38 Object relative clause, stretching and linking meets e (NP\S)/NP [NP] [NP] (NP\S) S (N\N) wh (N\N)/(NP\S)[N] N [(NP\S)] (NP*\S)\(NP\S)[(NP\S)] (NP*\S) today who Bill meets today Note: In a natural deduction type CG, a permutation operator is needed for this case, which adds power to the system.

39. lex-cg, Israel, June 95:39 How is B(PPT), finite set of basic partial proof trees, constructed? Unfold arguments of the type associated with a lexical item in a CG (AB) by introducing assumptions. No unfolding past an argument which is not an argument of the lexical item. If a trace assumption is introduced while unfolding then it must be locally discharged, i.e., within the basic PPT which is being constructed. While unfolding we can interpolate, say, from X to Y where X is a conclusion node and Y is an assumption node.

40. lex-cg, Israel, June 95:40 An example -- John tries to walk tries (NP\S)/Sinf[Sinf] (NP\S) [NP] S walk(inf) (NPpro)\Sinf[NPpro] Sinf John NP John tries to walk Note: Subject NP is an argument for tries. Hence, unfolding continues past NP in (NP\S).

41. lex-cg, Israel, June 95:41 Raising verbs -- subject NP is not an argument seems (NP*\S)/(NP\Sinf)[(NP\Sinf)] (NP*\S) Subject NP is not an argument of seems. Hence, unfolding does not continue past NP in (NP*\S).

42. lex-cg, Israel, June 95:42 Interpolation in a basic PPT Another basic PPT for walk(inf) walk(inf) (NP\Sinf) [(NP\S)] [NP] S Interpolation from (NP\Sinf) to (NP\S)

43. lex-cg, Israel, June 95:43 Interpolation and linking walk(inf) (NP\Sinf) [(NP\S)] [NP] S seems (NP*\S)/(NP\Sinf)[(NP\Sinf)] (NP*\S) John NP John seems to walk

44. lex-cg, Israel, June 95:44 Interpolation: extraction of an NP under a PP complement gives (NP\S)/PP/NP[NP][PP] (NP\S)/PP (NP\S) [NP] S

45. lex-cg, Israel, June 95:45 Interpolation: extraction of an NP under a PP complement to PP/NP[NP] e PP [S] [NP] NP\S Interpolation:From PP to S S Local discharge of trace assumption

46. lex-cg, Israel, June 95:46 Interpolation: extraction of an NP from a PP complement gives (NP\S)/PP/NP[NP][PP] (NP\S)/PP (NP\S) [NP] S to PP/NP[NP] e [S] [NP] S PP (NP\S) John NP books NP Mary NP Mary John gives books to

47. lex-cg, Israel, June 95:47 Formal representation of the inference rules u Rules for the three types of operations on PPTs -- linking, stretching, and interpolation -- are from proof trees to proof trees.  These operations are specified by inference rules that take the form of -operations, where the body of the -term is itself a proof.  A version of typed label-selective -calculus (Garrigue and Ait-Kaci 1994) –Arguments have both symbol and numeric labels “the use of labels for argument selection enhances clarity and obviates the need of argument-shuffling combinators” Garrigue and Ait-Kaci 1994

48. lex-cg, Israel, June 95:48 u Although arguments must be applied along the correct channels, it does not matter in what order they are applied -- two reductions of Bob likes Hazel  Stretching and linking can also be handled by  - reduction, where the proof tree to be stretched at a node becomes an abstraction over an inference rule -- higher-order  -reduction.  A similar higher-order  -reduction is used to handle interpolation. The inference rule abstraction for interpolation is done during the course of building the basic PPT. Formal representation of the inference rules

49. lex-cg, Israel, June 95:49 CG (PPT) is more powerful than CG (AB): A strictly non-context-free language generated by CG (PPT) a (S/C)/B[B][C] (S/C) S a (S/C*)/C/B/(S/C) [S/C] [B][C] (S/C*)/C/B (S/C*)/C (S/C*) b B c C L is the language generated by this CG(PPT) L  { a* b* c*} = {a n bnbn c n | n  1}

50. lex-cg, Israel, June 95:50 CG(PPT) and crossing dependencies a S/B[B] e S S/B a (S/B*)/B/(S/B)[B] e [S/B] (S/B*)/B (S/B) (S/B*)/B L = { a n b n | n  1} The dependencies are as follows. a a a... b b b b B Local discharge of the trsce assumption S (S/B*)

51. lex-cg, Israel, June 95:51 Parsing CG(PPT) l Given a string w determine how a proof tree t for w is built from the basic partial proof trees. l Analogous to parsing LTAGs. Hence, algorithms for parsing LTAGs can be extended to CG(PPT). l Complexity of parsing -- O(n 6 )

52. lex-cg, Israel, June 95:52 Summary CFGLTAG CG (AB) CG (PPT) Weak equivalence? Strong Lex-EDL, FRD The finite set of basic partial proof trees, B(PPT) is constructed with limited machinery. (1) unfolding and function application (2) local discharge of trace assumptions (3) interpolation Weak equivalence

53. lex-cg, Israel, June 95:53 Summary Inference rules from proof trees to proof trees Stretching and linking Interpolation and linking CG(PPT) is more powerful than CG(AB), both weakly and especially strongly. Linguistic adequacy Polynomial parsing? Giving up string adjacencygives more power, weak and strong Formal representation of inference rules using label selective -calculus and helps parsing

54. lex-cg, Israel, June 95:54 Summary -- Relationship to Bar-Hillel’s work Bar-Hillel et al. 1960 suggested that CFGs (and by implication grammars equivalent to CFGs) can be used for the so-called ‘kernel’ sentences of Chomsky Categorial Grammars with partial proof trees CG (PPT), the system presented here, can be thought of as related to this suggestion of Bar-Hillel et al. 1960 This relationship and Bar-Hillel’s strong interest in comparative studies of formal grammars are the basis for the second half of the title -- A story Bar-Hillel might have liked