1. Guarded Constraints in Natural Language Processing
Kathryn L. Baker
May 3, 2002
Committee:
Bob Carpenter, Chair
Teruko Mitamura, Co-Chair
Chris Manning
Carl Pollard
Rich Thomason

2. Overview Introduction and Thesis Statement Linguistic Theories
Guarded Descriptions and Guarded Rules
Implementation and Evaluation
Conclusions
7/8/2019

3. Feature Structures Feature-value pairs encode information
Framework for linguistic theories (FUG, LFG, HPSG…)
Widely used in NLP (frame grammars, interlinguas…)
Can use unification to combine them.
7/8/2019

4. Example: “The dog runs.”
7/8/2019

5. German determiners have features
nom
acc
dat
gen
masc
der
den
dem
des
fem
die
neut
das
plur
den is masculine accusative or dative plural.
den Mann is masculine accusative
den Männern is dative plural
7/8/2019

6. Constraints on German Determiners
Constraints narrow down set of answers, and hence the search space, for a problem. e.g. X < 5, X > 2, then X = {3,4}
Computational linguists can use constraints to narrow down an analysis for some piece of language.
Noun constrains a determiner in German.
Specifically, we can describe the information we get, such as “case=accusative”
7/8/2019

7. Underspecification
When more than one answer is still possible, a feature is considered underspecified.
The verb constrains sheep in these cases.
The sheep bleats.
The sheep graze.
The number feature is considered underspecified.
Types can also be underspecified, in a typed system.
7/8/2019

8. Typed Feature Structures
This feature structure is underspecified for number.
This feature structure has number specified.
7/8/2019

9. Implementing Feature-based Linguistic Theories
Linguists can use underspecification to great advantage. The benefit is generality.
Sometimes, an underspecified portion of a feature structure is shared.
Our goal in implementation is to maintain the elegance of the original theories, and parse with them.
7/8/2019

10. Specific Problems for NLP
Increased local ambiguity if lexicon contains a priori an entry for every possibility.
e.g. extracted complements for a sentence that is not gapped; every combination of binder + bound arguments.
Process may not terminate if arguments are underspecified.
e.g. applying a lexical rule over an uninstantiated list of complements
Constraints may be prematurely satisfied.
Vacuously true in absence of contradiction
7/8/2019

11. Solution: Guarded Constraints
Wait until we have more information, then process the underspecified part of the feature structure.
For example, describe what we want to know from the verb, in the case of the sheep bleats, before processing sheep
Waiting is also called delaying. Constraints that are delayed are called guarded constraints.
7/8/2019

12. Thesis Statement
Delays for processing a linguistic theory can be generalized with a specification for guarded constraints as descriptions of typed feature structures. My work enables the successful parsing of modern grammars as written, even with highly lexicalized constraints.
7/8/2019

13. Contributions of the Thesis
Guarded descriptions added to the theory of typed feature structures.
Extension to theory of Carpenter (1992)
Demonstrate theory in a logic programming system.
Extended ALE system of Carpenter & Penn (1994)
Identification of various linguistic phenomena which can be implemented using guarding.
7/8/2019

14. Linguistic Theories Linguistic Theories
Introduction and Thesis Statement
Linguistic Theories
Guarded Descriptions and Guarded Rules
Implementation and Evaluation
Conclusions
7/8/2019

15. Cross-language Phenomena as a Basis for Guarding
Argument Raising in German, French and Italian (Johnson 1986, Hinrichs & Nakazawa 1989, Monachesi 1993, Abeille & Godard 1994, Baker 1999)
Romance Clitics (Miller 1992, Monachesi 1993, Sag & Godard 1993, Abeille et al. 1995)
Morphology
German Passive (Kathol 1994, Pollard 1994)
Causative
Chicheŵa (Alsina 1992)
Japanese (Manning et al. 1999)
7/8/2019

16. Cross-linguistic Phenomena
Complement Extraction (Pollard & Sag 1994)
Quantifier Raising (Pollard & Yoo 1997)
Ordering Constraints (Erbach et al. 1995, Penn 1999)
7/8/2019

17. A Common Thread: Argument Sharing
Two heads have a common argument or arguments.
Causative: causee linked to both causative and embedded predicate
Verb and auxiliary share syntactic arguments in argument raising
Quantifier store of verb phrase derived from quantifiers of its arguments
During processing, one head might need to gain information from the other.
7/8/2019

18. Example: Argument Raising by Auxiliary
Wird Sandy Kim sehen?
Will Sandy Kim see
“Will Sandy see Kim?”
Arguments of the verb sehen are the subject Sandy and the object Kim.
Arguments of the auxiliary verb wird are Sandy, Kim, and the verb sehen.
7/8/2019

19. An auxiliary’s arguments are underspecified
Wird Sandy Kim sehen?
7/8/2019

20. Lexicalization of Constraints
Concurrent trend towards lexicalization of constraints, such as binding theory, quantifier scoping, and complement extraction.
e.g. for a verb, “Apply a binding constraint to my argument structure, however that is filled in.”
7/8/2019

21. Argument Raising + Complement Extraction
Kim wird Sandy sehen.
Kim will Sandy see
“Sandy will see Kim.”
The object Kim has been fronted (extracted).
A lexical rule for complement extraction says, “If I have a lexical entry with an object on the complements list of a verb, then I also have an entry with that object in the fronted position.”
7/8/2019

22. Lexically specified Complement Extraction (topicalization, or fronting) (Pollard & Sag, 1994)
Kim wird Sandy sehen.
7/8/2019

23. Raising by Auxiliary + Complement Extraction = Processing Problems
We know nothing about the verb. Thus, it is difficult to extract arguments from the auxiliary’s complements list.
If we know the verb is sehen, then we know there is a direct object.
Solution: Wait until we know what the verbs arguments are to process this lexically specified constraint.
7/8/2019

24. Guarded Descriptions and Guarded Rules
Introduction and Thesis Statement
Linguistic Theories
Guarded Descriptions and Guarded Rules
Implementation and Evaluation
Conclusions
7/8/2019

25. Feature Structure Descriptions
“third person singular”
The description is:
I used feature structure descriptions to guard a portion of a feature structure.
7/8/2019

26. Typed Feature Structure Descriptions
Descriptions are a shorthand for picking out a particular feature structure. (Rounds & Kasper 1986). Descriptions are relative to a particular type.
7/8/2019

27. Guarded Descriptions: Extending the Description Language
If the feature structure satisfies φ, then add the information in the description ψ. Else, if it doesn’t, add the description υ.
Example: verb are
NUMBER:plural  true; PERSON:second
7/8/2019

28. Satisfaction Conditions for Guarded Descriptions
If a feature structure satisfies a description φ, then it also satisfies ψ; else it satisfies υ.
7/8/2019

29. Negation
Negation is defined as failure if every extension of a feature structure satisfies a description, else true. Definition follows Moshier (1988).
7/8/2019

30. Inequations An instance of negation.
Compile the description that contains an inequation as in instance of guarding (transparent to the user).
7/8/2019

31. Example: Binding Theory
For non-reflexives, if the nominal objects are co-indexed, then fail, else true.
Johni likes himselfi
*Johni likes Johni.
Johni likes Johnj.
(Subj,loc:cont:((npro;ppro),index:(SubjInd,=\=Ind)))|
bind(Subj,(Arg,loc:cont:index:Ind))
(=\=Ind) compiled as SubjInd==Indfail;true
7/8/2019

32. Guarded Rules (Constraint Logic Programming -- CLP)
A constraint can be associated with a rule. This affects the order of goal resolution (constraint must be satisfied first).
I will use descriptions of feature structures as the guards on rules which have feature structure arguments.
7/8/2019

33. Guarded Rules Equivalent to using guarded descriptions.
With negation defined, we can guard rules using the regular description language.
7/8/2019

34. Argument Raising with Guards
Kim wird Sandy sehen.
Kim will Sandy see.
“Kim will see Sandy.”
Put a guard on argument raising. Each member of the verb’s complement’s list must satisfy at least the type subst (substantive) before the arguments can be raised by auxiliary.
A substantive is noun, verb, etc.
7/8/2019

35. Argument Raising (cont.)
Guard on the COMPS list of the subcategorized verb.
Comps:loc:cat:head:subst | aux_raising(Aux,Comps,Subj)
7/8/2019

36. Add Complement Extraction
Sandy sehen wird Kim.
Sandy see will Kim
“Kim will see Sandy.”
Know the semantic relation of the embedded verb before arguments can be extracted from an auxiliary.
Prop:nucleus:relation |
lexical_rule(aux(comps:PVPComps) 
aux(slash:(PVP,cont:Prop),comps:PVPComps)
7/8/2019

37. Suspending Goals
7/8/2019

38. Implementation and Evaluation
Introduction and Thesis Statement
Linguistic Theories
Guarded Descriptions and Guarded Rules
Implementation and Evaluation
Conclusions
7/8/2019

39. Logic Grammar Systems Early systems ALE (Carpenter & Penn 1994)
FUG (Kay 1983)
PATR-II (Shieber et al. 1985)
LOGIN (Ait-Kaci and Nasr 1986)
ALE (Carpenter & Penn 1994)
ALEP (Erbach et al. 1995)
CUF (Dorre & Dorna 1993)
ConTroll (Gotz et al. 1997)
FUF (Elhadad & Robin 1992)
7/8/2019

40. Implementation Extension to ALE (Carpenter & Penn 1994) Grammars:
Allow guards on the type of a typed feature structure.
Grammars:
Japanese causative and binding theory (Manning et al. 1999). Inequations are compiled as waits on disproving isomorphism.
German partial verb phrase fronting (Baker 1999)
Lexical quantifier scoping (Pollard & Yoo 1997)
7/8/2019

41. Other Possible Strategies
Brute Force or naïve approach (multiply out possibilities, esp. in the lexicon)
Accommodation (setting limits not in the original theory; edit program or grammar)
e.g. limit the number of a verb’s complements.
Hand-threading the grammar
e.g. put lexical constraints in the phrase-structure rules.
7/8/2019

42. Results of Grammar Comparison
Guarding may reduce lexicon size substantially, especially with respect to the brute force approach.
Accommodation strategy may prune solutions.
Hand threaded grammar performs fastest, but is knowledge-intensive.
Other: Slowness of guarding due to blocking and unblocking of guards (up to 10x slower than the “ideal” case). Implementation could be improved.
7/8/2019

43. Raising by Auxiliary+PVP Fronting (parse times, in msec)
Sentence
Naïve
Unguarded
Guarded
Thread
Sandy sieht Kim.
Sandy wird Kim sehen können.
Sandy sehen wird Kim.
Sandy wird Kim sehen.
Sandy sehen können wird Kim.
Wird Kim gehen können?
Wird Kim sehen können?
Wird Kim gehen?
wird Kim können (VP)
Wird Kim Sandy sehen?
Wird Kim Sandy sehen können? 70
5590
1070
1240
3450
2620
4260
540
1860
1220
5850 60
4990
470
810
2010
2120
3990
1460
940
1230
8240
3620
350
710
1110
2210
3270
430
580
1570
8900
840
200
280
390
330
610
120
160
320
880
7/8/2019

44. Advantages of the Approach
Theoretical relevance
Faithful rendering of both the linguistic theory and the theory I develop for guarded feature structures. Enables complex theories to be implemented as written.
Algorithm is parser-independent
In a study of quantifier raising, the guards are unblocked in exactly the same order, with three different parsers.
7/8/2019

45. Conclusions Conclusions Introduction and Thesis Statement
Linguistic Theories
Guarded Descriptions and Guarded Rules
Implementation and Evaluation
Conclusions
7/8/2019

46. Conclusions First generalization of delays over feature structures.
Flexible; guards can be applied to lexical rules, grammar rules, type constraints, etc.
Guarding is relevant for a wide range of linguistic phenomena and also across languages.
7/8/2019

47. Future Work
Speedier implementation so that guards are not unblocked every time the type variable changes.
Transfer the approach to non-typed formalisms (e.g. LFG).
7/8/2019

48. Extra slides begin here
7/8/2019

49. Example (CLP)
A program to append two lists together will go into an unbounded search if the first or third argument is a variable. Wait until either of these is nonvariable.
(nonvar(Xs);nonvar(Zs)) | append([X|Xs],Ys,[X|Zs])  append(Xs,Ys,Zs)
7/8/2019

50. Guarding on Typed Feature Structures
In implementing delays, we put a guard on the type of the information we are looking for.
We never delay any goals that could have been solved with a less specific type.
In asking whether a feature structure F satisfies a feature-value pair Feat:Desc, a wait is placed on the type of F if it does not already satisfy a type for which Feat is appropriate. Recursively, a wait is placed on the feature structure rooted at Feat with respect to Desc.
7/8/2019