1. Representation of Actions as an Interlingua
Karin Kipper & Martha Palmer
Presented in CIS630 by Sriram Venkatapathy

2. Framework Command Animation In Performed by Natural Language
a Human Agent
Command In
Natural Language
(“open a door”)

3. Primary Goal Command Animation In Performed by Natural Language
a Human Agent
Command In
Natural Language
(“open a door”)
Bridge the Gap

4. Parametrized Action Representation (PAR)
Used to bridge the gap between the command and the animation performed by the virtual human agent.
PARs make explicit many details that are underspecified in the human language.

5. PARs as an Interlingua
This detailed representation is well suited for an interlingua for MT applications because
Animations of Actions – and therefore the PARs that control them – will be equivalent for the same actions described in different languages.

6. PARs as an interlingua – contd.
Spanish
instruction
English
instruction
PARs
Animation

7. PAR Action’s participants (agent and objects)
Kinematic Properties like its path, manner and duration.

8. PAR – contd.
Traditional state-space properties of actions such as applicability conditions and preparatory actions.
Termination and Post assertions which determine when an action is concluded.

9. PAR ( for the action “contact”)

10. Hierarchy of actions
Verbs can be represented in a lattice that allows semantically similar verbs, such as motion verbs or verbs of contact, to be closely associated.
A common parent captures the properties that these verbs all share.

11. Verbs related to “contact”

12. Representation of “hammer”
Inherits PAR of hit , and ultimately the PAR
for contact.
with “forceful” manner.
and “hammer” as an instrument.

13. PAR – contd.
PAR is intended to provide slots for information that is typically conveyed in modifiers or adjuncts.
“John hit the ball”
“John hit the ball with a bat”
“John swung mightily and his bat hit the ball with a resounding sound”.
All map to the same PAR schema.

14. Generating animations

15. Deriving the PAR schema
Synchronous Tree-adjoining grammar is used
for parsing natural language instructions and obtaining the predicate-argument dependencies.
Assemble the corresponding schemas, and fill in the participants and modifiers to output the PAR schema.

16. Planner
These schemas might be underspecified for actions like “enter”, “put” and hence don’t have enough information for the animation.
So, a planner is used to
Select the way (activity) “enter by walking/swimming”
Determine preparatory actions “open the door”
Decompose complex actions “put the glass on the table”

17. PAR as Interlingua PAR representation is a general template.
It includes properties of the action that can
occur linguistically either as the main verb or as adjuncts to the main verb phrase.
Captures divergences such as, for verb-framed versus satellite-framed languages.

18. Verb-Framed vs Satellite-Framed
Satellite-Framed (English)
Manner -> Main verb.
Motion (path or path + ground location) -> Satellite
Ex. The bottle floated out.
Verb-Framed (Spanish)
Motion -> Main verb
Manner -> Satellite
Ex. La bottella salio flotanda.
(the bottle exited floating)

19. Verb-Framed vs Satellite-Framed
For the sentences,
The bottle floated into the cave. (English)
La bottella entro flotanda a la cueva (Spanish)
(The bottled entered floating the cave)

20. Verb-Framed vs Satellite-Framed
The PAR schemes don’t distinguish the representation for these sentences, because there is a single schema that includes both the manner and the path without specifying how they are realised linguistically.
EN float/[par:motion,activity:float]
into/[term:in(AG,OBJ]
SP entrar/[par:motion,term:in(AG,OBJ)
flotar/[activity:float]

21. Schema for the above sentences

22. Comparison with other work
Considerably different from the approach
outlined in Palmer et al. (1998) which discusses
the use of Feature-based Tree-adjoining
grammars.
It was a transfer-based mechanism expressed
in Synchronous TAG to capture divergences in
VFL and SFL through the use of semantic
features and links between the grammars.

23. Comparison with other work
Similar to Lexical Conceptual Structures (LCS)
approach.
LCS allows the separation of the concepts of
motion, direction, and manner of motion in the
sentence “John swam across the lake”.
(represented as GO,PATH,MANNER).
This approach allows for a similar representation
and the end result is the same.

24. Conclusions This work discusses a parameterized
representation of actions grounded by needs
of animations in a simulated environment.
Generalizations based on action classes
provide the basis for an interlingua approach
that captures the semantics without committing
to any language-dependent specification.
The PAR schema incorporates in a single
template both VFL and SFL languages.

25. Thank you.