1. Project-Options 1.Fridge 2.Workbench with Toolbox 3.Playschool

2. Fridge-Scenario  scenario is a 3-dimensional fridge  fridge contains shelves  shelves are 2-dim vertically arranged  each shelve is 2-dim horizontally  shelves have objects in 2-dim space  constraints on seeing objects (model "viewers perspective")  constraints on moving / getting / placing objects

3. Workbench-Scenario  scenario is a 2.5-dimensional workbench and a toolbox  items / objects in the scenario are tools  tools can be on workspace or in toolbox  tools on workspace can be under / on top of each other  tools in toolbox cannot be seen  inferences on where objects are (toolbox)  constraints on moving / getting / putting objects

4. Tools http://www.homedepot.ca/escalate/store/CategoryPage?pls=hd_canada_gift&bc=hd_canad a_gift&ctgpage=category&cc=Hand+Tools&clist=018980323807&ret=Hand+Tools&start Num=0&rangeNum=10

5. More Tools

6. Playschool-Scenario  scenario is a 2-dimensional ground  items / objects in the scenario are toys  toys have various arrangements on ground  toys can be moved around, removed from, and added to the scene  all toys can be seen (unless they are not in the scene; no hidden objects)  model spatial relations between toys based on absolute position on ground

7. Project Parts Description of scenario –Hierarchy of concepts with inheritance –Features (relations or functions) for each concept, e.g. size, colour, other physical characteristics related to spatial domain –Spatial relations between objects –Corresponds to T-Box

8. Project Parts Reasoning in scenario –based on characteristics and relations –draw conclusions about objects and features –e.g. use transitivity of relations, like besides –e.g. model occlusion (i.e. if a large object is in front of a small object, the small object cannot be seen) –modeled in A-Box –use ask and tell / assert and retrieve to access and store/modify values

9. Input Sentence Modeling  Select sentence types.  For each sentence type (question, command,... ?) determine prototypical frame for representing the input.  Subclassification of these frames might be suitable (e.g. different question forms).  Define procedures to fill these frames with values from the actual input sentence.  Use terminology / ontology (semantic basis).

10. Queries  Define frame for each question type.  Fill frame for specific NL input (question) using terminology / ontology from T-Box (mapping to synonyms; semantic basis; T-Box concepts).  Formulate retrieval operations for different query frames (may involve reasoning!).  Evaluate:  retrieve values from A-Box according to query specification.  Formulate answer (use templates, if feasible, add number/person agreement for NP - VP).

11. Commands  Define frames for different command types.  Connect frames to action-descriptions.  Fill frame for specific NL input (command sentence) using terminology from T-Box and action-frames.  Evaluate instantiated action-frame (see next slide).  Formulate answer (Either confirm "DONE" or give reason why it didn't work.).

12. Acting  Define frame for each prototypical action, based on verb semantics.  Fill frame for specific NL input (command).  Include preconditions and effects in frame, or associate action description with respective formulae.  Evaluate:  check satisfaction of precondition in A-Box.  make assertion to A-Box according to effect description.