1. Intelligent Robotics Jeremy Wyatt Thanks to: Nick Hawes, Aaron Sloman, Michael Zillich, Somboon Hongeng, Marek Kopicki

2. The Whole Iguana  AI commonly studies aspects of intelligence separately: narrow domain high performance  In 1976, philosopher Dan Dennett suggested putting it all together, but with a low level of performance  In fact people had been trying to build integrated systems for some twenty years by then

3. Shakey the robot  1970 - Shakey the robot reasons about its blocks Built at Stanford Research Institute, Shakey was remote controlled by a large computer. It hosted a clever reasoning program fed very selective spatial data, derived from weak edge-based processing of camera and laser range measurements. On a very good day it could formulate and execute, over a period of hours, plans involving moving from place to place and pushing blocks to achieve a goal. (Hans Moravec)

4. Shakey: key ingredients  World model used logical representations type(r1,room) in(shakey,r1) in(o1,r2) type(d1 door) type(o1 object) type(f3 face) type(shakey) at(o1 15.1 21.0) joinsfaces(d2 f3 f4) joinsrooms(d2 r3 r2) … shakey 30 20 10 0 01020 r3 f4 f3 d2 d1 f2 f1 r1 r2 o1

5. Planning  Shakey used a form of planning called goal regression  Idea: find an action that directly achieves your goal, and then actions to achieve the first action’s preconditions, etc…  e.g. Blocked(d1,X)  Let’s see Shakey solve a problemsolve a problem block_door(D,Y) preconditions:in(shakey,X) & in(Y,X) & clear(D) & door(D) & object(Y) delete list:clear(D) add list:blocked(D,Y) shakey 30 20 10 0 01020 r3 f4 f3 d2 d1 f2 f1 r1 r2 o1

6. Lessons from nature  Gannets – wings half open to control dive  Fold wings to avoid damage  Not at a constant distance, but at a constant time  Birds have detectors that calculate time to impact

7. Lessons from nature  All naturally occuring intelligence is embodied  So robots are in some ways similar systems  Robots, like animals exploit their environments to solve specific tasks “There are no general purpose animals … why should there be general purpose robots?” David MacFarland

8. Behaviour Based Robots  Inspired by simpler creatures than humans  Throw away most representations  Throw away most reasoning  Build your robot out of task specific behaviours

9. Pushing the behaviour based envelope  Behaviour based systems can display quite sophisticated behaviour, particularly for interaction  But they don’t have understanding because they don’t haverepresentations

10. The age of data  In the 1990s people were finally beginning to have success with representation driven approaches  One key has been the use of probabilistic methods  These are data intensive and require very strong assumptions about the learning task  Stanley Stanley

11. Robots that understand  Internal structure to represent the meaning of the utterance e.g. “The orange ball” B1: phys-object ^ ball C1: colour ^ orange

12. Learning object appearances

13. Learning names and appearances of objects

14. Cognition requires attention  Object recognition is unreliable and expensive  We can use bottom up salience to make it more efficient

15. Salience can be modulated by language Directing processing of the visual scene

16. The Whole Iguana: coming full circle  Collection of loosely coupled sub- architectures  Each sub-architecture contains processing elements that update structures within a working memory  WM are typically only locally read & write (bar privileged sub- architectures)  Processing controlled by local and global goals and managers  Knowledge management by local and global ontologies Sensor Actuator Processor Working Memory Manager

17. Movie goes here Illustration: Cross Modal Ontology Learning Architectures

18. Linguistically Driven Manipulation Illustration: Language Driven Manipulation Architectures  Goals are raised by language  Reference to objects by learned features  Robot plans intentional actions using logical planner  Intention shifting is handled via execution monitoring and continual replanning Communicatio n SA Binding SA Communicatio n SA Visual SA Communication SA Spatiotemporal SA Communication SA Coordinator SA Communicatio n SA Planning SA Communicatio n SA Manipulation SA

19. ROI 1 SO 1 ROI 2 SO 2 Inst 1 Qual Spatial Relations Inst 2 Object locations Communication SA Binding SA Visual SA “Put the blue thing to the right of the red thing” Parse + Dialogue Interpretation Coordination SA Spatial SA Raise Planning Goal Goal LF Planning SA Object locations Qual Spatial Relations Object locations Qual Spatial Relations ROI 1 SO 1 Inst 1 ROI 2 SO 2 Inst 2 Manipulation SA MAPL GoalPlanPlan Step Vis Servo Manip Goal Executed Execution Check ROI 1 SO 1 ROI 2 SO 2 Raise Manip Goal Inst 1 Inst 2

20. Movie goes here Illustration: Language Driven Manipulation Architectures

21. Wrap up  Robotics gets to the heart of big issues in AI  There are enormous engineering and scientific challenges  There is a tension between different approaches: Representation heavy, top-down approaches to cognition Representation light, bottom approaches  The fun is in linking these