1. From Teleoperation to Autonomy
Define Intelligent Robot
Be able to describe at least two differences between AI and engineering approaches to robotics
Be able to describe the difference between telepresence and semi-autonomous control
Have some feel for the history and societal impact of robotics
History
-AI
-Engineering
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

2. Intelligent Robot Mechanical creature which can function autonomously
Mechanical= built, constructed
Creature= think of it as an entity with its own motivation, decision making processes
Function autonomously= can sense, act, maybe even reason; doesn’t just do the same thing over and over like automation
History
-AI
-Engineering
Teleop
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

3. What are Robots? Autonomous mechanical creatures
Capek 1921: R.U.R.
Intelligent because teleoperation doesn’t work, doesn’t scale
Physically situated, but now software agents or softbots
Principles from robotics influenced AI community, esp. planning
Combines programming, networks, operating systems, algorithms, … everything about CS into a system (the ultimate software engineering project)
History
-AI
-Engineering
Teleop
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

4. Robots Constantly in the Press
History
-AI
-Engineering
Teleop
Case Studies
Programming
Summary
Review
courtesy of MIT AI Lab
courtesy of Honda
Introduction to AI Robotics (MIT Press)
Chapter 1

5. Less Famous Cousins at WTC
Inuktun microTracks
½ iRobot PackBot
Introduction to AI Robotics (MIT Press)
Chapter 1

6. Why Robots? Dirty, Dangerous, Dull Tasks
JV2010, TRADOC, JFCOM, all branches even down to the organic level
Reconnaissance, MOUT, denial of area, consequence management, logistics, demining
History
-AI
-Engineering
Teleop
Case Studies
Programming
Summary
Review
Replace Humans with Robots
Introduction to AI Robotics (MIT Press)
Chapter 1

7. Why Robots? Better Than Bio
Robots at WTC…
voids smaller than person could enter
voids on fire or oxygen depleted
NBC Response
Lose ½ cognitive attention with each level of protection
Level A=12.5% of normal ability
Void:1’x2.5’x60’
History
-AI
-Engineering
Teleop
Case Studies
Programming
Summary
Review
Void on fire
Do Things that Living Things Can’t
Introduction to AI Robotics (MIT Press)
Chapter 1

8. Major Robot Modalities: UAV, UGV, UUV
Unmanned Aerial Vehicles
drones since Vietnam: Global Hawk, UCAV
easy: nothing to hit
hard: mission sensing, human-in-the-loop control
Unmanned Ground Vehicles
since 1967
easy: can always stop and think, a priori maps
hard: perceiving, e.g., light vegetation vs. wall
Unmanned Underwater Vehicles
ROVs since 1960s
easy: run tethers
hard: platform operation in unfriendly environment
History
-AI
-Engineering
Teleop
Case Studies
Programming
Summary
Review
Mobility (platform), Perception,
Communications +HRI, Control (Intelligence), Power
Introduction to AI Robotics (MIT Press)
Chapter 1

9. A Brief History… Introduction to AI Robotics (MIT Press) Chapter 1
There’s some nice segments on industrial automation on “Robots” the History Channel 1999 video.
Introduction to AI Robotics (MIT Press)
Chapter 1

10. Industrial Manipulators
History
-AI
-Engineering
Teleop
Case Studies
Programming
Summary
Review
“Tommy” type of robots: deaf, dumb, and blind
High precision, fast repetition
Usually no sensing of the environment
Welding can be off by an inch…
From The Who rock opera…
Introduction to AI Robotics (MIT Press)
Chapter 1

11. 3 Ways of Controlling a Robot
“RC-ing”
you control the robot
you can view the robot and it’s relationship to the environment
ex. radio controlled cars, bomb robots
operator isn’t removed from scene, not very safe
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

12. 3 Ways of Controlling a Robot
teleoperation
you control the robot
you can only view the environment through the robot’s eyes
don’t have to figure out AI
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

13. 3 Ways of Controlling a Robot
semi- or full autonomy
you might control the robot sometimes
you can only view the environment through the robot’s eyes
ex. Sojouner with different modes
human doesn’t have to do everything
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

14. Components of a Telesystem (after Uttal 89)
Local
display
Local control device
Communication
Remote
sensor
mobility
effector
power
Display
Control
Sensor
Mobility
Effector
Power
Communi-
cation
Local
Remote
Introduction to AI Robotics (MIT Press)
Chapter 1

15. Example Remote Local History Teleop -Motivation -Components -Problems
-Alternatives
Case Studies
Programming
Summary
Review
This is a picture of Dr. Robin Murphy operating Cowboy, one of the first iRobot Urbans developed for the DARPA Tactical Mobile Robot program. The scene is the SRDR training site in Miami, Florida, and the task is to explore a collapsed building for survivors (urban search and rescue).
Remote
Local
Introduction to AI Robotics (MIT Press)
Chapter 1

16. Typical Run History Teleop -Motivation -Components -Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

17. Problems That You Saw
no feedback, couldn’t really tell that the robot was stuck but finally got free
robot doesn’t have “proprioception” or internal sensing to tell you what the flippers were doing. No crunching noises, no pose widget to show the flippers
no localization, mapping-> no idea how far traveled
partial solution: better instrumentation (but can’t do dead reckoning well)
operator doesn’t have an external viewpoint to show itself relative to the environment
solution: two robots, one to spot the other
communications dropout, even though ~3 meters away
lighting conditions went from dark to very bright
hard for computer vision or human to adjust
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

18. But good for unmodeled events
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

19. Communications is Important: DarkStar+7 seconds=DarkSpot
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
7 second lag time
also, though this didn’t play a role in this case, there’s a lot of cognitive literature on how people react when having to change tasks– when they’re doing one thing and then get interrupted to do something else. Usually takes a while to context-switch (page in)
7 second communications lag (satellite relay)
“interruption” lag on part of operator
Introduction to AI Robotics (MIT Press)
Chapter 1

20. Predator: ~7:1 human to robot ratio
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Leo’s unofficial
Predator page
4 people to control it (52-56 weeks of training)
one for flying
two for instruments
one for landing/takeoff
plus maintenance, sensor processing and routing
lack of self-awareness– in Kosovo, come along side in helicopter and shoot down
Introduction to AI Robotics (MIT Press)
Chapter 1

21. Summary of Teleop Problems
cognitive fatigue
communications dropout
communications bandwidth
communications lag
too many people to run one robot (hidden cost)
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

22. Telesystems Best Suited For:
the tasks are unstructured and not repetitive
the task workspace cannot be engineered to permit the use of industrial manipulators
key portions of the task require dexterous manipulation, especially hand-eye coordination, but not continuously
key portions of the task require object recognition or situational awareness
the needs of the display technology do not exceed the limitations of the communication link (bandwidth, time delays)
the availability of trained personnel is not an issue
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

23. Teleop Improvements: Telepresence
improves human control, reduces simulator sickness and cognitive fatigue by providing sensory feedback to the point that teleoperator feels they are “present” in robot’s environment
increases demands on bandwidth
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

24. Teleop Improvements: Supervisory Control
Semi-autonomous
Supervisory Control
human is involved, but routine or “safe” portions of the task are handled autonomously by the robot
is really a type of mixed-initiative
Shared Control/ Guarded Control
human initiates action, interacts with remote by adding perceptual inputs or feedback, and interrupts execution as needed
robot may “protect” itself by not bumping into things
Traded Control
human initiates action, does not interact
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

25. Teleop Improvements: Mixed-Initiative
Levels of Initiative
do only what told to do (teleoperation)
recommend or augment (cognitive augmentation)
act and report
act on own and supervise itself (autonomy)
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

26. “No Hands Across America”
1994
CMU NavLab
Pittsburgh to San Diego
2897 miles total
2849 autonomously
Autonomous or Mixed-Initiative?
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

27. Mixed-Initiative Matching Game
Level of Initiative
teleoperation
cognitive augmentation
act and report
act on own
Application
janitorial robot
medical robot (telemedicine)
high flying surveillance drone
combat aerial vehicle
resupply system for bringing water to fire fighters
guard dog robot
“nurse” robot
History
Teleop
-Motivation
-Components
-Problems
-Alternatives
Case Studies
Programming
Summary
Review
Break into teams and have them spend 10 minutes discussing which goes where
Introduction to AI Robotics (MIT Press)
Chapter 1

28. Collaborative Teleoperation1 3
mpg: June 2, 2000 SRDR Miami Beach: view from Inuktun as it falls
mpg: June 2, 2000 SRDR Miami Beach: view from Inuktun from hoisted position 2
Urban is stuck, Inuktun can’t help from current perspective
Driven off 3rd floor
Hoisted to 2nd floor by tether
Has better view, changing configuration & rocking extend view
still: June 2, 2000 SRDR Miami Beach
Introduction to AI Robotics (MIT Press)
Chapter 1

29. 2000 AAAI Mobile Robot
History
Teleop
Case Studies
Programming
Summary
Review
2 robots helping each other reduced collision errors, sped up time navigating confined space, righting
Introduction to AI Robotics (MIT Press)
Chapter 1

30. Example: Mixed-Initiative & Collab. Teleop
9/2000 DARPA Tactical Mobile Robots demonstration
Robot used an intelligent assistant agent to look for signs of snipers hiding in urban rubble
motion
skin color
difference in color
thermal (IR camera)
Human navigated mother robot using viewpoint of 2nd robot (not in picture)
Once deposited the human moved the daughter robot, and either saw a sniper or was alerted by the agent
History
Teleop
Case Studies
Programming
Summary
Review
This is USF’s marsupial team– a customized RWI ATRV robot that can carry three Urban robots. The mother is being teleoperated using a second daughter off to the side (collaborative teleoperation) that isn’t in the picture.
Note: the idea of the robot doing the perception and the human doing the navigation is pretty much the opposite of conventional semi-autonomy thinking where people do the perception and the robot tries to figure out where to go next. USF’s work in USAR with fire rescue workers suggested that they felt comfortable joysticking the robot around but joysticking and looking through a regular camera and an IR camera simultaneously was too much. They voted for having the robot do the vision part, trying to detect people.
Introduction to AI Robotics (MIT Press)
Chapter 1

31. AI provides the “other stuff”
knowledge representation
understanding natural langugage
learning
planning and problem solving
inference
search
vision
History
Teleop
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

32. Example User Expectation of AI
Proposed Goal: 1:1 soldier:any robot, where 1 soldier is responsible for 1 or more active robots but does not have to pay continuous attention to them.
Young Frankenstein
4 specialists:
1 vehicle
1 specialist:
1 modality
1 specialist
1 soldier
n vehicles
UAVs as
theater
assets
MAVs as
organic
assets
Field recon-
figurable
UUVs
MAV-UGV
cooperative
monitoring
Flocks
of MAVs
Introduction to AI Robotics (MIT Press)
Chapter 1

33. More Reasonable Expections
Consolidation
agents with
“tactical”
autonomy,
toolkits
Vehicle success is
still based on human,
but robot is “in front”
History
Teleop
Case Studies
Programming
Summary
Review
Dedicated Autonomy Systems
Human intermittent
attention as team
coordinator, not
with individuals
Mass-produced
dedicated
agents
Cooperating
“pack” or “herd”
agents
Reconfigurable AutonomySystems
Human primary
responsibility as a
tool builder, expert
advisor. Peer-level
communication
Field-
reconfigurable
agents
Introduction to AI Robotics (MIT Press)
Chapter 1

34. Programming Notes
You always need telesystem or human intervention as a backup
at some point a human will need to take control
embed in your design
“Roboticists automate what is easy and leave the rest to the human”- Don Norman
The user interface is absolutely critical
User interface make up 60% of commercial code
Useful= is the program purpose useful?
usually given to designer via specifications and requirements
Usable= can a human use it efficiently?
designer must conduct usability studies
avoid “if I can use it, some one else will”
History
Teleop
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

35. Example of How an “Internal” Display Can Hurt
History
Teleop
Case Studies
Programming
Summary
Review
iRobot PackBot
video, FLIR, 2 way audio
gamer joystick plus laptop with video & audio
robot state: battery, comms, orientation, camera, encoders
was not used on rubble pile at WTC because it scared off rescuers: too complicated, too long to boot, too toy
now integrated with Land Warrior– used in Afghanistan
Introduction to AI Robotics (MIT Press)
Chapter 1

36. Summary Teleoperation arose a partial solution to autonomy
cognitive fatigue, high comms bandwidth, long delays, and many:one human to robot ratios
Telepresence tries to reduce cognitive fatigue through enhanced immersive environments
Semi-autonomy tries to reduce fatigue, bandwidth by delegating portions of the task to robot
mixed-initiative
Teleop isn’t simple and improvements aren’t just “better user interfaces”
History
Teleop
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1

37. Review Questions What is an intelligent robot?
What is the difference between engineering and AI robotics?
What are 3 types of control?
What are the parts of a telesystem?
What are problems with teleoperation?
What’s the difference between telepresence and semi-autonomous control?
What are the levels of initiative (mixed-initiative)?
What are alternatives to traditional teleoperation?
History
Teleop
Case Studies
Programming
Summary
Review
Introduction to AI Robotics (MIT Press)
Chapter 1