1. Robot Metaphors and Models

2. Animatronic “Robot” or device brain effectors

3. Perceiving “Robot” brain sensors

4. Reactive Robot is the simplest behavioral robot Brain is a mapping sensors This is the simplest robot that satisfies the definition of a robot effectors

5. Reactive Robot in environment brain sensors This is the simplest robot that satisfies the definition of a robot effectors ENVIRONMENT is a feedback

6. Braitenberg Vehicles and Quantum Automata Robots

7. Another Example: Braitenberg Vehicles and Quantum BV

8. Braitenberg Vehicles

9. Emotional Robot has a simple form of memory or state Brain is a Finite State Machine sensors This is the simplest robot that satisfies the definition of a robot effectors

10. Behavior as an interpretation of a string Newton, Einstein and Bohr. Hello Professor Hello Sir Turn Left. Turn right. behavior

11. Behavior as an interpretation of a tree Newton, Einstein and Bohr. Hello Professor Hello Sir Turn Left. Turn right. behavior Grammar. Derivation. Alphabets.

12. Our Base Model and Designs

13. Neck and upper body movement generation

14. Robot Head Construction, 1999 Furby head with new control Jonas Jonas We built and animated various kinds of humanoid heads with from 4 to 20 DOF, looking for comical and entertaining values. High school summer camps, hobby roboticists, undergraduates

15. Mister Butcher 4 degree of freedom neck Latex skin from Hollywood

16. Robot Head Construction, 2000 Skeleton Alien We use inexpensive servos from Hitec and Futaba, plastic, playwood and aluminum. The robots are either PC-interfaced, use simple micro-controllers such as Basic Stamp, or are radio controlled from a PC or by the user.

17. Adam Marvin the Crazy Robot Technical Construction, 2001 Details

18. Virginia Woolf heads equipped with microphones, USB cameras, sonars and CDS light sensors 2001

19. Max Image processing and pattern recognition uses software developed at PSU, CMU and Intel (public domain software available on WWW). Software is in Visual C++, Visual Basic, Lisp and Prolog. BUG (Big Ugly Robot) 2002

20. Visual Feedback and Learning based on Constructive Induction 2002 Uland Wong, 17 years old

21. Professor Perky 1 dollar latex skin from China We compared several commercial speech systems from Microsoft, Sensory and Fonix. Based on experiences in highly noisy environments and with a variety of speakers, we selected Fonix for both ASR and TTS for Professor Perky and Maria robots. We use microphone array from Andrea Electronics. Professor Perky with automated speech recognition (ASR) and text-to-speech (TTS) capabilities 2002, Japan

22. Maria, 2002/2003 20 DOF

23. Construction details of Maria location of controlling rods location of head servos location of remote servos Custom designed skin skull

24. Animation of eyes and eyelids

25. Cynthia, 2004, June

26. Currently the hands are not moveable. We have a separate hand design project.

27. Software/Hardware Architecture Network- 10 processors, ultimately 100 processors. Robotics Processors. ACS 16 Speech cards on Intel grant More cameras Tracking in all robots. Robotic languages – Alice and Cyc-like technologies.

28. Face detection localizes the person and is the first step for feature and face recognition. Acquiring information about the human: face detection and recognition, speech recognition and sensors.

29. Face features recognition and visualization.

30. Use of Multiple- Valued (five- valued) variables Smile, Mouth_Open and Eye_Brow_Raise for facial feature and face recognition.

31. HAHOE KAIST ROBOT THEATRE, KOREA, SUMMER 2004 Sonbi, the Confucian ScholarPaekchong, the bad butcher Czy znacie dobra sztuke dla teatru robotow?

32. Editing movements

33. Yangban the Aristocrat and Pune his concubine The Narrator

36. We base all our robots on inexpensive radio- controlled servo technology.

37. We are familiar with latex and polyester technologies for faces Martin Lukac and Jeff Allen wait for your help, whether you want to program, design behaviors, add muscles, improve vision, etc.

38. New Silicone Skins

39. A simplified diagram of software explaining the principle of using machine learning based on constructive induction to create new interaction modes of a human and a robot.

40. Probabilistic and Finite State Machines

41. Probabilistic State Machines to describe emotions Happy state Ironic state Unhappy state “you are beautiful” / ”Thanks for a compliment” “you are blonde!” / ”I am not an idiot” P=1 P=0.3 “you are blonde!” / Do you suggest I am an idiot?” P=0.7

42. Facial Behaviors of Maria Do I look like younger than twenty three? Maria asks:  “yes”  “no” 0.3 0.7 Response: Maria smiles Maria frowns

43. Probabilistic Grammars for performances Who? What? Where? Speak ”Professor Perky”, blinks eyes twice Speak “In the classroom”, shakes head P=0.1 Speak “Was drinking wine” P=0.1 P=0.3 P=0.5 Speak ”Professor Perky” Speak ”Doctor Lee” Speak “in some location”, smiles broadly Speak “Was singing and dancing” P=0.5 P=0.1 …. P=0.1

44. Human-controlled modes of dialog/interaction Robot asks Human teaches Human commandsHuman asks Robot performs “Hello Maria” “Thanks, I have a question” “Thanks, I have a lesson” “Thanks, I have a command” “Lesson finished” “Questioning finished” “Command finished” “Stop performance” “Question”

45. Dialog and Robot’s Knowledge

46. Robot-Receptionist Initiated Conversation Robot What can I do for you? Human Robot asks This represents operation mode

47. Robot-Receptionist Initiated Conversation Robot What can I do for you? Human I would like to order a table for two Robot asks

48. Robot-Receptionist Initiated Conversation Robot Smoking or non- smoking? Human Robot asks

49. Robot-Receptionist Initiated Conversation Robot Smoking or non- smoking? Human I do not understand Robot asks

50. Robot-Receptionist Initiated Conversation Robot Do you want a table in a smoking or non-smoking section of the restaurant? Non-smoking section is near the terrace. Human Robot asks

51. Robot-Receptionist Initiated Conversation Robot Do you want a table in a smoking or non-smoking section of the restaurant? Non-smoking section is near the terrace. Human A table near the terrace, please Robot asks

52. Human-Initiated Conversation Robot Human Hello Maria Robot asks initialization

53. Human-Initiated Conversation Robot Human Hello Maria What can I do for you? Robot asks

54. Human-Asking Robot Human Question Human asks Question Robot asks

55. Human-Asking Robot Human Question Human asks Yes, you ask a question.

56. Human-Asking Robot Human What book wrote Lee? Human asks Yes, you ask a question.

57. Human-Asking Robot Human What book wrote Lee? Human asks I have no sure information.

58. Human-Asking Robot Human Try to guess. Human asks I have no sure information.

59. Human-Asking Robot Human Try to guess. Human asks Lee wrote book “Flowers”.

60. Human-Asking Robot Human This is not true. Human asks Lee wrote book “Flowers”.

61. Human-Teaching Robot Human Questioning finished Human teaches “Questioning finished” Robot asks Human asks Thanks, I have a lesson Human ends questioning

62. Human-Teaching Robot Human Questioning finished Human teaches “Questioning finished” Robot asks Human asks Thanks, I have a lesson Robot enters asking mode What can I do for you?

63. Human-Teaching Robot Human Thanks, I have a lesson Human teaches “Questioning finished” Robot asks Human asks Thanks, I have a lesson Human starts teaching What can I do for you?

64. Human-Teaching Robot Human Thanks, I have a lesson Yes Human teaches

65. Human-Teaching Robot Human I give you question- answer pattern Yes Human teaches

66. Human-Teaching Robot Human Question pattern: What book Smith wrote? Yes Human teaches

67. Robot Human Answer pattern: Smith wrote book “Automata Theory” Yes Human teaches Human-Teaching

68. Human-Teaching Robot Human Checking question: What book wrote Smith? Yes Human teaches

69. Human-Teaching Robot Human Checking question: What book wrote Smith? Smith wrote book “Automata Theory” Human teaches

70. Human-Teaching Robot Human I give you question- answer pattern Yes Human teaches

71. Human-Teaching Robot Human Question pattern: Where is room of Lee? Yes Human teaches

72. Human-Teaching Robot Human Answer pattern: Lee is in room 332 Yes Human teaches

73. Human-Checking what robot learned Robot Human Lesson finished Human asks Question Robot asks Human teaches “Lesson finished”

74. Human-Checking what robot learned Robot Human Lesson finished Human asks Question Robot asks Human teaches “Lesson finished” What can I do for you?

75. Human-Checking what robot learned Robot Human Question Human asks Question Robot asks Human teaches “Lesson finished” What can I do for you?

76. Human-Asking Robot Human Question Human asks Question Robot asks Human teaches “Lesson finished” Yes, you ask a question.

77. Human-Asking Robot Human What book wrote Lee? Human asks Yes, you ask a question.

78. Human-Asking Robot Human What book wrote Lee? Human asks I have no sure information.

79. Human-Asking Robot Human Try to guess. Human asks I have no sure information.

80. Human-Asking Robot Human Try to guess. Human asks Lee wrote book “Automata Theory” Observe that robot found similarity between Smith and Lee and generalized (incorrectly)

81. Behavior, Dialog and Learning The dialog/behavior has the following components: –(1) Eliza-like natural language dialogs based on pattern matching and limited parsing. Commercial products like Memoni, Dog.Com, Heart, Alice, and Doctor all use this technology, very successfully – for instance Alice program won the 2001 Turing competition. –This is a “conversational” part of the robot brain, based on pattern-matching, parsing and black-board principles. –It is also a kind of “operating system” of the robot, which supervises other subroutines.

82. (2) Subroutines with logical data base and natural language parsing (CHAT). –This is the logical part of the brain used to find connections between places, timings and all kind of logical and relational reasonings, such as answering questions about Japanese geography. Behavior, Dialog and Learning

83. (3) Use of generalization and analogy in dialog on many levels. –Random and intentional linking of spoken language, sound effects and facial gestures. –Use of Constructive Induction approach to help generalization, analogy reasoning and probabilistic generations in verbal and non-verbal dialog, like learning when to smile or turn the head off the partner. Behavior, Dialog and Learning

84. (4) Model of the robot, model of the user, scenario of the situation, history of the dialog, all used in the conversation. (5) Use of word spotting in speech recognition rather than single word or continuous speech recognition. (6) Continuous speech recognition (Microsoft) (7) Avoidance of “I do not know”, “I do not understand” answers from the robot. –Our robot will have always something to say, in the worst case, over-generalized, with not valid analogies or even nonsensical and random. Behavior, Dialog and Learning