1. The Robot of my dreams Marek Perkowski

2. A fairy tale with many answers and seven questions to the public.

3. The robot of my dreams is a theatre!!

4. Turing test Turing test has been treated as a test of intelligence, consciousness and “human- likeness” by several serious researchers. Turing test is very demanding and it is possible that it will be not passed in hundreds of years. Turing test is an “external test”. – Agent A is equal to Agent B if they behave externally the same way. – This test allows therefore “cheating”. As I am 64, Turing test is thus not a good idea for my personal research in robotics. I believe that simpler Turing-like tests should be created. Turing-like tests for narrower topics have been already created: – Chess (done), – Natural language (Loebner Price), – Mathematics (Automatic Theorem Proving), – Soccer (year 2050 a team of 11 robots is supposed to win with top professional human team).

5. Partial results of our pool 1.People seem to believe that it is easier to create a convincing robot actor than an intelligent robot. 2.Intelligent robot is less than an “artificial human” characterized by free will, consciousness, morality and soul. 3.It would be difficult to build a robot with “free will” or “consciousness” if it were not intelligent.

6. The robot of my dreams complete robot theatre. The robot of my dreams is a complete robot theatre. An actor in a standard theatre pretends. The play is not real life. It is about impression not reality. Simulating impressions is easier than simulating realities.

7. The robot of my dreams robot theatre is not a test of intelligence. The robot theatre is not a test of intelligence. – It is a test of emotions, esthetics and believeability.

8. The robot of my dreams – Turing–like tests Test 1 (relatively easy). Test 1 (relatively easy). A robot puppet is controlled by a human or by a computer. (Wilkowski) – Can one distinguish one from another? Test 2 (difficult). Test 2 (difficult). A theatre of human-like and other robots. The human-created scripted performance is compared with an improvisational interactive performance created by a computer. – Can we distinguish one from another? Test 3 (very difficult). Test 3 (very difficult). A realistic android robot plays in a play with humans. – Can we distinguish one from another? We work on Test 1 and Test 2. Can you propose other tests for robot theatre?

9. First question Can you propose other Turing-like tests for robot theatre?

10. The mechanical design concept Complete automated system of: – robots, – controlled cameras, – controlled furniture, smoke machines, fountains, – curtains, – lights and sounds. More than in standard theatre. Controlled by a centralized or distributed computer system. Actors are physical robots with replaceable standard components. – I could take their heads off. – I could take their hands off. – I want to create Lego-like system of components to build robots: Lego NXT, Tetrix, Lynxmotion, etc. – Connected to internet to acquire knowledge necessary for conversation and behavior – Use GSP, cameras, gyros, accelerometers and other sophisticated sensors for information acquisition.

11. Robot Design The system will be based on components. From inexpensive to expensive. – A cheap hand for waving hello – An expensive hand to grab items. The robots in the theatre will be seen by a camera and transmitted to world through Internet. People from outside will be able to control one or more robots and connect the robots to their autonomous or semi-autonomous software. Various variants of simplified Turing tests will be designed. No complicated wiring. Just snap-in design with connectors. – Immediate replacement of a broken hand.

13. Steve, Professor Perky and Normen at Intel's show.

14. The Hahoe Robot Theatre

15. Sonbi – the Confucian Scholar

16. Paekchong – the bad butcher

17. The interpreter robot without face skin

18. The same robot 2004, June

19. 2009 “What’s That? Schr ö dinger Cat” a debate between Einstein and Schroedinger Cat about quantum mechanics – an educational theatre.

20. Schroedinger Cat Dr. Niels Bohr Physics Debate Theatre of medium robots Professor Albert Einstein

21. Improvisational Theatre “What’s That? Schr ö dinger Cat” Professor Einstein Schr ö dinger Cat

22. 2010 - OUR RECENT BIPED THEATRE KHR-1iSOBOT Lynxmotion Systems Parts of Hahoe shown in 2010 and 2011

23. Walking biped robot can express the fullness of human emotions: – body gestures, – dancing, – jumping, – gesticulating with hands. Emotions can be: – Emergent - Arushi – Programmed – Martin Lukac ISMVL – Mimicked – ULSI – Learned – Martin Lukac Reed-Muller Fighting KHR-1 robots

24. iSOBOT robot

25. The stage of Portland Cyber Theatre in FAB building

26. Second question Can you propose other types of robots and devices that can create an interesting theatre?

27. Theory of Robot Theatre? 1.Motion Theory: – Motions with symbolic values 2.Theory of sign – Creation of scripts, generalized events, motions to carry meaning 3.Robot theories that may be used: 1.Machine Learning 2.Robot Vision 3.Sensor Integration 4.Motion: kinematics, inverse kinematics, dynamics 5.Group dynamics 6.Developmental robots

28. Types of robot theatre

29. Realizations of Robot Theatres Animatronic “Canned” Robot theatre of humanoid robots – Disneyworld, Disneyland, Pizza Theatre Theatre of mobile robots with some improvisation – Ullanta 2000 Theatre of mobile robots and humans – Hedda Gabler, Broadway, 2008 – Phantom in Opera, 2008 – Switzerland 2009

30. Animatronic Theatre Actors: robots Directors: none Public: no feedback Action: fixed Example: Disney World

31. Interaction Theatre Actors: robots Directors: none Public: feedback Action: not fixed Example: Hahoe

32. Input text from keyboard Face Detection and Tracking Face Recognition Facial Emotion Recognition Hand gesture recognition Behavior Machine Perception Machines Motion Machines Machines Output text i Output speech i Behavior Learning Architecture for Interaction Theatre Speech recognition Sonar, infrared, touch and other sensors Output robot motion i Output lights i Output special effects i Output sounds i Robot architecture is a system of three machines: motion machine, perception machine and brain machine

33. Improvisational Theatre Actors: robots Directors: humans Public: no feedback Action: not fixed Example: Schr ö dinger Cat

34. Motions of Einstein Motion e1 Improvisational Theatre “What’s That? Schr ö dinger Cat” SiddharArushi Professor Einstein Motion e2 Motion en Motions of Schr ö dinger Cat Motion c1 Motion cm Schr ö dinger Cat

35. Theatre of Robots and Actors (contemporary) Actors: robots Actors: humans Directors: humans Public: traditional feedback, works only for human actors Action: basically fixed, as in standard theatre

36. Theatre of Robots and Actors (future) Actors: robots Actors: humans + universal editors Directors: humans + universal editors : traditional feedback, like clapping, hecking, works for both robot and human actors Public: traditional feedback, like clapping, hecking, works for both robot and human actors : improvisational, as in standard improvisational theatre Action: improvisational, as in standard improvisational theatre

37. Third question Can you propose other types of robot theatre?

38. Motion Machine

39. Robot controller Canned code Robot controller Motion language Editor motion

40. Robot controller Motion language Editor motion Motion Capture Inverse Kinematics Forward Kinematics 1.A very sophisticated system can be used to create motion but all events are designed off-line. 2.Some small feedback, internal and external, can be used, for instance to avoid robots bumping to one another, but the robots generally follow the canned script. Evolutionary Algorithms

41. Robots controller Events language Universal Event Editor events Motion Capture Inverse Kinematics Forward Kinematics Lighting System Sound System Curtain and all equipment script Universal Event Editor Initial events

42. Universal Editors for Robot Theatre

43. Perception Editor Examples – input output pairs cameras Neural Nets Principal Component Analysis Various Feature Extracting Methods Constructive Induction Clustering Speech input Sensors Universal Perception Editor

44. Robot controller Robot controller Critic Feedback from the environment The environment includes: 1.Other robots 2.Human actors 3.Audience 4.The director

45. Mouth Motion text Hexapod walking Distance evaluation Biped walking Number of falls evaluation Biped Gestures Comparison to video evaluation Hand gestures Subjective human evaluation Learning problems in Human-Robot Interaction – Motion Generation problems Motion Problems = examples of correct motions – generalize and modify, interpolate Motion

46. The concept of generalized motions and universal event editor to edit: – robot motions, – behaviors, – lightings and automated events

47. Languages to describe all kinds of motions and events Labanotation DAP (Disney Animation Principles) and CRL (Common Robot Language)

48. KHR-1 and iSobot Motion Editor Interface

49. Editor with integrated video, text to speech and probabilistic regular expressions editing

51. Chameleon box converts sound to light and controlsUniversal motion editor MIDI Lights and controlled events

52. Generating Emotional Motions

53. Spectral filtering Matched filters Hermite interpolation Spline Interpolation Wavelets Repetitions Mirrors

55. Editor of wwaveforms Editor of wwaveforms

56. Theory of Event Expressions Tool to design motions directly from symbols. This theory is general enough to allow arbitrary motion to be symbolically described but is also detailed enough to allow the designer or the robot to precise the generated behavior to the most fundamental details. Our main concept is that the motion is a sequence of symbols, each symbol corresponding to an elementary action such as shaking head for answering “yes”. We will call them primitive motions. The complex motions are created by combining primitive motions.

57. Greeting_1 = (Wave_Hand_Up o Wave_Hand_Down ) (Wave_Hand_Up o Wave_Hand_Down ) *  Wave_Hand_Up o Say_Hello Which means, to greet a person the robot should execute one of two actions: – Action 1: wave hand up, follow it by waving hand down. Execute it at least once. – Action 2: Wave hand up, next say “Hello”. The same is true for any complex events. As we see, the semantics of regular expressions is used here, with atomic symbols from the terminal alphabet of basic events {Wave_Hand_Down, Wave_Hand_Up, Say_Hello}. The operators used here are: concatenation ( o ), union (  ) and iteration ( * ). Each operator has one or two arguments. So far, these expressions are the same as regular expressions. Initial state Final state Wave_Hand_Up Say_Hello Wave_Hand_Up Wave_Hand_Down  Wave_Hand_Up Wave_Hand_Down

58. Acceptor, generator and transformer Observe that this graph can be interpreted as an acceptor, when symbols Xi are inputs. It can be interpreted as a generator when symbols Xi are outputs. The graph can be thus used to recognize if some motion belongs to some language and can generate a motion belonging to the language. This graph is realized in software

59. Dance, rituality and regularity Dances of groups of robots are already very popular In most cases all robots do the same, or there are few groups of robots programmed identically. It would be interesting to investigate some recursive and iterative patterns, similar to behaviors of flocks of birds and of bees in which emergent behavior can adaptively change one form of regularity to another form of regularity.

60. Dance, rituality and regularity …….change one form of regularity to another form of regularity…….

61. A need for a better actuator

62. Problems with motion You can have small biped humanoid with no facial gesture, or large face with good gestures, but the robot will not walk. Solution is a mobile robot with upper human body.

63. Conclusions on motion 1.Motion can be generated based on splines, Spectral methods, regular expressions, grammars, forward and inverse kinematics. 2.Motion can be transformed from other motions or signals (sound, music, speech, light) 3.Motion can be acquired (from camera, from accelerometers, body sensors, etc).

64. Fourth question Can you propose other methods of robot motion acquisition, generation and transformation to be used in robot theatre?

65. Perception Machine

66. Face Image 1 Face Image 2 Face Image 3 Face Image 4 John Smith Marek Perkowski Face Recognition as a learning problem Perception

67. Face Image 1 Face Image 2 Face Image 3 Face Image 4 happy sad Face Emotion Recognition as a learning problem Faceperson Face Emotion (Gesture) Recognition as a learning problem

68. Faceperson Faceemotion Faceage Facegender Facegesture Learning problems in Human-Robot Interaction – Perception problems Recognition Problems = Who? What? How?

69. Face features recognition and visualization.

70. Recognizing Emotions in Human Face

71. PCA + NN software of Labunsky

74. Fifth question Can you propose other methods for robot perception to be used in robot theatre?

75. Brain Machine

76. Software Artificial and Computational Intelligence: – Search such as A*. – Natural language such as integrated chattbots. – Sophisticated vision and pattern recognition algorithms. – Evolutionary and Neural algorithms. – Multi-processor systems, multi-threading, CUDA and GPU like systems – Individual simple behaviors based on hierarchical architectures: Distance keeping, Tracking. Following

77. Behaviors 1.Tracking with whole body (mobile robot) 2.Tracking with upper body of humanoid robot 3.Keeping distance 4.Avoiding 5.Following 6.Following when far away, avoiding when close 7.Creating a line of robots 8.Dancing 9.Falling down 10.Standing up 11.Discussion 12.Fight

78. Input textOutput text Hexapod walking Distance evaluation Biped walking Number of falls evaluation Biped Gestures Comparison to video evaluation Hand gestures Subjective human evaluation Learning problems in Human-Robot Interaction – Motion Behavior (input/output) generation problems Behavior Problems = examples of correct motions – generalize and modify, interpolate How to evaluate?

79. Concepts for brain (implemented and what is wrong with them?) 1.Genetic algorithm 2.Genetic programming 3.Search such as A* 4.Neural Networks 5.Predicate Calculus Automatic Theorem Proving New integrated models of robot: 1.Emotional robot 2.Quantum robot 3.Moral robot

80. Emotional Robot

82. Emotion Recognition versus Emotion Generation

83. Quantum Robot

84. Quantum Logic Binary Logic Fuzzy Logic Quantum Signals and Automata 0, 1 [0,1] Hilbert Space, Bloch Sphere

85. Algorithm Logic circuit Finite State Machine Quantum Signals and Automata Quantum array Quantum state Machine Quantum Algorithm

86. Quantum Computing Intelligent Robotics Quantum Robotics Since 1999 Since 1969 Since 2004 Quantum Robot Theatre

87. Quantum Robotics Constraint Satisfaction Model for Grover Algorithm Quantum Fuzzy Logic Collaboration: Martin Lukac, Michitaka Kameyama, Tohoku University, Vamsi Parasa, Erik Paul Quantum Braitenberg Vehicles Quantum Emotions Quantum Initialization and Neural Networks Arushi Raghuvanshi Michael Miller, Univ. Victoria, BC Martin Lukac David Rosenbaum Quantum Robot Vision Siddhar Manoj Arushi Raghuvanshi

88. Our concept of quantum robot based on reducing all problems to constraint satisfaction solved on a quantum computer

89. QUBOT-1 – the world’s first quantum robot

90. PC The whole proposed PSU Quantum Robot system Bluetooth connection Personal computer CUDA/GPU supercomputer Orion interface Orion Quantum Adiabatic Computer in Vancouver BC, Canada

91. Oracle for Quantum Map of Europe Coloring Spain France Germany Switzerland Spain France Germany Switzerland  Good coloring Quaternary qudits Multiple-Valued Quantum Circuits

92. Spain France Germany Switzerland Spain France Germany Switzerland  Good coloring Constraints to be satisfied Energy to be minimized Count number of ones minimize the energy Constraint Satisfaction Problem is to satisfy the constraints and minimize the energy Adiabatic Quantum Computer

93. Constraints Satisfaction Problems S E N D + M O R E M O N E Y Graph coloring Cryptographic Problems

94. Constraint Satisfaction for Robotics Insufficient speed of robot image processing and pattern recognition. Insufficient speed of robot image processing and pattern recognition. – This can be solved by special processors, DSP processors, FPGA architectures and parallel computing. Prolog allows to write CSP programs very quickly. Prolog allows to write CSP programs very quickly. An interesting approach is to formulate many problems using the same general model. An interesting approach is to formulate many problems using the same general model. This model may be predicate calculus, Satisfiability, Artificial Neural Nets or Constraints Satisfaction Model. This model may be predicate calculus, Satisfiability, Artificial Neural Nets or Constraints Satisfaction Model. – Constraints to be satisfied (complex formulas in general) – Energies to be minimized (complex formulas)

95. SAT as a constraint satisfaction problem (a + b’ + c) * (b + d’) … = 1 a d c b (a + b’ + c) =1 =0=1 (b + d’) Yes Yes, do nothing to nodes No No, update nodes Highly parallel system of updating nodes

96. SAT as a constraint satisfaction problem (a + b’ + c) * (b + d’) … = 1 Constraints: (a + b’ + c) = 1 (b + d’) = 1 ….. Energy optimization: (a + b’ + c) = f1 (b + d’) = f2 …. Min ( f1’ + f2’ + …..) Orion programming is just writing equations for constraints and equations for energy

97. Conditional robot response based on camera input This one is really cool. At a high level the way it works is as follows. You have a camera trained on a human. The data taken by the camera is processed so as to detect features, which are generalized patterns of behavior. For example, a feature detector could be configured to detect the presence of anger in the human, for example by learning-based methods. In addition, there is a robot, which is connected to the data processing system connected to the camera. This robot has effectors which control its actions. In this application, the effector controls are functions of the processed input from the camera, where the rules connecting the two are user-determined.

98. This generic situation, where the robot’s behavior is conditioned upon the input from the feature detectors connected to the camera, maps to a constraint satisfaction problem as described here.as described here The way this would work is that the human / camera / robot system would generate optimization and satisfiability problems, to determine how the robot’s effectors should fire, and these problems can be remotely solved using Orion. For example, you could acquire a Hansen Robotics Einstein, sit it him on your desk, train a camera on your face, use an anger feature detector that causes the Einstein robot to laugh harder the angrier you get. Hansen Robotics Einstein

99. Moral Robot

100. Concepts of a moral robot 1.So far, moral robots are built for military (immoral robots?) 2.Our concept is a moral robot, helper, housekeeper and entertainer for an old demanding lady. 1.Path planning and tasks execution 2.Access to internet 3.Conversation and dancing 1.Robot integrates modal and deontic logic. 2.Principles of morality: 1.Be nice 2.Help and protect 3.Never harm by accident or mistake 4.Resolve conflicts of the lady with family and friends 3.Asimov’s Laws. 4.Exclusions and special cases.

101. Sixth question Can you propose morality rules for the assistive robot for the elderly?

102. Seventh question Can you propose other methods for robot brain to be used in robot theatre?

103. theory of robot theatre 1.Our goals are to both create a model innovative robot theatre and a theory of robot theatre that would be similar to the theory of film or theory of interactive computer games. 2.We believe that robot theatre will become a new art form and we are interested what are the basic questions related to the art of performing robots. 3.We hope to have an interesting feedback to our ideas from all groups of PSU researchers.Conclusions

104. More questions

105. New Research Area or only application? 1.What is Robot Theatre Theory? 2.What are its main methods? 3.How to evaluate Robot Theatres? 4.Is robot theatre only an application of robotics or is it more? What ? We see that humans can laugh looking at our theatre. Will we ever experience humans crying at robot performances? What can we learn from robot theatre that is not a standard robotics problem?

106. Final Fundamental Questions: 1.Will Robot theatre be ever as popular art form as film or theatre? 2.Will robots be popularly used in theatres? 3.Will we see robot theatre toys? 4.Will home robots become also robot entertainers?

107. In what year this will happen?