1. Behavior Based Systems Behavior Based Systems Lezione 6

2. Properties of Behaviors Behaviors are feedback controllers Behaviors are feedback controllers Behaviors achieve specific tasks/goals (e.g., avoid-others, find- friend, go-home)‏ Behaviors achieve specific tasks/goals (e.g., avoid-others, find- friend, go-home)‏ Behaviors are typically executed in parallel/concurrently Behaviors are typically executed in parallel/concurrently Behaviors can store state and be used to construct world models/representation Behaviors can store state and be used to construct world models/representation Behaviors can directly connect sensors and effectors (i.e., take inputs from sensors and send outputs to effectors) Behaviors can directly connect sensors and effectors (i.e., take inputs from sensors and send outputs to effectors) Behaviors can also take inputs from other behaviors and send outputs to other behaviors (this allows for building networks)‏ Behaviors can also take inputs from other behaviors and send outputs to other behaviors (this allows for building networks)‏ Behaviors are typically higher-level than actions (go-home rather than turn-left-by-37.5-degrees)‏ Behaviors are typically higher-level than actions (go-home rather than turn-left-by-37.5-degrees)‏ Behaviors are typically closed-loop but extended in time Behaviors are typically closed-loop but extended in time When assembled into distributed representations, behaviors can be used to look ahead but at a time-scale comparable with the rest of the behavior-based system When assembled into distributed representations, behaviors can be used to look ahead but at a time-scale comparable with the rest of the behavior-based system

3. A Generic Classification of Robot Behaviors Exploration/directional behaviors (heading based, wandering)‏ Exploration/directional behaviors (heading based, wandering)‏ Goal-oriented appetitive behaviors (discrete object attractor, area attractor)‏ Goal-oriented appetitive behaviors (discrete object attractor, area attractor)‏ Aversive/protective behaviors (avoid stationary objects,dodge moving objects, aggression)‏ Aversive/protective behaviors (avoid stationary objects,dodge moving objects, aggression)‏ Path following behaviors (road following, hallway navigation, stripe following)‏ Path following behaviors (road following, hallway navigation, stripe following)‏ Postural behaviors (balance, stability)‏ Postural behaviors (balance, stability)‏ Social/cooperative behaviors (sharing, foraging, flocking/herding)‏ Social/cooperative behaviors (sharing, foraging, flocking/herding)‏ Teleautonomous behaviors (influence, behavioral modification)‏ Teleautonomous behaviors (influence, behavioral modification)‏ Perceptual behaviors (saccades, visual search, ocular reflexes)‏ Perceptual behaviors (saccades, visual search, ocular reflexes)‏ Walking behaviors (gait control)‏ Walking behaviors (gait control)‏ Manipulator-specific behaviors (reaching)‏ Manipulator-specific behaviors (reaching)‏ Gripper/dextrous hand behaviors (grasping, enveloping)‏ Gripper/dextrous hand behaviors (grasping, enveloping)‏

4. Key properties Ability to act in real time Ability to act in real time Ability to use representations to generate efficient (not only reactive) behavior Ability to use representations to generate efficient (not only reactive) behavior Ability to use a uniform structure and representationthroughout the system (so no intermediate layer)‏ Ability to use a uniform structure and representationthroughout the system (so no intermediate layer)‏

5. Key Questions of Behavior-based Robotics What are the right behavioral building blocks for robotic systems? What are the right behavioral building blocks for robotic systems? What really is a primitive behavior? What really is a primitive behavior? How are these behaviors effectively coordinated? How are these behaviors effectively coordinated? How are these behaviors grounded to sensors and actuators? How are these behaviors grounded to sensors and actuators?

6. Design Paradigms Ethological guided/constrained Ethological guided/constrained Situated activity Situated activity Experimentally driven Experimentally driven

7. Ethologically Guided/constrained Design Methodology Ethologically guided (inspired) design methodology Ethologically guided (inspired) design methodology  We can use and mimic models generated by biological scientists;  we are free to take what we need and discard the rest. Ethologically constrained design methodology Ethologically constrained design methodology  We deliberately constrain behavioral models to match those of the animal under study as well as we can.

8. Design Methodology

9. Situated Activity Design Methodology Situated activity-based design methodology Situated activity-based design methodology  Situated activity means that a robot's actions are predicated upon situations in which it finds itself.  Hence the perception problem is reduced to recognizing what situation(s) the robot is in and then choosing one action (out of many) to undertake.

10. Situated activity-based design methodology

11. Experimentally Driven Design Methodology Experimentally driven behaviors are invariably created in a bottom-up manner. The basic operating premise Experimentally driven behaviors are invariably created in a bottom-up manner. The basic operating premise  is to endow a robot with a limited set of capabilities (competencies),  run experiments in the real world,  see what works, and what does not,  debug imperfect behaviors, and then  add new behaviors iteratively until the overall system exhibits satisfactory performance. "Do whatever works.." methodology. Very much like the software development write, run, crash, debug, write,... "Do whatever works.." methodology. Very much like the software development write, run, crash, debug, write,...

12. Experimentally Driven Design Methodology

13. Types of Behavior Notation Stimulus-Response Stimulus-Response Diagrams Functional Notation for Behaviors Diagrams Functional Notation for Behaviors Finite State Acceptor Diagrams Finite State Acceptor Diagrams

14. Stimulus-response Diagrams Stimulus-Response (SR) diagrams are the most intuitive and the least formal method of expression. Stimulus-Response (SR) diagrams are the most intuitive and the least formal method of expression.

15. Stimulus-response Diagram for a Robot Going to a Classroom

16. Functional Notation for Behaviors We can use functional notation to describe behaviors b(s) = r We can use functional notation to describe behaviors b(s) = r where b is the behavior, s the stimulus and r the response. where b is the behavior, s the stimulus and r the response. In a purely reactive system, time is not an argument for the behavior b. In a purely reactive system, time is not an argument for the behavior b.

17. Functional Example coordinate-behaviors [ move-to-classroom ( detect-classroom-location ), avoid-objects ( detect-objects ), dodge-students ( detect-students ), stay-to-right-on-path ( detect-path ), defer-to-elders ( detect-elders ) ] = motor-response coordinate-behaviors [ move-to-classroom ( detect-classroom-location ), avoid-objects ( detect-objects ), dodge-students ( detect-students ), stay-to-right-on-path ( detect-path ), defer-to-elders ( detect-elders ) ] = motor-response

18. Finite State Acceptor Diagrams Finite state diagrams (FSD) or finite state acceptor (FSA) diagrams can be used to describe aggregations and sequences of behaviors (also called as behavior assemblages). Finite state diagrams (FSD) or finite state acceptor (FSA) diagrams can be used to describe aggregations and sequences of behaviors (also called as behavior assemblages). FSA are best to specify complex behavioral control systems where entire sets of primitive behaviors are swapped in and out of execution during the accomplishment of some high-level goal. FSA are best to specify complex behavioral control systems where entire sets of primitive behaviors are swapped in and out of execution during the accomplishment of some high-level goal. In some cases formal languages like the Specification and Description language (SDL) can be used to create/generate code for real implementations In some cases formal languages like the Specification and Description language (SDL) can be used to create/generate code for real implementations

19. FSA representation for the classroom navigational task

20. Behavioral encoding To encode the behavioral response that the stimulus should evoke, we must create a functional mapping from the stimulus plane to the motor (response) plane. To encode the behavioral response that the stimulus should evoke, we must create a functional mapping from the stimulus plane to the motor (response) plane. A behavior can be expressed as a triple (S,R, β ) where S denotes the domain of all interpretable stimuli. R denotes the range of possible responses, Behavior β denotes the mapping β : S → R. A behavior can be expressed as a triple (S,R, β ) where S denotes the domain of all interpretable stimuli. R denotes the range of possible responses, Behavior β denotes the mapping β : S → R.

21. Two components of robot’s motor response We can factor the robot's (motor) response into two orthogonal components: We can factor the robot's (motor) response into two orthogonal components:  Strength: Denotes the magnitude of the response. Denotes the magnitude of the response. The strength may be entirely independent of the strength of the stimulus yet modulated by exogenous factors like intention The strength may be entirely independent of the strength of the stimulus yet modulated by exogenous factors like intention  Orientation: Denotes the direction of action for the response. Denotes the direction of action for the response. The realization of this directional component requires knowledge of the robot's kinematics. The realization of this directional component requires knowledge of the robot's kinematics. It may or may not be dependent on the stimulus's strength. It may or may not be dependent on the stimulus's strength.

22. The instantaneous response r The instantaneous response r (where r ∈ R) of a behaviorbased reactive system can be expressed as a six- dimensional vector: The instantaneous response r (where r ∈ R) of a behaviorbased reactive system can be expressed as a six- dimensional vector: r =[x,y,z,θ,φ,ψ], where the first three components represent the translational degrees of freedom (x,y,z in three dimensional Cartesian coordinates), and the last three components encode the three rotational degrees of freedom. The dimensionality is often considerably less than six DOFs. 2D-mobile robot has: r=[x, y, θ]. r =[x,y,z,θ,φ,ψ], where the first three components represent the translational degrees of freedom (x,y,z in three dimensional Cartesian coordinates), and the last three components encode the three rotational degrees of freedom. The dimensionality is often considerably less than six DOFs. 2D-mobile robot has: r=[x, y, θ]. The response r may be limited by the physical limitations of the robot; the robot is non-holonomic, or has restrictions the way it can move. The non-holonomic constrains can be dealt with The response r may be limited by the physical limitations of the robot; the robot is non-holonomic, or has restrictions the way it can move. The non-holonomic constrains can be dealt with  during the generation of the response or  by including them in β or  after r has been computed, translating the desired response to be within limitations.

23. The stimulus domain S consists of the domain of all perceivable stimuli. Each individual stimulus s (s ∈ S) is represented as a binary tuple: s = (p,l), where p is a particular type or perceptual class and l is a property strength. If l is nonzero, the stimulus specified by p is present to some degree. S consists of the domain of all perceivable stimuli. Each individual stimulus s (s ∈ S) is represented as a binary tuple: s = (p,l), where p is a particular type or perceptual class and l is a property strength. If l is nonzero, the stimulus specified by p is present to some degree. The presence of a stimulus is necessary but not sufficient to evoke a motor response in a behavior-based robot. The presence of a stimulus is necessary but not sufficient to evoke a motor response in a behavior-based robot. A threshold value t is defined, above which a response is generated. Habituation and inhibition (and other exogenous factors) might influence the threshold t value. A threshold value t is defined, above which a response is generated. Habituation and inhibition (and other exogenous factors) might influence the threshold t value.

24. The behavioral mapping β For each individual active behavior we can formally establish the mapping between the stimulus and response range that defines a behavioral function β: β (s) → r. For each individual active behavior we can formally establish the mapping between the stimulus and response range that defines a behavioral function β: β (s) → r. β can be defined arbitrarily, but it must be defined over all relevant p in S. Where a specific stimulus threshold, t, must be exceeded before a response is produced for a specific s = (p,l); r =[0,0,0,0,0,0], l < t β: (p,l) r=some function(p, l), otherwise β can be defined arbitrarily, but it must be defined over all relevant p in S. Where a specific stimulus threshold, t, must be exceeded before a response is produced for a specific s = (p,l); r =[0,0,0,0,0,0], l < t β: (p,l) r=some function(p, l), otherwise The functional mapping between the strength of stimulus and the magnitude (strength) and direction of robotic motor response defines the design space for a particular robot behavior. The functional mapping between the strength of stimulus and the magnitude (strength) and direction of robotic motor response defines the design space for a particular robot behavior.

25. Strength multiplier Associated with a particular behavior, b, may be a scalar gain value g, a strength multiplier. The strength multiplier modifies the magnitude of the overall response r: r' = g r. Associated with a particular behavior, b, may be a scalar gain value g, a strength multiplier. The strength multiplier modifies the magnitude of the overall response r: r' = g r. The gain values are used to compose multiple behaviors by specifying their strengths relative to one another. In the extreme case, g can be used to turn off a behavior by setting it to zero, thus reducing r' to 0. The gain values are used to compose multiple behaviors by specifying their strengths relative to one another. In the extreme case, g can be used to turn off a behavior by setting it to zero, thus reducing r' to 0. Learning and habituation can change the strength multiplier for a response. Learning and habituation can change the strength multiplier for a response.

26. Different types of behavioral mappings b The behavioral mappings, b, of stimuli onto responses fall into three general categories: The behavioral mappings, b, of stimuli onto responses fall into three general categories:  Null: the stimulus produces no motor response.  Discrete: The stimulus produces a response from an enumerated set of prescribed choices. Typically R is a bounded set of stereotypical responses enumerated for the stimulus domain S and specified by b.  Continuous: The stimulus domain produces a motor response that is continuous over R.

27. Null Encoding For all s, b, s → 0. For all s, b, s → 0. A very useful behavior! A very useful behavior!

28. Discrete Encoding We use the "Situated-action methodology" We use the "Situated-action methodology" One possibility is that the behavioral mapping b consists of a finite set of (situation,response) pairs. Sensing provides the index into a lookup table for finding the appropriate situation and corresponding response. One possibility is that the behavioral mapping b consists of a finite set of (situation,response) pairs. Sensing provides the index into a lookup table for finding the appropriate situation and corresponding response. Second possibility is to define the discrete encoding by using rule- based systems. Here b is represented as a collection of IF-THEN rules. The rules take the general form: IF THEN. Second possibility is to define the discrete encoding by using rule- based systems. Here b is represented as a collection of IF-THEN rules. The rules take the general form: IF THEN. If more than one rule applies, conflict resolution must be used to clarify the situation and to decide what rule to use. If more than one rule applies, conflict resolution must be used to clarify the situation and to decide what rule to use.

29. Continuous Functional encoding Continuous response allows a robot to have an infinite space of potential reactions to its world. This might be a good thing or bad thing. Continuous response allows a robot to have an infinite space of potential reactions to its world. This might be a good thing or bad thing. A mathematical function transforms the sensory input into a behavioral reaction. A mathematical function transforms the sensory input into a behavioral reaction. Potential fields method is one of the most common (and promising) methods for implementing continuous response. Potential fields method is one of the most common (and promising) methods for implementing continuous response. Here we borrow ideas from physics: Coulomb's electrostatic attraction, gravitation fields. These fields are typically combined through super- positioning to yield a single global field. Here we borrow ideas from physics: Coulomb's electrostatic attraction, gravitation fields. These fields are typically combined through super- positioning to yield a single global field. Potential fields have some problems; local minima (robots get stuck) and oscillatory behaviors (robots go in a circle)‏ Potential fields have some problems; local minima (robots get stuck) and oscillatory behaviors (robots go in a circle)‏

30. Continuous Functional encoding (cont.). Navigational templates (NATs) are one way to generate continuous response. Navigational templates (NATs) are one way to generate continuous response. NATs are defined as arbitrary functions to characterize navigational space that do not necessary have any correlation with typical potential field methods. NATs are defined as arbitrary functions to characterize navigational space that do not necessary have any correlation with typical potential field methods. Example: spin-based obstacle treatment; "whenever there is an obstacle we need to avoid, start circumventing the object". Example: spin-based obstacle treatment; "whenever there is an obstacle we need to avoid, start circumventing the object". By using NATs we can solve some local minima problems found intraditional potential fields methods. By using NATs we can solve some local minima problems found intraditional potential fields methods.