KINESTHETIC DISPLAYS FOR REMOTE & VIRTUAL ENVIRONMENTS -Blake Hannaford and Steven Venema Presented By Subhashini Ganapathy Sasanka V. Prabhala

Contents Introduction Characteristics of the Kinesthetic Channel Simulation Types of kinesthetic Displays Kinesthetic Displays and Selection issues Safety Issues Application Implementation and Design Approach

Introduction Different ways of perceiving the environment Haptic Displays –Kinesthetic Display –Tactile Display Kinesthesis –Sensations derived from muscles, tendons and joints and stimulated by movement and tension

HISTORY In virtual reality the perceptions from a simulated environments are conveyed to the user Hence teleoperation and virtual reality share the same user interface issues Thus much of the information presented is drawn from the field of teleoperation

Characteristics of the Kinesthetic channel Two different roles of kinesthetic sensations Body position sense Contact between the body and the external environment

Comparison between Kinesthetic Information and other modalities Kinesthetic DisplaysVisual & Auditory Displays Kinesthetic sensations involve bi-directional flow of energy between the humans and the environment Kinesthetic sensations cannot be reproduced by the information system alone To sense the environment is to modify the environment The visual and the auditory channels are one- way, information-only flows. The environment can only be recorded & synthesized, replicated by the information system Environment cannot be modified

Physics: Position/Force Simultaneity The problem with the Virtual environment and the teleoperations is how to produce the bi-directional properties of mechanical energy flow One way is to use “FORCE FEEDBACK” technique in which velocity is sensed and to apply the appropriate force to the operator and vice versa The usual implementation constrain is to sense and leave one variable and to control the other variable and the physical restriction is to interface the energy system to the information-only system The kind of information needed to conveyed to reproduce the kinesthetic sensation or contact is difficult to deduce as there may be many possible modes of contact between objects and the existence of multiple contact points

Simulation Simulation is studied using three levels of realism  Second order linear system  Point contact ( Continuous)  Point contact ( Discontinuous)

Second order linear systems A second order linear system is permanently attached to the users hand at the kinesthetic display The kinesthetic display will appear to have certain mass, damping, and spring like behavior and the control system necessary to produce this effect can be produced by “IMPEDENCE CONTROL”. It is global model in that it applies to all values of position

Point contact continuous and discontinuous This form of realism applies to the local region of space (soft and hard surfaces) The rapidity with which a display can calculate and apply forces to the human hand determines the level of realism Soft surfaces will cause force to increase gradually as contact is made while those with hard surfaces will cause discontinuous force trajectories

Body Reference Frame/Object Extent Visual sensations appear to exist in space which is external to the observer where as the kinesthetic sensations are always perceived with respect to a body reference as opposed to the world reference frame When viewing a complex scene the eye movement generates a scan path during which our retina image a sequence of detailed spots on the scene where as kinesthetic contact sensations are spatially localized to specific object

Types of kinesthetic Displays Contact Modeling Force-Feedback Displays Displacement-Feedback Displays Cross-Modal Displays

Contact Modeling Net work theory allows to model the interaction between the operator and the simulated or remote environment using the network theory which is applicable to both mechanical and electrical energy transmission The equation shows that it is possible to control at most one of the two mechanical system variables force and velocity

CONTACT MODELLING F1F1 Z 1 (v) Z 2 (v) F2F2 Human OperatorKinesthetic Display One-port model of a kinesthetic display and human operator F 1 – Z 1 (V p )-Z 2 (V p ) = F p Vp Fp

Forced-Feedback Displays The most common approach to implement kinesthetic interaction is to sense the operators velocity/position and to apply force at the point where the velocity is sensed assuming the contact point to be the operators hand This mode of kinesthetic display should have the following characteristics: –Must produce accurately the forces intended to be applied –Should have high bandwidth –Capable of sufficiently large forces so that the contact can be simulated

Displacement-Feedback displays In this method of implementing the kinesthetic interaction is to sense applied force and to impose a controlled displacement on the display This displacement is calculated by a dynamic world model from the response of the simulated object to the measured operator contact force This mode of kinesthetic display should have the following characteristics: 1.It should accurately reproduce the displacement intended to be applied 2.It must have high bandwidth 3.It must be rigid enough to completely block the operators hand when contact is meant to be conveyed with a rigid object

Cross-Modal Displays This type of display keeps the user feedback in the information domain and thereby avoids the difficulties of reproducing or simulating bilateral energy flow This can be achieved if one of the variable from the simulated or remote contact is controlled by the operator, and the other is displayed to the operator at a different point or through a different sensory modality The power at each port is zero but information about the simulated or remote interaction is conveyed

Kinesthetic Display Design & Selection Issues Kinematics Degrees of Freedom Workspace Singularity Analysis Human Interface Dynamics

Requirements : Kinematics Kinesthetic display must be capable of exchanging energy with an operator using the mechanical system variables “force” and “velocity” A common “ground” or reference must exist between the operator and the display

Kinematics The requirements are met by using kinematically articulated mechanism with joints and articulated links configured with one end connected to the “ground” and the other connected to the operator’s hand A display’s kinematic parameters are describe the interrelation between the display’s DOF or joints

Degrees of Freedom DOF(Degrees of freedom) allows motion along or around a single axis The number of positions and orientations that a mechanism with a single prismatic DOF can only achieve motion along a single line in space Though the increase in DOF gives complete freedom of motion there are a lot of restrictions like the increase in DOF increases the complexity and cost of the display

Workspace When all DOFs are considered simultaneousely the ranges of motion describe the “ workspace” of the display The work space of higher-DOF display is difficult to describe mathematically due to higher dimensionality The size of the workspace depends on the type of task that is to be done

Singularity Analysis A display mechanism is said to be “singular” when one or more joints is at a motion limit or when two or more joint axes become parallel The display mechanism should be designed such that no singularities are encountered within the workspace that the operator is expected to use

Human Interface The kinesthetic display must be suitable for human use and comfort The display performance is constrained by the human arm that is grasping the mechanism The region of operation is very essential to determine the manipulation activities

Dynamics The fidelity of a force-feedback display system is inherently limited by the display mechanism itself The force available to the operator is reduced by terms accounting for the friction and inertia of the display mechanism The effects of mass, friction and stiffness on the transfer of force from display to operator and the transmission of velocity from the operator to the display

Dynamics FaFa M Force Display Command k x, v F f (x, v) Operator F a = The accurate force k = The mechanism stiffness F f (x, v) = The friction of the transmission system F 0 = The force at the mechanism/operator interface x = the position of the mechanism/operator interface v = The velocity of the mechanism/operator interface Dynamic model of one-dimensional kinesthetic display

Kinesthetic Display Examples Salisbury/JPL force-feedback display –An important design feature of it is the location of the singular point of its wrist mechanism. –Force-Feedback displays Utah Displacement Display –Displacement-Feedback display –The system senses force applied by the human operator to the joints of the display mechanism, and control the device to achieve a displacement based on that of a kinematically identical slave arm.

Safety Issues for Kinesthetic Interfaces Kinesthetic displays have a great display and are finding application in a wide variety of rehabilitation related applications Examples: »Force-reflecting joysticks for wheel-chair control »Six DOF head-input devices for improved assistive robot dexterity

Safety Issues If the force-feedback is uncontrolled or improperly limited, the applied forces and moments may represent a potential hazard to the operator or people nearby

Application Field of Biochemistry Field of microteleoperation

Implementation Issues and Design Approach The “hard-contact” problem Real-time dynamic modeling Mechanism design

Conclusion While much research remains to be done in this area, the current state of the art allows at least rudimentary forms of these displays to be added to existing virtual environment implementations

References Safety Issues for Kinesthetic Interface in Assistive Robotics, RESNA’96 Proceedings gypsy.rose.utoronto.ca/#research Ergonomics Teleoperation & Control Laboratory Clark, F.J. and Horch, K. W. (1986) Kinesthesia, in Handbook of Perception and Human Performance, Boff et al.(Eds), New York: Wiley-Interscience Hannaford,B.(1989) A design framework for teleoperators with kinesthetic feedback, IEEE Trans. Robot. Autom., 5, B.Hannaford, ‘Kinesthetic Feedback Techniques in Teleoperated Systems,’ In “Advances in Control and Dynamic Systems”, pp. 1-32, C.Leondes, Ed., Academic Press, San Diego, 1991