Natural User Interfaces: Touchless Management of Medical Equipment in Sterile Environments. J Rafael Sádaba MSc FRCS CTh. Dpt of Cardiac Surgery. CH de Navarra. Pamplona, Spain.

J. Rafael Sádaba MSc, FRCS CTh   I have no relevant financial relationships

THE PROBLEM Access to patient information during surgical procedures Imaging Medical records Management of imaging/equipment during interventional procedures Let me first discuss the difficulties we may have during surgical or interventional procedures during which the operator, sometimes a surgeon, sometimes an interventionalist, needs to have access to diagnostic images obtained before the procedure or to other type of information, for instance from medical records. Traditionally, when this is the case, one has to ask somebody else, who is not scrubbed to look into the computer and obtain the information or images required. This is most of the times cumbersome, takes time, and the result may no be ideal In other occasions, the scrubbed up operator may need to manipulate imaging equipment during procedures. Traditionally he or she needs to ask somebody else to do this. Again, the procedure may be cumbersome, complex and time consuming.

THE SOLUTION: NATURAL USER INTERFACE Natural user interfaces allows for touchless (with natural gestures or voice commands) direct communication between the operator and the imaging equipment without the need of intermediaries. Controlling medical equipment and imaging devices through natural gestures and voice commands : No need to install software in it. Compatible with any sensor device (LEAP, Kinect, MYO…) Fully customizable (operating theatre, hybrid room, cath lab, etc) Speed and simplicity

THE TECHNOLOGY TedCube (Gateway): Information is processed Recognises pre-defined and customized gestures Information is processed Translates it into a protocol that the computer understands it is a “translator” for my gestures and voice commands for the computer, making the mouse and keyboard redundant

Sensors Kinect Leap Motion Myo

SENSORS SUPPORTED Leap Motion Kinect Myo Feature Accuracy High Medium 3D interaction through 3D interface Yes Fast and Easy training DICOM viewer support Full Touchless control Line of sight required No Distance to operate 20-40 cm + 1,4 m No minimun limit The Leap Motion has a stereoscopic camera (i.e. two twin calibrated cameras), which allows simulation of the human binocular vision, and therefore gives it the ability to capture three-dimensional images. It is ideal for executing highly accurate commands, such as performing measurements. The Microsoft Kinect 2.0 sensor relies upon a novel image sensor that indirectly measures the time it takes for pulses of laser light to travel from a laser projector, to a target surface, and then back to an image sensor. It’s suited for specialists who need to control their computers or equipment from a distance. The function of the Myo bracelet in based on electromyography which detects the electrical activity produced by skeletal muscles in the forearm. It also contains an accelerometer and a gyroscope. It is ideal for specialists who are required not to take their hands off the patient during the procedure.

Use of voice commands technology

Seeing is believing…showtime!

The Team Área del Corazón. CHN Tedcas: Rafael Sádaba MSc FRCS (CTh) Virginia Álvarez Asiain MD Nuria Basterra MD Tedcas: Jesús Perez Llano PhD Gerardo Caballero PhD Enrique Muñoz PhD Daniel Calvo PhD Elisa Novoa PhD