Implementing Tactile Behaviors Using FingerVision Akihiko Yamaguchi and Christopher G. Atkeson Presentation* by Jacob Best *Special thanks to Google Slides “Science Fair” Template
Outline What is FingerVision? How does it work? Grasping Strategies Experiments
What is FingerVision?
FingerVision FingerVision is a vision-based tactile sensor consisting of soft skin and cameras
FingerVision Soft layer, made of silicone, has markers allowing the cameras to detect skin deformation Transparency of layers allows cameras to calculate and track proximity objects in the real world It is cheap! ($50/sensor) and easy! (all calculations done with OpenCV) Y X Z
How does it work? i.e. how does it actually perform grasping tasks?
Multimodality Two Basic Functions of FingerVision: Proximity Vision - detection of the object being grasped and its movement through images Object shape/texture Slip Movement Force Distribution - tracking the markers on the soft layer to understand the force of the grip in x, y, and z dimensions
Proximity Vision All images represented as histograms of colors Movement Detection Uses cv::BackgroundSubstractorMOG2 Object Model Construction Originally constructs a model of the background Before gripping new object, uses movement detection to model image Erosion and dilation to refine object Tracking On new frame, apply back projection method (cv::calcBackProject) on histogram of object Then use background subtraction again
Force Distribution Two Methods for Object Tracking: cv::MeanShift cv::SimpleBlobDetector Two Stages for Object Tracking: Calibration - Find markers by placing image against white background Tracking - Follow blobs by selecting ROI around each blob Calculation of Force: Right: MeanShift Left: SimpleBlobDetector When normal force is applied:
Grasping Strategies Types of grasps and how to utilize Vision and Force to accomplish these tasks
Types of Grasps Gentle Grasp Holding Handover Uses array of force and some threshold for the sum of the norm of the forces to know when to stop gripping Holding Increases grip on the object when there is slippage Slip defined as number of moving points on object > threshold Handover Open gripper when there is a certain change in force or slip In-Hand Manipulation (controlled slip) Slowly decrease grip until slip is detected then increase grip to a hold Repeat that process until target angle/orientation of object is achieved
Experiments Types of grasps and how to utilize Vision and Force to accomplish these tasks
Pouring Water into A Grasped Container
Gentle Grasp: Coke Can + Business Card
Holding
Holding for Lightweight Objects: Oragami Bird
Handover
In-Hand Manipulation
Conclusion The manipulation strategies were effective with FingerVision (Though In-Hand Manipulation not as effective) Very impressive and effective grasping technique in general Creative but computationally and conceptually simple Next steps? Would be interested to see how it performs on heavier and more irregularly shaped objects Practicality? Gentle grip has potential for complicated robot tasks
Video