1. Implementing Tactile Behaviors Using FingerVision
Akihiko Yamaguchi and Christopher G. Atkeson
Presentation* by Jacob Best
*Special thanks to Google Slides “Science Fair” Template

2. Outline What is FingerVision? How does it work? Grasping Strategies
Experiments

3. What is FingerVision?

4. FingerVision
FingerVision is a vision-based tactile sensor consisting of soft skin and cameras

5. FingerVision
Soft layer, made of silicone, has markers allowing the cameras to detect skin deformation
Transparency of layers allows cameras to calculate and track proximity objects in the real world
It is cheap! ($50/sensor) and easy! (all calculations done with OpenCV) Y X Z

6. How does it work? i.e. how does it actually perform grasping tasks?

7. Multimodality Two Basic Functions of FingerVision:
Proximity Vision - detection of the object being grasped and its movement through images
Object shape/texture
Slip
Movement
Force Distribution - tracking the markers on the soft layer to understand the force of the grip in x, y, and z dimensions

8. Proximity Vision All images represented as histograms of colors
Movement Detection
Uses cv::BackgroundSubstractorMOG2
Object Model Construction
Originally constructs a model of the background
Before gripping new object, uses movement detection to model image
Erosion and dilation to refine object
Tracking
On new frame, apply back projection method (cv::calcBackProject) on histogram of object
Then use background subtraction again

9. Force Distribution Two Methods for Object Tracking: cv::MeanShift
cv::SimpleBlobDetector
Two Stages for Object Tracking:
Calibration - Find markers by placing image against white background
Tracking - Follow blobs by selecting ROI around each blob
Calculation of Force:
Right: MeanShift Left: SimpleBlobDetector
When normal force is applied:

10. Grasping Strategies
Types of grasps and how to utilize Vision and Force to accomplish these tasks

11. Types of Grasps Gentle Grasp Holding Handover
Uses array of force and some threshold for the sum of the norm of the forces to know when to stop gripping
Holding
Increases grip on the object when there is slippage
Slip defined as number of moving points on object > threshold
Handover
Open gripper when there is a certain change in force or slip
In-Hand Manipulation (controlled slip)
Slowly decrease grip until slip is detected then increase grip to a hold
Repeat that process until target angle/orientation of object is achieved

12. Experiments
Types of grasps and how to utilize Vision and Force to accomplish these tasks

13. Pouring Water into A Grasped Container

14. Gentle Grasp: Coke Can + Business Card

15. Holding

16. Holding for Lightweight Objects: Oragami Bird

17. Handover

18. In-Hand Manipulation

19. Conclusion
The manipulation strategies were effective with FingerVision (Though In-Hand Manipulation not as effective)
Very impressive and effective grasping technique in general
Creative but computationally and conceptually simple
Next steps?
Would be interested to see how it performs on heavier and more irregularly shaped objects
Practicality?
Gentle grip has potential for complicated robot tasks

20. Video