1. Dimitris Valeris Thijs Ratsma
Synthesis of Detailed Hand Manipulations Using Contact Sampling
Yuting Ye & C. Karen Liu [2012]
Dimitris Valeris
Thijs Ratsma

2. Table of contents Introduction Related work Short overview
Technical explanation
Results
Limitations
Future development
Questions/discussion

3. Introduction
Problems
Subtle realism
Tracking body + hands

4. Introduction Current solutions Manually animate hand
Capture hand separately
Wearable device

5. Introduction Proposed solution Synthesize hand manipulation Input
Full body motion + wrist movement
Object motion
Output
Set of widely varied, realistic hand motions

6. Related Work Hand motion
Grasping motions using kinematics (Koga et al. [1994]; Huang et al. [1995]; Aydin and Nakajima [1999])
Sampling
Desired joint angles (Liu et al. [2010])
Initial joint configuration (Sok et al. [2007])

7. Related Work Physical Simulation
Grasp controllers (Kry and Pai [2006])
Detailed Manipulations
Robotics (Ciocarlie and Allen [2008]; Hamer et al. [2011])

12. Technical Explanation
Search for Contact Point Trajectories
Basic algorithm:
Depth-first search on a tree structure.
At each level of the tree there are two main tasks for searching:
Generate a small set of nodes as potential expansion of the tree.
Test feasibility of a random node from set.

13. Technical Explanation
Notations
Definition of state s={q,P,F}
Additional definitions:
contact point p∈P (pair of p.o and p.h)
contact force f∈F (in Cartesian space: fC = f ⊥n+f //)
guiding configuration c (max one contact point for each finger)
guiding points, p

14. Technical Explanation
Algorithm 2

15. Technical Explanation
Generate New Nodes (Initial case)
Identify a frame in two steps:
Use the wrist motion to compute a reachable volume for each finger.
Intersect volumes with the surface of the object.
Call the intersecting areas on the surface of the object,
contact patches.
Create uniform point samples on the contact patches.
Pair each contact patch with a finger tip to form a
Guiding point

16. Technical Explanation
Generate New Nodes (General case)
Produce a set of new guiding points based on contact dynamics (static, sliding, breaking) by the following steps:
Select some seed points from
From and we can create a new guiding point

17. Technical Explanation
Generate New Nodes (General case)
Rules to create a new guiding point, for the current level:
If is within friction cone, the current contact must remain static.
If is on the boundary of friction cone, the current contact can remain static or slide.
If = 0, the finger can remain static, or release contact and move to a neighboring location.

18. Test feasibility (kinematics)
Consider a random guiding configuration to test its feasibility. c is kinematically feasible if a penetration-free hand pose can be recovered from c, by solving a sequence of IK problems.
(1) is the forward kinematic function that computes the position of in the local coordinates of the object given pose q.
outputs the normal direction of the nail on the finger that contains
The third equation prevents large change of hand poses.

19. Test feasibility (kinematics)
Penetrations are considered only on Distal and Intermediate phalanges.
Apply collision detection to these phalanges to find the pair of contacts of the deepest penetration.
Build a problem by substitute the pair into Equations 1,2.
The solution to the problem can reduce penetrations.
Iterate until to generate a penetration-free hand pose q.

20. Test feasibility (dynamics)
Test whether the contact points P can produce sufficient contact forces F.
If a feasible solution cannot be found, we consider that c fails the dynamic feasibility test.
(5)The inertial and the gravitational forces in the generalized coordinates are included in G
(6) Constrain the friction direction to be perpendicular to the contact normal.
(7) Friction within Coulomb friction cone.

21. Create Diverse Solutions
Sparse Exploration
For small window we can keep the same manipulation strategy.
To generate more variations, use a random windows size (parameter ε).
Avoid similar paths, because sibling nodes often represent similar states.
No more than 2,5 feasible paths can be created for each initial sample.

22. Create Diverse Solutions
Informed backtracking
Propagate backward the possible failures, to use this information to expand nodes with higher likelihood of success.
Check infeasibility in three different situations:
Frame fails on kinematic test for a particular finger,
Inter-penetration between two fingers,
Frame fails on the dynamics test.

23. Create Diverse Solutions
Manipulation strategy preference:
Instead of picking a random strategy, we can give high priority to a preferred manipulation strategy for each finger.
Contact force preference
The possible manipulation strategies are determined by contact forces. By adjusting the Equation 4, we can indirectly favor different strategies.

24. Reconstruct hand motion
After we obtain feasible trajectories of contact points, we apply a spacetime optimization to create smooth hand animations with respect to the contact points.
E1 encourages smooth change of joint angles by minimizing acceleration (Eq. 10).
E2 favors joints on the same finger having similar bending angles (Eq. 11)

25. Reconstruct hand motion
Fingers in contact
Fingers not in contact
Attaching hands to wrists of character

26. Results Performance Kinematic test: 200ms Dynamic test: 5-10ms
Variable parameters

27. Demonstration

28. ε T
Time (sec)
Solution
Dead end
Total frame
Success frame
Success ratio
Solution ratio
Milk box (turning) 10 71
834
241
2710
661
24.4%
4.0%
Milk box (pick up) 20
101
917
2208
26588
974
3.7%
0.45%
Spatula
226
1323
285
6324
3735
59.1%
6.6%
Pot (with wine)
120
636
340
3235
1144
35.4%
2.9%
Two-hand
(small box) 15
141
666 35
273
8833
4901
55.5%
11%

29. Evaluation Comparison to close-range motion capture data
Frequent contact movements and complex contact relations
Path variability

31. Limitations Rigid hand model Independant finger movements
No collision with environment
Sliding friction constraints
Plenty of other possible solutions

32. Limitations
Kinematic test
Small objects
Limited planning capability

33. Future development Grasp planning in robotics (Song et al [2016])
Virtual grasp synthesis (Linssen [2018])

34. Questions?

35. Discussion
Aside from robotics and as input for grasp controllers, what could be other useful applications for this technique?
We already proposed some improvements that arose from the limitations, what other improvements to the system could be made?
Do you think this technique has such a significant impact on the animation that it can replace existing approaches?