1. Putting the Turing into Manufacturing: Algorithmic Automation and Recent Developments in Feeding and Fixturing Ken Goldberg, UC Berkeley IEEE ICRA 2007 Plenary: Rome, Italy

2. thanks to: matt mason, frank van der stappen, ruzena bajcsy, john canny, howard moraff, mike erdmann, russ taylor, george bekey, ari requicha, randy brost, yu wang, mike peshkin, bud mishra, kevin lynch, srinivas akella, alan christiansen, barak perlmutter, bruce donald, karl bohringer, lydia kavraki, dan halperin, mark overmars, hugh durrant whyte, anil rao, jeff wiegley, rick wagner, hadi moradi, brian carlisle, john craig, steve holland, elon rimon, mike zhang, mark moll, dez song, ron alterovitz, allison okamura, peter luh, dick volz, hubert dreyfus, eric paulos.

3. Alan Turing (1912 – 1954)

4. Turing Machines Precise vocabulary: 0, 1 Class of primitive operations: Read Write Shift Left Shift Right Well Formed Sequences Correctness Completeness Equivalence Complexity

5. Assembly Lines

6. Put the Turing into Manufacturing Automation Algorithmic Part Feeding –2D Polygonal Parts –3D Polyhedral Parts –Traps –Blades Algorithmic Fixturing –Modular Fixtures –Unilateral Fixtures –D-Space and Deform Closure Related Work and Open Problems

7. Robotics and Automation Both involve: computers, physical world, geometry Both engage many disciplines “ robota” coined in 1920 (Capek) –Emphasizes unpredictable environments like homes, undersea “ automation” coined in 1948 (Ford Motors) –Emphasizes predictable environments like factories, labs roboticsautomation

8. Emphasis on efficiency, quality, productivity, and reliability New Applications and Methods Central to the IEEE RA Society Flagship journal (T-ASE), Flagship conference (CASE) Attracting leading researchers from Automation www.ieee.org/t-ase

9. Status Quo in Manufacturing Design is a “Black Art” Part Geometry is Known Motions are Repetitive Design changes Intermittently Criteria: Cost Throughput Reliability

10. Algorithmic Automation: Define Admissible Inputs Define Admissible Operations Output: all solutions or negative report Complexity as function of input size

11. Two Problems in Manufacturing: Part Feeding Part Fixturing and Holding

12. Putting the Turing into Manufacturing Automation Algorithmic Part Feeding –2D Polygonal Parts –3D Polyhedral Parts –Traps –Blades Algorithmic Fixturing –Modular Fixtures –Unilateral Fixtures –D-Space and Deform Closure Related Work and Open Problems

13. 50% of assembly setup time goes into Parts Feeding. (nevins and whitney, 78) Part Feeding (Orienting)

14. Vibratory Bowl Feeders Effective for over a century. However: Set Up Time Design of Tracks is a Black Art Part Damage, Contamination Bulky, Noisy Cost: $5-20K

15. Goal: A Universal “Turning” Machine

16. Related Work: Sensorless Manipulation Pushing (Jaws, Fences, Robot arms,…) –Mason, 1986 –Mason & Erdmann, 1988 –Peshkin & Sanderson, 1988 –Akella & Mason, 1992 –Brokowski, Peshkin & Goldberg, 1995 –Lynch & Mason, 1996 –Wiegley, Goldberg, Peshkin & Brokowski, 1997 –Berretty, Goldberg, Overmars & van der Stappen, 1998 –Akella, Huang, Lynch & Mason, 2000 Squeezing (Parallel jaw grippers) –Goldberg, 1993 –Chen & Ierardi, 1995 –Moll, Erdmann, Fearing & Goldberg, 2002

17. Sensorless Manipulation Toppling –Lynch, 1999 –Zhang, Smith, Berretty, Overmars & Goldberg, 2000 Pulling –Berretty, Goldberg, Overmars & van der Stappen, 2001 Tapping –Huang & Mason, 1998 Dropping –Kriegman, 1997 –Moll & Erdmann, 2002

18. Sensorless Manipulation Wobbling –Erdmann, Mason & Vaněček, 1993 Rolling –Marigo, Ceccarelli, Piccinocchi, & Bicchi, 2000 Vibrating (Vibratory surfaces, force fields) –Reznik, Canny & Goldberg, 1997 –Luntz, Messner & Choset, 1998 –Bohringer, Bhatt, Donald & Goldberg, 2000 –Böhringer, Donald, Kavraki, Lamiraux, 2000

19. Modular Feeder Hardware –Maul & Goodrich, 1983 –Jaumard, Shi-Hui-Lu & Sriskanarajah, 1990 –Goldberg and Canny, RISC, 1994 –Jonega & Lee, 1997 –Lee, Lim, Ngoi, & Tan, 1993 & 1998 –Tay, Chua, Sim & Gao, 2004 –Cullinan & Phelan, 2004 Cullinan & Phelan

20. Modeling Part Behavior –Boothroyd, Redford, Poli & Murch, 1972 –Mirtich & Canny, 1995 –Mirtich, Zhuang, Goldberg, Craig, Zanutta, Carlisle, Canny, 1996 –Rosario & Hernández-Coronas, 1997 –Huang, 2003 –Khakbaz-Nejad & Maul, 2003 –Cordero, 2004 Abigail Cordero

21. Simulation of Feeders Vibratory bowl & Devices –Jakiela & Krishnasamy, 1993 –Berkowitz & Canny, 1996 –Maul & Thomas, 1997 –Jiang, Chua & Tan, 2003 –Silversides, Dai & Seneviratne, 2005 –Selig & Dai, 2005 Jiang, Chua & Tan

22. Heuristic Design of Feeders Computer-aided framework –Lim, Ngoi, Lee, Lye, & Tan, 1994 Genetic algorithms –Christiansen, Edwards, Coello, 1996 –Edwards, 2004 Visualization configuration space –Caine, 1994 Edwards

23. Algorithmic Part Feeding with the Parallel-Jaw Gripper

29. Can this approach be generalized to arbitrary polygons?

30. First, a Mechanical Problem:

31. Solution: Kinematically Yielding Gripper ( US Patent 5,098,145)

32. Problem Statement Given a list of n vertices describing a planar part. Find the shortest sequence of gripper actions guaranteed to orient the part or report that no such sequence exists. Assumptions: Contacting surfaces are frictionless. All motion in the plane. Part is rigid. Inertial forces are negligible. Orients up to symmetry in convex hull

33. Algorithm: Width Function

34. Algorithm: step function

35. Algorithm: backchaining preimages

36. Example: computing S-intervals

37. Example: resulting 3-step plan

39. An Implementation in JavaImplementation (boo and goldberg, 2005) Feeding Polygonal Parts without Sensors http://goldberg.berkeley.edu/java-applets.html

40. Theorem (Completeness): A sensorless plan exists for any polygonal part. Theorem (Correctness): The algorithm will always find the shortest plan. Theorem (Complexity): For a polygon of n sides, the algorithm runs in time O(n 2 ) and finds plans of length O(n). Extensions: Stochastically Optimal Plans Extension to Non-Zero Friction Geometric Eccentricity and constant time result (van der Stappen) Pulling with point jaws inside concavities, Sorting with wedges

41. Algebraic parts (Rao)

42. fences on conveyor belt Peshkin, Sanderson, Wiegley, Brownowski, Mason, Akella, Lynch, Huang, Beretty, Overmars, Van der Stappen, De Berg

43. MEMS and other arrays [BDMM94; LMC2001; MB2004] Flexible vibrating plates [C1787; BBG95] Universal parts feeding [BDKL99; LK2000] Vibration of rigid 3-DOF plate [RC98] Rigid 6-DOF plate [VUL2007] Planar Force Fields Tom Vose, Paul Umbanhowar, Kevin Lynch, Northwestern U.

45. Putting the Turing into Manufacturing Automation Algorithmic Part Feeding –2D Polygonal Parts –3D Polyhedral Parts –Traps –Blades Algorithmic Fixturing –Modular Fixtures –Unilateral Fixtures –D-Space and Deform Closure Related Work and Open Problems

46. Feeding 3D Polyhedra

47. Pivoting grasps (using gravity)

48. Algorithm to compute m x m Transition Matrix in time O (m 2 n log n) (with Rao and Kriegman)

49. Adept FlexFeeder (1995) Estimating Pose Statistics (1996) Tuning Belt Velocities (1997)

50. Estimating Part Pose Statistics 12345 Orange Insulator Cap (1036 Physical Drop Tests)

51. Feeding 3D Polyhedra with Fences Inclined plates with fences (Berretty, Overmars, van der Stappen)

52. Putting the Turing into Manufacturing Automation Algorithmic Part Feeding –2D Polygonal Parts –3D Polyhedral Parts –Traps –Blades Algorithmic Fixturing –Modular Fixtures –Unilateral Fixtures –D-Space and Deform Closure Related Work and Open Problems

53. Design of Bowl Feeders “Bowl toolers are artists [...] creativity plays an important role” Boothroyd, 2005

54. Algorithmic Trap Design Berretty, Van der Stappen, Goldberg, Cheung, Levandowsky, Smith, Overmars, Goemans Input: Polygonal Part. Output: Polygonal Trap

55. Trap Design: Static Analysis P is safe if there are three supported points t 1, t 2, and t 3 in P such that the Center of Mass (CoM) lies inside the triangle t 1 t 2 t 3. R P x

56. Static Analysis Lemma: P is safe if CoM lies in CH(S). CH(S)S Define: Support S: area of part supported by the track.

57. Static Analysis Theorem: Checking if P is safe takes time O((n+m+k) log (n+m)), where k is the number of intersections between the part and the trap. Proof –Compute the vertices of P that do not lie in R: O((n+m) log (n+m)) –Compute the intersections between the boundaries of P and R, then take convex hull. –Check whether the CoM lies inside it: O(n+m+k).

58. Translating Part Analysis Convex Part and Convex Trap Events occur only when: –A vertex of P crosses an edge of R, or vice versa: O(n+m) events. –Each event update: O(log (n+m)) time. –Analysis takes time: O((n+m) log (n+m)).

59. Complete algorithms : Trap Design –Berretty, Goldberg, Overmars & van der Stappen, 1999 & 2001 –Agarwal, Collins, Harer, 2001 –Goemans, Levandowski, Goldberg & van der Stappen, 2005 Berretty et al. Agarwal et al. Complete design Complexity bounds

60. Toppling (with K. Lynch, M. T. Zhang)

61. Toppling Shortest sequence of pins and their heights [Zhang, Lynch, Goldberg, Smith, Berretty, Overmars 01].

62. O. Goemans, A.Frank van der Stappen, K. Goldberg, Marshall Anderson Blades: A Geometric Primitive for Feeding 3D Parts on Vibratory Tracks

63. New Primitive: Blades

64. Blades Combines reorientation and rejection functionality

65. Algorithm blade angle  blade width w blade height h INPUT Polyhedral part P & Center of mass C OUTPUT Set of blades b( ,h,w)

66. Assumptions Parts –Rigid and identical polyhedra –Quasi static motion –Singulated Zero friction No toppling Locally linear track

67. Part Geometry

68. Stable Blade Poses Fixed orientation –Floor contact –Blade contact

69. Part Rejection Rejection –Blade width

70. 3D “Blade Space” Blade width Blade height Blade angle  h A w Point represents Blade Surfaces subdivide space

71. Criticalities in Blade Space Blade width Blade height Blade angle h w crit  One track pose One blade height and angle Critical blade width

72. Critical Surface in Blade Space Blade width Blade height Blade angle Critical surface for Part Pose i All Blades above Critical Surface reject Part Pose i S

73. Intersection of Critical Surfaces Blade width Blade height Blade angle Set of solutions : All points above all but one critical surface or more formally : 1-level of arrangement of critical surfaces P1P1 P2P2 P3P3 B

74. 1-Level Arrangement Blade height Blade angle Blade width

75. 1-Level Arrangements Blade height Blade angle Blade width Complexity bounds O(n 6 ) by Sharir ´93, Elegant algorithm by Agarwal, Schwarzkopf & Sharir ´96 Divide & Conquer, Vertical Decomposition

76. Algorithm Input: –CAD model of polyhedral n-sided part Output: all blades that feed part Time complexity: O(n 6 )

77. Hardware Prototype

79. Putting the Turing into Manufacturing Automation Algorithmic Part Feeding –2D Polygonal Parts –3D Polyhedral Parts –Traps –Blades Algorithmic Fixturing –Modular Fixtures –Unilateral Fixtures –D-Space and Deform Closure Related Work and Open Problems

80. Bulky Complex Multilateral Dedicated Expensive Long Lead time Black Art Fixtures

81. 3L1C [Brost & Goldberg 96] Vises [Wallack & Canny] 4C [Wentink 97] Linear elements [WSO] 3D: Ponce and others Modular Fixturing

82. Form closure [Reuleaux 1876] four points sufficient [MSS 87, MNP 90] 2nd-order immobility [RB 98] three points sufficient synthesis of all fixtures [SWO 00] Form closure and immobility

83. Modular fixturing toolkits Reusability Amenable to Analysis T-slot locator (L) clamp (C)

84. Modular fixtures: synthesis

86. Problem Statement Given a list of n vertices describing a planar part. Find all combinations of 3 pins and one clamp guaranteed to hold the part in form closure or report that no such combination exists. Assumption: Contacting surfaces are frictionless.

87. Case study: glue gun

89. a b

90. b l max l min a

91. l max x y

93. Java Implementation: FixtureNet (brost, wagner, goldberg, 1996) http://goldberg.berkeley.edu/java-applets.html

94. complete algorithm for fixturing given polygonal part with n edges (d diameter in lattice units): computes all feasible fixture arrangements In O(n 6 d 6 ) brost, goldberg, bekey, requicha, mishra, wagner, overmars, van der stappen

95. Negative existence result: we can construct an infinite set of parts that cannot be fixtured on the lattice (1994).

96. 4 locators 3 locators, 1 clamp 4 clamps all polygonal parts without parallel edges, regardless of size. All parts can be held by four fingers on two perpendicular lines! [Zhuang & Goldberg 96] Other existence results (van der stappen)

97. Gripper Design (with Mike Tao Zhang)

98. Putting the Turing into Manufacturing Minimalism Algorithmic Part Feeding –2D Polygonal Parts –3D Polyhedral Parts –Traps –Blades Algorithmic Fixturing –Modular Fixtures –Unilateral Fixtures –D-Space and Deform Closure Related Work and Open Problems

99. “Unilateral” Fixtures for sheet metal parts with holes coaxial cone contacts

100. Fixture Locator Optimization the precise localization objective dependent only on locators position best locator solution for optimal localization observations: clustering, symmetry (M.Y.Wang, et al.) (Zhu and Ding)

101. Putting the Turing into Manufacturing Automation Algorithmic Part Feeding –2D Polygonal Parts –3D Polyhedral Parts –Traps –Blades Algorithmic Fixturing –Modular Fixtures –Unilateral Fixtures –D-Space and Deform Closure Related Work and Open Problems

102. Elastic Fingers, Soft Contacts [Hanafusa and Asada, 1982] [Salisbury and Mason, 1985] [Kim, Hirai, Inoue, 2003] Physical Models [Joukhadar, Bard, Laugier, 1994] Bounded Force Closure [Wakamatsu, Hirai, Iwata, 1996] Learned Grasps of Deformable Objects [Howard, Bekey, 1999] Robot manipulation [Henrich and Worn, 2000] Robust manipulation with Vision [Hirai, Tsuboi, Wada 2001] Related Work: Holding Deformable Parts

103. Deformable Parts Path Planning for Elastic Wires, Sheets and Bodies [Kavraki et al, 1998, 2000] [Amato et al, 2001] [Moll and Kavraki, 2004]

104. Inspiration FEM for Surgery simulation:

105. Deformable parts “Form closure” based on immobility For deformable parts, how to define immobility? The part can always escape:

106. Introduce FEM Mesh Define “C-Space” based on node displacements Characterize potential energy Idea [with K. “Gopal” Gopalkrishnan]

107. Deformation Space (D-Space) Each node has 2 DOF Analogous to configurations in C-Space D-Space: 2n-dimensional space of node positions. point q in D-Space is a “deformation” q 0 is initial (undeformed) point (30-dimensional D-space)

108. D-Space Example: 2 fixed nodes 1 moveable node: 2 dimensional D-Space x y Physical space D-Space q0q0

109. D T : Topology Preserving Subspace x y Physical space D-Space D T  D-Space. DTDT DTC:DTC:

110. D-Obstacles x y Physical space D-Space Like C-Obstacles, a physical obstacle A i defines a deformation- obstacle DA i in D-Space. Collision of any mesh element with obstacle. A1A1 DA 1

111. D-Space: Example Physical space x y D-Space Like C free, we define D free. D free = D T  [  (DA i C )]

112. Potential Energy Assume Linear Elasticity, Zero Friction K = FEM stiffness matrix. Forces at nodes: F = K X. Potential Energy: U(q) = (1/2) X T K X

113. Potential Energy Needed to Escape from a Stable Equilibrium Consider: Stable equilibrium q A, Equilibrium q B. “Capture Region”: K(q A )  D free, such that any configuration in K(q A ) returns to q A. Saddlepoints [Rimon, Blake, 1995] q A qBqB q U(q) K( q A )

114. where U is at a strict local minimum: U A (q A ) = Increase in Potential Energy needed to escape from q A. = minimum external work needed to escape from q A. U A : Quality Measure q A qBqB q U(q) UAUA K( q A ) Define: “Deform Closure” Grasps

115. U A : Example U A = 4 JoulesU A = 547 Joules

116. Putting the Turing into Manufacturing Automation Algorithmic Part Feeding –2D Polygonal Parts –3D Polyhedral Parts –Traps –Blades Algorithmic Fixturing –Modular Fixtures –Unilateral Fixtures –D-Space and Deform Closure Related Work and Open Problems

117. Synthesis of All Grasps (van der Stappen, Overmars) Polygonal parts –Form closure Three and four fingers in roughly O(n 2 ) time [van der Stappen et al 00] [Cheong et al 03] Two fingers in O(n 4/3 ) time [Cheong et al 03] –2nd order immobility Three fingers in roughly O(n 2 ) time [Cheong et al 03] Semi-algebraic parts –Form closure [Cheong & van der Stappen 06] Four fingers in O(n 8/3 ) time Three fingers in in O(n 5/2 ) time …

118. Caging (Rimon, Blake, van der Stappen, Overmars) Fingers prevent to take part to an arbitrary position even though it may be possible to move about [Kuperberg 90] Characterization of solution set [Rimon & Blake 95] Synthesis of two-finger caging grasps of polygons –In O(n 2 log n) time [Sudsang & Pipattanasomporn 06] [Vahedi & van der Stappen 06] Synthesis of three-finger caging grasps of polygons –Convex polygons in O(n 6 ) time [Erickson et al. 03] –Non-convex polygons in O(n 6 log n) time [Vahedi & van der Stappen 06]

119. Sensorless Parts Sorting (de Berg, van der Stappen, Overmars) Sorting scenario: Forking conveyor belt sorts parts of types P and Q onto different sub-belts [de Berg et al. 05]. Algorithmic foundation: Optimal push plan achieving stable orientations φ and ψ for P and Q with n vertices each computed in O(n 4 log 2 n) time. Orienting with fences. Generalizes to k parts.

120. Form-closure fixturing of an assembly [WT94; BMK97; CGOvdS2002] Fixtures against transport forces [BMK97] Pushing assemblies [BL2005; HNMK2006] Time-optimal transport: “the waiter’s problem” [BL2004,6; Shiller89] Transport of Assemblies Jay Bernheisel, Kevin Lynch, Northwestern U.

121. Algorithmic Automation: Define Admissible Inputs Define Admissible Operations Output: all solutions or negative report Complexity as function of input size

122. Open Problems in Algorithmic Automation –feeding, fixturing –tangling –tolerancing –assembly line layout –redesign parts for manufacture

123. Grand Challenge: 1770: Interchangeable Parts 1910: Assembly Lines 20??: Algorithmically Configuring Assembly Lines from Interchangeable Parts

124. Factories are Fun

125. Putting the Turing into Manufacturing Automation Algorithmic Part Feeding –2D Polygonal Parts –3D Polyhedral Parts –Traps –Blades Algorithmic Fixturing –Modular Fixtures –Unilateral Fixtures –D-Space and Deform Closure Related Work and Open Problems

126. Thank You. Putting the Turing into Manufacturing: Algorithmic Automation and Recent Developments in Feeding and Fixturing Ken Goldberg, UC Berkeley goldberg@berkeley.edu http://goldberg.berkeley.edu

128. Group Elevator Scheduling with Advance Information Conventional Group Elevators With Destination Entries Luh, Xiong, and Chang, UConn

129. Kineo (jean paul laumond, nic simeon)