4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch1 Interactive Multiresolution Modeling and 3D Painting with a Haptic Interface Arthur D. Gregory, Stephen A. Ehmann, Ming C. Lin {gregory, ehmann, Brought to you by :

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch2 Abstract 1) Intuitive 3D interface 2) Interactive multiresolution edit of polygonal mesh 3) “Directly” painting on surface 4) Haptic feedback result  create interesting and complex 3D objects ! Rooster created with inTouch

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch3 Motivation What’s in it for us soon-to-be VR experts ? We need models! humans objects places

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch4 Motivation What’s in it for us soon-to-be VR experts ? © 1996 Brian Heston Virtual painting the next killer app ? Maybe not, but I still think its cool! We need models!

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch5 Motivation What’s in it for us soon-to-be VR experts ? We need models! Virtual painting the next killer app ? Maybe not, but I still think its cool! Haptics state of the art technologies Phantom H-collide

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch6 3D painting How it all began… –Pat Hanrahan and Paul Haeberli, ‘Direct WYSIWYG Painting and Texturing on 3D Shapes’ (SIGGRAPH 1990) [HH90] [HH90] allows painting in object space, but… Model must be divided into micro-polygons Change colors and material per vertex Painting is done in 2D parameter space

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch7 3D painting a la inTouch Use texture maps instead of per-vertex colors Use a 3D “brush function” instead of 2D inTouch allows direct painting onto object space…

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch8 3D painting a la inTouch For each triangle within the brush radius… –Compute the edge equations in texture space –Use them to define a plane equation in object space –Incrementally find 3D position of each texel while rasterizing –Apply brush function at this position and modify the texel  direct painting into texture map… (u,v) (x,y,z) triangle in object spacetriangle in texture space

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch9 Multiresolution Mesh Editing Multi-level model representation related by Loop subdivision rules Mesh editing by dragging with force feedback at any of the levels Two deformation styles - slide and stick

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch10 Haptics hardware Sensable Technologies PHANTOMsSensable Technologies –3 or 6 DOF force feedback device

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch11 Interfacing with PHANTOM GHOST toolkit H-Collide library –collision detection between a haptic probe and environment –need 1000Hz updates! –Hierarchical scene representation –Uses frame-to-frame coherence

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch12 Haptic Display Mesh modification –Triangle, movement vector, magnitude from H-Collide –Compute new position of triangle and use subdivision rules to update finer mesh levels –Pass new mesh to H-collide and compute coarser mesh levels by averaging force

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch13 Haptic Display Painting –H-collide establishes point of contact of probe with model and computes the force –Probe becomes brush with specified color, radius and fall-off –Push harder  increase brush radius, deposit more paint

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch14 User Interface 2D menu at edge of 3D scene Use PHANTOM stylus as mouse Pear by Gentaro Hirota

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch15 Prototype system IR2 as graphics server PC as haptics server VRPN for communication Display on PIT or monitor (not 3D)

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch16 Conclusions User comments: –Easy to modify globally or in detail at a point –Intuitive way of “sculpting” a 3D model –Good haptic feedback –Intuitive way of painting the model directly Fox, by Sarah Hoff (starting model = white icosahedron)

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch17 Results “Garl” - Observe the deformation Young Silvia the butterfly, 80k triangles Sylvia painted with inTouch Models and painting by Arthur Gregory

4/19/2000Deepak Bandyopadhyay / COMP239 / inTouch18 Future Work –Two handed, multi-user, … –More interesting brush functions… –More interesting force feedback... –6 DOF... –Paint in “real” 3D space (my project) Potentially moving real object Moving paintbrush