High Resolution Avatars from 3D Body Scans David Bruner, [TC] 2 Siggraph 2008 Booth #136

1999 Levis San Francisco 2000 Lands End Mobile Tour [TC] 2 History of Major Body Scanning Events [TC] 2 Scanner NX-12 -Volume Deployments 2007 NX-16 Scanner 2008 ImageTwin Real Time Avatars

NX-16 Scanner - Changing Room Size Body Scanner - 5X4 feet Safe white light Stand at floor level Enhanced privacy –In changing room area –Self-operated scanning mode

Scan time 7 seconds 3D Point Cloud 12 seconds Landmark detection, body segmentation and data reduction 16 seconds Measurement 3 seconds waist = 37.7-in hip = 42.8-in seat = 41.8-in thigh = 22.7-in knee = 15.0-in sideseam = 39.7-in inseam = 29.5-in crotch length = 26.0-in collar = 16.7-in front neck to waist = 20.2-in back neck to waist = 20.8-in cross shoulder = 19.1-in chest = 46.2-in cross chest = 16.3-in cross back = 16.0-in sleeve length = 33.8-in Instructions and Light Optimization 20 sec Total Duration 60 seconds Avatar Mesh 2 seconds

Digital Humans (Avatars) from 3D body scans Body Scanning enables the possibility of high quality Avatars that actually look like the consumer

Digital Humans (Avatars) from 3D body scans Virtual Communities – Social Networks Private Environment – personal Virtual Dressing, Virtual Fitness Assessment Entertainment Networks – multi-player gaming, content creation Business Networks –Online Conferences –Speeches –Job Interviews –Product Research Industrial/Technical Applications -Ergonomics

Allen, Curless, Popovic – Siggraph 2003 University of Washington

Fitting High Resolution 3D scans to a reference mesh Consistent data organization (polygon count and organization identical to reference mesh) for all individuals Reference mesh pre-made to be ideal for human shape

Allen - weighted optimization problem with 3 component objective function Three components of objective function: 1. Data Error 2. Smoothness Error 3. Marker Error Allen proposes solving in 4 stages in which involves updating the objective function component weighting and mesh resolution (from low to high) to avoid local minima in solution convergence and reduce solution time. Processing Time ([TC] 2 implementation of Allen solution) ~ 1 hour per body scan (700,000 scan data points, 30,000 vertex template mesh) ~ Allen processes 200 scans and creates new meshes using Principal Compoment Analysis using this reference population

Long time vs Real time [TC] 2 – new direct transformation using markers only ~ 2 seconds Key Similarity – morphing a pre-existing template mesh to 3D scan data Key Difference – Allen uses sparse (70) manually placed landmark markers vs. [TC] 2 dense (3,000) automatic landmark markers Key Difference – [TC] 2 easy integration of any reference mesh with wide ranging mesh densities (~ 4K – 80K vertices) Key Difference – [TC] 2 inclusion of joints/skeleton structure in the skin transformation

Automatic Dense (~3000) Geometry Markers ~ 1 second Landmarks intelligently placed based on Body Geometry-Shape Features

Reference Mesh – Scan Subject

Basic idea – drag template mesh to the 3D scan geometry using a common distribution of dense landmarks as the guide

Landmark to Mesh Relationship on Template Mesh

Distance Based Weighting

For b references over N vertices with affine combination constraint Weight Formulation (Pre-computed) *Similar to James, Twigg Skinning Mesh Animations, Carnegie Mellon

Marker Landmark (reference mesh) to Marker Landmark (body scan) Transformation Vectors T j = l ref – l scan j landmark markers j~ 3000

Updated Vertex Locations X i = X i ref + T

Accuracy

Template Mesh Resulting Avatar

All Shapes and Sizes

Morph Skeleton and Joints along with Skin

Virtual Dressing without draping– texture garments No additional processing time

Augmented Morphing - Virtual Fashion without Draping

Face Texture from Photo

Face Texture from Photo Combining image warping with 3D Mesh Morphing Image Features 10 clicks < 1 minute Scan Features 10 clicks < 1 minute

X3D H-ANIM -Any format template (mesh, skeleton/joints, texture), only vertex locations are updated from the template mesh (polygon count and organization unchanged) -For now – VRML, X3D, and.obj

X3D H-Anim Using Flux Studio – Flux Player from Vivaty

Obj Animation in Maya

Simulated Humans using PCA Need reference population of scan data Personal Modifications (weight loss, fitness enhancement, etc) Synthetic humans using PCA and limited data on the individual (height, weight, few measurements).

Scan Avatars + Principal Components Weight loss/gain Simulation Motivator lbs Current +20 lbs +55

Principal Components – Weight loss

Initial and 20 lb loss Predicted vs Actual

You possess and control your encrypted 3D data Use it where you want Patent Pending Data network architecture

Currently 25 locations around the world