ETHAN BROWN / KATE RAFTER DIGITAL DANCING ETHAN BROWN / KATE RAFTER
What I learned…
SEEING THE WORLD IN THREE DIMENSION Humans are good at understanding depth from a multitude of cues: DEPTH PARALLAX PERSPECTIVE RELATIVE SIZE FAMILIAR SIZE STEREOPSIS OCCLUSION CONVERGENCE MANY MORE…. Computers: not so much.
INFRARED DEPTH MEASUREMENT: A CLEVER CHEAT Infrared laser projects a pattern of infrared dots Depth inferred from multiple cues Kinect uses proprietary technology developed by PrimeSense
HARDWARE Microsoft Occipital KINECT 1 KINECT 2 STRUCTURE SENSOR
SKELETON TRACKING Complicated problem Uses techniques from artificial intelligence, machine learning, and computer vision Requires massive amounts of training data
SOFTWARE TOOLKITS OpenFrameworks (C++) Cinder (C++) Kinect .NET SDK (C#) Point Cloud Library (C++) OpenNI (C++) OpenCV (C++)
Edge.js makes Kinect SDK available THREE.js / Canvas 3D / CubicVR NODE / JAVASCRIPT Edge.js makes Kinect SDK available THREE.js / Canvas 3D / CubicVR Distinct lack of libraries (OpenCV, PCL)
DISPLAYING POINT DATA OpenFrameworks / OpenGL Kinect provides world coordinates; store as vectors (ofVec3f) Convert point cloud vectors to mesh (ofMesh) Some effects “accidental” artifacts of mesh rendering
Plane detection with RANSAC algorithm, courtesy PCL FLOOR REMOVAL Plane detection with RANSAC algorithm, courtesy PCL Hooray math! Point/plane distance Expensive filtering: optimization pays
“EXPLODING” SHADOWS Easy centroid calculation: Subtract centroid from each point and normalize: unit vector away from centroid Scale unit movement vector and add to point to move away from centroid
KATE RAFTER ON TECHNOLOGY AND PERFORMANCE
LEARNING MORE Trigonometry (Kahn Academy) Linear Algebra (Kahn Academy) The C++ Programming Language, 4th Edition Making Things See by Greg Borenstein Books on OpenFrameworks and Cinder from Pact Publishing
ONWARD…. Projector/real-world correspondence Identify real-world features (with depth and/or visual data)
THANK YOU www.popart.com zepln.com/wdne ethan.brown@popart.com @EthanRBrown k.e.rafter@gmail.com www.katerafter.com www.popart.com zepln.com/wdne