1. ETHAN BROWN / KATE RAFTER
DIGITAL DANCING
ETHAN BROWN / KATE RAFTER

2. What I learned…

3. SEEING THE WORLD IN THREE DIMENSION
Humans are good at understanding depth from a multitude of cues:
DEPTH
PARALLAX
PERSPECTIVE
RELATIVE
SIZE
FAMILIAR
SIZE
STEREOPSIS
OCCLUSION
CONVERGENCE
MANY MORE….
Computers: not so much.

4. INFRARED DEPTH MEASUREMENT: A CLEVER CHEAT
Infrared laser projects a pattern of infrared dots
Depth inferred from multiple cues
Kinect uses proprietary technology developed by PrimeSense

5. HARDWARE
Microsoft
Occipital
KINECT 1
KINECT 2
STRUCTURE SENSOR

6. SKELETON TRACKING Complicated problem
Uses techniques from artificial intelligence, machine learning, and computer vision
Requires massive amounts of training data

7. SOFTWARE TOOLKITS OpenFrameworks (C++) Cinder (C++)
Kinect .NET SDK (C#)
Point Cloud Library (C++)
OpenNI (C++)
OpenCV (C++)

8. Edge.js makes Kinect SDK available THREE.js / Canvas 3D / CubicVR
NODE / JAVASCRIPT
Edge.js makes Kinect SDK available
THREE.js / Canvas 3D / CubicVR
Distinct lack of libraries (OpenCV, PCL)

9. DISPLAYING POINT DATA OpenFrameworks / OpenGL
Kinect provides world coordinates; store as vectors (ofVec3f)
Convert point cloud vectors to mesh (ofMesh)
Some effects “accidental” artifacts of mesh rendering

10. Plane detection with RANSAC algorithm, courtesy PCL
FLOOR REMOVAL
Plane detection with RANSAC algorithm, courtesy PCL
Hooray math! Point/plane distance
Expensive filtering: optimization pays

11. “EXPLODING” SHADOWS Easy centroid calculation:
Subtract centroid from each point and normalize: unit vector away from centroid
Scale unit movement vector and add to point to move away from centroid

12. KATE RAFTER ON TECHNOLOGY AND PERFORMANCE

13. LEARNING MORE Trigonometry (Kahn Academy)
Linear Algebra (Kahn Academy)
The C++ Programming Language, 4th Edition
Making Things See by Greg Borenstein
Books on OpenFrameworks and Cinder from Pact Publishing

14. ONWARD…. Projector/real-world correspondence
Identify real-world features (with depth and/or visual data)

15. THANK YOU www.popart.com zepln.com/wdne ethan.brown@popart.com
@EthanRBrown
zepln.com/wdne