Virtual painting project By: Leetal Gruper Tsafrir Kamelo Supervisor: Michael Kolomenkin Advisor from 3DV systems: Sagi Katz

The following project introduces a new type of interface based on the user's hand and body movements as captured by the novel 3DV systems Zcam™ Introduction

Project goal User will be able to paint on a virtual surface The program is intended for small children The user will stand in front of the camera, paint with any part of their body they wish, and see the painting on the computer screen

System description The system consists of the 3DV systems Zcam™ and a program that runs on windows The program includes inside: –Ogre – a 3D engine. –Depth camera interface. –Upper body tracking interface.

The virtual paint software

The program flow The user stands in front of the Zcam™ Selects a color to paint Starts painting with every part of his body that cross the depth threshold

The program Starts with a configuration dialog to setup

Modes of operation Painting mode Color selection mode: –Red, green and blue –Cyan, magenta and yellow –White, orange and brown The user can paint in both modes. The only difference is in the interface presented on the screen

First color selection screen

Second color selection screen

Third color selection screen

Passing between modes To move from color selection mode to the painting mode, one needs only to select a color or the eraser To move from the painting mode to the color selection mode, one should just press the arrow button To move between different color options, one needs to press the 2 arrows appearing on each side Press the

Passing between painting and color selection select a color or the eraser Press the Press the Press the Press the

Fine resolution painting An entire hand is to crude for fine painting Need for doing some pre-processing on the input depth image

Fine resolution painting cont. The algorithm we finally used was based on the assumption that we mainly paint homogenous objects The number of layers removed (which translate to a more or less finer resolution) is controlled from the keyboard numbers. 9 is the finest resolution possible and 1 is not removing layers at all

The algorithm 1.Scan the frame in an interlace order 2.For every pixel in depth range 1.Has at least one pixel from 4 surrounding neighbors that are at a certain distance that isn't in our depth range? 1.Don’t paint the pixel 2.Else 1.Paint the pixel

Conclusions We created software that uses an RGBD camera as an input interface and creates a fun game intended for children This project was our first encounter with computer graphics We have acquired knowledge in basic computer graphics We have also learned how to use a rendering platform as OGRE to create all our graphical needs For the students that will follow us and want to use OGRE, we have created a tutorial comprising all our knowledge

Thank you Michael and Sagi for all your help and guidance