Crayon3D Team

Outline  Overview  Features  Creating a Scene  Creating Nodes  Collision Events  Using a Wiimote  Using the Helper Classes  Class Diagram  Glove and IR Glasses

Overview Crayon3D provides a set of tools allowing aspiring game developers to easily use Wiimotes as an input device Crayon3D also provides a very simple interface to the physics engine, Open Dynamics Engine (ODE) “Make development easier for BVW so students can focus more on making their worlds entertaining”

Overview  Crayon3D Classes Crayon3DScene Crayon3DNode Wiimote Handler Cursor Gesture Recognition Handler

Overview Minimum Requirements – OS: Windows XP or Higher – Bluetooth Support – Files: wiiuse.dll wiiuse_al.dll wiiuse_al.py wiimote_handler.py Crayon3DScene.py Crayon3DGesture.py

Crayon3D Platform Features  Full integration with the Wiiuse Library Support for: ○ Accessing wiimote data (buttons, accelerometer, IR camera) ○ Accessing wiimote expansion devices (nunchuk, classic controller, guitar hero 3 controller)  Includes helper classes for: Controlling and managing wiimote resources IR Point tracking as a cursor (for finger tracking) Button controls (for use with IR Point cursor) IR Point tracking for Fish Tank VR Gesture Recognition  Simple interface to the ODE library  Well-documented source code  Online API Reference  Expandable

Creating a Scene  What is a scene? A stage setting, a place of an occurrence or action Merriam-Webster Dictionary The scene is the Virtual World ○ It contains everything in the world Environment Objects

Creating a Scene To create a scene: 1. Specify the environment model and position it in the world 2. Specify the physics properties of the world a. Default – use the default Crayon3D properties b. Manual – specify properties manually 3. Specify the collision geometry of the model a. Mesh – based on the polygon data of the model b. Plane – normal plane extending to infinity c. Manual – specify specific ODE collision geometry

Creating a Scene # Load the model of the environment to be used env_model = loader.loadModel(‘environment.egg'); # Scale it appropriately and place it in the world env_model.setPos(0, 0, 0); # Create the Crayon3DScene using the environment model we just created self.scene = Crayon3DScene(env_model); # Use default physics for our scene self.scene.setDefaultPhysics(); # Set the collision geometry of the environment to the scene # Use the actual polygons of the model to create the collision geometry self.scene.setEnvGeomMesh();

Creating a Scene  Physics in the scene can be started or stopped self.scene.startPhysics() self.scene.stopPhysics()  Before starting the physics in the scene, make sure to wait for at least half a second to make sure that all objects are properly placed in the world

Creating Nodes  What are nodes? Nodes are the objects in the scene Nodes must belong to a scene Scene Object1 Node

Creating Nodes To create a node: 1. Create a node in the scene 2. Specify the model to be used for that node 3. Specify the Physics properties for the node a. Box – the node will be treated like a box b. Sphere – the node will be treated like a sphere c. Manual – specify the physics properties manually Manipulating nodes is similar to how you manipulate nodes in Panda3D. – Note that scaling and shearing the nodes is not yet supported. Calling these functions will not do anything.

Creating Nodes # Load the model to be used by the node model = loader.loadModel(‘model.egg’); # Create the new node from the scene and give it a unique name box = self.scene.newNode(‘model_name’); # Set the model of the node to the model that we just created box.setModel(model); # Set the collision geometry and physics properties of the node by # giving the length, width and height of the bounding box of the node. box.setDefaultPhysicsBox(1, 1, 1);

Collision Events  Collision events can be handled by specifying an event handler for the event: “ode-collision” self.accept("ode-collision", onCollision)  The event handler should accept an additional parameter This additional parameter will contain information about the nodes that collided To get the information, do this: (assume the additional parameter is called entry) onCollision(entry): # This is the first node crayonNode1 = self.scene.findNodeByGeom(entry.getGeom1()); # This is the second node crayonNode2 = self.scene.findNodeByGeom(entry.getGeom2());

Using a Wiimote  Connecting a wiimote to the PC requires Bluetooth connection  To connect to a wiimote using Blue Soleil: Press and hold buttons 1 and 2 simultaneously Search for nearby Bluetooth devices. You should see a Nintendo RVL-CNT-01 Connect to this device Once connected release the buttons you are holding. Do this for all wiimotes you want to use

Using a Wiimote  Note for Blue Soleil: Ensure that you unplug all wiimotes that are not being used. ○ Right click on the Nintendo RVL-CNT-01 and check if there is an unplug option If there is click it to unplug If there is none then the device is already unplugged

Using a Wiimote  Crayon3D provides several classes that will make it easier to access data from wiimotes and its possible extensions (nunchuk, classic controller, guitar hero 3 controller) Wiiuse Abstraction Layer ○ Python interface to the wiiuse library Wiimote Handler ○ Helper class for managing and accessing wiimote information

Using a Wiimote Wiimote Handler – Specify the types of wiimotes that you need WIIMOTE_TRACKER – only accepts the IR camera information of the wiimote WIIMOTE_BUTTON – only accepts button input from the wiimote (and expansion input) WIIMOTE_ACCEL – only accepts accelerometer input from the wiimote WIIMOTE_BUTTON_ACCEL – ony accepts button and accelerometer input from the wiimote (and expansion input) WIIMOTE_ALL – accept all inputs from the wiimote

Using the Wiimote  Wiimote handler can theoretically support up to 15 wiimotes simultaneously

Using the Wiimote 1. Create a wiimote handler object 2. Start the wiimote handler object and specify the wiimotes that you want to use 3. Wait for button events or get wiimote data using the accessor functions provided 4. Before the application quits, make sure to stop the wiimote handler.

Using the Wiimote  Note that the wiimote handler object (and consequently the helper classes that require it) should be started last during the initialization sequence of the application This ensures that the system is not busy when initializing the wiimotes

Using the Wiimote Handling Wiimote Button Press and Release events – Just specify an event handler for the events: WM_BUTTON_PRESSED WM_BUTTON_RELEASED The handler functions should accept three more parameters, btn, dev and index: btn tells the function which button was pressed or released dev tells the function if the button is a wiimote button or an expansion button (necessary if you’re using expansions) Index tells the function which wiimote sent the event (necessary if you’re using multiple wiimotes)

Using the Wiimote  Getting IR Information from the wiimote: IR Structure ○ dot : contains the four IR source information ○ num_dots : number of IR sources seen by the wiimote ○ x, y : Corrected XY coordinates of the wiimote with respect to the IR sources ○ z : Arbitrary Z distance of the wiimote with respect to the IR sources – a value will only be computed if there are at least two IR sources visible ○ aspect : Aspect ratio of the screen, 0 for 4:3 and 1 for 16:9 – this determines the size of the XY coordinate space (560x420 and 660x370, respectively)

Using the Wiimote The Dot structure: ○ visible : Determines if the dot is visible or not ○ x, y : Corrected XY coordinates of the IR source ○ rx, ry : Raw XY coordinates of the IR source as reported by the wiimote ○ XY coordinates for the dots are on a fixed virtual screen resolution of 1024x768

Using the Wiimote IR Structure (sample code) # Get IR structure of the first wiimote ir = wmHandler.getIRData(0); # print X and Y position of the wiimote with respect to the IR sources print ir.x print ir.y # print the arbitrary Z distance print ir.z # Get the individual dot information for i in range(4): # Print the X and Y position of the IR dot if it is visible if(print ir.dot[i].visible): print ir.dot[i].x; print ir.dot[i].y;

Using the Wiimote  Getting Accelerometer Data Orient Structure ○ roll : Contains roll rotation information of the wiimote ○ pitch : Contains the pitch rotation information of the wiimote ○ yaw : Contains the yaw rotation information of the wiimote

Using the Wiimote Orient Structure (sample code): # Get the accelerometer data of the first wiimote orient = wmHandler.getAccelData(0); # Print the roll print orient.roll; # Print the pitch print orient.pitch; # Print the yaw print orient.yaw;

Using the Wiimote  Getting Nunchuk Data Nunchuk Structure ○ js : Joystick information ang : Angle of the joystick (0 – 360 degrees) mag : Magnitude of the joystick (value is 1 if the player is pushing the joystick to the maximum, 0 if the player is not pushing the joystick) ○ orient : Accelerometer information of the nunchuk (similar to the Wiimote accelerometer information)

Using the Wiimote Nunchuk Structure (sample code) # Get the nunchuk structure connected to the first wiimote nc = wmHandler.getNunchukData(0); # Print the joystick information of the nunchuk print nc.js.ang; # angle print nc.js.mag; # magnitude # Print the accelerometer information print nc.orient.roll; print nc.orient.pitch; print nc.orient.yaw;

Using the Helper Classes  Crayon3D provides a helper class if you want to use the wiimote as an IR tracker (ie. For finger tracking) Cursor Class  In addition to the IR tracker, Crayon3D also provides a helper class for recognizing gestures drawn using the Cursor helper class Gesture Recognition Handler

Using the Helper Classes Cursor – To create a cursor for finger tracking 1. Create a wiimote handler object and start it with at least one WIIMOTE_TRACKER 2. Create a cursor object: a. Specify the images to be used by the cursor (200x200 images) – you can specify multiple images here if you want to b. Specify the wiimote handler object c. Specify the index of the WIIMOTE_TRACKER that is used to track the IR points self.wmHandler = WiimoteHandler(); self.wmHandler.start([WIIMOTE_TRACKER, WIIMOTE_BUTTON]); self.cursor = Cursor([‘cursor.png'], self.wmHandler, 0);

Using the Helper Classes  The cursor class can be switched to use keyboard and mouse input if you don’t specify the wiimote handler and tracker index self.cursor = Cursor(['textures/cursors/cursor_hand.png'])

Using the Helper Classes Gesture Handler – The gesture handler class allows the users to specify specific gestures to recognize and when these gestures are recognized, an event will be sent for notification – Gestures are specified using a pattern of numbers Look at the keypad: – 8 means up, 7 means up-left, 4 means left, 1 means down-left, etc… – 5 and 0 doesn’t mean anything For example, for a square drawn like this: up, right, down, left the pattern will be “8624” The gesture handler uses the Levenshtein algorithm to find the best match

Using the Helper Classes Pattern = “8624”

Using the Helper Classes To create a gesture handler object: 1. Specify a name for the gesture handler object, this name will also be used to generate events when a gesture is recognized 2. Specify the pattern of gestures and their identifiers. These identifiers will be passed to the event handler. 3. Specify the IR cursor object that will be used to draw 4. Specify if the gestures will be visible 5. Specify the color of the gestures Using the gesture handler 1. Start drawing 2. Stop drawing 3. Handle gesture recognition event

Using the Helper Classes self.gestureHandler = GestureRecognition("GESTURE_EVENT", [("369871", 0), # fish ("866224", 1), # square (" ", 2)], # circle self.cursor, True, 1, 0, 0); To start drawing: self.gestureHandler.beginGesture(); To stop drawing: self.gestureHandler.endGesture();

Using the Helper Classes  Gesture Handler Handling gesture recognition events: ○ Create an event handler for the name specified during creation of the gesture handler object ○ The event handler should accept three parameters: evt_data : this is the identifier for the gesture that was recognized x : this is the x position in 2D coordinates of the approximate mid-point of the whole gesture y : this is the y position in 2D coordinates of the approximate mid-point of the whole gesture

Wiimote I/F Physics I/F Wiimote Handler Helper Classes Wiiuse Wiiuse Abstraction Layer Crayon3D Scene Crayon3D Node ODE Cursor Gesture Recognition Used for finger tracking Handles polling and managing Wiimote information. Crayon3D Class Diagram

Glove and IR Glasses  Gloves: Buy from Giant Eagle / CVS ○ Black Stanley Gloves Buy from RadioShack ○ 3 High Output 5mm Infrared LED ○ SPST Submini PC Board Toggle Switch ○ Round PCB Kit

Glove and IR Glasses

 IR Glasses Buy from Amazon.com ( ○ AO Safety Safety Glasses V2 Light Vision Buy from RadioShack ○ High Output 5mm Infrared LED Replace the regular LEDs with the Infrared LED from RadioShack

Questions?  More information is available in the API reference and the wiiuse.net website