Wekinator www.wekinator.org

Wekinator Anything that can output OSC can be used as a controller Anything that can be controlled by OSC can be controlled by Wekinator

Wekinator The Wekinator allows anyone to use machine learning to build new musical instruments, gestural game controllers, computer vision or computer listening systems, and more. users to build new interactive systems by demonstrating human actions and computer responses, instead of writing programming code.

Wekinator – I/O Wekinator receives inputs, processes them using computational functions called models, and those models produce outputs: Inputs could be sent from any real-time input, such as game controllers, webcam input, motion tracking, audio input, sensors connected to Arduino, etc. Outputs could be sent to any real-time process, such as music (Max/MSP, ChucK, PD, SuperCollider...), animation (Processing, OpenFrameworks, ...), games (Unity3D), robots or physical actuators (e.g. using Arduino), etc.

Wekinator - ML Wekinator builds these models using supervised machine learning algorithms.  Instead of creating the model function by writing code, you create models by providing training examples of inputs and the outputs that should correspond to them. A learning algorithm builds a model from the training examples

Install from Lab NAS PATH : \\10.187.34.200\Public\Software\Wekinator

Wekinator : INTERFACE

Wekinator – Eyw : OSC PORTS

Wekinator – Eyw : SEND AND RECEIVE GENERATING INPUT RECEIVING OUTPUT

Wekinator – Eyw : I/O MAPPING

Wekinator – Eyw : INPUT NAMES

Wekinator – Eyw : ML ALGORITHMS

Wekinator – Eyw : INPUT RULES Send inputs process : Inputs has to be into a single OSC message, with each input sent as a double/float The number of double/floats in this message must always match the number of inputs Wekinator is expecting Any sending rate is accepted

Wekinator – Eyw : OUTPUT TYPES Classification outputs: These are discrete categories, such as “Gesture 1”, “Gesture 2,” “Gesture 3.” You’ll need to tell Wekinator how many categories to use. Wekinator will sent outputs as numbers, such as “1” “2” “3” for categories 1, 2, and 3. Continuous outputs: These are numeric values. There are two types of numeric outputs: Real-valued numeric outputs can take on any number value (possibly limited to a certain range). For example, you might want to control “audio gain” with a real-valued output limited between 0 and 1. Integer-valued numeric outputs can take on any integer value (possibly limited to a certain range). For example, you might want to control “MIDI pitch” with an integer-valued output between 0 and 127. DTW outputs : These are discrete categories as classification outputs, but based on the DTW techniques. Mainly they are used for gesture recognition.

1) MOVEMENT TO COLOR OSC OSC Gesture Color Gesture Color

1) MOVEMENT TO COLOR 12 Inputs from KinectV2 Skeleton: OSC OSC 3D coordinates of RightElbow 3D coorinates of Right Wrist 3D coordinates of Left Elbow 3D coorinates of Left Wrist Outputs: Continuous : moving the hands the user can control the color that is visualized on the screen Discrete : the user moves and changes the text description and color visualized on the screen. Four possible movements : REST, UP, RIGHT, LEFT

ftp://ftp.infomus.org/pub/Wekinator/weki_eyw_video.mp4 OSC OSC KinectV2 Eyesweb Wekinator

2) SQUARE GAME OSC OSC OSC

2) SQUARE GAME

2) SQUARE GAME

2) SQUARE GAME Change the output name of the OSC messages that we want to send to our application

2) SQUARE GAME Change the output name of the OSC messages that we want to send to our application

2) SQUARE GAME

2) SQUARE GAME Press the recording button and hold it to record a new trial.

2) SQUARE GAME Map the gestures in the game to control the square

RUN IT!