1. Wekinator

2. Wekinator Anything that can output OSC can be used as a controller
Anything that can be controlled by OSC can be controlled by Wekinator

3. Wekinator The Wekinator allows
anyone to use machine learning to build new musical instruments, gestural game controllers, computer vision or computer listening systems, and more.
users to build new interactive systems by demonstrating human actions and computer responses, instead of writing programming code.

4. Wekinator – I/O
Wekinator receives inputs, processes them using computational functions called models, and those models produce outputs:
Inputs could be sent from any real-time input, such as game controllers, webcam input, motion tracking, audio input, sensors connected to Arduino, etc.
Outputs could be sent to any real-time process, such as music (Max/MSP, ChucK, PD, SuperCollider...), animation (Processing, OpenFrameworks, ...), games (Unity3D), robots or physical actuators (e.g. using Arduino), etc.

5. Wekinator - ML
Wekinator builds these models using supervised machine learning algorithms. 
Instead of creating the model function by writing code, you create models by providing training examples of inputs and the outputs that should correspond to them.
A learning algorithm builds a model from the training examples

6. Install from Lab NAS
PATH : \\ \Public\Software\Wekinator

7. Wekinator : INTERFACE

8. Wekinator – Eyw : OSC PORTS

9. Wekinator – Eyw : SEND AND RECEIVE
GENERATING
INPUT
RECEIVING
OUTPUT

10. Wekinator – Eyw : I/O MAPPING

11. Wekinator – Eyw : INPUT NAMES

12. Wekinator – Eyw : ML ALGORITHMS

13. Wekinator – Eyw : INPUT RULES
Send inputs process :
Inputs has to be into a single OSC message, with each input sent as a double/float
The number of double/floats in this message must always match the number of inputs Wekinator is expecting
Any sending rate is accepted

14. Wekinator – Eyw : OUTPUT TYPES
Classification outputs: These are discrete categories, such as “Gesture 1”, “Gesture 2,” “Gesture 3.” You’ll need to tell Wekinator how many categories to use.
Wekinator will sent outputs as numbers, such as “1” “2” “3” for categories 1, 2, and 3.
Continuous outputs: These are numeric values. There are two types of numeric outputs:
Real-valued numeric outputs can take on any number value (possibly limited to a certain range). For example, you might want to control “audio gain” with a real-valued output limited between 0 and 1.
Integer-valued numeric outputs can take on any integer value (possibly limited to a certain range). For example, you might want to control “MIDI pitch” with an integer-valued output between 0 and 127.
DTW outputs : These are discrete categories as classification outputs, but based on the DTW techniques. Mainly they are used for gesture recognition.

15. 1) MOVEMENT TO COLOR
OSC
OSC
Gesture
Color
Gesture
Color

16. 1) MOVEMENT TO COLOR 12 Inputs from KinectV2 Skeleton: OSC OSC
3D coordinates of RightElbow
3D coorinates of Right Wrist
3D coordinates of Left Elbow
3D coorinates of Left Wrist
Outputs:
Continuous : moving the hands the user can control the color that is visualized on the screen
Discrete : the user moves and changes the text description and color visualized on the screen. Four possible movements : REST, UP, RIGHT, LEFT

17. ftp://ftp.infomus.org/pub/Wekinator/weki_eyw_video.mp4
OSC
OSC
KinectV2
Eyesweb
Wekinator

18. 2) SQUARE GAME
OSC
OSC
OSC

19. 2) SQUARE GAME

20. 2) SQUARE GAME

21. 2) SQUARE GAME Change the output name of the OSC
messages that we want to send to our application

22. 2) SQUARE GAME Change the output name of the OSC
messages that we want to send to our application

23. 2) SQUARE GAME

24. 2) SQUARE GAME
Press the recording button and hold it to record a new trial.

25. 2) SQUARE GAME
Map the gestures in the game to control the square

26. RUN IT!