Networks ∙ Services ∙ People Domenico Vicinanza TNC2015 Creating Interactive Art with Prototype Boards and R&E Networks Porto, Portugal, 17 June 2015 Product manager, GÉANT, Cambridge Office Senior Lecturer and Director of the Electronics and Sound Engineering Research Group at Anglia Ruskin University

Networks ∙ Services ∙ People Outline Introduction to embedded systems Embedded systems and the arts Adding the network Case studies Conclusions

Networks ∙ Services ∙ People Designing complex devices made easy Inexpensive micro-processor boards now available Big impact in higher-education Smoother learning curve Easy access to network resources Create working prototypes 3 Inexpensive micro-processor boards A detail of the Arduino platform The ARM mbed (source mbed.org)

Networks ∙ Services ∙ People Electronic system that includes: Microprocessor -> Computing capabilities I/O Interfaces -> Communication with external world Ethernet interface -> Network connectivity! 4 What is an embedded device?

Networks ∙ Services ∙ People Arduino Open Source Hardware Atmega32U4 AVR Low-power 8-bit RISC microcontroller 16MHz, 2.5KB SRAM, 1KB EEPROM, USB KB on-chip flash program memory C++ based code with open source IDE

Networks ∙ Services ∙ People ARM mbed High performance 32-bit ARM® Cortex™-M3 Core 96MHz, 32KB RAM, 512KB FLASH Ethernet, USB Host/Device, 2xSPI, 2xI2C, 3xUART, CAN, 6xPWM, 6xADC Built-in USB drag 'n' drop FLASH programmer Lightweight Online Compiler with high level C/C++ SDK

Networks ∙ Services ∙ People Ready-to-use prototyping boxes

Networks ∙ Services ∙ People What is available Microphone Temperature Accelerometer Photoresistor

Networks ∙ Services ∙ People Code example: LED blink #include "mbed.h" DigitalOut myled(LED1); int main() { while(1) { myled = 1; wait(0.2); myled = 0; wait(0.2); }

Networks ∙ Services ∙ People Democratisation of electronic prototyping Covering everything from design to implementation Usable, interactive, customised electronic devices Availability of open sources platforms No expensive closed development environment Attracted students, researchers, engineers and, quite naturally, artists 10 Effects of cheap development boards

Networks ∙ Services ∙ People Let’s add network! Network brings people together Network allows people to cooperate and collaborate, to exchange data, experience and information.

Networks ∙ Services ∙ People Adding the network Creating/controlling art using sensors Light Acceleration Temperature Noise level Sending data through the network Ethernet WiFi

Networks ∙ Services ∙ People Network

Networks ∙ Services ∙ People Network (ARM Mbed) HTTP Client - GET and POST requests HTTP Client Data Containers HTTP Server - handle HTTP requests SMTP Client - a simple client supporting plain authentication NTP Client - set the RTC Twitter - post to twitter SuperTweet - post to Twitter via SuperTweet MySQL Client - connect to MySQL

Networks ∙ Services ∙ People Museum and galleries are becoming increasingly interested in computational/microprogrammable electronic art. Remotely controlled microprocessor-powered Drawing Bot 4-wheeled graffiti machine, with a pendulum arm and spray cans to create strokes as it moves up and down in a gallery space 15 Examples from museums The Senseless Drawing Bot (So Kanno and Takahiro Yamaguchi)

Networks ∙ Services ∙ People Museums leading hardware hacking sessions Participants with no prior experience learn to make working simple micro- programmed circuits V&A Museum: 16 V&A Museum

Networks ∙ Services ∙ People 17 Ultrasound (distance measurement) Accelerometer Ethernet connection ARM Mbed microcontroller ARM board + Ethernet + Sensors Electronics and Sound Engineering Research Group (ESERG)

Networks ∙ Services ∙ People Pure-data (pd) Sending ArtistReceiving Artist Connect with another artist using the network

Networks ∙ Services ∙ People From physical measurements to music notes Microcontroller programming language: map map(variable-to-map, from-min, from-max, to-min, to-max); Acceleration Min -150 For example mapping acceleration (-150, 150) to MIDI pitch (20,60): Pitch = map(zAxis, -150, 150, 20, 60); Max 150 Lower 20 Upper 60 Note pitch

Networks ∙ Services ∙ People Open Sound Control (OSC) Protocol for networking sound synthesizers, computers, and other multimedia devices Designed for networked musical performances Used by visual artists and light designers Bringing the benefits of modern networking technology to the world of electronic musical instruments Advantages: Open Interoperable Flexible Lightweight

Networks ∙ Services ∙ People 21 Receiving data from remote sensors

Networks ∙ Services ∙ People 22 Music from acceleration - using OSC

Networks ∙ Services ∙ People 23 Use case Creating music from distributed sensors across two continents

Networks ∙ Services ∙ People 24 NYU Citygram (

Networks ∙ Services ∙ People Mapping measurements to Music Physical ParametersSound energy Min Max

Networks ∙ Services ∙ People 26 Microcontroller+network Data repository GSM Real-time Sonification

Networks ∙ Services ∙ People 27 The final result

Networks ∙ Services ∙ People What can we do combining sensors+microcontrollers+network Control/play an instrument, synthesizer, drum-machines Distributed performances Control light consoles Light mixers controlled by remote sensors Theatre sound consoles, sound playback controllers, virtual audio matrices and switchers, video playback systems Video Real-time video generation/manipulation

Networks ∙ Services ∙ People Conclusions Microcontroller, sensors and network A natural synergy for the arts Network a natural enabler for the arts: Allowing remote collaboration Allowing access to remote data and information Using information collected in remote locations to create local arts Research and Education networks are natural partners

Networks ∙ Services ∙ People Thank you Networks ∙ Services ∙ People 30

Networks ∙ Services ∙ People Elisha Gray: “Musical Telegraph” 1876 Image source:

Networks ∙ Services ∙ People Image source:

Networks ∙ Services ∙ People First public demonstration, Highland Park, Illinois on December 29, 1874 E. Gray transmitted “familiar melodies through telegraph wire” according to a newspaper announcement

Networks ∙ Services ∙ People 34 Create/control sound generation or manipulation Example from LHC data Controlling MIDI parameters J. Cherston (MIT) E. Hill (UVIC), D. Vicinanza, 2015

Networks ∙ Services ∙ People 35 The score Example from LHC data Controlling MIDI parameters J. Cherston (MIT) E. Hill (UVIC), D. Vicinanza, 2015

Networks ∙ Services ∙ People 36 Use case Creating music from acceleration

Networks ∙ Services ∙ People Example: using acceleration to create music across the network Measuring acceleration using miniature accelerometer (ADXL345) in Cambridge (Anglia Ruskin University) Sending measurement through networks using a prototype board Receiving measurements in Italy where they are converted into music and played live

Networks ∙ Services ∙ People Example: using acceleration to create music across the network ARM LPC1768 board with ADXL345 3-axis accelerometer The final sonification

Networks ∙ Services ∙ People Result: acceleration to music Acceleration Sonification