2. Networks ∙ Services ∙ People www.geant.org Domenico Vicinanza TNC2015 Creating Interactive Art with Prototype Boards and R&E Networks Porto, Portugal, 17 June 2015 Product manager, GÉANT, Cambridge Office Senior Lecturer and Director of the Electronics and Sound Engineering Research Group at Anglia Ruskin University

3. Networks ∙ Services ∙ People www.geant.org Outline Introduction to embedded systems Embedded systems and the arts Adding the network Case studies Conclusions

4. Networks ∙ Services ∙ People www.geant.org Designing complex devices made easy Inexpensive micro-processor boards now available Big impact in higher-education Smoother learning curve Easy access to network resources Create working prototypes 3 Inexpensive micro-processor boards A detail of the Arduino platform The ARM mbed (source mbed.org)

5. Networks ∙ Services ∙ People www.geant.org Electronic system that includes: Microprocessor -> Computing capabilities I/O Interfaces -> Communication with external world Ethernet interface -> Network connectivity! 4 What is an embedded device?

6. Networks ∙ Services ∙ People www.geant.org Arduino Open Source Hardware Atmega32U4 AVR Low-power 8-bit RISC microcontroller 16MHz, 2.5KB SRAM, 1KB EEPROM, USB 2.0 32KB on-chip flash program memory C++ based code with open source IDE

7. Networks ∙ Services ∙ People www.geant.org ARM mbed High performance 32-bit ARM® Cortex™-M3 Core 96MHz, 32KB RAM, 512KB FLASH Ethernet, USB Host/Device, 2xSPI, 2xI2C, 3xUART, CAN, 6xPWM, 6xADC Built-in USB drag 'n' drop FLASH programmer Lightweight Online Compiler with high level C/C++ SDK

8. Networks ∙ Services ∙ People www.geant.org Ready-to-use prototyping boxes http://arduino.cc/en/Main/arduinoBoardEsplora

9. Networks ∙ Services ∙ People www.geant.org What is available Microphone Temperature Accelerometer Photoresistor

10. Networks ∙ Services ∙ People www.geant.org Code example: LED blink #include "mbed.h" DigitalOut myled(LED1); int main() { while(1) { myled = 1; wait(0.2); myled = 0; wait(0.2); } www.batsocks.co.uk

11. Networks ∙ Services ∙ People www.geant.org Democratisation of electronic prototyping Covering everything from design to implementation Usable, interactive, customised electronic devices Availability of open sources platforms No expensive closed development environment Attracted students, researchers, engineers and, quite naturally, artists 10 Effects of cheap development boards

12. Networks ∙ Services ∙ People www.geant.org Let’s add network! Network brings people together Network allows people to cooperate and collaborate, to exchange data, experience and information.

13. Networks ∙ Services ∙ People www.geant.org Adding the network Creating/controlling art using sensors Light Acceleration Temperature Noise level Sending data through the network Ethernet WiFi

14. Networks ∙ Services ∙ People www.geant.org Network

15. Networks ∙ Services ∙ People www.geant.org Network (ARM Mbed) HTTP Client - GET and POST requests HTTP Client Data Containers HTTP Server - handle HTTP requests SMTP Client - a simple email client supporting plain authentication NTP Client - set the RTC Twitter - post to twitter SuperTweet - post to Twitter via SuperTweet MySQL Client - connect to MySQL

16. Networks ∙ Services ∙ People www.geant.org Museum and galleries are becoming increasingly interested in computational/microprogrammable electronic art. Remotely controlled microprocessor-powered Drawing Bot 4-wheeled graffiti machine, with a pendulum arm and spray cans to create strokes as it moves up and down in a gallery space 15 Examples from museums The Senseless Drawing Bot (So Kanno and Takahiro Yamaguchi)

17. Networks ∙ Services ∙ People www.geant.org Museums leading hardware hacking sessions Participants with no prior experience learn to make working simple micro- programmed circuits V&A Museum: 16 V&A Museum http://www.vam.ac.uk/blog/digital-media/blogdigital-media-vadigital-media-museum-computer-network

18. Networks ∙ Services ∙ People www.geant.org 17 Ultrasound (distance measurement) Accelerometer Ethernet connection ARM Mbed microcontroller ARM board + Ethernet + Sensors Electronics and Sound Engineering Research Group (ESERG)

19. Networks ∙ Services ∙ People www.geant.org Pure-data (pd) Sending ArtistReceiving Artist Connect with another artist using the network

20. Networks ∙ Services ∙ People www.geant.org From physical measurements to music notes Microcontroller programming language: map map(variable-to-map, from-min, from-max, to-min, to-max); Acceleration Min -150 For example mapping acceleration (-150, 150) to MIDI pitch (20,60): Pitch = map(zAxis, -150, 150, 20, 60); Max 150 Lower 20 Upper 60 Note pitch

21. Networks ∙ Services ∙ People www.geant.org Open Sound Control (OSC) Protocol for networking sound synthesizers, computers, and other multimedia devices Designed for networked musical performances Used by visual artists and light designers Bringing the benefits of modern networking technology to the world of electronic musical instruments Advantages: Open Interoperable Flexible Lightweight

22. Networks ∙ Services ∙ People www.geant.org 21 Receiving data from remote sensors

23. Networks ∙ Services ∙ People www.geant.org 22 Music from acceleration - using OSC

24. Networks ∙ Services ∙ People www.geant.org 23 Use case Creating music from distributed sensors across two continents

25. Networks ∙ Services ∙ People www.geant.org 24 NYU Citygram (http://citygram.smusic.nyu.edu/)

26. Networks ∙ Services ∙ People www.geant.org Mapping measurements to Music Physical ParametersSound energy Min Max

27. Networks ∙ Services ∙ People www.geant.org 26 Microcontroller+network Data repository GSM Real-time Sonification

28. Networks ∙ Services ∙ People www.geant.org 27 The final result

29. Networks ∙ Services ∙ People www.geant.org What can we do combining sensors+microcontrollers+network Control/play an instrument, synthesizer, drum-machines Distributed performances Control light consoles Light mixers controlled by remote sensors Theatre sound consoles, sound playback controllers, virtual audio matrices and switchers, video playback systems Video Real-time video generation/manipulation

30. Networks ∙ Services ∙ People www.geant.org Conclusions Microcontroller, sensors and network A natural synergy for the arts Network a natural enabler for the arts: Allowing remote collaboration Allowing access to remote data and information Using information collected in remote locations to create local arts Research and Education networks are natural partners

31. Networks ∙ Services ∙ People www.geant.org Thank you Networks ∙ Services ∙ People www.geant.org 30

32. Networks ∙ Services ∙ People www.geant.org Elisha Gray: “Musical Telegraph” 1876 Image source: http://120years.net/

33. Networks ∙ Services ∙ People www.geant.org Image source: http://120years.net/

34. Networks ∙ Services ∙ People www.geant.org First public demonstration, Highland Park, Illinois on December 29, 1874 E. Gray transmitted “familiar melodies through telegraph wire” according to a newspaper announcement

35. Networks ∙ Services ∙ People www.geant.org 34 Create/control sound generation or manipulation Example from LHC data Controlling MIDI parameters J. Cherston (MIT) E. Hill (UVIC), D. Vicinanza, 2015

36. Networks ∙ Services ∙ People www.geant.org 35 The score Example from LHC data Controlling MIDI parameters J. Cherston (MIT) E. Hill (UVIC), D. Vicinanza, 2015

37. Networks ∙ Services ∙ People www.geant.org 36 Use case Creating music from acceleration

38. Networks ∙ Services ∙ People www.geant.org Example: using acceleration to create music across the network Measuring acceleration using miniature accelerometer (ADXL345) in Cambridge (Anglia Ruskin University) Sending measurement through networks using a prototype board Receiving measurements in Italy where they are converted into music and played live

39. Networks ∙ Services ∙ People www.geant.org Example: using acceleration to create music across the network ARM LPC1768 board with ADXL345 3-axis accelerometer The final sonification

40. Networks ∙ Services ∙ People www.geant.org Result: acceleration to music Acceleration Sonification