Lonce Wyse Arts and Creativity Lab National University of Singapore Net-Music 2013: The Internet as Creative Resource in Music

 Some new emerging standards from the World Wide Web Consortium  What’s so exciting about standards?  Functionality that becomes standard in the browser  Becomes cross platform  Runs natively  Requires no plug-in (end user effort)  Related technology developments

 Native audio engine in browsers with API called through javascript.  “Node” / connection flow-graph construction model  Create “Nodes”  Osc.connect(Filter);  Filter.connect(Conext.destination)

 Sample-accurate scheduling for built-in components (nodes, parameter automation)  Spatialization (sound cones, obstruction, oriented “listeners”)  Buffer Nodes, Gain Nodes  Oscillators, Delay, Channel management, Common Filter types, compression, waveshaping  Convolution, Time and frequency analysis  (Predefined) Parameters with automation

 “Roll your own” ScriptProcessorNodes for generation and processing of audio.  Not treated the same as system Audio Nodes  Timing latency and jitter  Can’t attach WebAudio Paramters to them.  Callbacks from audio processing thread  One API fits all (sound designers – sound users)  Future (ever-increasing number of node types?)

 WebRTC (also W3C emerging standard)  Oriented toward P2P media movement (eg video chat).  Access to user media – audio and video  No more plug-ins!  Plays nicely with WebAuido.  Also running on some mobile browsers

 First introduced in 1995 to add short interpreted scripts in HTML for dynamic elements – it has come a long way in a short time.  Google V8 engine (chrome, safari, node.js) brought huge increases in speed. Image:  Excellent browser debugging tools.  Libraries – thousands upon thousands.  Phone sensor data, large project support.  Still some issues that make it a less-than- ideal educational tool.

 Server side applications  Javascript on Google V8 virtual machine  Single-threaded, asynchronous event handling  Works with socket.io (a wrapper for web sockets) for real-time apps with two-way open communications.  Now one language for network, clients, app, sound synthesis….

 jsaSound  Sound development tools  Anticipatory Score  Shared graphical scrolling score paradigm  Network control signals with node.js server  Augmented Storytelling  Inter-browser communication for synthesis and control

 Library for creating WebAudio sound models.  Building tools  Simple and consistent API  play(), stop(), release(), setParam(), setParamNorm(), getParam(["name", "type", "val", "normval", "min","max"]  In action: animatedsoundworks.com/jsaSound*animatedsoundworks.com/jsaSound  Open source  [* Tested with Canary Browser]

Synth1 Patch Synth2 Patch Server Patch Synth2 Patch Synth1 Patch Server Patch Node.js Server Anticipatory Scores

 Putting sound interaction and voice transformation in to the hands of the storyteller  Oral traditions:  Shareabiltiy  Mutability

Story Telling & Sharing Sound Server Sound Server * Controller [* Tested with Canary Browser]

 And a much less than perfect one, at that.  But … Creative possibilities  The integration with web brings  Accessibility (no apps or plug-ins to download)  Cross platform (or will our hearts be broken again…)  Possibilities for massive participation  Access to unprecedented data flows to integrate into artworks  Dynamic and programmable at all levels, not just audio  … and it even says right in the WebAudio specification: Artistic Audio Exploration There are many creative possibilities for artistic sonic environments for installation pieces.

 W3C Web Audio draft spec   Working grouop mailing list  HTML5 Audio Blog   Node.js 