Visa-Valtteri visy Pimiä or how fumbling my way through pixel-based effects made learning WebGL a lot easier

Started doing simple effects with QBASIC Worked my way to Turbo Pascal and VGA programming Got sidetracked to NES and Atari 2600 development Learned OpenGL mostly professionally, then started using GPU-accelerated graphics

Processing makes it possible

Access the screen through a framebuffer, an array of pixel color values, palette index values or bitplane toggles Set pixel color values according to position, time, function value etc.

I started making small games and demo-style little programs Everything was always coded for one single production, with no mind put into reusability of the tools This got old pretty quickly and I found myself toying with the idea of a visual editor for demos

Trilobit ScriPtable Lua Interactive MultiplatForm Tackles a multitude of problems I had with development: Inherently multiplatform (OSX, Win32, Linux) because all the GL calls are coded with Lua. All visual code is reloadable without relaunching or recompiling the program The demo player is seekable and supports setting sync points The demo is the editor Always open source

Timeline file millisecond syncing of demo parts FFT beat sync for effects Seeking for testing

Fixed OpenGL version, upgrading would require recompiling the Lua libraries for all the three platforms No shader support Terrible hacks for texture handling The OpenGL / Lua bound version I used was found to be rather incomplete

Counter Intelligence Program by Bilotrip

After I heard about WebGL I got the idea of doing something like S.P.L.I.F in the browser with easily reloadable and tweakable visuals WebGL has shaders, so the natural choice was to enable the editing of shader parameters (in addition for JavaScript variables) realtime

WebGL canvas for the output Pure JavaScript + HTML5 + CSS3 for the UI Textures loaded from PNG files HTML5 audio from OGG file Timeline.js used for the keyframe editor

No real timeline dumping (cut & paste from export windows) Shader code still requires reloading of the page HTML5 audio support is still rather flaky cross browsers, and require some time to mature No cross-browser FFT support for sound analysis WebGL is still not available for mobile browsers or Internet Explorer

WebGL _is_ a W3 standard, and pretty much all major browser except Internet Explorer have support for it in development roadmaps No plugins required anymore for GPU accelerated content and complex video/audio data processing and playback No third party components required

Pixel shaders are pretty similar to framebuffers in many ways Can set single resolution independent fragments instead of toggling actual pixels on the screen Many fragments are calculated concurrently on the GPU, so the shader program is run in parallel.

I tend to avoid calculating complex geometries and like to play around with fragments on a single quad Complex blending and processing passes possible with the GPU -> enables effects that are just simply impossible to do on a framebuffer with a fast FPS

Demonstration MKULTRA by Bilotrip

Thank you for listening!