Martin Kruliš by Martin Kruliš (v1.1)1

 Images and CSS3 ◦ Wise application of CCS3 can produce unexpected effects, including animations ◦ HTMLImageElement DOM API var img = new Image(); img.src = "image.png"; elem.appendChild(img); ◦ Important properties and events  complete – is image loaded and ready  naturalWidth, naturalHeight – real proportions  onload – event fired when loading completes by Martin Kruliš (v1.1)2 Example 1

 Creating Images from Blob Data var xhr = new XMLHttpRequest(); xhr.open("GET", "picture.jpg", true); xhr.responseType = "arraybuffer"; xhr.onload = function(e) { var abView = new Uint8Array(this.response); var blob = new Blob( [ abView ], { type: "image/jpeg" } ); var urlCreator = window.URL || window.webkitURL; var imageUrl = urlCreator.createObjectURL(blob); var img = new Image(); img.src = imageUrl; parentElem.appendChild(img); }; xhr.send(); by Martin Kruliš (v1.1)3 Create temporary URL, which is valid only within the page

 Creating Animations ◦ Script can request a callback for next animation frame by window.requestAnimationFrame(anim)  Invocation returns request ID (similar to timers)  window.cancelAnimationFrame(id) aborts request ◦ The callback itself can update the page just before the window is re-painted  It gets time argument, which gets current time in miliseconds from window.performance.now()  The callback is invoked only once, but it can register new animation frame request by Martin Kruliš (v1.1)4 Example 2

 HTML Element ◦ Creates embedded screen for video playback Browser does not support HTML5 video.  The controls attribute makes controls visible  If the controls are hidden, the element may be controlled by Javascript by Martin Kruliš (v1.1)5 The same video in different formats

 HTML Element DOM API ◦ Important Methods and Properties  load(), readyState  play(), pause(), paused, ended  duration, currentTime, seekable  volume, muted ◦ Events  loadstart, loadeddata, canplay, canplaythrough  pause, play, playing, progress, stalled  seeking, seeked  volumechange by Martin Kruliš (v1.1)6

 Scalable Vector Graphics ◦ Quite complex vector graphics and animations ◦ HTML5 allows direct embedding of SVG code ◦ The SVG data are visible through DOM API document.getElementById("circle1")… by Martin Kruliš (v1.1)7 Example 3

 The canvas Element ◦ HTML5 element for JS drawing Your browser does not support HTML5 canvas.  Canvas can be sized by CSS, but width and height define the resolution of the viewport  Canvas is cleared when resized ◦ Drawing is performed through context var context = mycanvas.getContext("2d");  Various types of context (for drawing, 3D graphics, …)  Animations use timers or animation frame events by Martin Kruliš (v1.1)8

 2D Context Features ◦ Drawing rectangles  strokeRect(), fillRect(), clearRect() ◦ Drawing paths  beginPath(), closePath()  moveTo(), lineTo()  arc(), quadraticCurveTo(), bezierCurveTo()  stroke(), fill() ◦ Setting drawing styles  strokeStyle, fillStyle, globalAlpha  lineWidth, lineCap, lineJoin by Martin Kruliš (v1.1)9 Example 4

 2D Context Features ◦ Color gradients  createLinearGradient(), createRadialGradient()  gradient.addColorStop()  Can be used in strokeStyle, fillStyle ◦ Patterns var img = new Image(); img.src = "fill-pattern.png"; ctx.strokeStyle = ctx.createPattern(img,"repeat"); ◦ Text rendering  fillText(), strokeText(), font by Martin Kruliš (v1.1)10

 2D Context Features ◦ Transformations  Global transformation matrix for all drawings  translate(), rotate(), scale()  transform(), setTransform() ◦ Compositing  How is new drawing combined with existing drawings  globalCompositeOperation ◦ Clipping paths  Path terminated with clip(), instead of closePath()  All subsequent drawing is clipped accordingly by Martin Kruliš (v1.1)11

 2D Context Features ◦ Context state stack  save(), restore() ◦ Retrieving/drawing image data  ImageData object – raw RGB data in binary form  createImageData()  getImageData(), putImageData() ◦ Saving canvas as image  canvas.toDataURL() ◦ Drawing images/video on canvas  drawImage(imgElem, …) by Martin Kruliš (v1.1)12 Example 4

 KineticJS ◦ The programmer does not draw, but create objects that are automatically rendered  Similar to DOM elements  Supports geometric shapes, images, …  Modifying properties causes automatic repainting ◦ Complex object management  Layers, object groups, … ◦ Events  Extending event model to graphic objects  Drag and drop support by Martin Kruliš (v1.1)13 Example 5

 Web API for OpenGL ◦ A way to render 3D graphic in web browser  Based on OpenGL ES 3.0 specification ◦ canvas element is used as viewport  Using ‘webgl’ context, which provides bindings to OpenGL functions var canvas = document.getElementById('glcanvas'); var gl = canvas.getContext('webgl'); gl.viewport(0, 0, canvas.width, canvas.height); gl.clearColor(0.0, 0.0, 0.0, 1.0); by Martin Kruliš (v1.1)14

 WebGL Context ◦ Data buffers (vertices, colors, …) var buf = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, buf); var v = new Float32Array([ 1.0, -1.0,... ]); gl.bufferData(gl.ARRAY_BUFFER, v, gl.STATIC_DRAW); ◦ Vertex and fragment shaders  May be stored along with HTML page in elem. var shader = gl.createShader(gl.FRAGMENT_SHADER); gl.shaderSource(shader, shaderSourceCode); gl.compileShader(shader); by Martin Kruliš (v1.1)15 Example 6

 Three.js ◦ Lightweight 3D graphic library  Primarily uses webgl, but can fallback to SW rendering ◦ Object based approach  3D entities are JS objects  Camera, lights, scene, …  Hides complexities of 3D math ◦ Additional features  Object materials – different surface effects  Import/export 3D data  … by Martin Kruliš (v1.1)16 Example 7

 HTML Element ◦ Similar to element ◦ Controlled from user panel or from Javascript  Web Audio API ◦ Playback from media files, oscillators, and binary data buffers ◦ Adding audio filters and effects ◦ Related to WebRTC media streams  Recording from microphone  Sending audio to RTC channel by Martin Kruliš (v1.1)17

 Audio Context ◦ A context object must be created first var audioCtx = new (window.AudioContext || window.webkitAudioContext)(); if (!audioCtx)... // no audio ◦ Within the context a pipeline of audio nodes has to be assembled by Martin Kruliš (v1.1)18 Inputs Effects Destination Example 8

 Audio Nodes ◦ Audio Sources  OscillatorNode  AudioBuffer, AudioBufferSourceNode  MediaElementAudioSourceNode – e.g.,  MediaStreamAudioSourceNode – WebRTC media stream ◦ Audio Destinations  AudioDestinationNode  audioCtx.destination – default destination (speakers)  MediaStreamAudioDestinationNode – WebRTC channel by Martin Kruliš (v1.1)19

 Audio Nodes ◦ Effects and Filters  DelayNode  GainNode  … ◦ Other Nodes  AnalysisNode – analysis and visualization  ChannelSplitterNode, ChannelMergerNode  AudioListener – position the sound in space  PannerNode – signal behavior in space by Martin Kruliš (v1.1)20

 Audio Buffer Playback Example var audioCtx = new (window.AudioContext || window.webkitAudioContext)(); var channels = 2, sampleRate = 44100; var nextTime = 0; function enqueueAudioData(dataArrayBuffer) { var data = new Int16Array(dataArrayBuffer); var buf = audioCtx.createBuffer(channels, data.length / channels, sampleRate); //... Copy samples from data to buf... if (nextTime == 0) nextTime = audioCtx.currentTime ; var source = audioCtx.createBufferSource(); source.buffer = buf; source.connect(audioCtx.destination); source.start(nextTime); nextTime += buf.duration; } by Martin Kruliš (v1.1)21 Small constant to prevent data jittering BufferSource wraps the buffer for playback nextTime is used to mark, when the playback starts Puts data from ArrayBuffer to playback queue

by Martin Kruliš (v1.1)22