Comp311/Comp511 Lecture01 - Introduction

Unit Plan Text Book: Fundamentals of Multimedia by Ze-Nian Li and Mark S. Drew Theme - ”Platform independent multimedia” What we will not do: – How to become computer graphic artists! – How to drive commercial multimedia suits!

Contents Foundational aspects: Our study about this is minimal but there is five lectures covering Foundational aspects: Our study about this is minimal but there is five lectures covering Human Vision and Colour models.Human Vision and Colour models. Image data representations.Image data representations. Data compression.Data compression. Transform coding.Transform coding. Video compression.Video compression.

Contents Rest of the lectures are on programming aspects including Rest of the lectures are on programming aspects including Java Media APIs (Java2D, JAI, JMF, little bit of Java3D)Java Media APIs (Java2D, JAI, JMF, little bit of Java3D) SMIL – Synchronized Multimedia Integration languageSMIL – Synchronized Multimedia Integration language A XML based language. A XML based language. VRML/X3D – Virtual Reality Markup LanguageVRML/X3D – Virtual Reality Markup Language X3D is the current XML based VRML specification X3D is the current XML based VRML specification Basic Java Game programming.Basic Java Game programming.

Assessment   Assessment: 60% Assignments and 40% Exam o oMust obtain at least 50% from each component to pass the unit. o oThere are 3 assignments covering programming and multimedia development. o o The first assignment will be released today.   Lecturer: Ashoka Jayawardena o oRoom 171 o oExtension: 2267   Tutor: TBA

Multimedia Multimedia involves multiple modalities of text, audio, images, drawings, animation, and video.

Applications Video teleconferencing. Distributed lectures for higher education. Tele-medicine. Co-operative work environments. Searching in (very) large video and image databases for target visual objects. etc. etc.

Hypermedia and Multimedia A hypertext system: meant to be read nonlinearly, by following links that point to other parts of the document, or to other documents. Hypermedia: not constrained to be text-based, can include other media, e.g., graphics, images, and especially the continuous media – sound and video: The World Wide Web (WWW) – the best example of a hypermedia application. Multimedia means that computer information can be represented through audio, graphics, images, video, and animation in addition to traditional media.

Multimedia Software Tools Music Sequencing and Notation Digital Audio Graphics and Image Editing Video Editing Animation Multimedia Authoring

Music Sequencing and Notation Cakewalk: now called Pro Audio. o oThe term sequencer comes from older devices that stored sequences of notes (events, in MIDI). o o MIDI – Musical Instrument Digital Interface. o o MIDI is a protocol adopted by electronic music industry that enables computers, synthesizers, keyboards, and other musical devices to communicate with each other. o o It is a scripting language that codes events that stand for the production of certain sounds. o oIt is also possible to insert WAV files and Windows MCI commands (for animation and video) into music tracks (MCI is a ubiquitous component of the Windows API.) Cubase: another sequencing/editing program, with capabilities similar to those of Cakewalk. It includes some digital audio editing tools. Macromedia Soundedit: mature program for creating audio for multimedia projects and the web that integrates well with other Macromedia products such as Flash and Director.

Digital Audio Digital Audio tools deal with accessing and editing the actual sampled sounds that make up audio: o oCool Edit: a very powerful and popular digital audio toolkit; emulates a professional audio studio —multi-track productions and sound file editing including digital signal processing effects. o oSound Forge: a sophisticated PC-based program for editing audio WAV files. o oPro Tools: a high-end integrated audio production and editing environment — MIDI creation and manipulation; powerful audio mixing, recording, and editing software.

Graphics and Image Editing Adobe Illustrator: a powerful publishing tool from Adobe. Uses vector graphics; graphics can be exported to Web. Adobe Photoshop: the standard in a graphics, image processing and manipulation tool. o oAllows layers of images, graphics, and text that can be separately manipulated for maximum flexibility. o oFilter factory permits creation of sophisticated lighting- effects filters. Macromedia Fireworks: software for making graphics specifically for the web. Macromedia Freehand: a text and web graphics editing tool that supports many bitmap formats such as GIF, PNG, and JPEG.

Video Editing Adobe Premiere: an intuitive, simple video editing tool for nonlinear editing, i.e., putting video clips into any order: o oVideo and audio are arranged in ”tracks”. o oProvides a large number of video and audio tracks, super-impositions and virtual clips. o oA large library of built-in transitions, filters and motions for clips ) effective multimedia productions with little effort. Adobe After Effects: a powerful video editing tool that enables users to add and change existing movies. Can add many effects: lighting, shadows, motion blurring; layers. Final Cut Pro: a video editing tool by Apple; Macintosh only.

Multimedia Authoring Multimedia authoring is the creation of multimedia productions, sometimes called ”movies” or ”presentations”. We are mostly interested in interactive applications.

Multimedia Authoring Metaphors Scripting Language Metaphor: use a special language to enable interactivity (buttons, mouse, etc.), and to allow conditionals, jumps, loops, functions/macros etc. Slide Show Metaphor: A linear presentation by default, although tools exist to perform jumps in slide shows. Hierarchical Metaphor: User-controllable elements are organised into a tree structure – often used in menu-driven applications. Iconic/Flow-control Metaphor: Graphical icons are available in a toolbox, and authoring proceeds by creating a flow chart with icons attached. Frames Metaphor: Like Iconic/Flow-control Metaphor; however links between icons are more conceptual, rather than representing the actual flow of the program. Card/Scripting Metaphor: Uses a simple index-card structure – easy route to producing applications that use hypertext or hypermedia; used in schools.

Multimedia Authoring Metaphors Cast/Score/Scripting Metaphor: o oTime is shown horizontally; like a spreadsheet: rows, or tracks, represent instantiations of characters in a multi- media production. o oMultimedia elements are drawn from a cast of characters, and scripts are basically event procedures or procedures that are triggered by timer events. o oDirector, by Macromedia, is the chief example of this metaphor. Director uses the Lingo scripting language, an object-oriented event-driven language. Hypermedia Linkage – The Hypermedia Linkage paradigm is similar to the Frame paradigm in that it shows conceptual links between elements; however, it lacks the Frame paradigm’s visual linkage metaphor. Tagging – The Tagging paradigm uses tags in text files (for instance, SGML/HTML, SMIL (Synchronised Media Integration Language), VRML) to link pages, provide interactivity and integrate multimedia elements.

Multimedia Authoring Tools Macromedia Flash: allows users to create interactive movies by using the score metaphor, i.e., a timeline arranged in parallel event sequences. Macromedia Director: uses a movie metaphor to create interactive presentations—very powerful and includes a built-in scripting language, Lingo, that allows creation of complex interactive movies. Authorware: a mature, well-supported authoring product based on the Iconic/Flow-control metaphor. Quest: similar to Authorware in many ways, uses a type of flowcharting metaphor. However, the flowchart nodes can encapsulate information in a more abstract way (called frames) than simply subroutine levels.

Programming vs. Authoring Authoring involves the assembly and bringing together of Multimedia with possibly high level graphical interface design and some high level scripting. Programming involves low level assembly and construction and control of Multimedia and involves real languages like C++ and Java. Later in this course we will study: o oJava media programming (Java2D, JAI, JMF (Java Media Framework), Basic Java game programming, Java3D. o oSMIL - Synchronized Multimedia Integration Language. o oVRML/X3D - Virtual Reality Modelling Language.