Computing Multimedia Technology

A multimedia application might be defined as an interactive piece of software communicating to the user using several media, for example text, images (photographs, illustrations), audio (music, sounds), video and animation. What is multimedia?

Multimedia data is either captured from real life (digitised) or created in the computer (synthesised) a bitmap (discrete) type of data or a vector (object oriented) type of data static (like an image) or time-dependent (like a video) You should be able to apply 3 of these terms to every data type.

The Software Development Process and Multimedia Applications Analysis Design Implementation Testing Documentation Evaluation Maintenance

The Software Development Process and Multimedia Applications Analysis: The main purpose of this stage is to be absolutely clear about what the multimedia project is supposed to do.

The Software Development Process and Multimedia Applications Design: Producing a detailed plan which defines what the different parts of the project are and how they are linked together.

The Software Development Process and Multimedia Applications Implementation: This is the point at which the multimedia authoring, or web-page authoring, package is used to turn the design into a working project.

Testing: A series of practical tests are carried out to check that the multimedia project functions properly Documentation: User Guide and Technical Guide. The Software Development Process and Multimedia Applications

Evaluation: Checks that the finished application meets the user’s requirements. Includes an assessment of the HCI design Maintenance: Fixing bugs and adapting the design to suit client needs and the demands of new technology.

Methodologies used in creation of multimedia applications Text editor WYSIWIG editors Multimedia authoring packages Presentation software

Methodologies used in creation of multimedia applications Text editor: Creating simple web pages with HTML using Notepad or TextPad WYSIWYG editors HTML – HyperText Markup Language

Methodologies used in creation of multimedia applications WYSIWIG editors: Web page/site development using applications such Dreamweaver and Freeway Dreamweaver Freeway

Methodologies used in creation of multimedia applications Multimedia authoring packages, e.g. Mediator 9 and Blender Mediator 9BlenderBlender Example videoExample video

Methodologies used in creation of multimedia applications Presentation software, e.g. PowerPoint and Keynote KeynotePowerPoint

Methodologies used to view multimedia applications Displaying multimedia applications in a browser. Displaying multimedia applications using a file player or viewer.

Methodologies used to view multimedia applications Multimedia application as an executable file

Bit-mappedBit-mapped Graphic Data Capturing still images using a digital camera Using an array of Charged Coupled Devices to capture light coming in through a lens CCD: a series of sensors to capture light Capturing images using a scanner: also uses CCD

Bit-mapped Graphic Data Storing graphics as a bit map: each pixel in the image is represented by a binary number. Uncompressed bit-map format: a file which holds a binary number for each pixel in an image. Large file size: main limitation of bit-map format. Need for compression: to relieve demands on storage and transmission times.

Bit-mapped Graphic Data Compression using GIF format Lossless compression: GIF format compresses graphic data without losing any information about the image. It compresses by encoding repeated patterns of data.Lossless compressionGIF Limited number of colours: limited to 2 8, 256 colours Transparency: colours set as transparent let the background colours and patterns show through. Used for storing cartoons, and line drawings. Replacing GIF is PNGPNG

Bit-mapped Graphic Data JPEG Joint Photographic Experts Group: a commonly used method of lossy compression for digital photography (image). parts of the graphic are cut out, e.g. shades of colour. At low rates of compression this is not noticeable. JPEG

Bit-mapped Graphic Data JPEG JPEG format suitable for storing photographs and paintings. JPEG – 24 bit colour know as True Colour giving 16,777,216 colours

RGB colour codes The RGB colour code is a system for representing colours in a computer system using a combination of the three primary colours red, green and blue RGB typically uses 24 bits to represent a colour by specifying the amount of each colour in 8 bits RGB(255,255,0) is yellow (red and green) RGB(255,105,180) is “hot pink”- mainly red with some green and blue RGB colour coder Colour chart

Bit-mapped Graphic Data Trading quality for file size Resolution: increasing resolution increases the number of pixels, can improve the quality of a graphic but increases the file size. Colour depth: increasing colour depth increases the number of colours or shades of grey, can improve the quality of a graphic but increases the file size. Lossy compression: reduces file size and, providing the rate of compression is not too high, does not affect the quality of the graphic.

Bit-mapped Graphic Data Simple bit-map editing and creation software Painting programs Fill tool: a feature for pouring colour into a graphic. Paintbrush tool: for more precise application of colour.

Bit-mapped Graphic Data Main features of image editing programs Decrease resolution Alter colour depth Crop Alter brightness and contrast Insert graphic Re-size.

Technical terms Dithering Anti-aliasing Re-sampling

Bit-mapped Graphic Data Hardware for displaying 2D graphics CRT monitor: bulky, heavy and consume more power the LCD/TFT screens LCD and TFT screens: flat, light, and need less power than CRT monitors CRT in specialised markets LCDTFT

Bit-mapped Graphic Data Hardware for displaying graphic data Need for graphics cards to store and process graphic data, relieving the main processor of the task.

Digitised Sound Data Hardware for capturing sound Microphone Sound cards: to sample, store and process audio data.

Digitised Sound Data Uncompressed sound data RAW: Uncompressed samples of sound wavesRAW RIFF: Resource Interchange File FormatRIFF WAV: Microsoft's format for sound files, part of RIFFWAV

Digitised Sound Data Lossy compression: reduces file sizes by cutting out some of the data. MP3: uses lossy compression without noticeable loss of sound quality.

Digitised Sound Data Balancing file size and sound quality Sampling depth: increased sound depth = greater range of values = better sound quality and greater file size. Sampling frequency: The higher the sampling frequency, the better the sound quality, the greater the file size. Sound time: affects file size but not quality.

Digitised Sound Data Simple sound editing software Reducing sample frequency, e.g. from 44.1 KHz to KHz, reduces file size and audio quality. Reducing sample depth, e.g. from 16 bits to 8 bits per sample, reduces file size and audio quality.

Digitised Sound Data Editing sound file features: Volume Effects Echo Reverse

Digitised Sound Data Sound cards and playback: Sound cards needed to change the digital audio data into analogue signals to control output from speakers.

Video Data Hardware for capturing images Digital video camera Web cam

Video Data Storing video data Uncompressed format Uncompressed video data = Large file sizes 1 second of uncompressed wide-screen video can take up 53 Megabytes of storage AVI: Audio Visual Interleave, an uncompressed format.

Video Data Compressing video data MPEG Lossy compression: cuts out unnecessary parts of a video clip

Video Data Compressing video data Using MPEG compression: Reduces file sizes With no noticeable loss of video quality Only key frames are stored in full as a JPEG Then only the differences between key frames are stored

Calculating the file size of video Find the file size of a single frame –Use resolution and colour depth Multiply the size of a single frame by the frame rate and number of seconds in the clip (R*CD) * (fps*L)

Calculating the file size of video 20 minute video A single frame 768*576 = *24 = /8 = (bytes) /1024 = 1296/1024 = MB Frame rate and length *25= *20*60 = /1024 = Gigabytes

Video Data Video quality and file size Colour depth: increasing colour depth improves quality and file sizes. Resolution: increasing resolution improves quality and increases file sizes. Frame rate: measured in frames per second, fps. 30fps is the rate for a video clip. Increasing frame rate increases file size. Lower frame rates reduce file size but make video clip ‘jerky’.

Video Data Video quality and file size Video time: increasing or reducing the time of a video is the obvious way to affect the file size. Quality of the display of the clip is not affected. Lossy compression: Using MPEG compression reduces file sizes without affecting quality.

Video Data Video editing software features and applications used with single video clips Cropping: cutting unwanted data from the beginning and end of a clip. Add effects, titles, sound tracks. The need for graphics cards to process and output video data.

Vector Graphics Data VG store an image as a list of individual layered objects with each object being described by its attributes

3D objects in vector graphics The attributes will be the same as flat (2D) objects but with a few extra ones angle of rotation, the x, y and z axis need to be defined surface texture, the surface appearance of the 3D object shadow, the position and intensity of the light source

Vector Graphics Data Standard formats for vector graphics Scalable Vector Graphics (SVG) format Scalable: resolution independent Vector: represents objects by defining a series of attributes

Vector Graphics Data Standard formats for vector graphics Virtual Reality Markup Language A standard language used to model and animate geometric shapes Used to define 3D environments for the WWW.

A comparison of Vector and Bitmapped Graphics

Synthesised Sound Data Musical Instrument Digital Interface: MIDI Common attributes of notes stored as MIDI data: –Instrument: defines which instrument is playing –Pitch: defines the height of the note –Volume: determines the amplitude –Duration: determines the length of the note.

Synthesised Sound Data Common attributes of notes stored as MIDI data Duration: determines the length of the note. Tempo: the speed at which a piece of music is played.

MIDI hardware/software MIDI keyboards, or other instruments, are used to create sound data in MIDI format Or just use virtual instruments through MIDI software

Advantages of MIDI Music can be produced at the computer with an easy to use interface no instruments required Individual instruments and notes can be easily edited. MIDI format has much smaller file size than digitised sound data as it is the attributes of the sound that is stored not every sound sample. The MIDI format is a common standard which allows the musical data to be passed between devices made by different manufactures of musical instruments and a computer

Disadvantages of MIDI Synthesised sound, poor representation of human voice and other naturally occurring non uniform sounds Limited realism, sound can be “electronic” sounding Sound quality, dependent on quality of soundcard

Converging technologies Converging technologies means two or more technologies being combined, some examples are smartphones, Pocket PCs, digital television and tablet computers.

Implications: Multimedia Technologies Converging contemporary technologies Smartphone: merging technologies of a mobile phone and a laptop. Pocket PC: merging technologies of a laptop, mobile phone and desktop operating system and application software.

Implications of Multimedia Technologies Contemporary technologies Digital television: an interactive multimedia device which, because of the digital nature of its signals, is easily integrated into your digital computer and home networks.

Implications of Multimedia Technologies Contemporary technologies Virtual reality The ultimate multimedia experience where the user is immersed in the world of the computer and can journey through, and interact with, a computer generated 3- dimensional multimedia world.

Virtual reality Used for For training e.g. pilots Creating and inspecting and testing a 3D CAD model Simulating scientific processes Forecasting e.g. weather Gaming

Implications of Multimedia Technologies Immersive VR Output :Using speakers, stereo screens, headsets Input: sensors in gloves, headsets and suits