Class 11 LBSC 690 Information Technology Multimedia

Agenda Questions Images Audio Transmission Virtual reality Midterm structure Midterm review

Basic Image Coding Raster of picture elements (pixels) –Each pixel has a “color” Binary - black/white (1 bit) Grayscale (8 bits) Color (each pixel has three dots) –Red, green, blue Screen –A 1024x768 image requires 2.4 MB So a picture is worth 400,000 words!

Some Questions Use this to answer: –How many images can a 2 GB hard drive store? Nobody would use images! –How long does it take to send one by modem? Imagine how slowly web pages would load But real images don’t have these problems –How do we get around these problems?

Compression General goal: reduce redundancy –Send the same information using fewer bits more telephone calls in one cable more faxes per minute more images stored per disk better quality video images Two basic strategies: –Lossless, Lossy –The two strategies can be combined

Lossy Compression Example - Palette selection –No picture uses all 16 mission colors –Select a palette of 256 colors Can represent with 1byte instead of 3 Then look up each color in the palette JPEG (.jpg) –Standard lossy compression for images –Eliminates detail that’s not seen by humans –Uses frequency representation

Lossless Compression Run Length Encoding (RLE) –Pixels are organized into lines –Most pixels are the same as the one before That can be coded in 1 bit (1/24 the space) –Smaller files take less time to transmit GIF (.gif) –Standard lossless compression format

Moving Images One image frame is much like the next –An additional source of redundancy MPEG-1 (.mpg) can handle small screens –Compression requires extensive computation Special purpose hardware needed to run in real-time –Pentium processors can decode it MPEG-2 is needed for full-screen video –Not yet widely used

Audio Sample at twice the highest frequency –One or two bytes per sample –Voice (0-4 kHz) requires 8 kB/s –Music (0-22kHz) requires 44 kB/s Compression strategies –Lossy is pretty good for voice Only some of the frequencies are actually used –Lossless is better for music

Transmission MIME –Attach a standard format to message Formats include.gif,.jpg,.mpg, and.au Messages include and web pages –The whole file is sent first, then played

Streaming Audio and Video Streaming protocols –Replay starts almost immediately –RealVideo has emerged as the standard Streaming video challenges –Sent in small packets –Sometimes arrive out of order –Compensate by storing some in a buffer Introduces a delay Modems carry audio better than video –Video data requires high “bandwidth” –Real Video compensates with lower frame rates

Virtual Reality Displays Visual –Position/motion, color/contrast, symbols Auditory –Position/motion, tones, speech Haptic –Mechanical, thermal, electrical, kinesthethic Olfactory –Smell, taste Vestibular

Types of Displays Image display –Fixed view, movable view, projection Acoustic display –Headphones, speakers, within-ear monitors Tactile display –vibrotactile, pneumatic Force feedback –dexterous handmaster, joystick, pen

More Display Types Inertial Display –Motion-based simulators Olfactory Display –Chemical (requires re-supply) Locomotive display –Stationary bicycle, treadmill,... Temperature Display

Wearable Visual Interfaces Head mounted displays –Two liquid crystal (camcorder-like) displays –Cables for video and position tracking Binocular omni-oriented monitor –Simpler and more comfortable –Limited range of motion Shuttered glasses –Used with fixed and projection displays

Some Input Techniques Arms and hands –Aimed movement, grasp, press, tap, sign/gesture, write/draw/paint/sculpt, crawl/swing, rub Legs –Walk/run/skip/hop/crawl, swim, aimed movement Face –Expression, speak, sing/hum, breathe Body –Posture, appearance/clothing

Input Devices Position tracking –Mechanical, magnetic, acoustic, GPS, inertial Gloves and suits –Fiber optic, flexible strip, exoskeleton Locomotive –Stationary bicycle, treadmill, stairs, shoes, rowing Conventional input devices (haptic, speech) Image processing (position, gesture) Facial sensing, eye tracking, biosignal

Position Tracking Magnetic –Three fixed emitters, three movable sensors Acoustic –Time delay systems have variable update rates –Phase coherent systems can jump lanes Differential GPS –Requires unobstructed sight lines to satellites Inertial –Large, short term, requires initial calibration

Exam Structure One hour and 15 minutes Approximately 4 questions –Each may have multiple parts Open Book (Oakman only) –You may hand write anything in your Oakman –No extra pages of notes The software you may use will be specified You may bring a calculator

Exam Advice The only goal is to get points! –Spend each minute in the best place Develop a strategy for each question type –Guessing CAN hurt on multiple choice –Don’t write a page when a sentence will do Study concepts, not details –Grading rewards conceptual understanding –Don’t expect a clone of the sample exams