Class 13 LBSC 690 Information Technology More Multimedia Compression and Recognition, and Social Issues

Agenda (fall 99 only) Questions Finish Multimedia (week 9) Multi-modal recognition NetMeeting Risks Analysis Social and Economic Impact

Moving Images One image frame is much like the next –An additional source of redundancy MPEG-1 (.mpg) can handle small screens –Compression requires extensive computation Special purpose hardware needed to run in real-time –Pentium processors can decode it MPEG-2 is needed for full-screen video –Not yet widely used by computers Try video from learn.umd.edu site

Audio In most cases, people care more about high quality audio than high quality images Sample at twice the highest frequency –One or two bytes per sample –Voice (0-4 kHz) requires 8 kB/s –Music (0-22kHz) requires 44 kB/s Compression strategies –Lossy is pretty good for voice Only some of the frequencies are actually used –Lossless is better for music

Transmission - Streaming Audio and Video Streaming protocols –Replay starts almost immediately –RealVideo has emerged as the standard Streaming video challenges –Sent in small packets –Sometimes arrive out of order –Compensate by storing some in a buffer Introduces a delay Modems carry audio better than video –Video data requires high “bandwidth” –Real Video compensates with lower frame rates

Representation and Recognition Semantic representations versus pixels –(MPEG7) Good representations can lead to good recognition Approaches to recognition –Bottom-up –Top-down –Up-down (model-based recognition)

Document and Optical Character Recognition (OCR) Accuracy –99.5% accuracy required for library quality –Some effective retrieval with 60-70% accuracy Cross-Language –Can’t necessarily match character templates –Need to start with character fragments (glyphs)

Speech and Language Recognition Phonemes –Building blocks of a spoken language –about 56 in English (long-a, vs short-a) Statistical language processing vs. algorithmic language processing

Visual Object and Scene Recognition What are good visual representations? –Text descriptions –Templates –Elemental Visual Features Scene representations –Scene graph

Gesture and Video Recognition Structured vs. unstructured video Characterizing the context and time-course of motion Multi-modal recognition –face recognition vs. person recogntion

Simulations Models vs. Representations How to show a process Interactive simulations –for training –Avatars

Summary Compression is needed to make multimedia manageable. More “semantic” representations are now possible because of more computer power at the received (client).

Review Social Issues (week 7) Control over information Copyright Authentication

Risk Analysis Pick one from your birthday for Summarize for the class in 1 minute the issues and what you think a solution would be.

Complex System Issues Critical system availability –Who needs warfare - we do it to ourselves! Understandability –Why can’t we predict what systems will do? Nature of bugs –Why can’t we get rid of them? Audit-ability –How can we learn to do better in the future?

Big Picture Social Issues Many important social institutions (e.g, libraries, government, schools) are based on managing the flow of information. How will they be changed by electronic information technology? –Direct democracy? Every citizen could vote via the web End of privacy? Personal data about you will be more public –e.g., your purchases with grocery electronic coupons Almost every action may be tracable –current example is EZ Pass records

Economic Questions Economy of attention? End of money? “Content is king” ?

Project Demonstrations Scheduled in advance Targeted for 30 minutes Web pages in my office Databases anywhere you need to be Sometime during Finals week –Get your reservations in early! At least one person from a team must come –But everyone is welcome!

Project Demonstrations Starts with a brief structured test –I’ll pick a spec requirement and use the test plan Then you can show me around –You guide, I’ll drive I’ll run off the road a lot, though, to see what happens For web pages, I may try another browser or two I’ll award the group’s grade on the spot –Perfection is not required, only greatness!