Engaging Technolog(ies) for Effective Interaction Jeremy R. Cooperstock Centre for Intelligent Machines McGill University Centre for Interdisciplinary Research in Music Media and Technology McGill University

A Brief Roadmap  Shared Reality  Intelligent Classrooms  Ultra-Videoconferencing

Inspiration Mark Weiser “The most profound technologies are those that disappear.” Jef Raskin “Graphical User Interfaces are not human-compatible.”

Classroom Technology

“High-tech” touch-screen control…

Simplified button control light controls motor controls context- sensitive volume control

I don’t want to control all of that!

Solution: Intelligent Classroom “Reactive” Technology  room senses events….  and responds automatically

Full-featured button panel left right

Simplifying Lecture Capture

What about the cameraman?

Student Questionnaires  technology viewed favourably by most users  :  90 survey responses, 83% positive  :  115 survey responses, 77% positive  (most negative comments related to technical problems)  :  110 survey responses, 85% positive

Videoconferencing: the past (AT&T PicturePhone – 1956)

Videoconferencing: the future? Zack Snyder’s “Online Jam”, October 2001

For those with DSL at home... yeah, right!

You don’t say! "As depicted in the spot, it's a bit of a stretch for the home consumer to run that instantaneously and smoothly; that's more the realm of the professional musician…. there's a little more to it than just going on the net."

"Playing Together" experiment New York-Ottawa, Remote Masters class, Dec. 8, 2000 Pinchas Zuckerman, Columbia U, NYC Wu Ji, Canarie ARDNOC, Ottawa

Latency: The Interaction-Killer or put another way...

Tolerated Synchronicity compress transmit ms speed of light: Mtl-LA net latency Mtl-LA networks small ensemble large ensemble lip synch musical activity * This is a theoretical “best-case” scenario. Actual performance may vary. The manufacturer makes no warranties, either express or implied, and will not be held liable for damage resulting from use of the product. Certain restrictions may apply. Offer void where prohibited by law. Batteries not included. MPEG* acquire

Distributed Violin Duet McGill-Université de Montréal for RISQnet Conference, Nov. 21, 2001

"The medium is the message" - Marshall McLuhan Cooperstock's Corollary: "The medium must be faster than the message.” * * but between protocol deficiencies (TCP, UDP), competing data flows, process scheduling on a budget, ARQ (retransmission management) without QoS, interface HW/SW “uncooperativeness”, aggressive router policies, multi-format support, echo suppression, and interface complexity… it’s much more than just the medium

Note:look of anguish

The Brass Montreal-Ottawa (NRC) Trial with Pace Sturdevant, February 2, 2002

Violin Masters Class Montreal-Ottawa (NRC) Trial with Pinchas Zuckerman, Feb. 20, 2002

Size does matter

Distributed Jazz Jam Montreal-Stanford (CCRMA), Research Demonstration, Oct. 24, 2002

Remote Video Interpretation (Sign Language) Montreal-Fredericton (UNB), research trials, 2003 physician's setup patient's setup

Shared Reality and Shared Spaces: Toward Immersive Telepresence

Vibrosensory Transmission VRQ demo, March 8, 2004

… and another application

Next Steps: Multi-screen High-Def Funded by Canarie’s Advanced Applications Program

Conference Room Configuration

New challenge: Real-time mosaicing

Sample Result: Can your camera do this?

And going deep under… Funded by Canarie’s Intelligent Infrastructure Program and NEPTUNE

Thank you... software download: Shared Reality Lab: