Mixed Reality University of Central Florida Media Convergence IST School of Electrical Engineering & Computer Science Digital Media Department.

Slides:



Advertisements
Similar presentations
SEMINAR ON VIRTUAL REALITY 25-Mar-17
Advertisements

EU funded FP7: Oct 11 – Sep 14 Co-evolution of Future AR Mobile Platforms Paul Chippendale, Bruno Kessler Foundation FBK, Italy.
IT Issues and Support Structures Simulation Education and Complex Technology Based Practice.
Mixed Reality Reality – Virtuality Continuum Antonio Camurri Casa Paganini – InfoMus Lab –
Augmented Reality David Johnson.
Lecture 9: Virtual and augmented environments for design Dr. Xiangyu Wang.
WHAT IS HAPPENING TODAY!? SIT BACK, RELAX, AND STEP INTO THE WONDERFUL WORLD OF SHANNON…IF YOU DARE!
Interactive Systems Technical Design
Ch 1 Intro to Graphics page 1CS 367 First Day Agenda Best course you have ever had (survey) Info Cards Name, , Nickname C / C++ experience, EOS experience.
FYP Project LYU0303: 1 Video Object Tracking and Replacement for Post TV Production.
Tangible Flags Collaborative Educational Technology to enhance grade school field trips Gene Chipman PhD Candidate in Computer Science
© 2010 Pearson Addison-Wesley. All rights reserved. Addison Wesley is an imprint of 1-1 HCI Human Computer Interaction Week 10.
Introduction to Virtual Environments CISE 6930/4930
Introduction to Virtual Environments CIS 4930/6930
Integration Of CG & Live-Action For Cinematic Visual Effects by Amarnath Director, Octopus Media School.
Introduction to Graphics and Virtual Environments.
©2005 University of Central Floridawww.mcl.ucf.eduMedia Convergence Laboratory Provocative Statement: Truly interactive stories have yet to be successfully.
Audio Visual and Classroom Technology Support Computer and Networking Services IST.
People in multimedia Systems. Multimedia Systems Multimedia systems are designed by a team of people who specialise in a particular field, For example:
CHAPTER 2 Communications, Networks, the Internet, and the World Wide Web.
Bringing Real Context with a Virtual World: New Rehabilitation Strategies for Parkinson Disease Angela Roberts-South, Mandar Jog National Parkinson Foundation.
Welcome to CGMB574 Virtual Reality Computer Graphics and Multimedia Department.
Where’s the Reality in Augmented Reality? Mark Billinghurst HIT Lab NZ University of Canterbury.
E-LIXIR PRESENTATION We help you with both your internal and external communication projects PR programmes, conferences, exhibitions, road shows, seminars.
Jeanne Corcoran, OTD, OTR/L October 6 th, The mission of Envision Center for Data Perceptualization is to serve, support, and collaborate with faculty,
Introduction to Virtual Environments Slater, Sherman and Bowman readings.
3D COMPUTER GRAPHICS IMD Chapter 1: 3D Computer Graphics Chapter 1: 1 Lecturer: Norhayati Mohd Amin.
Augmented and mixed reality (AR & MR)
2 CEVI (Interactive Rendering Center) is a research center linked to the Language and Information Systems department in the Universitat Jaume I. Research.
Welcome to 14:332:476 Virtual Reality Spring 2008 Grigore C. Burdea Ph.D. Director, Human–Machine Interface Laboratory, CAIP Center, Rutgers University.
 Multi (Latin): Many/Much  Media (Latin): An intervening substance through which something is transmitted on.
Chapter 8 Augmented reality and computer supported cooperative work 1.
Tangible Bits Next Generation HCI
Miguel Tavares Coimbra
GENESIS OF VIRTUAL REALITY  The term ‘Virtual reality’ (VR) was initially coined by Jaron Lanier, founder of VPL Research (1989)..
Encuentro/Rencontre/Meeting VR and IG Girona, Spain, Dec 19-20, 2007 An INRIA Project-team in partnership with four other institutions Stéphane Donikian.
Modeling - Simulation and AI Software ©Ideler2002.
WP Strategic Objective Networked Audio Visual Systems and Home Platforms.
Augmented reality Prepared by: Khyati Kataria 08DCE001
1 Introduction to Virtual Environments User Interfaces and Usability Fall 09 John Quarles
W E L C O M E. A U G M E N T E D R E A L I T Y A SEMINAR BY JEFFREY J M EC7A ROLL NO:
HCI 입문 Graphics Korea University HCI System 2005 년 2 학기 김 창 헌.
Mixed Reality Trompe l’oëil in the 21st Century
FYP Project LYU0304: “Monster Battle”: A Prototype of Augmented Reality Card Game.
© Anselm Spoerri Lecture 9 Augmented Reality –Definition –Visualization Approaches –Applications –Tools and Links.
Augmented Reality Authorized By: Miss.Trupti Pardeshi. NDMVP, Comp Dept. Augmented Reality 1/ 23.
Special Topic: Virtual Reality, Mixed Reality, Augmented Reality, Augmented Virtuality – based Collaborative Virtual Environments Dr. Xiangyu Wang.
TELE IMMERSION AMAN BABBER
Scientific Visualization Facilities The Digital Worlds Institute Andy Quay Associate Director Digital Worlds Institute University of Florida.
Multimedia. A medium (plural media) is something that a presenter can use for presentation of information Two basic ways to present information are: –Unimedium.
Mixed Reality Conferencing Hirokazu Kato, Mark Billinghurst HIT Lab., University of Washington.
3D Animation 1. Introduction Dr. Ashraf Y. Maghari Information Technology Islamic University of Gaza Ref. Book: The Art of Maya.
Virtual Reality - by Arpit Gupta. Introduction  Virtual reality is an artificial environment that is created with software and presented to the user.
Crowds (and research in computer animation and games)
Mixed Reality Benjamin Lok.
Augmented Reality.
Introduction to Virtual Environments & Virtual Reality
Augmented Reality And Virtual Reality.
M.Tech Technical Seminar Presentation
Crowds (and research in computer animation and games)
Image Based Modeling and Rendering (PI: Malik)
Virtual Reality.
Smart Learning concepts to enhance SMART Universities in Africa
Experience the non-Existent through Mixed Reality Paradigm
Spatial Augmented Reality Oliver Bimber
Lecture 9: Virtual and augmented environments for design
February 27th, 2004 Benjamin Lok
Spatial Augmented Reality Oliver Bimber
December 12th, 2003 Benjamin Lok
Presentation transcript:

Mixed Reality University of Central Florida Media Convergence IST School of Electrical Engineering & Computer Science Digital Media Department

Media Convergence Laboratory Using mixed reality to solve real problems

The faces of MR Physical Reality (PR) – real world Virtual Reality (VR) – purely synthetic Augmented Reality (AR) – virtual assets registered in real world Augmented Virtuality (AV) – real (people, props) layered in virtual space

Mixed Reality Properties Blends the real and the synthetic into a single landscape & experience Addresses multiple senses Requires proper registration of real and virtual, relative to each other Typically requires complex behaviors of virtual characters Enables buy-in through imagination

Limitations along the continuum PR is constrained by physical space VR limits person-to-person expression and context of PR AR often limits escape from real world AV allows rich layering In general, we want to move smoothly along the MR continuum

Visual capture Optical Video HMDs Light capture 3d laser mocap

Visual rendering Dome Screens Flat World Demo Dome MR Windows HMDs

Audio capture / rendering Surround Hydrophones Holophone Mixing Designing Delivery in constrained settings

Pipeline: Audio for VR/MR Planning & Prescripting Capturing Synthesizing, Mixing, and Mastering Designing Sound & Integrating Delivering

Special effects - Colorkinetics SmartJack3 (USB to DMX) - Colorkinetics JuiceBox2 / iColor MR Lights - Gilderfluke MP3-50/ Channel Dimmer Packs - Pneumatic / Smoke System - Sound Transducers (”Bass Shakers”) - Colorkinetics SmartJack3 (USB to DMX) - Colorkinetics JuiceBox2 / iColor MR Lights - Gilderfluke MP3-50/ Channel Dimmer Packs - Pneumatic / Smoke System - Sound Transducers (”Bass Shakers”)

Tracking Technologies Magnetic Optical Acoustical Inertial Vision (often with markers/features) Hybrid (hardware and soft/hardware)

What we currently use Wide field of view: 51 degree in horizontal direction, 37 degree in vertical direction-high resolution: VGA (640 × 480) weight: 286 g standard camera I/O: NTSC Coastar (Co-Optical Axis for AR) InterTrax2 Wireless Tracker IS 900 Wide Area Tracker

Registration / Illumination Virtual and real must be properly placed relative to each other Inter-occlusion must be properly managed Mutual shadowing must occur, including shadows from real caused by virtual light The effects of ambient light (real and virtual) must be rendered

Story Virtual characters must have appropriate behaviors, reactive and proactive Virtual characters often need to mimic those of real characters Capture, replay must be provided for entertainment, after-action review, etc.

Putting it together

Content is king Create experiences that have long-term impact Experiential movie trailers Situational awareness training Creative collaboration Cognitive and physical rehabilitation Teacher screening and training Free-choice learning … Imagination is the most important dimension

Siggraph ‘03 – experiential trailer

Situational awareness MR MINI-MOUT

Peer Collaboration

Along the Continuum Start in PR (look at current plant) Move to AR (add new equipment and new windows) Individual jumps out to VR to privately review designs Move to AV as all are surrounded by new design, but still see each other

What We Must Support Jointly visualized “what if?” scenarios Face-to-face interactions Alternative and often opposite POVs Personal and group creation Enablers Tangible and tactile components Constructive distractions (sandbox) Shared display (1 st & 3 rd person views) Shared interfaces Scalability, interoperability & portability

Cognitive rehabilitation Patient and therapist in context of patient’s kitchen but in safety of clinic patient at home reconfigurable kitchen as seen thru HMD movement for after action review real items to use tethered for now

Pre vs. Post Milk Spoon Cereal Bowls Make Cereal Bowls Cereal Spoon Make Cereal Pre-testing HomePost-testing Home

Stuttering

Teacher screening and training Cost to hire a teacher is $15,000 Cost to remove can be much higher Turnover in inner city is 50% in 3 years Many last less than a year Children are unintended guinea pigs Want to select those who can succeed Want to help those who want to succeed

Front room – early prototype

Back room -- puppeteers

Interaction with a Virtual Class In front of class At desk of student Personal contact AI for all but student being directly addressed Vision-based tracking

Free-choice learning

Water’s journey Coming to a science center near you Initial focus is on the Everglades But, water is central to all ecologies Move clock backwards or forwards Real blends to virtual User-chosen policies affect what is experienced

Forest fire visualizations Fire spread model from Farsite Tree rendering from SpeedTree Fire and smoke visualization is ours Used in environmental economics & Everglades museum projects

Some basic research Blending of real and virtual in changing illumination Overcoming HMD camera limitations Real-time video chroma keying in presence of noise and inconsistent lighting LOD management (forests) Neoroevolution for virtual character behaviors Painting in HDR

Lab people characteristics Original director from theater and theme parks Lab manager from museum industry Chief programmer from Lego Content delivery people from digital media, children’s literature, audio, theater, museum, … Content providers from all over campus PhD students have interests in graphics, neuroevolution, vision, ecology, software engineering, simulation & modeling Undergraduates are mainly from CS and DM

Charles E. Hughes Professor & Associate Director School of Electrical Engineering & Computer Science Professor Digital Media Director, Media Convergence Lab Home Page: Graphics Lab: Media Convergence Lab: Contact information

J. Michael Moshell Professor Digital Media Professor School of Electrical Engineering & Computer Science Co-founder, Media Convergence Lab Home Page: Media Convergence Lab: Contact information