Lecture1. Introduction to Virtual Reality CSCE 458 Fall 2004 Chang-Hun Kim Department of Computer Science Korea University

SIGGRAPH 2004 Video Review Electronic theater program Go To Sleep [5:21], The Mill 3D music video Rock the World [2:35], School of Vision Art Political satire Rockfish [8:28], Blur Studio Realistic rendering Animation theater program Oddworld Stranger CG Intro [2:49], Oddworld Inhabitants Commercial animation

SIGGRAPH 2004 Video Review Animation theater program Riba [5:57], One Plus One / Supinfocom Non-photorealistic rendering (NPR) “Muscle System and Skin Solver” on Hellboy [2:17], Tippett Studio Muscle-based animation “Machine City” in the Matrix Revolutions [3:18], Tippett Studio Realistic rendering Massive Arabesque [2:50], Microsoft Research Augmented Reality

Contents What is VR? Related Areas History State of the Art and Research Issues

What is VR? Glove  CG Machine  HMD Concept This is not exactly VR!!! Concept High level: Replication of reality Low level: 3D interface / interaction Studies on reality  Computational reality Analogous to AI AI as a science / engineering Science: Computational model for intelligence Engineering: Making “intelligent-looking” machines

Target of VR VR = Computational reality Seek for the computational model of reality Apply the model to the VR system The essence of science: Search for models Astrology: Heavenly bodies (stars, sky) Physics: Nature Philosophy: Human mind Virtual reality: Computational model of reality

Understanding VR in the Context of Modeling Efforts Intelligence  AI Life  Artificial life Reality  Artificial (or virtual) reality Economy  Computational economy

VR Terminologies (1/2) Virtual reality Computer generated environment that is ... Immersive ex) IMAX, dome projection Interactive ex) Computer game Multi-sensory Viewer-centered 3D Combination of various technologies To build the environments above Physical participation in the interactive simulation

VR Terminologies (2/2) Virtual environment (VE) Cyberspace A computational model or Realization of it by VR techniques Cyberspace A computer environment spanning ... Multiple computers, Multiple users, and Multiple sets of data

VR Devices

VR as a Computer Technology Ultimate computer interface Human body and its sensor Turn the whole body into an input device Technological trends Powerful: Supercomputing, parallel computing Smart: AI Physical: VR Primary concern Software Other important issues Hardware, human factors, social issues, infrastructure

VR vs. Existing Media Immersive Interactive 3D Multi-modal Mediated Information is not sent back and forth directly Mediated virtual environments

Virtual Presence (1/2) Sense of “being there(or presence)” Effectiveness of communication depends on VP Medium create VP: Movie, poem, novel, TV, ... Feeling of Virtual Presence Reading a book BBS, chatting Listening to live recording of music Virtual Presence = f(Vividness, Interactivity) Vividness = f(Sensory breadth, Sensory depth)

Virtual Presence (2/2) Sensory breadth Number of sensory modalities Some of modalities may be redundant, but enhance the presence ex) Rainstorm scene: Raindrops, its images & sound, smell (Printer, Telephone, TV) vs. (Motion platform, Sensorama) Sensory depth: (Amount / Quality) of information Movies : 35mm, Cinemascope, 70mm, IMAX,… Interactivity Response time Range: Amount of changes that can be effected on the VEs Mapping: Human actions to virtual actions Vividness vs. Interactivity

HCI Conceptual Model Basic components of VR system Issue Machine Interface Human Issue How to put them together? How to conceptualize the system HCI

New Paradigm for HCI Conventional Computer VR-based Computer Computer World Conventional Computer Virtual World VR-based Computer

Related Areas Tele-Operation, Simulator Technology Computer Graphics, Artificial Intelligence

1. Training Simulation (1/2) ex) Flight simulation Modern flight simulator is considered to be the most sophisticated application of a virtual environment. This photograph depicts a state-of-art, Boeing 777, full-flight simulator with a 6 DOF motion platform, panoramic collimated display, 3D sound system and force feedback flight controls with the ability to take-off or land at any international airport in the world. (Courtesy Thomson Training & Simulation Ltd)

1. Training Simulation (2/2) Characteristics Reconfigurable by changing software May include highly unnatural environment Highly interactive and adaptive A wide variety of Human sensing modalities and Sensory-motor systems Highly immersive Difference Near field can be real, far field is synthetic VR: Near field is simulated

2. Tele-operation (1/2)

2. Tele-operation (2/2) Tele-operator Tele-presence A machine is controlled by a human at a distance Tele-presence Sense of being “Physically Present” With virtual objects at the remote tele-operator site

Various Tele-operators Directly (manually) controlled tele-operator Continuously controlled by human Master-slave : Kinematically equivalent Rate-controlled : Controls velocity Tele-robot Limited sensing or manipulation tasks autonomously Needs tele-robot’s intelligence Supervisory control Needs: tele-sensor, tele-manipulator RE, VE, Tele-oper Tele+VE: overcome time delay VE+RE: Augmented Reality Tele-presence vs. Virtual Presence Sence of being physically present with virtual objects at remote TO site

3. Computer Graphics Modeling Motion control (animation) Rendering User interface

4. Artificial Intelligence Get: Studies on perception and cognition Conditions for perceiving VE as real as possible Perception and cognition for the virtual agent Give: Testbed for AI research

History 1st Stage: Some Visionaries 2nd Stage: Technology for Specific Purposes 3rd Stage: VR as the General-Purpose Technology Next Stage: Toward a Scientific Discipline

1st Stage: Some Visionaries Morton Heilig ExperienceTheater (1962), Sensorama Simulator (1962) Ivan Surtherland “Sketchpad: a man machine graphical communication system”(1963) Ph.D thesis Head-mounted display (HMD) (1966) Myron Krueger “Artificial Reality” (1972) -- new word William Gibson “Cyberspace” (1984)

Heilig’s HMD (1960) U.S. Patent no. 2,955,156

Heilig’s Sensorama (1962) U.S. Patent no. 3,050,870

2nd Stage: Technology for Specific purposes Training simulator Edwin Link: Vehicle simulation (1920s) Evans&Sutherland, Martin-Marietta, GE, Westinghouse, Thomson Space exploration NASA for simulation for astronaut training First VR system -- VIEW(Virtual Interface Environment Workstation) Tele-operation Began as early as 1940 Head Controlled TV system by Argon Lab.

Training Simulator

3rd Stage: VR as the General-Purpose Technology Jaron Lanier VPL: Dataglove, EyePhone, VR system (1987) Coined “virtual reality”(1989) VR as CHI perspective VR industry Frontiers: Division, Sense8(WorldToolKit), WorldDesign, W-industry(game) Merge into industry: Proslovia, EAI VR academia MIT, UNC(University of North Carolina), UW(University of Washington), Naval Postgraduate School, Tokyo University

Next Stage: Toward a Scientific Discipline Computational Reality A new computing paradigm A new media: 3D in the web A new art form Representation, Creation and Operation of Virtual World

Current Research Issues Computer generation of VEs Design of tele-robots Improvement of Multi-Modal Interfaces Study of human behavior Development of communicating systems for VE Merging VE and RE (Real Environment)

Information Resources Journals Conference Proceedings Magazines and Newsletters Books Web Sites

Journals Presence International Journal of Virtual Reality MIT Press, quarterly published since 1992. tele-robotics, psychophysics, human factors, HCI, not much computer graphics. International Journal of Virtual Reality England

Conference Proceedings (1/2) IEEE VR (formerly Virtual Reality Annual International Symposium (VRAIS)) VRAIS – held in 1993, 1995, 1996, 1997 IEEE VR – since 1998 Virtual Reality Software Technology (VRST) Conference ACM SIGCHI and Pacific-Rim countries, since 1994. VR software. International Conference on Virtual Sysytems and Multimedia (VSMM) City of Gifu, Japan, since 1995. VR, multimedia. International Conference on Artificial Reality and Tele-existence (ICAT) Japan, held annually since 1993. S. Tachi at Tokyo University tele-robotics, VR.

Conference Proceedings (2/2) Symposium on the VRML (Held since 1995.) ACM SIGGRAPH ACM SIGGRAPH, held annually since 1978. computer graphics, recently VR. EuroGraphics (Since 1979.) PacificGraphics (Since 1993.) Symposium on Interactive 3D Graphics Graphics Interface ACM SIGCHI ACM SIGCHI, held annually since 1984. ACM UIST IEEE Visualization

Magazines and Newsletters Cyber Edge Journal (terminated) published bi-monthly. mostly covers VR industry and applications. Real Time Graphics focused on so-called "image generation" technology. Virtual Reality World (terminated) all kinds of non-serious aspects of VR and entertainment.

Books There are numerous VR books, most of them are junk. At this moment, there is no book that can be adequately adapted as a textbook. Artificial Reality II Myron Krueger, Addison-Wesley 1991. Well-thought essay on artificial reality and electronic culture. (If you are interested in the electronic culture, you must purchasea copy of "Being Digital" authored by N. Negroponte.) Better read end-to-end if you have time. Pictorial Communication in Virtual and Real Environments, Stephen Ellis, Taylor & Francis 1990. collections of papers on vision-related psychophysics. Most of papers are not relevent to VR, but some are very useful. Virtual Reality: Scientific and Technological Challenges Nathaniel Durlach and Anne Mavor (eds.), National Academy Press 1994. report by the Committee on VR Research and Development, established by the NRC (National Research Council). good reference for overview, state of the art, R&D issues. If you are writing a VR proposal, this is the one you want!

Web Sites General resources in virtual reality Research centers http://www.3dsite.com/cgi/virtual-reality-index.html http://www.stars.com/WebStars/VR.html http://www.yahoo.com/Entertainment/Virtual_Reality/ http://curry.edschool.virginia.edu/~lha5w/vr/ http://www.cc.gatech.edu/gyu/people/Phd/Benjamin.Watson/links/vr.html http://www.nist.gov/itl/div878/ovrt/hotvr.html Research centers http://www.hitl.washington.edu/projects/knowledge_base/research.html Conferences http://www.hitl.washington.edu/projects/knowledge_base/conf.html Distributed virtual reality http://www.csh.rit.edu/~airwick/dist.html http://www-dse.doc.ic.ac.uk:80/~np2/virtual_reality/distributed.html http://www.nta.no/telektronikk/4.93.dir/Loeffler_C_E.html VRML http://www.3dsite.com/cgi/VRML-index.html http://vrml.wired.com/ http://www.chaco.com/vrml/links.html