Virtual Reality Lecture1. Introduction to Virtual Reality 고려대학교 그래픽스 연구실

Contents What is VR? Conceptual Model of VR VR Related Areas History State of the Art and R&D Issues Some Application Examples

Science vs. Engineering What is not VR. high road - replication of reality low road - 3-D interface / interaction VR = studies on reality computational reality analogous to AI –lots of hypes –AI as a science / engineering

What is not VR.

VR as computational reality –To seek for the computational model of reality. –To apply the model to the VR system. VR can be understood in the context of modeling efforts. –intelligence –linguistics –emotion –life –reality –compuational “X”

Definition : What is VR? Computer generated environment that is … –Immersive (like IMAX,dome projection), –Interactive (like computer game), –Multi-sensory, –Viewer-centered, –3-D, and –The combination of technologies required to build such environments.

VR as a computer technology Technological trend –powerful –smart –physical computer - human interaction primary concern --- software other important issues --- hardware, human factors, social issues, infrastructure

Interacting with computers Computer World

New Paradigm for HCI Conventional Computer Computer World VR-based Computer Computer World

VR as a media All medium attempt to create the “virtual presence”. –Theater, poem, fine arts, novel, telephone, movies, TV,... VR vs. existing media –immersive –interactive –3-D –multimodal –mediated Information is not sent back and forth. Mediated environments are created and then experience.

effectiveness of communication depends on... the sense of “being there”. virtual presence virtual presence depends on... –sensory breadth –sensory depth –interactivity

VR Application Education/ Training Design Engineering Medicine Scientific Visualization Entertainment Communication Products Engineering Museum Arts VR application

VR technologies Character CRT Graphic CRT Virtual console Virtual Reality Tele-conference TV phone Telephone Keyboard Mouse tablet 3D mouse Computer simulation/ visualization Tele-Existence Tele-Operation Computer graphics Realtime CG Video arts 3D CAD Computer Aided Design Virtual products design All technologies meet together at VR !!

VR in Real World vs. Virtual World Virtual Reality Tele-Existence in Real World Tele-Existence in Virtual World Physical World Quasi Physical World Non Physical World Standard Tele-Existence Augmented Tele-Existence Size Sensation Time

로보트 인간에의 임장감 제시 시각, 청각, 촉각, 미각, 후 각 체성감각 힘감각 인간의 상태추정 운동 음성 몸통 다리 팔 눈 머리 뇌파, 심전, 근전, 맥박, 혈압, 발한, etc 감각정보 제어정보 다른 로보트 상호 작용 상호 작용 가상인간 다른 가상인간 다른 Tele Existens system 으로 VR system 의 구성

Conceptual Model of VR 3D image Large-scale display, Head Mounted Display Sound field by DSP Force beedback mechanism Tactile display Simulation System Computer Display system Sensing system Non-contact type magnetic field supersonic wave infrared light Contact type optical fiber strain gauge potentio-meter Human

VR Related areas 1. Training simulation 2. Tele-operation 3. Computer graphics 4. Artificial intelligence

Training simulation Differences –reconfigurable by changing software –may include highly unnatural environment –highly interactive and adaptive –use of a wide varielty of human sensing modalities and sensorimotor systems –highly immersive –near-field is synthetic; far-field is synthetic.

Tele-operation for at least 30 years. Tele-operator –directly (manually) controlled tele-operator –tele-robot Tele-operation vs. Virtual reality Tele-presence vs. Virtual presence

Computer graphics Modeling Motion control (animation) Rendering User interface

Artificial intelligence Studies on perception and cognition Testbed for AI research

History 1’st stage: some visionaries –Morton Heilig : ExperienceTheatre(1962) –Ivan Surtherland : Sketchpad(1963), HMD(1966) –Myron Krueger : Artificial Reality(1972) –William Gibson : “Cyberspace” in Neuromancer(1984) 2’nd stage: technology development for specific purposes –training simulator : Earlier works –space exploration : NASA for astronaut simulation –tele-operation

History(con’t) 3’rd stage: VR as the general-purpose technology –Jaron Lanier : VPL(1987) – Dataglove,EyePhone,VR system –VR industry : Division Ltd. Sense8, WorldDesign(production house,W-Industry(game) –VR academia : MIT, UNC, UW,Tokyo U. Next stage: Toward a scientific discipline –computational reality –a new computing paradigm –a new media –a new art form –representation, creation and operation of virtual worlds

State of the Art & Issues Reference –Virtual Reality: Scientific and Technological Challenges”, pp , National Research Council, National Academic Press, Areas of the study –application domains –psychological issues –VR technologies –evaluation of VR systems

1. Application domains design, manufacturing & marketing medicine, health care hazardous operations training entertainment, military experimental psychology education information visualization tele-communication, tele-travel

2. Psychological topics human performance characteristics alteration of sensori-motor loops developing the cognitive model cognitive side-effect

3. VR technologies Gap between the current technology the required technology (exception -- entertainment, tele-operation) (1) human-machine interface (2) computer generation of VE (3) tele-robotics (4) network

(1) Human-machine interface 3D image Large-scale display, Head Mounted Display Sound field by DSP Force beedback mechanism Tactile display Simulation System Computer Display system Sensing system Non-contact type magnetic field supersonic wave infrared light Contact type optical fiber strain gauge potentio-meter Human

Human-machine interface (cont’) visual channelvisual channel auditory channelauditory channel haptic channelhaptic channel motion interfacemotion interface position trackingposition tracking video camera microphone others

Visual channel visual display –HMD –OHD (off-head display) perceptual effects –mis-registration –sensori-motor alteration –distortion –time-delay –noise

research issues –ergonomics –improvement of resolution and fov –wireless –integration of visual, auditory, position tracking –sun glass-like –see-through option –exploiting foveal and peripheral vision

Auditory channel Current hardware is adequate. Research issues –perceptual issues similar to the visual channel use for sensory substitution (for visual, haptic) –auditory scene analysis –hear-through display

Position tracking and mapping tracking = finding a point mapping = finding a 3D surface (e.g., environmental mapping) tracking mechanisms –mechanical linkage –magnetic –optical –acoustic –intertial

eye tracking research issues –tracking –mapping

Haptic channel force, pressure, tactile feedback unique characteristics –Haptic interface requires manipulation and sensing mechanism –body-based -- glove, exoskeleton –ground-based -- joystick

Research issues –haptic science = study on the human haptics (bio-mechanical, psychophysical, cognitive) –tool-hand system (which takes its metaphor from real tools.) –creating the haptic illusion –the interaction effects of haptic and vision –texture, temperature devices

Motion interface motion –whole-body motion passive -- e.g., motion platform active -- e.g., locomotion –part-body motion passive active

motion cues –vestibular system -- inertial –motor –visual –auditory –proprioceptive / kinesthetic -- muscle –tactile

motion interface –inertial system moves the body (e.g., treadmill, motion platform) –non-inertial system simulates motion

Other types of interfaces olfactory (smell) gustatory (taste) heat, wind, humidity speech direct physiological sensing and control

VR technologies (1) human-machine interface (2) computer generation of VE (3) tele-robotics (4) network

(2) Generation of virtual environments 3D image Large-scale display, Head Mounted Display Sound field by DSP Force beedback mechanism Tactile display Simulation System Computer Display system Sensing system Non-contact type magnetic field supersonic wave infrared light Contact type optical fiber strain gauge potentio-meter Human

Generation of VE(cont’) the core issue general-purpose VR system? trade-off between realism and interactivity requirements –frame rate –response time –scene quality

hardware interaction and navigation VE management –simulation –rendering modeling autonomous agent hypermedia interaction OS Generation of VE(cont’)

VE management - simulation Task : simulating everyday world Traditional simulation methods do not work. (requires pre-processing) Need : “meta-modeling”

VE management - rendering Issue : load balancing –1. partitioning VE –2. LOD Much work has been done on static scene. Research issues –1. dynamic scene –2. parallel rendering

OS real-time, multi-modal requirements very high-resolution time slicing atomic, transparent distribution of tasks large number of light-weighted processors, communicating by means of shared memory support for time-critical computing: –negotiated, graceful degradation –guaranteed frame rate, lag time

VR technologies (1) human-machine interface (2) computer generation of VE (3) tele-robotics (4) network

(3) Tele-robotics tele-robotics and VR hardware time-delay problem distributed tele-robots

(4) Network The future is here! applications –distance learning –group entertainment –distributed training –distributed design current future What is needed

Research Organizations International Efforts U.S.A. defense, space, visualization, medicine U.K.education, training, entertainment Germany JapanVR as a logical extension of robotics, automation, HDTV.

Academia HIT Lab, University of Washington University of North Carolina Media Lab, MIT Georgia Institute of Technology Naval Postgraduate School University of Pennsylvania University of California at Berkeley University of Illinois - Chicago Columbia University University of Toronto

And, Some VR Example … (Video)