1. Virtual Reality Lecture1. Introduction to Virtual Reality 고려대학교 그래픽스 연구실

2. Contents What is VR? Conceptual Model of VR VR Related Areas History State of the Art and R&D Issues Some Application Examples

3. Science vs. Engineering What is not VR. high road - replication of reality low road - 3-D interface / interaction VR = studies on reality computational reality analogous to AI –lots of hypes –AI as a science / engineering

4. What is not VR.

5. VR as computational reality –To seek for the computational model of reality. –To apply the model to the VR system. VR can be understood in the context of modeling efforts. –intelligence –linguistics –emotion –life –reality –compuational “X”

6. Definition : What is VR? Computer generated environment that is … –Immersive (like IMAX,dome projection), –Interactive (like computer game), –Multi-sensory, –Viewer-centered, –3-D, and –The combination of technologies required to build such environments.

7. VR as a computer technology Technological trend –powerful –smart –physical computer - human interaction primary concern --- software other important issues --- hardware, human factors, social issues, infrastructure

8. Interacting with computers Computer World

9. New Paradigm for HCI Conventional Computer Computer World VR-based Computer Computer World

10. VR as a media All medium attempt to create the “virtual presence”. –Theater, poem, fine arts, novel, telephone, movies, TV,... VR vs. existing media –immersive –interactive –3-D –multimodal –mediated Information is not sent back and forth. Mediated environments are created and then experience.

11. effectiveness of communication depends on... the sense of “being there”. virtual presence virtual presence depends on... –sensory breadth –sensory depth –interactivity

12. VR Application Education/ Training Design Engineering Medicine Scientific Visualization Entertainment Communication Products Engineering Museum Arts VR application

13. VR technologies Character CRT Graphic CRT Virtual console Virtual Reality Tele-conference TV phone Telephone Keyboard Mouse tablet 3D mouse Computer simulation/ visualization Tele-Existence Tele-Operation Computer graphics Realtime CG Video arts 3D CAD Computer Aided Design Virtual products design All technologies meet together at VR !!

14. VR in Real World vs. Virtual World Virtual Reality Tele-Existence in Real World Tele-Existence in Virtual World Physical World Quasi Physical World Non Physical World Standard Tele-Existence Augmented Tele-Existence Size Sensation Time

15. 로보트 인간에의 임장감 제시 시각, 청각, 촉각, 미각, 후 각 체성감각 힘감각 인간의 상태추정 운동 음성 몸통 다리 팔 눈 머리 뇌파, 심전, 근전, 맥박, 혈압, 발한, etc 감각정보 제어정보 다른 로보트 상호 작용 상호 작용 가상인간 다른 가상인간 다른 Tele Existens system 으로 VR system 의 구성

16. Conceptual Model of VR 3D image Large-scale display, Head Mounted Display Sound field by DSP Force beedback mechanism Tactile display Simulation System Computer Display system Sensing system Non-contact type magnetic field supersonic wave infrared light Contact type optical fiber strain gauge potentio-meter Human

17. VR Related areas 1. Training simulation 2. Tele-operation 3. Computer graphics 4. Artificial intelligence

18. Training simulation Differences –reconfigurable by changing software –may include highly unnatural environment –highly interactive and adaptive –use of a wide varielty of human sensing modalities and sensorimotor systems –highly immersive –near-field is synthetic; far-field is synthetic.

19. Tele-operation for at least 30 years. Tele-operator –directly (manually) controlled tele-operator –tele-robot Tele-operation vs. Virtual reality Tele-presence vs. Virtual presence

20. Computer graphics Modeling Motion control (animation) Rendering User interface

21. Artificial intelligence Studies on perception and cognition Testbed for AI research

22. History 1’st stage: some visionaries –Morton Heilig : ExperienceTheatre(1962) –Ivan Surtherland : Sketchpad(1963), HMD(1966) –Myron Krueger : Artificial Reality(1972) –William Gibson : “Cyberspace” in Neuromancer(1984) 2’nd stage: technology development for specific purposes –training simulator : Earlier works –space exploration : NASA for astronaut simulation –tele-operation

23. History(con’t) 3’rd stage: VR as the general-purpose technology –Jaron Lanier : VPL(1987) – Dataglove,EyePhone,VR system –VR industry : Division Ltd. Sense8, WorldDesign(production house,W-Industry(game) –VR academia : MIT, UNC, UW,Tokyo U. Next stage: Toward a scientific discipline –computational reality –a new computing paradigm –a new media –a new art form –representation, creation and operation of virtual worlds

24. State of the Art & Issues Reference –Virtual Reality: Scientific and Technological Challenges”, pp. 35-66, National Research Council, National Academic Press, 1995. Areas of the study –application domains –psychological issues –VR technologies –evaluation of VR systems

25. 1. Application domains design, manufacturing & marketing medicine, health care hazardous operations training entertainment, military experimental psychology education information visualization tele-communication, tele-travel

26. 2. Psychological topics human performance characteristics alteration of sensori-motor loops developing the cognitive model cognitive side-effect

27. 3. VR technologies Gap between the current technology the required technology (exception -- entertainment, tele-operation) (1) human-machine interface (2) computer generation of VE (3) tele-robotics (4) network

28. (1) Human-machine interface 3D image Large-scale display, Head Mounted Display Sound field by DSP Force beedback mechanism Tactile display Simulation System Computer Display system Sensing system Non-contact type magnetic field supersonic wave infrared light Contact type optical fiber strain gauge potentio-meter Human

29. Human-machine interface (cont’) visual channelvisual channel auditory channelauditory channel haptic channelhaptic channel motion interfacemotion interface position trackingposition tracking video camera microphone others

30. Visual channel visual display –HMD –OHD (off-head display) perceptual effects –mis-registration –sensori-motor alteration –distortion –time-delay –noise

31. research issues –ergonomics –improvement of resolution and fov –wireless –integration of visual, auditory, position tracking –sun glass-like –see-through option –exploiting foveal and peripheral vision

32. Auditory channel Current hardware is adequate. Research issues –perceptual issues similar to the visual channel use for sensory substitution (for visual, haptic) –auditory scene analysis –hear-through display

33. Position tracking and mapping tracking = finding a point mapping = finding a 3D surface (e.g., environmental mapping) tracking mechanisms –mechanical linkage –magnetic –optical –acoustic –intertial

34. eye tracking research issues –tracking –mapping

35. Haptic channel force, pressure, tactile feedback unique characteristics –Haptic interface requires manipulation and sensing mechanism –body-based -- glove, exoskeleton –ground-based -- joystick

36. Research issues –haptic science = study on the human haptics (bio-mechanical, psychophysical, cognitive) –tool-hand system (which takes its metaphor from real tools.) –creating the haptic illusion –the interaction effects of haptic and vision –texture, temperature devices

37. Motion interface motion –whole-body motion passive -- e.g., motion platform active -- e.g., locomotion –part-body motion passive active

38. motion cues –vestibular system -- inertial –motor –visual –auditory –proprioceptive / kinesthetic -- muscle –tactile

39. motion interface –inertial system moves the body (e.g., treadmill, motion platform) –non-inertial system simulates motion

40. Other types of interfaces olfactory (smell) gustatory (taste) heat, wind, humidity speech direct physiological sensing and control

41. VR technologies (1) human-machine interface (2) computer generation of VE (3) tele-robotics (4) network

42. (2) Generation of virtual environments 3D image Large-scale display, Head Mounted Display Sound field by DSP Force beedback mechanism Tactile display Simulation System Computer Display system Sensing system Non-contact type magnetic field supersonic wave infrared light Contact type optical fiber strain gauge potentio-meter Human

43. Generation of VE(cont’) the core issue general-purpose VR system? trade-off between realism and interactivity requirements –frame rate –response time –scene quality

44. hardware interaction and navigation VE management –simulation –rendering modeling autonomous agent hypermedia interaction OS Generation of VE(cont’)

45. VE management - simulation Task : simulating everyday world Traditional simulation methods do not work. (requires pre-processing) Need : “meta-modeling”

46. VE management - rendering Issue : load balancing –1. partitioning VE –2. LOD Much work has been done on static scene. Research issues –1. dynamic scene –2. parallel rendering

47. OS real-time, multi-modal requirements very high-resolution time slicing atomic, transparent distribution of tasks large number of light-weighted processors, communicating by means of shared memory support for time-critical computing: –negotiated, graceful degradation –guaranteed frame rate, lag time

48. VR technologies (1) human-machine interface (2) computer generation of VE (3) tele-robotics (4) network

49. (3) Tele-robotics tele-robotics and VR hardware time-delay problem distributed tele-robots

50. (4) Network The future is here! applications –distance learning –group entertainment –distributed training –distributed design current future What is needed

51. Research Organizations International Efforts U.S.A. defense, space, visualization, medicine U.K.education, training, entertainment Germany JapanVR as a logical extension of robotics, automation, HDTV.

52. Academia HIT Lab, University of Washington University of North Carolina Media Lab, MIT Georgia Institute of Technology Naval Postgraduate School University of Pennsylvania University of California at Berkeley University of Illinois - Chicago Columbia University University of Toronto

53. And, Some VR Example … (Video)