Virtual Experiences Courses and Research Benjamin Lok March 22 nd, 2005

Outline  Topics Interactive Computer GraphicsInteractive Computer Graphics Human Computer InteractionHuman Computer Interaction Virtual RealityVirtual Reality  Jobs  Research  Skills

Interactive Computer Graphics

Motivation  We want to understand how to do:

Motivation

Why should I take this course?  An excuse to build that program/game/project you always wanted  A Computer Science course that involves a variety of skills (other than coding!) ArtArt PsychologyPsychology SoundSound CreativityCreativity  To play with fun toys  To show off your projects!

Human Computer Interaction  What is a user interface?  Why do we care about design?  We see this all the time. What’s good about the design of this error box?What’s good about the design of this error box?  The user knows there is an error What’s poor about the design of this error box?What’s poor about the design of this error box?  Discouraging  Not enough information  No way to resolve the problem (instructions or contact info)

My Choice  iPod by Apple Computers  Pros: portableportable powerpower ease of useease of use # of controls# of controls  Cons: scratches easilyscratches easily no speech for car useno speech for car use proprietaryproprietary

HCI Community  Academics/Industry Research TaxonomiesTaxonomies TheoriesTheories Predictive modelsPredictive models  Experimenters Empirical dataEmpirical data Product designProduct design  Other areas (Sociologists, anthropologists, managers) MotorMotor PerceptualPerceptual CognitiveCognitive Social, economic, ethicsSocial, economic, ethics

HCI Tools  Sound  3D  Animation  Video  Devices Size (small->very large)Size (small->very large) Portable (PDA, phone)Portable (PDA, phone) PlasticityPlasticity  Context sensitive/aware  Personalizable  Ubiquitous

Usability Requirements  Goals: UsabilityUsability UniversalityUniversality UsefulnessUsefulness  Achieved by: PlanningPlanning Sensitivity to user needsSensitivity to user needs Devotion to requirements analysisDevotion to requirements analysis TestingTesting

Bad Interfaces  Encumbering  Confusing  Slow  Trust (ex. windows crashing)  What makes it hard? Varies by cultureVaries by culture Multiple platformsMultiple platforms Variety of usersVariety of users  Think of a game you’ve played with a bad interface UNIXUNIX

Example  Five fastest places to click on for a right-handed user?

Example  What affects time?

Virtual Reality Definition  What is virtual reality? Virtual –Virtual –  Being in essence or effect, but not in fact  Example VRAM Reality –Reality –  The state or quality of being real. Something that exists independently of ideas concerning it. Something that constitutes a real or actual thing as distinguished from something that is merely apparent.” What was the first VR?What was the first VR?

What was the first VR?

Ivan Sutherland’s The Ultimate Display “Don’t think of that thing as a screen, think of it as a window, a window through which one looks into a virtual world. The challenge to computer graphics is to make that virtual world look real, sound real, move and respond to interaction in real time, and even feel real.”

Key Elements of Virtual Reality Experience  Virtual World - content of a given medium screen play, script, etc.screen play, script, etc. actors performing the play allows us to experience the virtual worldactors performing the play allows us to experience the virtual world  Immersion – sensation of being in an environment mental immersion – suspension of disbeliefmental immersion – suspension of disbelief physical immersion – bodily entering the mediumphysical immersion – bodily entering the medium Related to presence – (mentally immersed) the participant’s sensation of being in the virtual environment (Slater)Related to presence – (mentally immersed) the participant’s sensation of being in the virtual environment (Slater) Walking Experiment at UNC – Chapel Hill

Augmented Reality  A combination of a real scene viewed by a user and a virtual scene generated by a computer that augments the scene with additional information. Ultrasound Visualization Research at UNC – Chapel Hill All Virtual ObjectsAll Real Objects

Immersive Technology  Head-mounted Display Optical SystemOptical System Image Source (CRT or LCD)Image Source (CRT or LCD) Mounting ApparatusMounting Apparatus EarphonesEarphones Position TrackerPosition Tracker

Immersive Technology  Multi-screen Projection of stereoscopic images (CAVE)

Immersive Technology  Single large stereoscopic display Projection-based Head-tracked Possible tracking of hands and arms. Brings virtual objects into the physical world

User’s perspective  Setting  Objects in world  Other participants  Active/Passive Factory Simulation Architectural Walkthrough

Immersive Virtual Characters for Educating Medical Communication Skills J. Hernendez, A. Stevens, D. S. Lind Department of Surgery (College of Medicine) Department of Surgery (College of Medicine) M. Duerson Department of Community Health and Family Medicine (College of Medicine) Department of Community Health and Family Medicine (College of Medicine) K. Johnsen, A. Raij, R. Dickerson, B. Lok Department of Computer and Information Science and Engineering (College of Engineering) R. Ferdig College of Education The University of Florida, Gainesville, FL The University of Florida, Gainesville, FL J. Jackson, M. Shin Department of Computer Science The University of North Carolina at Charlotte, Charlotte, NC Sebastian Foti, Juan Cendan, Peggy Werner Associated Faculty

Future Work  Future collaborators UF College of EducationUF College of Education Medical College of Georgia (MCG)Medical College of Georgia (MCG)  Now n=20, future n=120 students / year (MCG)  Evaluate VP vs SP  Non-verbal communication Body leanBody lean Eye gazeEye gaze  Real Speech vs Text to Speech  Negative training transfer

Driving Application: Engineering Design Evaluation At assembly design stage : Several constructed partsSeveral constructed parts Many CAD models for parts yet to be constructedMany CAD models for parts yet to be constructed Unverified assembly plansUnverified assembly plans How can we verify the assembly plan? To be built in 2008 We only have this in 2005 Motivation : Finding design flaws earlier will save time and moneyFinding design flaws earlier will save time and money Virtual environments have difficulty simulating hands-on assemblyVirtual environments have difficulty simulating hands-on assembly Support team based interactionSupport team based interaction

Capturing the Physical Model Scanning Define color markers Interaction in ME Reconstruction from multiple scans

Skill Set  “Oh I don’t want to program, I just want to create stuff”  Architect or Car analogy  Programming -> tool to create what is in your imagination  Programming proficiency goal: be able to code anything that you imagine  To do what you saw before we use: C/C++, OpenGL, many different librariesC/C++, OpenGL, many different libraries  Math, math, and yet more math!!! Differential equations (simulations, physics engines), linear algebra (transforms, 3D manipulations), statisticsDifferential equations (simulations, physics engines), linear algebra (transforms, 3D manipulations), statistics

Graduate School  Programming Team Lead at RockStar Games Worked on Midnight Club 2Worked on Midnight Club 2  Renderman Developer at Pixar Worked on the Jellyfish of Nemo and The IncrediblesWorked on the Jellyfish of Nemo and The Incredibles  Software Engineer for Interactive Media for Disney Worked on Toon Town, online capable titlesWorked on Toon Town, online capable titles  Worked at 989 Sports, EA Sports, Freedom Force, etc.  What did they all have in common? Published scientific journals in computer sciencePublished scientific journals in computer science Graduate school in computer scienceGraduate school in computer science Worked on efficient collision detection, particle systems, curved surfaces, virtual reality, etc.Worked on efficient collision detection, particle systems, curved surfaces, virtual reality, etc. All hadAll had  GRE scores in the top 10%  Went to a top graduate school (admission is tough)  Strong math and science and coding  Why would a CS professor want to work with you?

Interested? Take the following…  CAP4730 Computational Structures in Computer Graphics (Fall ’05)  CAP4930/6930 Design and Creation of Virtual Environments (Fall ’05)  CAP4930/6930 Human-Computer Interaction (Spring ’06?)  Computational Geometry (Spring ’06)?  Computer Simulation Concepts (Fall ’05)

Virtual Experiences Group 2006  PhD Students Kyle JohnsenKyle Johnsen John QuarrelsJohn Quarrels Andrew RaijAndrew Raij Xiyong WangXiyong Wang Aaron KotranzaAaron Kotranza Brent RossenBrent Rossen  Undergraduates Harold RodriguezHarold Rodriguez Joshua HortonJoshua Horton  Alumni Robert Dickerson (BS)Robert Dickerson (BS) Sayed Hashimi (BS)Sayed Hashimi (BS) Art Homs (BS)Art Homs (BS) Cyrus Harrison (MS)Cyrus Harrison (MS) George Mora (MS)George Mora (MS) Andrew Joubert (BS)Andrew Joubert (BS) Samuel Preston (BS)Samuel Preston (BS) John Samuelsen (BS)John Samuelsen (BS) Thank you! Questions?

Virtual Experiences Group 2004  PhD Students (4) Kyle JohnsenKyle Johnsen John QuarrelsJohn Quarrels Andrew RaijAndrew Raij Xiyong WangXiyong Wang  Undergraduates (4) Robert DickersonRobert Dickerson Sayed HashimiSayed Hashimi Andrew JoubertAndrew Joubert Art HomsArt Homs  Alumni (4) Cyrus Harrison (MS)Cyrus Harrison (MS) George Mora (MS)George Mora (MS) Andrew Joubert (BS)Andrew Joubert (BS) Samuel Preston (BS)Samuel Preston (BS) John Samuelsen (BS)John Samuelsen (BS) Thank you! Questions?