1. ES305 Virtual Tools in Engineering Design Introduction to Virtual Reality and Virtools 11-04-2009 Prof. Searleman jets@clarkson.edu

2. Overview ■ Introduction to Virtual Reality □ Key Elements □ Virtual Product Development ■ Introduction to Virtools Homework: Exercise 1 (could complete today), and Exercise 2 to be completed on 11/10/09 (note that these are to be done individually)

3. What is Virtual Reality? ■ VR is the use of computer technology to create the effect of a 3D environment containing 3D objects which have a strong sense of spatial presence with respect to the user. ■ In virtual reality, you have a sense of and interact with three-dimensional things as opposed to pictures or movies. ■ Control of and interaction with the environment is typically through direct manipulation of objects in the real world.

4. 4 Key Elements of VR ■ Key Element 1: 3D Virtual World ■ Key Element 2: Immersion ■ Key Element 3: Sensory Feedback ■ Key Element 4: Interactivity

5. Key Element 1: Virtual World Three-dimensional “virtual world” or “virtual environment”:

6. Key Element 2: Immersion ■ Mental Immersion - sense of “presence”; state of being deeply engaged; suspension of disbelief; involvement ■ Physical Immersion - bodily entering into a medium; synthetic stimulus of the body’s senses via the use of technology Frequently embodied by an “avatar” – a virtual object used to represent a participant or physical object in a virtual world

7. The perception of being in a particular space or place. Immersion: Sense of “Presence” Physically immerse the participant in a computer-generated space. Provide computer-generated sensation to one or more of the human senses.

8. Key Element 3: Sensory Feedback ■ visual ■ aural ■ haptic & tactile ■ vestibular (balance: equilibrium, acceleration, and orientation wrt gravity) ■ olfactory, taste, magnetoreception

9. Visual: VR Display Technology Computer Monitor “Fishtank VR” –Pros: Cheap –Cons: Least immersive

10. Visual: VR Display Technology ■ Head-Mounted Displays (HMDs) □ The display and a position tracker are attached to the user’s head □ Pros: More immersive □ Cons: Individual experience ■ Head-Tracked Displays (HTDs) □ Display is stationary, tracker tracks the user’s head relative to the display. □ Example: CAVE, Workbench, Stereo monitor

11. Visual: VR Display Technology Projection-Based: - rear-projection screens, e.g. CAVE, ICUBE - large, flat-panel monitors - table-top displays –Pros: Most immersive –Cons: Expensive

12. Projection-Based VR ICUBE in Advanced Visualization Lab (ADVIL) at Clarkson (CAMP 155) ImmersaDesk Microsoft Surface

13. Virtusphere website: www.virtusphere.com There will be a demonstration of a Virtusphere on Thursday, 11/05/09, from 1:00 to 2:00 pm in the CAMP Atrium. 10’ diameter sphere, wireless HMD (800x600), wireless controller, inverted mouse base, one person/sphere but can have multiple spheres share the same virtual environment, ~485 lb., Cortona VRML technology from ParallelGraphics

14. Haptics: Sense of touch ■ Includes touch feedback (taction) and force feedback (kinesthesia) 3 DOF cylindrical robot Max force output 250 N Stiffness 5x104 N/m Uses force-in, position-out arrangement

15. Haptic Interface for Surgery Training VEST: Virtual Endoscopic Surgery Training Simulation Scene Haptic instrument interface box

16. Other Examples of Sensory Stimulation ■ Proprioception – ability to sense the position, location, orientation and movement of the body and its parts (stimulation comes from within the body); includes both: □ kinesthesia – sense of movement of the limbs □ ability to sense body posture

17. Key Element 4: Interactivity ■ responsiveness to user actions □ the ability to affect a virtual world □ the ability to change one’s viewpoint within a world ■ collaborative environments □ multiple users interacting within a virtual world and among themselves ■ alternative realities □ games, computer simulations of natural and artificial phenomenu, flight simulators

18. What is VR? (revisited) virtual reality a medium composed of interactive computer simulations that sense the participant’s position and actions and replace or augment the feedback to one or more senses, giving the feeling of being mentally immersed or present in the simulation (a virtual world)

19. Recall: Product Delivery Process – Design Core ■ a market (user need) product design specification (PDS) ■ conceptual design ■ detailed design ■ manufacturing ■ sales – the ending point Virtual Product Development can be used throughout each phase of this process

20. Virtual Product Development ■ Virtual Product Development (VPD) is an environment where all phases of the product design process use an integrated combination of both simulation software technology and traditional techniques. ■ Through VPD, cost effective designs with higher levels of reliability can now be achieved in less time than using only traditional physical processes.

21. Use of VR in Design, Manufacturing & Marketing ■ Developing design requirements; e.g. architectural walkthroughs of spatial designs such as kitchens, aircraft interiors, etc. ■ Building prototypes ■ Configuration & management of production lines; e.g. plant layouts ■ Marketing – how customers actually experience a product

22. Example: Virtual Prototyping “Think Virtual do Real”: digital prototyping as key factor for innovation Caterpillar, Inc.

23. Example: Space Planning & Marketing An early space planning tool, developed by Virtalis (Virtual Presence Ltd) and used for retail by Sainsbury’s supermarkets (1993)

24. A multidisciplinary team of students designed and built an immersive virtual reality power W/C simulator modeled after flight simulators VR Wheelchair Project The user wears an HMD that displays a virtual world through which the user has to maneuver the virtual power W/C Feedback from the joystick and virtual environment drive the control system for the platform

25. Software 3DVIA Virtools 5.0 from Dassault SystèmesDassault Systèmes

26. What is 3DVia Virtools? 3DVia Virtools 5.0 is a development platform for 3D visualization It consists of the following components: 1. an Authoring application 2. a Behavioral Engine (CK2) 3. a Rendering Engine 4. a Web Player (3D VIA player) 5. a Software Development Kit (SDK)

27. Virtools Training Notes Media Design 3DS, Maya, Lightwave Sound Design.wav,.mp3 Texture, 2D Interface.jpg,.bmp Specific BB, Interface Visual C++ Virtools Application Integration tool Behavioral tool

28. Virtools Libraries In addition to the basic functionality provided by Dev 5.0, there are a number of add-on libraries. These include: ■ VR Library ■ Physics Library ■ AI Library ■ PSP and Wii Solutions ■ Multiuser Server ■ 3D XML Virtools Plugin

29. 1. Virtools Authoring ■ The authoring application allows you to create “compositions” (CMOs) where 3D objects are given behaviors and attributes. ■ Virtools is not a modeling application. Other applications (such as 3D Studio Max, Maya, Rhino, AutoCAD, etc.) can be used to create models, which then can be imported into Virtools.

30. 2. Behavioral Engine ■ The behavior engine, CK2, processes the behaviors of how each object acts in a given environment. ■ Behavior Building Blocks (BB’s) are used to create behaviors. Virtools has an extensive library of BB’s. ■ Managers are used to support the behavior engine. Examples are the LevelManager, ResourceManager and the SoundManager

31. 3. Render Engine ■ This performs the graphics rendering, and can be customized with the SDK; it consists of 2 parts: - CK2_3D determines what should be drawn (list of visible items) - Virtools Rasterizers, which support Direct X and OpenGL, draw the items on-screen

32. 4. Web Player ■ The 3D Via Player allows playback of a virtools application via a web browser. ■ Virtools compositions (.cmo) can be exported to the Web Player (.vmo)

33. 5. Software Development Kit The SDK allows you to customize the Behavioral Engine and the Render Engine. For example, you can write your own functions in C++ and convert them to a building block.

34. Getting Help ■ Online reference: documentation.chm ■ Virtools MiniSite 1. Technical Samples 2. Sample Resources 3. Tutorials 4. Web Player Commands 5. Demos ■ Contact me: jets@clarkson.edu

35. Basic Concepts in Virtools ■ Level Manager □ organizes the composition □ use this to set up your objects, lights and cameras □ THEN SET INITIAL CONDITIONS ■ Schematic □ use to create, edit, and manage “scripts”; a script describes how an object behaves □ based on “Building Blocks” (BBs)

36. Basic Concepts in Virtools (cont.) ■ Building Blocks (BBs) □ a library of behaviors, organized by the type of behavior and/or by the type of object to which the behavior can be applied ■ Media objects: □ e.g. model, sound or image □ created in other applications (e.g. 3ds, Maya, etc.) ■ Data Resources □ e.g. VirtoolsResources □ used to import media objects into the VE

37. 3D Coordinate System ■ defaults to metric (1 unit = 1 meter) ■ X – horizontal; Y – vertical; Z – depth ■ Local coordinate system – set when a 3D entity is created; typically this is 1 corner of the bounding box or is the barycenter (geometric center) of the entity ■ World coordinate system (absolute coordinates); shared by all elements in a composition

38. Object – Mesh - Material ■ 3D mesh is the “wireframe” shape for an object □ A mesh is often shared by several different objects to save space ■ A material is wrapped around the mesh □ materials can be shared □ color parameters: − ambient: multiplies the scene’s ambient color − diffuse: default surface color, depends on lights − specular: shiny highlights − emmisive: self-illumination

39. Building Block blue: Behavior Inputs (bIn) green: Behavior Outputs (bOut) red: Parameter Inputs (pIn) violet: Parameter Outputs (pOut)

40. BB Operating Cycle 1.BB is inactive 2.At least 1 input is activated => BB is activated 3.BB reads the input parameters & performs its function 4.At least 1 output is activated 5.Some BBs stop here (go back to 1) and others reactivate themselves even if no input is active (go back to 3)

41. Getting Started ■ Save your composition often (say every 15 mins or so, or when you make a change in the objects) ■ Set initial conditions whenever you change any setting about an object, before you run the composition ■ Hot Keys □ F1, Ctrl-double click ■ Shortcuts □ Trace mode

42. Steps to create a composition 1.Import an object (Resources|Import) 2.Use the Level Manager to set up the object □ rename it □ double-click the object to get 3D Object Setup □ adjust the world coordinates (0,0,0) □ rotate it as desired □ change the color (if desired) 3.Add lighting: spot & directional 4.Add a camera & adjust it to view the object 5.SET INITIAL CONDITIONS

43. Virtools Exercises ■ add comment box containing name, date & exercise# ■ schematics should be neat & easy to read ■ make sure all pIns are labelled ■ print the scripts (landscape mode) ■ turn in printouts & also copy your composition to the ES305 drive in your individual directory

44. Virtools Exercises ■ Virtools Exercise 1 □ complete by today, in class ■ Virtools Exercise 2 □ to be completed on Monday, 11/09/09