1. Rapidly Incorporating Real Objects for Evaluation of Engineering Designs in a Mixed Reality Environment Xiyong Wang, Aaron Kotranza, John Quarles, Benjamin Lok Computer and Information Sciences and Engineering Department University of Florida [xw3, akotranz, jpq, lok@cise.ufl.edu] lok@cise.ufl.edu B. Danette Allen NASA Langley Research Center Danette.Allen@nasa.gov Danette.Allen@nasa.gov

2. Driving Application: Engineering Design Evaluation At assembly design stage : Several constructed parts Several constructed parts Many CAD models for parts yet to be constructed Many CAD models for parts yet to be constructed Unverified assembly plans Unverified assembly plans How can we verify the assembly plan? To be built in 2008 We only have this in 2005 Motivation : Finding design flaws earlier will save time and money Finding design flaws earlier will save time and money Virtual environments have difficulty simulating hands-on assembly Virtual environments have difficulty simulating hands-on assembly Support team based interaction Support team based interaction

3. Previous Work Engineering Design Zachmann--Virtual Assembly Simulation: Pure VE technique used in Assembly [ GMD IMK, Oct 2000 ]Virtual Assembly Simulation Jack – commercial software to do ergonomics (www.ugs.com) EDS Jack – commercial software to do ergonomics (www.ugs.com) EDS Interfaces in VR Bowman (Siggraph 2001 course notes) – comprehensive list of VR interaction LaViola, Bowman (Siggraph 2001 course notes) – comprehensive list of VR interactionCollaboration Mortensen 2002, Collaboration in tele-immersive environments. [Eurographics] Mortensen 2002, Collaboration in tele-immersive environments. [Eurographics] Mortensen Pinho 2002 Cooperative Object Manipulation [VRST2002] Pinho 2002 Cooperative Object Manipulation [VRST2002] Image from Zachmann’s website, only the gloves (or hands) were tracked [http://web.informatik.uni-bonn.de/II/ag- klein/people/zach/projects/fhg/ems.html]

4. Mixed Environment Solution Mixed Environments (ME): Components Components Physical objects CAD models Physical correlates Effective Interaction Effective Interaction Haptic feedback Visualization of Articulated models Challenges Capturing physical models Capturing physical models Tracking real objects Tracking real objects Supporting teamwork Supporting teamwork

5. Capturing the Physical Model Scanning Define color markers Interaction in ME Reconstruction from multiple scans

6. Tracking Solution Object tracking STRAPS - color tracking STRAPS - color tracking Web cams Attributes Low cost Low cost Moderate latency Moderate latency Moderate accuracy Moderate accuracy Many sensors Many sensors Limited marker space Solution: Multi-blob tracking Head tracking Worldviz PPT Worldviz PPT Intersense Interiacube2 Intersense Interiacube2Attributes Expensive Expensive Low latency, Low latency, High accuracy High accuracy Few sensors Few sensors

7. Components: Real circuit board Real circuit board Fuel tank physical correlate Fuel tank physical correlate Virtual fuselage CAD model Virtual fuselage CAD model Experiment Assembly task Abstracted task Abstracted task CAD models and parts from NASA CAD models and parts from NASA

8. Assembly Task Video

9. Team Based Visualization Tracked Tablet PC For local team members For local team membersEngineersSupervisorsHMD For primary technician For primary technicianDesktop For remote observers For remote observers

10. Magic Lens Video

11. Conclusion Results The user: The user: Found the design flaw Reordered the assembly plan dynamically Completed the assembly task successfully Takes two hours to incorporate a real object into the ME Takes two hours to incorporate a real object into the MEProblems Tracking TrackingOcclusionLatency Future work: Improve tracking Improve trackingRobustness Additional cameras Formal study with NASA engineers Formal study with NASA engineers Avatars Avatars Automate pipeline -- Reduce incorporating time to 15 minutes!! Automate pipeline -- Reduce incorporating time to 15 minutes!! More tracking camera from different location make more complicate tasks possible. But complicated mechanism is needed to combine the information from multi channel

12. Acknowledgements Funding University of Florida University of FloridaPeople NASA Langley Research Center NASA Langley Research Center UNCC UNCC Dr. Shin Jonathan Jackson Dr. Hodges CISE, UF CISE, UF Robert Dickerson Art Homs