Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007
Source Environment : 32-bit Vista, VS 2005,.NET 3.0, Logitech QuickCam Pro Web Cam for Notebooks
brains-N-brawn.com Pervasive Computing –Tablet PC (MVP 03) –Compact Framework (MVP 04) –Advanced Web Services (MVP 05) –Media Center (MVP 06) –Speech / Office Communication Server (MVP 07) –Location Based Services –Artificial Intelligence –3D –Image Processing –Augmented Reality
Audience DirectShow? 3D? WPF? Augmented Reality?
Outline Intro to Augmented Reality DirectShow AR Libraries WPF 3D
Introduction Augmented Reality (AR) - is the combination of real world data with virtual data.
Audience What are some current examples?
Current Examples Pilot HUDs QuickCam video conferencing PS3 Eye of Judgment Sports : Football, Hockey, Nascar Aircraft maintenance ARQuake Star Wars chess …
Videos QuickCam Eye of Judgment
Steps Acquire real-world data (camera, HMD, mobile devices) Recognize real-world data (computer vision, face detection, fudicial markers) Determine relative position of real-world marker to input source. Display composite of real-world data and virtual object (monitor, HMD, mobile device, projector).
Acquire real-world data OpenCV / SharperCV DirectShow / DirectShow.NET Windows Media Foundation …
DirectShow Used by Windows Media Player, Media Center Edition, Windows Media Encoder, Movie Maker, … User for audio/video playback, DVD, TV tuners, web cams, media capture, media transcoding, …
DirectShow COM GraphEdt tool Graph, Filter, Pin, Configuration Filters (Source, Transform, Sink) DirectShow.NET (C# wrapper) – Learning the filters is key Not WPF 3D friendly
DirectShow.NET C# wrapper Wraps a majority of DirectShow Many samples available, along with GraphEdt.exe, and MSDN documentation Also has wrappers for Windows Media Lib and Windows Media Foundation
Demo GraphEdt /dshowPlay /wpfArBasic - capture
Recognize real-world data OpenCV Face detection AR Libraries (fudicial markers) …
AR Libraries ArToolKit ArTag ArToolKitPlus – graz.ac.at/handheld_ar/artoolkitplus.phphttp://studierstube.icg.tu- graz.ac.at/handheld_ar/artoolkitplus.php Studierstube Tracker Goblin XNA (coming)
ArToolKitPlus Optimized for mobile devices C++ Lib GPL, no longer under development Not well documented, but ArToolKit is Single, Multi-config, Multi-distinct tracking Markers (ID, BCH, Patterns)
Videos ARTag Magic Lens Studierstube Mobile
Demo /wpfArBasic – AR tracking /wpfArTest
Composite real and virtual data GDI OpenGL / Tao Direct3D XNA WPF 3D …
WPF 3D Viewport3D MatrixCamera Light ModelVisual3D / Model3DGroup Material Transform3D HitTest
WPF 3D 3D Programming for Windows (Petzold) Windows SDK Samples
Demo /wpfArBasic – 3D model /wpfArVideo
Future Augmented Home (book) –Mostly OpenGL and C++, but at least one C# and Tao sample. Based on ArTag library. Goblin XNA (AR library) –For PC use, not on 360 until XNA is extended …
Questions