Mobile Augmented Reality Game Development Jeff Chastine Southern Polytechnic State University Georgia State University, Mar. 27, 2013

Hi!

Overview What got us here? Paradigm Shift! Case Study: Qualcomm Challenges of AR Game Dev If time: code review

A Little AR History 1968 Ivan Sutherland Optical Combiner

A Little AR History 1992 Tom Caudell and David Mizell coin the term “Augmented Reality”

1994 Virtual Reality (VR) attempts to completely synthesize Augmented Reality (AR) integrates virtual objects into the physical world A Little AR History Real Environment Virtual Environment (VR) Mixed Reality Augmented Virtuality (AV) Augmented Reality (AR)

A Little AR History 1996 – Rekimoto tracks in 6 DoF! –x, y, z –yaw, pitch, roll

1997 Azuma states three requirements: 1.Combines the real and virtual worlds 2.Interactive in real time 3.Registered in 3D A Little AR History

1997 Feiner et. al create the “Touring Machine” First mobile augmented reality system

A Little History 1999 ARToolkit Free/Open Source Kato/Billinghurst Good for intro to AR

Visual Tracking (ARToolkit)

A Little History 2000 Landmark breakthrough! AR with Duran Duran

Hardware Requirements HMDs/Displays Rendering Tracking

Tabletops Tablets & Phones HMDs –Optical (see-through) Stereo? –Video (closed-view) Display

GPS, HA-GPS, GPS/RTK Acoustic (IS-900) Electro-magnetic (Polhemus) Visual (ARToolkit) Tracking

AR Examples

Medical Scientific Visualization Expert/Technician 17 Scenarios

Medical Scientific Visualization Expert/Technician 18 Scenarios

Medical Scientific Visualization Expert/Technician 19

AR in Education

Navigation

Nightvision Commercial

Military

Tracking Accuracy Optical vs. Video –Optical – High Resolution/FoV. Synching/Delay issues, brightness issues –Video – safety: camera vs. eye offset Registration Problems –location –lag in rendering Weight / Battery Life Interaction –Techniques –Physical objects General Problems

Other Issues: Interaction

Paradigm Shift! Note: –Media is cool –Mobile is cool –Media+Mobile must be really cool Unity supports scripting –Basic programming concepts –Reinforces OO –Javascript or C# (or Boo) Provides scaffolding to remove barriers (or at least lower them)

Moving into Mobile Nearest Tube Layar

Moving into Mobile (2007) Magic Books

Moving into Mobile ARToolkit on an iPhone (2008)

Moving into Mobile (2009) I MAED A GAM3 W17H AR 1N 1T – (not really, it’s called AAAARRRRRGH!)

Moving into Mobile

Let’s make a Magic Book Mark Billinghurst et al. The hard part is finding a tracker…

Need a good tracker –GPS + Gyros is NOT good enough! –Qualcomm has released their free tracker! Need a good/cheap display –Phone is a window into the virtual world Need a way to make games/render Mobile AR Gaming + = Easy AR!

Not anymore…

AR Preprocessed Images (pictures + tracking information) Into the ImageTargetTextures folder Tracking XML file and data file“Page” Textures Into the Streaming Assets / QCAR folder

Very Cool Unity Mobile API Input.acceleration –Returns orientation as a Vector3 Input.GetKeyDown(KeyCode.Mouse0) –Screen tap event iPhoneUtils.Vibrate() –Makes the phone vibrate GPS/Location –Unity 3.2 or Java LocationManager interface and native wrapper

AR Demo

Using accelerometers

ARToolkit –Free –Written in C –Uses fiducials – What you’ll need: –A webcam –A printer –A compiler –Graphics card? A safe place to get started

Free Works with Android –Java –JNI –C++ Runs on Qualcomm chipsets –HTC Incredible –Nexus One –Others The Qualcomm AR SDK

Others Dekko SDK –No training! 13 th Lab’s PointCloud –Dynamic building of the scene

The Burden of AR Gaming Printing out fiducials –Is this really “mobile”? User training –Do your parents know what AR is? –How do you interact in AR environments? –Don’t block the tracker! –Don’t shake the camera!

The Burden of AR Gaming How do you interact? –Point and click? –Tactile feedback? Will people get physically tired?

User Acceptance is Key Tracking is a challenge Lazy players AR must “go beyond” ( and why AR?) Where’s the killer app?

5 Common AR Mistakes (if you don’t believe me) 1.Poor education 2.Not investing in content 3.Different AR app for every campaign 4.AR and 3D for the sake of AR 5.Make content engaging

The Future of AR? Lots of hype right now Project Glass?

Thank you! Questions?