Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum ( )Siu Ho Tung ( )

2Outline Background Information & Objective Summary of Works Done in Semester 1 Works in Semester 2 Game Flow Programming Tricks Difficulties Conclusion Q & A

3Objectives Demonstrate how a game can “recognize” its external environment for Augmented RealityDemonstrate how a game can “recognize” its external environment for Augmented Reality Develop a mobile game, Virtual Dart, which illustrates the proposed methodologyDevelop a mobile game, Virtual Dart, which illustrates the proposed methodology

4Definition Augmented Reality is... a combination of real world and computer generated data the use of video and which is augmented by addition of computer graphics

5 Background Information Programming Sybmian C++ C# Target Mobile Phone Nokia N90 (S60 2 rd Edition FP3) Nokia N80 (S60 3 rd Edition) (Nokia 6600)

6 Summary of Works Done in Semester 1 Feature Selection Feature Recognition Motion Tracking on the featue

7 Summary of Works Done in Semester 1 Program Core 1.Implementation of Feature Selection Algorithm 2.Implementation of Initial Feature Recognition Algorithm 3.Implementation of Enhanced Feature Recognition Algorithm 4.Modification of mVOTE Motion Tracking Engine 5.Implementation of mobile game, Virtual Dart, in prototype form Virtual Dart Studied Materials 1.Existing mVOTE Engine & its related papers 2.Symbian C++ & Nokia S60 3rd edition programming guides 3.Corner Detection Algorithms such as FAST Corner Detector and Harris Corner Detector 4.Block Matching technique

8 Problem Discovered in Semester 1 Problem Enhanced Feature Recognition Algorithm takes more than 30 seconds to recognize the saved features Cause 1.Nokia N80 has only limited computational power 2.The algorithm maybe too general to recognize features Solution Design a new and specific algorithm (To be discussed later)

9 What does the program need to recognize? LED Recognition Bar Code Recognition Common in indoor environment Can emit light + Goal: Use the simplest method to build the recognizer

10 How to locate LED? Filter out the surrounding environment 1.Locate a block which contains LED 2.If the block contains more than a threshold of non- black pixels 1. Filter for LED2. Find LED

11 LED Recognizer – Filter Use Green / Red LED as target object Goal: Separate the target object from its surrounding environment

12 Add constraint to other color channels Result image of G > 230 and B < 204 LED Recognizer – Filter Set a lower bound for Green color channel of the pixel

13 LED Recognizer – Filter Extension Use similar technique to build a filter for Fluorescent Light, Light Bulb

14 LED Recognizer – Version 1 Check every block in the screen Prevent block overlapping

15 Limitations of LED Recognizer – Version 1 Problem 1.Slow 2.Target object at the corner of the block 3.Select more than one blocks for same object

16 LED Recognizer – Version 2 Search the center region of the block only Distance check Stop after finding 2 feature blocks

17 Comparison of 2 versions of LED Recognizer Version 1Version 2

18 LED Recognizer – Final Version LED Recognizer – Version 2 RemoveDistanceCheck Remove Block Overlapping Check Stop After Finding First Feature Simplify by Reduce the number of tracking point to 1 + Final Version of LED Recognizer

19 Motion Tracking by mVOTE Engine Problem The motion tracking of the LED would be lost very easily CauseSolution 1.Detect if the motion tracking of the LED is lost or not 2.Relocate the LED if the motion tracking is lost Areas around the LED are flat regions. mVOTE Engine cannot work well in flat region

20 Virtual Dart Modeling Dart Throwing Preview Scene

21 Virtual Dart Modeling X 39 pixels Meter 4 pixels Meter = m = m 1.6 m

22 Game Flow

23 Programming Tricks A lot of Square Root calculation in Projectile Motion Found a fast Square Root calculation by John Carmack Newton Approximation Method

24 Programming Tricks

25Difficulties Build 3D model of dart images Insufficient support from emulator Colors in images captured change in different platforms Unclear explanations from Nokia S60 documentations “Active Object” in Symbian platform

26Conclusion Successfully recognized Green / Red LED for Augmented Reality Developed a mobile game with Augmented Reality – Virtual Dart

27 Virtual Dart End

28 LED Recognizer – Filter Before FilteringAfter Filtering

29 LED Recognizer – Filter Before FilteringAfter Filtering

30 LED Recognizer – Filter Use similar technique to build filter for Red LED Set lower bound for Red color channel and upper bound for Blue color channel

31 Projectile Motion

32 Programming Tricks floatCos(int aAngle) { switch(aAngle) { case(0): return 1.0; break; case(1): return ; break; case(2): return ; break; case(3): return ; break;... case(12): return ; break; case(13): return ; break; case(14): return ; break; case(15): return ; break; default: return 1.0; break; }