ART: Augmented Reality Table for Interactive Trading Card Game Albert H.T. Lam, Kevin C. H. Chow, Edward H. H. Yau and Michael R. Lyu Department of Computer Science and Engineering The Chinese University of Hong Kong

Presentation Outline Introduction Objective Augmented Reality Table System Design Implementation Demonstration Conclusion

Trading Card Game Two players play a match. Players play cards to fight with each other. It includes summoning monsters, casting spells, or setting traps, etc. We choose “ YU-GI-OH ” as our implementation

Game Mat Game Mat of YU-GI-OH Different kinds of card must be put inside particular card zones

Traditional Card Game Computer Card Game Traditional Card Game –Interaction with real people –Lack of sound and visual effect –Require players ’ own imaginations Computer Card Game –3D animations –Handle complicated rules –Interaction with computer

Objective To develop a generic Interactive Trading Card Game Interface which allows: –Interaction with people and computer –Playing naturally in the original way –Visual and sound effect –Complicated calculation

Augmented Reality Table (ART) ART is a prototype platform –Employ Augmented Reality technology –Users can play trading card games on it –Provide an interactive environment and interface –Realize the game with 3D graphics and sound effect

Hardware Setup Overhead camera –The only input device Plasma monitor –Act as a table –Display 3D-animation –Produce sound effect Computer –Recognize valid inputs –Maintain game rules –Generate outputs

System Architecture Perception Module –Calibrate –Extract card and input commands from the camera Database Module –Access card information –Access game and card rules –Identify cards Game Enhancement Module –Display game mat –3D animation –Sound effect GameCore Module –Control game flow –Store game state –Rule-based game engine

System Architecture

Perception Read and process the raw video perceived from the camera Detect card inputs and commands Some assumptions –Fixed camera, fixed table –Camera is approximately right above the table –Cards can only be placed in predefined regions called card zones

Card Locator Locate search window –The screen is partitioned into some fixed small search windows as card zones identified in calibration –Select those with significant number of changed pixels as candidates –Search windows just stop changing will be processed –This minimizes both search space and frequency Image differenceBinary image difference

Card Locator Locate four corner card positions given a search window –Apply Canny Algorithm –Extract Contours –Search for contours which contain 4 points contain nearly right angle corners have area within a certain threshold

Card Detector Detect the card input including: –Location (in which card zone) –Orientation (vertical or horizontal) –Unique card ID (to query for card information) Compute an undistorted version of card image by geometric transformation followed by bilinear interpolation

Card Detector Identify card image by 2 steps: –Image retrieval Classify and retrieve cards of the same type as candidates –Image matching Use color-based matching since higher level features are lost due to low resolution of query image (about 40 x 40 pixels) Find the best match card according to pixel difference: Reject the image if the pixel difference is larger than the threshold Accept the image with minimum pixel difference Improve the accuracy by using block matching

Database The Database Module stores all game information It provide necessary information for the other three modules Image Database –Contain all card images Card Database –Contain all card information Rule Database –Contain all game rules and card rules

Rule-based Game Engine The system is driven by rules Define a rule: –Premise-conclusion form –e.g. “ If in main phase and player put a card, then summon the monster. ” –The action may be to load another rule, remove a rule, reset a rule, update game states, or show some outputs, etc. MainPhasePutCardSummon ActionLIst Display Monster premise 1 premise 2conclusion action list

Main PhasePut CardSummon DestroyerSummonDestroy Card premise 1 premise 2conclusion Rule-based Game Engine Maintain a pool of rules which loads and stores rules dynamically during the game Receive and raise appropriate events according to the game rules, thus, maintaining the game logic Mechanism: forward checking until no more conclusion is reached

Why Rule-based? Extensible –New cards can be extended by adding new rules and new card information to the database Flexible –Card games are very flexible and some card effect are unpredictable, of which brute-force programming cannot handle it Generic –New card games can be implemented by using a new set of game rules

Game Enhancement Provide game mat to place cards Provide buttons for command input Enhance visual and sound effects Display game states and hints

Game Mat Game statesCard zonesButtonsBattle field

Screenshot during Battle

Demonstration

Conclusion We developed a prototype of the Augmented Reality Table for trading card games With AR techniques, the system enhances existing trading card games, while remaining much of original playing style The system provided an interactive interface for players and performs player ’ s effort to maintain game rules and calculations

Q&A

Improved Block Matching Algorithm Solution, use an Improved Block Matching Algorithm –Divide the image into 9 squares –Apply Block Matching Algorithm to each squares 