AR 2 kanoid - Augmented Reality Arkanoid Bax Smith, BSc, BEng, PhD Candidate Supervisor: Ray Gosine

Topics of Talk What is Arkanoid? What is AR 2 kanoid? AR 2 kanoid Screenshots Component Overview of AR 2 kanoid Difficulties in Implementation Conclusions and Future Work

What is Arkanoid? Video game invented in 1986 AKA Breakout, Bricks

What is AR 2 kanoid? Augmented Reality version of Arkanoid developed in

AR 2 kanoid Screenshot

Component Overview of AR 2 kanoid Physics Vision Joystick Render Sound Calibration Kalman Filter

Component Overview of AR 2 kanoid - Calibration Mode 1 – Camera Calibration Segment Circles from Several Images Compute Lens Distortion Parameters Save Calibration Parameters Mode 2 – Apply Camera Calibration Load Calibration parameters Segment Circles from Image Get the Position and Orientation of Surface from Circles

Difficulties in Implementation Problem: Camera Distorts Image Solution: Calibrate Camera

Component Overview of AR 2 kanoid - Vision Responsibilities Interface to a camera Initialization and Shutdown Set the video format (FPS, Size, Color Encoding) Resize Image using Bilinear Interpolation Send Image to Calibration System Returns Position and Orientation of Surface Supports USB and Firewire Cameras

Difficulties in Implementation Problem: TOO MUCH DATA!!!!!!!! Image Size: 640x480x3 = ~1MByte 4 Rows x 5 Columns of Circles Looked Real, Accurate Position Measurement Very Slow ~1 Frames/second CPU Intensive ~100% Solution: Reduce Image Size and Number of Circles

Difficulties in Implementation Problem: Small Image Size, Less Circles 160x120x3 = ~60KBytes 2 Rows x 3 Columns of Circles Fast ~30 Frames/second Not CPU Intensive ~20% Image Blockie, Noisy Position Measurement Solution: Bilinear Interpolation & Kalman Filter Better Image, Better Position Estimation

Component Overview of AR2kanoid – Kalman Filter Used to Predict Position of Surface and Position of Camera Simple Model Distance = Velocity*Time

Component Overview of AR 2 kanoid - Joystick Initialize Joystick using DirectX Supports all Windows Joysticks Get X-Y Position and Button Status Shutdown Joystick

Component Overview of AR 2 kanoid - Sound Initialize Sound using DirectX Load Sounds from Disk Play Sounds when requested Puck Hovering Puck Colliding with a Wall or Paddle Puck Colliding with a Brick Puck Colliding with the Goal Line Shutdown Sound

Component Overview of AR 2 kanoid - Physics Detect Puck Collision Brick – Brick explodes, puck changes direction Paddle – Puck changes direction Side Walls – puck changes LR direction Top Wall – puck changes UD direction Goal Line – no. of pucks decrements, play resets Get Joystick Status and Move Paddle Move Puck Puck Position,Velocity, Collision, Joystick

Component Overview of AR 2 kanoid - Render Gets Puck and Paddle Position from Physics System Gets Position and Orientation of Surface from Vision System Gets Image from Vision System Renders Scene on top of Image

Conclusion and Future Work Version 1 Completed! Future Work Better Memory Management/Optimizations Better Position Extraction/Image Processing Algorithms Better Model for the Kalman Filter Better/More Detailed Graphics Camera with wider FOV