1. AR 2 kanoid - Augmented Reality Arkanoid Bax Smith, BSc, BEng, PhD Candidate Supervisor: Ray Gosine

2. Topics of Talk What is Arkanoid? What is AR 2 kanoid? AR 2 kanoid Screenshots Component Overview of AR 2 kanoid Difficulties in Implementation Conclusions and Future Work

3. What is Arkanoid? Video game invented in 1986 AKA Breakout, Bricks

4. What is AR 2 kanoid? Augmented Reality version of Arkanoid developed in 2002-2003

5. AR 2 kanoid Screenshot

10. Component Overview of AR 2 kanoid Physics Vision Joystick Render Sound Calibration Kalman Filter

11. Component Overview of AR 2 kanoid - Calibration Mode 1 – Camera Calibration Segment Circles from Several Images Compute Lens Distortion Parameters Save Calibration Parameters Mode 2 – Apply Camera Calibration Load Calibration parameters Segment Circles from Image Get the Position and Orientation of Surface from Circles

12. Difficulties in Implementation Problem: Camera Distorts Image Solution: Calibrate Camera

13. Component Overview of AR 2 kanoid - Vision Responsibilities Interface to a camera Initialization and Shutdown Set the video format (FPS, Size, Color Encoding) Resize Image using Bilinear Interpolation Send Image to Calibration System Returns Position and Orientation of Surface Supports USB and Firewire Cameras

14. Difficulties in Implementation Problem: TOO MUCH DATA!!!!!!!! Image Size: 640x480x3 = ~1MByte 4 Rows x 5 Columns of Circles Looked Real, Accurate Position Measurement Very Slow ~1 Frames/second CPU Intensive ~100% Solution: Reduce Image Size and Number of Circles

15. Difficulties in Implementation Problem: Small Image Size, Less Circles 160x120x3 = ~60KBytes 2 Rows x 3 Columns of Circles Fast ~30 Frames/second Not CPU Intensive ~20% Image Blockie, Noisy Position Measurement Solution: Bilinear Interpolation & Kalman Filter Better Image, Better Position Estimation

16. Component Overview of AR2kanoid – Kalman Filter Used to Predict Position of Surface and Position of Camera Simple Model Distance = Velocity*Time

17. Component Overview of AR 2 kanoid - Joystick Initialize Joystick using DirectX Supports all Windows Joysticks Get X-Y Position and Button Status Shutdown Joystick

18. Component Overview of AR 2 kanoid - Sound Initialize Sound using DirectX Load Sounds from Disk Play Sounds when requested Puck Hovering Puck Colliding with a Wall or Paddle Puck Colliding with a Brick Puck Colliding with the Goal Line Shutdown Sound

19. Component Overview of AR 2 kanoid - Physics Detect Puck Collision Brick – Brick explodes, puck changes direction Paddle – Puck changes direction Side Walls – puck changes LR direction Top Wall – puck changes UD direction Goal Line – no. of pucks decrements, play resets Get Joystick Status and Move Paddle Move Puck Puck Position,Velocity, Collision, Joystick

20. Component Overview of AR 2 kanoid - Render Gets Puck and Paddle Position from Physics System Gets Position and Orientation of Surface from Vision System Gets Image from Vision System Renders Scene on top of Image

21. Conclusion and Future Work Version 1 Completed! Future Work Better Memory Management/Optimizations Better Position Extraction/Image Processing Algorithms Better Model for the Kalman Filter Better/More Detailed Graphics Camera with wider FOV