AR2kanoid - Augmented Reality Arkanoid

Bax Smith, BSc, BEng, PhD Candidate

Supervisor: Ray Gosine

Topics of TalkWhat is Arkanoid?What is AR2kanoid?AR2kanoid ScreenshotsComponent Overview of AR2kanoidDifficulties in ImplementationConclusions and Future Work

What is Arkanoid?Video game invented in 1986AKA Breakout, Bricks

What is AR2kanoid?Augmented Reality version of Arkanoid developed in 2002-2003

AR2kanoid Screenshot

AR2kanoid Screenshot

AR2kanoid Screenshot

AR2kanoid Screenshot

AR2kanoid Screenshot

Component Overview of AR2kanoid

Physics

Vision

Joystick

Render

Sound

Calibration

KalmanFilter

Component Overview of AR2kanoid - Calibration

Mode 1 – Camera Calibration Segment Circles from Several Images Compute Lens Distortion Parameters Save Calibration Parameters

Mode 2 – Apply Camera Calibration Load Calibration parameters Segment Circles from Image Get the Position and Orientation of Surface from

Circles

Difficulties in Implementation

Problem: Camera Distorts Image

Solution: Calibrate Camera

Component Overview of AR2kanoid - Vision

Responsibilities Interface to a camera Initialization and Shutdown Set the video format (FPS, Size, Color Encoding) Resize Image using Bilinear Interpolation Send Image to Calibration System

Returns Position and Orientation of Surface

Supports USB and Firewire Cameras

Difficulties in Implementation

Problem: TOO MUCH DATA!!!!!!!! Image Size: 640x480x3 = ~1MByte4 Rows x 5 Columns of Circles

Looked Real, Accurate Position MeasurementVery Slow ~1 Frames/secondCPU Intensive ~100%

Solution: Reduce Image Size and Number of Circles

Difficulties in Implementation

Problem: Small Image Size, Less Circles 160x120x3 = ~60KBytes 2 Rows x 3 Columns of Circles

Fast ~30 Frames/second Not CPU Intensive ~20% Image Blockie, Noisy Position Measurement

Solution: Bilinear Interpolation & Kalman Filter Better Image, Better Position Estimation

Component Overview of AR2kanoid – Kalman Filter

Used to Predict Position of Surface and Position of CameraSimple ModelDistance = Velocity*Time

Component Overview of AR2kanoid - Joystick

Initialize Joystick using DirectXSupports all Windows Joysticks

Get X-Y Position and Button StatusShutdown Joystick

Component Overview of AR2kanoid - Sound

Initialize Sound using DirectXLoad Sounds from DiskPlay Sounds when requestedPuck HoveringPuck Colliding with a Wall or PaddlePuck Colliding with a BrickPuck Colliding with the Goal Line

Shutdown Sound

Component Overview of AR2kanoid - Physics

Detect Puck Collision Brick – Brick explodes, puck changes direction Paddle – Puck changes direction Side Walls – puck changes LR direction Top Wall – puck changes UD direction Goal Line – no. of pucks decrements, play resets

Get Joystick Status and Move PaddleMove Puck Puck Position,Velocity, Collision, Joystick

Component Overview of AR2kanoid - Render

Gets Puck and Paddle Position from Physics SystemGets Position and Orientation of Surface from Vision SystemGets Image from Vision SystemRenders Scene on top of Image

Conclusion and Future Work

Version 1 Completed!Future WorkBetter Memory Management/OptimizationsBetter Position Extraction/Image

Processing AlgorithmsBetter Model for the Kalman FilterBetter/More Detailed GraphicsCamera with wider FOV