Transcript BARViS – Bax’s Augmented Reality Vision System

AR2kanoid Augmented Reality
Arkanoid
Bax Smith, BSc, BEng, PhD Candidate
Supervisor: Ray Gosine
Topics of Talk
What is Arkanoid?
What is AR2kanoid?
AR2kanoid Screenshots
Component Overview of AR2kanoid
Difficulties in Implementation
Conclusions and Future Work
What is Arkanoid?
Video game invented in 1986
AKA Breakout, Bricks
What is AR2kanoid?
Augmented Reality version of Arkanoid
developed in 2002-2003
AR2kanoid Screenshot
AR2kanoid Screenshot
AR2kanoid Screenshot
AR2kanoid Screenshot
AR2kanoid Screenshot
Component Overview of
AR2kanoid
Vision
Kalman
Filter
Render
Physics
Calibration
Sound
Joystick
Component Overview of
AR2kanoid - Calibration
Mode 1 – Camera Calibration



Segment Circles from Several Images
Compute Lens Distortion Parameters
Save Calibration Parameters
Mode 2 – Apply Camera Calibration



Load Calibration parameters
Segment Circles from Image
Get the Position and Orientation of Surface from
Circles
Difficulties in
Implementation
Problem: Camera
Distorts Image
Solution:
Calibrate Camera
Component Overview of
AR2kanoid - Vision
Responsibilities





Interface to a camera
Initialization and Shutdown
Set the video format (FPS, Size, Color Encoding)
Resize Image using Bilinear Interpolation
Send Image to Calibration System

Returns Position and Orientation of Surface
Supports USB and Firewire Cameras
Difficulties in
Implementation
Problem: TOO MUCH DATA!!!!!!!!
Image Size: 640x480x3 = ~1MByte
 4 Rows x 5 Columns of Circles

Looked Real, Accurate Position Measurement
 Very Slow ~1 Frames/second
 CPU Intensive ~100%

Solution: Reduce Image Size and
Number of Circles
Difficulties in
Implementation
Problem: Small Image Size, Less Circles


160x120x3 = ~60KBytes
2 Rows x 3 Columns of Circles



Fast ~30 Frames/second
Not CPU Intensive ~20%
Image Blockie, Noisy Position Measurement
Solution: Bilinear Interpolation & Kalman
Filter

Better Image, Better Position Estimation
Component Overview of
AR2kanoid – Kalman
Filter
Used to Predict Position of Surface and
Position of Camera
Simple Model

Distance = Velocity*Time
Component Overview of
AR2kanoid - Joystick
Initialize Joystick using DirectX

Supports all Windows Joysticks
Get X-Y Position and Button Status
Shutdown Joystick
Component Overview of
AR2kanoid - Sound
Initialize Sound using DirectX
Load Sounds from Disk
Play Sounds when requested
Puck Hovering
 Puck Colliding with a Wall or Paddle
 Puck Colliding with a Brick
 Puck Colliding with the Goal Line

Shutdown Sound
Component Overview of
AR2kanoid - Physics
Detect Puck Collision





Brick – Brick explodes, puck changes direction
Paddle – Puck changes direction
Side Walls – puck changes LR direction
Top Wall – puck changes UD direction
Goal Line – no. of pucks decrements, play resets
Get Joystick Status and Move Paddle
Move Puck

Puck Position,Velocity, Collision, Joystick
Component Overview of
AR2kanoid - Render
Gets Puck and Paddle Position from
Physics System
Gets Position and Orientation of
Surface from Vision System
Gets Image from Vision System
Renders Scene on top of Image
Conclusion and Future
Work
Version 1 Completed!
Future Work
Better Memory Management/Optimizations
 Better Position Extraction/Image
Processing Algorithms
 Better Model for the Kalman Filter
 Better/More Detailed Graphics
 Camera with wider FOV
