Virtual Dart: An Augmented Reality Game on Mobile Device Supervisor: Professor Michael R. Lyu Prepared by: Lai Chung Sum Siu Ho Tung

Virtual Dart: An Augmented Reality Virtual Dart: An Augmented Reality Game on Mobile DeviceGame on Mobile DeviceSupervisor: Professor Michael R. LyuSupervisor: Professor Michael R. Lyu

Prepared by:Prepared by:

Lai Chung SumLai Chung Sum

Siu Ho TungSiu Ho Tung

OutlineOutline

Background InformationBackground InformationMotivationMotivationObjectiveObjectiveMethodsMethodsResultsResultsFuture WorkFuture WorkQ & AQ & A

What is Augmented Reality (AR)?What is Augmented Reality (AR)?

A combination of real world and computer A combination of real world and computer generated datagenerated data

Add computer graphic into videoAdd computer graphic into video

Background InformationBackground Information

Most mobile phones equipped with camerasMost mobile phones equipped with camerasGames written in J2ME & proprietary Games written in J2ME & proprietary

development platformdevelopment platform


Typical mobile gamesTypical mobile games


Mobile games employed Augmented RealityMobile games employed Augmented Reality

MotivationMotivation

How can the game “remember” external How can the game “remember” external environment?environment?

Save external environment informationSave external environment information

ObjectivesObjectives

Demonstrate how a game “remember” its Demonstrate how a game “remember” its external environment for Augmented Reality external environment for Augmented Reality (AR)(AR)

Virtual Dart is just a game for demonstration Virtual Dart is just a game for demonstration of the proposed methodologyof the proposed methodology

Problems to be solved…Problems to be solved…

1.1. What information should we store?What information should we store?

2.2. How does the game recognize the How does the game recognize the information?information?

3.3. How does the game perform motion How does the game perform motion tracking?tracking?

Introduction to Mobile Video Object Introduction to Mobile Video Object Tracking Engine (mVOTE)Tracking Engine (mVOTE)

Convert the camera movement into translational Convert the camera movement into translational movement and degree of rotationmovement and degree of rotation

Feature Selection Feature Selection (Find a feature to trace)(Find a feature to trace) Motion Tracking of Translational MovementMotion Tracking of Translational Movement Motion Tracking of Rotational Movement Motion Tracking of Rotational Movement

What is a feature?What is a feature?

Section of an image that is easily highlighted Section of an image that is easily highlighted for the purpose of for the purpose of detectiondetection and and trackingtracking

Have a Have a high contrasthigh contrast in relation to its in relation to its immediate surroundingsimmediate surroundings

X

What does our program need?What does our program need?

Functions needed for our programFunctions needed for our program Can mVOTE do this?Can mVOTE do this?

Feature SelectionFeature Selection(What information should we store?)(What information should we store?) Feature RecognitionFeature Recognition(How does the game recognize the information?)(How does the game recognize the information?) Motion Tracking of Translational Motion Tracking of Translational MovementMovement(How does the game perform motion tracking?)(How does the game perform motion tracking?)

Program Flow – Initial AlgorithmProgram Flow – Initial Algorithm

Program Flow – Initial AlgorithmProgram Flow – Initial Algorithm

Experiment of Feature SelectionExperiment of Feature Selection

Feature Selection in mVOTE Feature Selection in mVOTE VSVS FAST FAST Corner Detection AlgorithmCorner Detection Algorithm

Testing EnvironmentTesting Environment1.1. Normal lightingNormal lighting

2.2. Insufficient lightingInsufficient lighting

Normal Lighting ConditionNormal Lighting Condition

Feature Selection in mVOTEFeature Selection in mVOTE FAST Corner DetectorFAST Corner Detector





Insufficient Lighting ConditionInsufficient Lighting Condition


Insufficient Lighting ConditionInsufficient Lighting Condition


AnalysisAnalysis

Normal LightingNormal Lighting

Both algorithms worked reasonably wellBoth algorithms worked reasonably well

Insufficient LightingInsufficient Lighting

Only mVOTE’s Feature Selection could produce outputOnly mVOTE’s Feature Selection could produce output

Occasionally, Feature Selection in mVOTE selected Occasionally, Feature Selection in mVOTE selected some flat regions as featuressome flat regions as features

FAST Corner worked better in terms of accuracyFAST Corner worked better in terms of accuracy

Experiment of Initial ApproachExperiment of Initial Approach

Selected FeaturesSelected Features Recognized FeaturesRecognized Features





Initial Feature Recognition ConclusionInitial Feature Recognition Conclusion

Accuracy?? LOW!Accuracy?? LOW!


Program Flow – Enhanced Feature Program Flow – Enhanced Feature Recognition Algorithm Recognition Algorithm

Enhanced Feature RecognitionEnhanced Feature Recognition

Set 1Set 1 Set 2Set 2





Algorithms Comparison Algorithms Comparison

Initial Feature Recognition Initial Feature Recognition VSVS Enhanced Enhanced Feature RecognitionFeature Recognition Initial Approach: 3 FeaturesInitial Approach: 3 FeaturesNew Approach: Whole selection areaNew Approach: Whole selection area

Reason for LOW accuracy: Reason for LOW accuracy: (Initial Approach)(Initial Approach)

Features may not be descriptive enoughFeatures may not be descriptive enough

Improvement of Feature SelectionImprovement of Feature Selection

Two conditions of a “Good” Feature:Two conditions of a “Good” Feature:Descriptive Descriptive Large internal intensity differenceLarge internal intensity difference

Corner Detector can help us to find out Corner Detector can help us to find out good featuresgood features

FAST Corner DetectorFAST Corner Detector

Examine a small patch of imageExamine a small patch of imageConsidering the Bresenham Circle of radius Considering the Bresenham Circle of radius rr

around the candidate pixel which is called around the candidate pixel which is called pp Intensities of Intensities of nn continuous pixels on the circle continuous pixels on the circle

are larger than are larger than pp or smaller than or smaller than pp by by barrierbarrier Potential cornerPotential corner

215215 255255 200200

3939 167167

2020 132132

1616 6565 152152

9191 153153

101101 165165

114114 182182 167167

e.g. r = 3, n = 12, barrier = 25

215 – 65 = 150 > 25 = barrier Marked by red

65 – 39 = 26 > 25 = barrier Marked by Blue

215215 6363 2121

8484 5757

108108 1212

6565 9090 5252

3131 1515

3535 4343

5757 5858 6262

e.g. r = 3, n = 12, barrier = 25

215215 6363 2121

8484 5757

108108 1212

6565 9090 9090

3131 1515

3535 4343

5757 5858 6262

e.g. r = 3, n = 12, barrier = 25


The typical values of The typical values of rr and and nn are 3 and 12 are 3 and 12 respectivelyrespectively

For the value of For the value of barrierbarrier, we did an , we did an experiment to choose the valueexperiment to choose the value

We chose “25” after the experiment (for We chose “25” after the experiment (for what?)what?)


Advantage:Advantage:FastFast

Disadvantages:Disadvantages:Cannot work well in noisy environmentCannot work well in noisy environmentAccuracy depends on parameter Accuracy depends on parameter – – barrierbarrier


barrier barrier = 10= 10 barrier barrier = 40= 40

How does Feature Recognition How does Feature Recognition works?works?

Full screen as search windowFull screen as search windowUse Sum Square Difference (SSD) to Use Sum Square Difference (SSD) to

calculate the similarity of blockscalculate the similarity of blocksStill slow in current stage (~20 – 60sec)Still slow in current stage (~20 – 60sec)Tried to use a smaller image and scale up to Tried to use a smaller image and scale up to

full screenfull screenScaling step is too time consumingScaling step is too time consuming

Motion Tracking during the gameMotion Tracking during the game

Keep track of three featuresKeep track of three featuresUse two features to locate dart boardUse two features to locate dart boardThe last feature point is used for backupThe last feature point is used for backupUse if either one of the feature points failUse if either one of the feature points failCondition for a feature point failureCondition for a feature point failure

Feature point is at the edge of the screen Feature point is at the edge of the screen Two feature points are too close Two feature points are too close

Future WorksFuture Works

Allow users to load saved featuresAllow users to load saved features Increase the speed of feature recognitionIncrease the speed of feature recognitionAdd physical calculation engineAdd physical calculation engine

Q & AQ & A

Documents

Virtual Dart: An Augmented Reality Game on Mobile Device Supervisor: Professor Michael R. Lyu Prepared by: Lai Chung Sum Siu Ho Tung