View
220
Download
0
Category
Preview:
Citation preview
LYU 0602 Automatic PhotoHunt Generation1
Automatic Automatic PhotoHunt PhotoHunt GenerationGenerationShum Hei Lung To Wan Chi
Supervisor: Prof. Michael R. Lyu
LYU 0602 Automatic PhotoHunt Generation2
• Background• Objectives• Previous Work• Newly Developed Module
– Image Analysis – Image Warping– Object Appending
• Enhanced Module– Elimination– Game Engine
• Evaluation• Conclusion
AgendaAgenda
LYU 0602 Automatic PhotoHunt Generation3
BackgroundBackground
• PhotoHunt is …– A Spot-the-difference game
– Classic yet evergreen
– Popular in electronic game centers all over the world
However…It is limited by man power
LYU 0602 Automatic PhotoHunt Generation4
ObjectivesObjectives
• Develop real-time Image Generation Engine– Employ image processing techniques
– Mimic human behavior
• Develop PhotoHunt game– Make use of generation engine
– Implement more unique features
LYU 0602 Automatic PhotoHunt Generation5
Objectives Objectives Image Generation EngineImage Generation Engine
• To generate an image for PhotoHunt game– Effects that may be applied:
• Elimination • Image Warping• Object Appending• Color Change
Definition of well generated image: • The effects should be
“NOT OBVIOUS YET DISCOVERABLE”
LYU 0602 Automatic PhotoHunt Generation6
Previous WorkPrevious Work
ApplicationsApplications
• Semi Automatic PhotoHunt
• Game Engine
Automatic PhotoHunt Generation Automatic PhotoHunt Generation
Image Generation EngineImage Generation Engine
• Segmentation Module• Modification Module -Elimination -Color change• Smoothing module
Image Processing FoundationImage Processing Foundation
LYU 0602 Automatic PhotoHunt Generation7
Gaussian Pyramid
Previous WorkPrevious WorkSegmentation Module Elimination Module Smoothing Module
• To detect and extract segment from the input image• Three Phases:
– Pyramid Segmentation
– Constraint Checking
– Reference image building
Game Engine
LYU 0602 Automatic PhotoHunt Generation8
Previous WorkPrevious WorkSegmentation Module Elimination Module Smoothing Module Game Engine
• Direct Copy Algorithm
• Horizontal Gradient Algorithm
• Nearest Boundary Algorithm
• Enhanced Nearest Boundary Algorithm
LYU 0602 Automatic PhotoHunt Generation9
Previous WorkPrevious WorkSegmentation Module Elimination Module Smoothing Module Game Engine
• To reduce noise and distortion
• To make the image more realistic
• Gaussian Filter (Neighbor size=3, sigma=1)
LYU 0602 Automatic PhotoHunt Generation10
Previous WorkPrevious WorkSegmentation Module Elimination Module Smooth Module Game Engine
LYU 0602 Automatic PhotoHunt Generation11
Last semesterLast semesterSegmentation
Module
Modification Modules
Smooth Image
Game Engine
LYU 0602 Automatic PhotoHunt Generation12
This SemesterThis SemesterSegmentation
Module
Enhanced Elimination
Smooth Image
EnhancedGame Engine
Image Analysis Modules
Image Warping Color Change
Object Appending
LYU 0602 Automatic PhotoHunt Generation13
Image Analysis Module
LYU 0602 Automatic PhotoHunt Generation14
Image Analysis Module
• Purpose– To extract useful information from the image in
order to assist the generation process
Segmentation Module Image Analysis Module
Color Change
Elimination
Image Warping
LYU 0602 Automatic PhotoHunt Generation15
Image Analysis Module > Function 1Image Analysis Module > Function 1Screening out undesirable segmentScreening out undesirable segment
• Definition of undesirable segment– Regions that are wrongly segmented
• Cause of undesirable segment– The engine uses an optimized threshold to segment all
images
• AssumptionSegment & Surrounding havesimilar color and brightness
Undesirable Segment
RejectSegment
Come fromSame Object
LYU 0602 Automatic PhotoHunt Generation16
Image Analysis Module
Functions:
• Screening out undesirable segment
• Deciding modification to be applied
• Providing suggestion on replacement color
LYU 0602 Automatic PhotoHunt Generation17
Image Analysis ModuleScreening out undesirable segmentScreening out undesirable segment
• Procedures: 1. Compute Mean of Object –(1)
2. Compute Mode of Background –(2)
3. Compare (1) and (2)
Object Mean : [66 72 72] T Bg Mode : [220 231 228] T Diff : 475>60 Accept
Object Mean : [66 72 72] T Bg Mode : [220 231 228] T
Diff : 284>60Accept
Object Mean : [217 92 76 ] T Bg Mode : [207 81 65] T
Diff : 32<60Object Mean : [3 83 148 ] T Bg Mode : [10 99 163] T
Diff : 39<60
Reject
Reject
LYU 0602 Automatic PhotoHunt Generation18
Image Analysis Module > Function 2> Function 2Deciding modification to be appliedDeciding modification to be applied
Segment
Color Change
Image Warping
Elimination
Single Colored
Regular in shape
Any shape The property of image for the specified effect:
LYU 0602 Automatic PhotoHunt Generation19
Image Analysis Module > Function 2> Function 2Deciding modification to be appliedDeciding modification to be applied
Object Neighbor Difference<Threshold1 Object Variance < Threshold2
Color Change
yesNo
Object occupied Area>70%
yesNo
Image Warping Elimination
LYU 0602 Automatic PhotoHunt Generation20
Deciding modification to be appliedDeciding modification to be applied
LYU 0602 Automatic PhotoHunt Generation21
Image Analysis Module > Function 3> Function 3 Suggestion on replacement coloruggestion on replacement color
Parameter Meaning
Background
Neighbor Difference
The complexity of background
Background Mode The major color of background
Background Mean The texture property of the background
LYU 0602 Automatic PhotoHunt Generation22
Image WarpingImage WarpingModuleModule
LYU 0602 Automatic PhotoHunt Generation23
Image WarpingImage Warping
• Produce Distortion by applying geometric transformation.
LYU 0602 Automatic PhotoHunt Generation24
LYU 0602 Automatic PhotoHunt Generation25
Image WarpingImage WarpingForward mapping algorithmForward mapping algorithm
Transformation1
Quadratic of y
Transformation2
linear of x
• Transformation Equation (General)
54322
12
0' bybxbxybybxby
xx '
LYU 0602 Automatic PhotoHunt Generation26
• Transformation 1
• Transformation 2
Transformation EquationTransformation Equation
minX
Δy=0Mid pt
Δy=max
maxX
Δy=0
Δymax
1. Quadratic equation of root x=minX or maxX
2. Substituting (midptX, Δymax) into the equation to acquire the weight to control curvature
1. Flip the upper part, y>mid pt y
2. Enlarge the curve with a ratio proportional to distance between mid pt of y
maxY
minY
midptY
LYU 0602 Automatic PhotoHunt Generation27
LYU 0602 Automatic PhotoHunt Generation28
Object AppendingObject AppendingModuleModule
LYU 0602 Automatic PhotoHunt Generation29
Object Appending Object Appending
• To append an object from our database to the original image
• Unable to carry out object recognition– Only generic objects are inserted to engine
LYU 0602 Automatic PhotoHunt Generation30
Examples:Examples:
LYU 0602 Automatic PhotoHunt Generation31
Enhanced Enhanced Elimination ModuleElimination Module
LYU 0602 Automatic PhotoHunt Generation32
Elimination ModuleElimination Module
• Hybrid Elimination
Makes use of statistic data that came from the image analysis module
Information Needed:– Background Mode – Background Neighbor Difference– Background Mean
LYU 0602 Automatic PhotoHunt Generation33
Hybrid Elimination AlgorithmHybrid Elimination Algorithm
• Check Background Neighbor Difference- To check whether the background is single colored
Case 1: Use the Background Mode to replace
Case 2: Apply texture from surrounding – Select the suitable surrounding region – Apply Direct copy Algorithm
LYU 0602 Automatic PhotoHunt Generation34
Game EngineGame Engine
LYU 0602 Automatic PhotoHunt Generation35
System OverviewSystem Overview
PHP S cript
U s e rs
Au tom at i cPh oto
Ge n e rati onEn gin e
P h o to S c o re
Gam eplayi n g
a ppl i cationwri tte n in
Flash
G a m e E n g i n e fo r W eb A p p lica tio n
im a g e s
o rig i n a l p h o to
w i th g en e ra ted p h o to
g e n e r at io n r e q u e s t
r e su l t in g p h o t o
p la y ing v iew
gen e
rat e
d p h
oto
s av e
d ph
o to s
p la yin g sc ore
p la y in g s co re
p la y in g sco re
p la y in g req u e st
u p lo a d ed im a g e s
LYU 0602 Automatic PhotoHunt Generation36
System Interface System Interface • In later demo session
LYU 0602 Automatic PhotoHunt Generation37
EvaluationEvaluation
LYU 0602 Automatic PhotoHunt Generation38
Acceptance RateAcceptance Rate
Assume Acceptance rate= 75% = 988/1318 accepted imagesThus, x-axis is about 200 in upper limit and 4500 in lower limit.
Accepted Image varies Threhold
950
960
970
980
990
1000
1010
1020
1030
1040
1050
1060
100 120 140 160 180 220 240 260
Acc
ep
ted
Ima
ge
Lower Limit=5000
Lower limit=4500
LYU 0602 Automatic PhotoHunt Generation39
Processing TimeProcessing Time
• Run one set of 1318 images for 17 times
• Average Processing time per set: – 00:15:50
• Average Processing time per image– 0.721 second
LYU 0602 Automatic PhotoHunt Generation40
Image QualityImage Quality• Survey Result • Total Visitors : 91
• Total Hits: 1109• Total Images : 264• Received Survey : 121• Total Segments : 605
Satisfy 76%
Not Satisfy24%
Characteristics image shared for achieving good effects:
• Many objects within the image• Sharp Edge of object • Less noise• Maybe a cartoon
LYU 0602 Automatic PhotoHunt Generation41
DemoDemo
LYU 0602 Automatic PhotoHunt Generation42
ConclusionConclusion
• Developed the image generation engine
• Developed the game engine
• Carried out testing and analysis on the system
• Published the product to the public
• We are still watching the statistic and looking at feedback to improve our system
LYU 0602 Automatic PhotoHunt Generation43
Q & AQ & A
LYU 0602 Automatic PhotoHunt Generation44
The EndThe End
Thanks for your kind attention.
Recommended