LYU 0602 Automatic PhotoHunt Generation
Automatic Automatic PhotoHunt PhotoHunt GenerationGenerationShum Hei Lung To Wan Chi
Supervisor: Prof. Michael R. Lyu
LYU 0602 Automatic PhotoHunt Generation
• Background
• Objectives
• System Implementation
• Segmentation Module
• Elimination Module
• Other Module
• Web-based PhotoHunt game
• Conclusion
OverviewOverview
LYU 0602 Automatic PhotoHunt Generation
BackgroundBackground
• PhotoHunt is …– a Spot-the-difference game
– Classic yet evergreen
– Popular in electronic game centers all over the world
However…It is limited by man power
LYU 0602 Automatic PhotoHunt Generation
Basic techniques to edit image Basic techniques to edit image • Elimination
• Color modification
• Cloning
• Transformation
LYU 0602 Automatic PhotoHunt Generation
ObjectivesObjectives
• To implement the PhotoHunt Generation Engine – Focus on eliminating object
• Develop an engine to support real-time image generation for PhotoHunt game
LYU 0602 Automatic PhotoHunt Generation
System ImplementationSystem Implementation
Image Generation
Engine
Edited Photo
Game Engine
LYU 0602 Automatic PhotoHunt Generation
Image Generation EngineImage Generation Engine
• To generate an image for PhotoHunt game– Effects that may be applied:
• Elimination
• Color Change
• Duplication
Definition of well generated image: • The effects should be
“NOT OBVIOUS YET DISCOVERABLE”
LYU 0602 Automatic PhotoHunt Generation
Image Generation Engine – The ProcessImage Generation Engine – The Process
Segmentation Module
Modification Modules
Smooth Image
GAME Game Engine
LYU 0602 Automatic PhotoHunt Generation
Segmentation ModuleSegmentation Module• To detect and extract segment from the input image
• Three Phases:1. Segmentation
2. Constraint CheckingArea of Input Image/10000<Area of Segment < Area of Input Image/500
3. Area of Segment Reference image building
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
1
1
LYU 0602 Automatic PhotoHunt Generation
Segmentation AlgorithmSegmentation Algorithm
The Segmentation algorithms we have examined:
• Pixel-based Algorithm
• Edge Detection Algorithm
• Region-based Algorithm
LYU 0602 Automatic PhotoHunt Generation
Gaussian Pyramid
Pyramid SegmentationPyramid Segmentation
• The Pyramid Segmentation Algorithm• Step 1 Generation of the Gaussian pyramid
• Step 2 Segmentation by pyramid-linking and averaging linked-pixel
LYU 0602 Automatic PhotoHunt Generation
Step 1- Generation of the Gaussian pyramidStep 1- Generation of the Gaussian pyramid
)2()2()2()1()2()0()2()1()2()2()( 11111 migwmigwmigwmigwmigwig llllll
W(-2)
W(-1)
W(0) W(1) W(2)
5 5 8 7 6 5 4
+
g(i)Father node
Level l-1
Level l
LYU 0602 Automatic PhotoHunt Generation
Step 1- Generation of the Gaussian pyramidStep 1- Generation of the Gaussian pyramid
m n
ll njmignmwjig )2,2(),(),( 1
LYU 0602 Automatic PhotoHunt Generation
Step 2 Segmentation by pyramid-linking Step 2 Segmentation by pyramid-linking and averaging linked-pixel and averaging linked-pixel
• ct(i,j,l) : local characteristics
• ft(i,j,l) : The potential father node
• st(i,j,l) : segment property
• areat(i,j,l) : area of the segment
LYU 0602 Automatic PhotoHunt Generation
1. Set ft(i,j,l)
2. Initializeareat(i,j,0) =1
ct(i,j,l) =c0(i,j,0)
3. for l = 0 to L
st(i,j,L) = ct(i,j,L)
for l = L downto 0
st(i,j,l) = ct(i”,j”,l+1)
),,(
)1,','()1,','((),,(
ljiarea
ljicljiarealjic
t
tt
)1,,(),,( ljiarealjiarea tt
Step 2 Segmentation by pyramid-linking Step 2 Segmentation by pyramid-linking and averaging linked-pixel and averaging linked-pixel
LYU 0602 Automatic PhotoHunt Generation
22 4
1215 12
13
17
2117 17
17 17 17 17 17 4 4 22
Step 2 Segmentation by pyramid-linking Step 2 Segmentation by pyramid-linking and averaging linked-pixel and averaging linked-pixel
LYU 0602 Automatic PhotoHunt Generation
Elimination AlgorithmElimination Algorithm
• Direct Copy Algorithm
• Horizontal Gradient Algorithm
• Nearest Boundary Algorithm
• Enhanced Nearest Boundary Algorithm
LYU 0602 Automatic PhotoHunt Generation
Direct Copy AlgorithmDirect Copy Algorithm• Directly copy the upper pixel to current pixel
iSyxCforyxCyxC ),()1,(),(
Where Si is segmented region of the image for i = 0,1,2,……
LYU 0602 Automatic PhotoHunt Generation
Horizontal Gradient AlgorithmHorizontal Gradient Algorithm
• Determine color line by line
• Smoothing the changes of color from one side to another
)()()()()(PQ
NP
PQ
NQ
d
dQC
d
dPCNC
Where C(M) is the color verctor of pixel M and dXY is the distance between X and Y
LYU 0602 Automatic PhotoHunt Generation
Nearest Boundary AlgorithmNearest Boundary Algorithm
min
min
min
min
)(
)(
)(
)(
)(
ddSC
ddRC
ddQC
ddPC
NC
SN
RN
QN
PN
Where dmin = min(dPN, dQN, dRN, dSN)
• Replace the color by the color of nearest boundary
LYU 0602 Automatic PhotoHunt Generation
Enhanced Nearest Boundary Algorithm Enhanced Nearest Boundary Algorithm
• Improved from Nearest Boundary Algo.
),(),()(
),(),()(
),(),()(
),(),()(
)(
min
min
min
min
QPSSRSandddifSC
QPSSRSandddifRC
SRSQPSandddifQC
SRSQPSandddifPC
NC
SN
RN
QN
PN
Where dmin = min(dPN, dQN, dRN, dSN) S(A,B) is Euclidean distance between A and B
LYU 0602 Automatic PhotoHunt Generation
Enhanced Nearest Boundary AlgorithmEnhanced Nearest Boundary Algorithm
LYU 0602 Automatic PhotoHunt Generation
Enhanced Nearest Boundary AlgorithmEnhanced Nearest Boundary Algorithm
LYU 0602 Automatic PhotoHunt Generation
Color Change ModuleColor Change Module
Segmentdominate color
Operation apply
Red Swap red and blue channel
Green Swap red and green channel
Blue Swap green and blue channel
Dominate color segment: % of Pixel of dominate color> 75%Dominate color pixel: DChannel >180 and otherChannel<50
LYU 0602 Automatic PhotoHunt Generation
Smooth ImageSmooth Image
• To reduce noise and distortion
• To make the image more realistic
• Gaussian Filter (Neighbor size=3, sigma=1)
LYU 0602 Automatic PhotoHunt Generation
Semi-Automatic Semi-Automatic PhotoHunt GenerationPhotoHunt Generation
LYU 0602 Automatic PhotoHunt Generation
Semi-Automatic PhotoHunt GenerationSemi-Automatic PhotoHunt Generation
LYU 0602 Automatic PhotoHunt Generation
Game EngineGame Engine
PHP S cript
U s e rs
Au tom at i cPh oto
Ge n e rati onEn gin e
P h o to S c o re
Gam eplayi n g
a ppl i cationwri tte n in
Flash
G a m e E n g i n e fo r W eb A p p lica tio n
im a g e s
o rig i n a l p h o to
w i th g en e ra ted p h o to
g e n e r at io n r e q u e s t
r e su l t in g p h o t o
p la y ing v iew
gen e
rat e
d p h
oto
s av e
d ph
o to s
p la yin g sc ore
p la y in g s co re
p la y in g sco re
p la y in g req u e st
u p lo a d ed im a g e s
LYU 0602 Automatic PhotoHunt Generation
LimitationLimitation
• Only elimination and limited color change effect supported
• No artificial intelligence control to the Applied effect
• Modified area found noise and distortion
• Not all user input can be segmented
LYU 0602 Automatic PhotoHunt Generation
Looking Forward…Looking Forward…
• Improve the segmentation algorithm
• Implement the Duplication and Transformation module
• Support numbers of object recognitions
• More Features on the Web-based game
• Ultimately, to achieve the complete automatic PhotoHunt generation
LYU 0602 Automatic PhotoHunt Generation
A B
C D
A1 Example of Pyramid SegmentationA1 Example of Pyramid Segmentation
21 25 2 0
22 15 1 1
13 11 2 3
13 12 12 2
c0(0)= 21 c0(1)= 25 c0(2)= 2 c0(3)= 0 c0(4)= 22 c0(5)= 15 c0(6)= 1 c0(7)= 1 c0(8)= 13 c0(9)= 11 c0(10)= 4 c0(11)= 5 c0(12)= 13 c0(13)= 12 c0(14)= 12 c0(15)= 2
c0(A)= 21.2 c0(B)= 2.2 c0(C)= 14.8 c0(D)=3.2
f0(0)= A f0(1)= A f0(2)= B f0(3)= B f0(4)= A f0(5)= C f0(6)= B f0(7)= B f0(8)= C f0(9)= C f0(10)= B f0(11)= D f0(12)= C f0(13)= C f0(14)= C f0(15)= D
a0(A)= 3 a0(B)=5 a0(C)= 6 a0(D)= 2
c0(A)= 22.6 c0(B)=1.2 c0(C)= 12.6 c0(D)= 2.5
LYU 0602 Automatic PhotoHunt Generation
A3 Pixel-based Segmentation -ThreshA3 Pixel-based Segmentation -Thresholding olding
0
10000
20000
30000
40000
50000
60000
70000
80000
1 21 41 61 81 101 121 141 161 181 201 221 241
Fig 3.2b Histogram of data generated by ThresSeg.cpp
Intensity (0-28)
Frequency
Threshold
otherwise
Tyxcyxc
1
),(0),('
10 2 DT
2/)( 1,1, ibioi mmT
LYU 0602 Automatic PhotoHunt Generation
A4 Edge Detection Segmentation-A4 Edge Detection Segmentation-Canny Edge DetectionCanny Edge Detection
• Phase1 - Gaussian convolution S(x,y)=smoothed image
• Phase 2 - Edge strength and direction
|M|=|H|+|V|
• Phase 3 - Non-maximum suppression- Non-maximum suppression trace along the edge in the edge direction and suppress any pixel value that is
not considered to be an edge to zero.
• Phase 4 - Hysteresis- continuing the tracking until the Threshold falls behind the lower second threshold
2/)),1()1,1(),()1,((),( yxSyxSyxSyxSyxH 2/))1,()1,1(),(),1((),( yxSyxSyxSyxSyxV
otherwiseyxHyxV
yxandVyxH
yxandVyxH
yx
)),(/),((tan
0),(0),(2
0),(0),(0
),(
1
),(*),(),( yxcyxGyxS
LYU 0602 Automatic PhotoHunt Generation
A5 Gaussian Filter CharacteristicsA5 Gaussian Filter Characteristics1. Separable (when it is in 2 dimension0)
(x,y) = (x) (y)
2. Normalized
3. Symmetric
4. Equal contribution
Each pixel should have equal contribution to the father pixel in the upper level.
1)(^
m
mw
)()(^^
mwmw