-
Proceedings of IEEE CCIS2011 CAMERA SELF-CALIBRATION METHOD
BASED
ON GA-PSO ALGORITHM Jing Li, Yimin Yang, Genping Fu
School of Automation, Guangdong University of Technology,
Guangzhou, China [email protected]
Abstract This paper proposes a new algorithm(GA-PSO) by
combining genetic algorithm and particle swarm optimization to
improve the accuracy of camera self-calibration based on the Kruppa
equation. Firstly, the simplified Kruppa equations based on the SVD
of the fundamental matrix is converted into the optimized cost
function. Secondly, the minimum value of the optimized cost
function is calculated by GA-PSO.Finally, the intrinsic parameters
of the camera is obtained. The experimental results show that it is
accurate, and the accuracy of the proposed method is obviously
improved compared with the single optimization methods. Keywords:
Camera self-calibration; Kruppa equation; genetic algorithm;
particle swarm optimization; GA-PSO
1 Introduction Camera calibration is an indispensable step to
obtain 3D information from 2D images. Recently, many researchers
have proposed their methods, and these technologies can be divided
into three categories as follows: standard calibration method,
self-calibration method based on active vision system, and
self-calibration method (no need any calibration pattern, the
camera to be calibrated can be moved, but it does not assume that
the camera motion is known. The only assumption is that for every
displacement of the camera the corresponding points between two
images can be established). Because of simple and convenient
manipulation, much attention had been paid to self-calibration
method [1]-[5]. As the first reported approach for camera
self-calibration in [6], the Kruppa equations, which are derived
from the epipolar geometry of the Absolute Conic, are solved for
identifying the dual of the Image of the Absolute Conic, from which
the camera intrinsic parameters are retrieved. Yang G. et al
proposed an analytic solution of the self-calibration based on the
Kruppas equations[7],the technique allows camera self-calibration
under the least pairs of views of a
scene. Guo Q. Y. et al proposed to convert the equations into
the form of cost function according to the Hartleys deduction and
calculate the cost function[8],the method uses the genetic
algorithm to obtain the minimum value of the cost function,but this
method doesn't deal with local optimal solution,so the accuracy of
the camera self-calibration is not high. In order to improve the
accuracy of the camera self-calibration, we propose the GA-PSO
algorithm to obtain the intrinsic parameters by minimizing the cost
function.
2 The cost function based on Kruppa equations The pinhole camera
model is used in this paper. In this model, the perspective
projection of TZYXM )1,,,(= onto Tyxm )1,,(= is given by PMm = .
The matrix P is the projection that can be factorized into the
rigid motion from the world to the camera reference frame and a 33
upper-triangular matrix K called the intrinsic parameters matrix.
The intrinsic parameters matrix
K is given by
=
1000 0
0
vfusf
K vu
.
Given two images taken at two different positions with the
intrinsic camera parameters fixed, from the epipolar geometry of
absolute conic, the following relation can be obtained: [ ] [ ]
FKKFeKKe TTTT '' (1)
The simplification of the Kruppas equations is given in (2)by
Hartley[9] which is based on the singular value
decomposition(SVD)of the fundamental matrix F . Consider the SVD of
a fundamental matrix TUDVF = . Here D is a diagonal matrix of the
singular values( 1 , 2 ,0) The column vectors of U are 1u , 2u , 3u
and the column vectors of V are 1 , 2 , 3 . Give the simplification
of Kruppas equations as follows
___________________________________ 978-1-61284-204-2/11/$26.00
2011 IEEE
-
11
2222
21
2121
22
1121
CuuC
CuuC
CuuC
T
T
T
T
T
T =
= (2)
Where TKKC = , If C is found out to make those three equations
are equal, then the intrinsic parameters matrix is obtained, so the
question of camera self-calibration can be solved. Supposing that
the three expressions in Eq.(2) are denoted with 1y , 2y and 3y
respectively:
22
1121
1 CuuCy T
T = ,
21
21212 Cuu
Cy TT
=
,11
2222
3 CuuCy T
T = .
The self-calibration problem can be solved by converting the
Kruppas equations to the cost function which is just a mathematic
problem. This cost function can be used for one pair of images or
multiple pairs of images. For one pair of images, the cost function
),,,,( 00 svufff vu is denoted as
232
231
22100 )()()(),,,,( yyyyyysvufff vu ++= (3)
For the multiple pairs of images, Eq.(4) should be multiplied
its corresponding weight factor i which represents the reliability
of fundamental matrix iF which is corresponding to the ith pair of
images. The total cost function is shown as
),,,,(),,,,( 001
00 svufffsvuffF vuiN
iivu
=
= (4)
Where i is the weight factor of the ith image pairs, and belongs
to[0,1], ),,,,( 00 svufff vui is the ith cost function of
corresponding fundamental matrix. The intrinsic parameters in the
cost function are uf , vf , 0u , 0v , s . In this paper, we propose
the GA-PSO algorithm to obtain the intrinsic parameters by
minimizing the total cost function(or to 0).
3 Camera self-calibration method based on GA-PSO algorithm
3.1 Particle swarm optimization
PSO is a stochastic, population-based evolutionary optimization
algorithm. PSO is initialized with a group of random particles
(solutions) and then searches for optima by updating generations.
In every iteration, each particle is updated by following two best
value. The first one is its personal best position which is labeled
as pBest corresponding to the best fitness value obtained so far in
the flying. The other is the global best particle with the best
fitness value of all the particles corresponding to the current
optimum solution, which is marked with gBest. Each particle updates
the velocity and position according to pBest and its own gBest.
The
optimal solution can be found after a lot of times of
iteration.
Particle i updates the thd -dimension of its velocity vector and
position vector in every iteration as follows:
)()( 22111 t
idtgd
tid
tid
tid
tid xPrcxPrcvv ++=+ (5)
11 ++ += tidtid
tid vxx (6)
The inertia weight controls the influence of previous velocity a
particle, resulting in a memory effect. Study factors 1c , 2c play
a role in adjusting a particle's social swarm experience and its
own experience. Here 1r and 2r represent random variables following
the uniform distribution between 0 and 1.The max velocity must be
set at advance i.e.if maxvvid > , then we can
set maxvvid = [10].
3.2 GA-PSO algorithm
Genetic algorithm (GA)has the better global search
capability,but it cannot make use of the feedback information,so it
leads to the massive redundancy iteration. The improved GA
generally considers the problem of premature and convergence,but
ignores the condition of every individual.However,PSO can make good
use of the information of every individual, it can improve the
performance of every individual by updating the the adaptation
value and the best location according to equation(5)and
equation(6). In other words, PSO will perform best in environments
where the average local gradient points towards the global optimum,
but may have difficulties in finding optima when the average local
gradient points in the wrong direction. In contrast, GAs
exploration direction is rather uncertain, it can adjust the search
space by its mutation operator.Thus, GA will be able to perform
better in such an environment, since it does not have a strictly
defined exploration area. Therefore, hybrid GA-PSO has been
proposed by combining advantages of PSO and GA.In our proposed
hybrid GA-PSO, selection, crossover and mutation operation are
included, which can improve the performance of individuals and
increase the diversity of individuals, PSO is considered as the
improvement operator to improve the performance of individuals
again. The GA-PSO algorithm makes full use of the individual
information in every iteration and has strong global search
capability, the performance of GA-PSO is obtained greatly improved,
and the following generation which is generated by GA-PSO in every
iteration is more outstanding than
-
GA and PSO, so the optimization result of GA-PSO will be more
accurate than GA and PSO. Our GA-PSO algorithm is proposed as
follows: Initialization; Calculating fitness value for GA and PSO;
Selection; Crossover; mutation; Applying PSO to improve the
performance of the individuals; Until the termination criterion is
reached. The GA-PSO algorithm flow chart is showen in Figure 1.
Figure 1 The flow chart of GA-PSO
3.3 Camera self-calibration method based on GA-PSO algorithm
Camera self-calibration method based on GA-PSO algorithm
describe as follows: Detect the corner points for each image of the
sequence images. Match the corner points for each pair of images.
Compute all the fundamental matrices, calculate the SVD of all the
fundamental matrices and the corresponding Kruppa equations. Write
out the fitness function about Eq.(4), then compute the intrinsic
parameters by using GA-PSO algorithm as follows: Step1: Initial
population.It generates 2 groups of individuals, Set the initial
position of particles and initial velocity in specified range.Each
ndividual is a combination of uf , vf , 0u , 0v , s which are real
number encoding, Population size labeled as popsize is set at 200
and each individual is randomly initialized in search space. Step2:
Calculate the fitness value of each individual.For each individual,
calculate the adaptation value and preserve its best location and
the corresponding optimal adaptation value. From
the obtained optimal adaptation values, choose the optimal
particle as gbest. Step3: Selection operator. Random traversal
sampling method is chosen. Step4: Crossover operator. This operator
chooses the line recombination between pairs of individuals and
returns the new individuals after mating. Probability of crossover
occurring between pairs of individuals is not used, only for
compatibility. Step5: Mutation operator. the mutation probability
labeled as Pm is set as Pm=0.01. We control the mutation range in
the mutation operation according to the generation and the fitness
value. Step6: Improvement operator. We calculate the new speed of
each particle according to equation(5)and the new location of each
particle according to equation(6). For each particle,update the
adaptation value and the best location. Inertia weight: =0.6, Study
factors: 1c = 2c =1.4962. Step7: If it reaches the maximum
iteration number 200, the best individual of the last generation is
considered as the optimum results of one time, otherwise, go to
step 2. The GA-PSO will run 100 times like this and the average of
the results of all times is calculated.
4 Experiment and analysis of the results In order to demonstrate
the performance of our method, we conduct experiments with real
images for camera calibration. Three real images are used here
Figure 2. The Sequence images of template which can be downloaded
from the internet (http://research.
microsoft.com/en-us/um/people/zhang/calib/). Feature points are
automatic picked and matched using the method proposed by lowe et
al [11]. The fundamental matrices are computed by RANSAC method. We
have also used GA and PSO algorithm to calculate the Objective
function which is defined by Eq.(5). The performance of three
algorithms is showed in Table 1. The calibration parameters are
showed in Table 2.
Figure 2 The sequence images From Table 1, we can see that the
performance of GA-PSO is best among the three algorithms.The
minimum value and the average value both get close to 0,success
rate is 100%.
-
From Table 2, we can see that this papers result is similar to
Zhang. This result is obviously improved compared with the single
optimization methods. Due to the GA-PSO algorithm which is combined
by GA and PSO, its running time is more than GA and PSO, but it is
worth to obtain the accurate result at the expense of a few seconds
time. In order to get more stable results, we can add the
population size properly, but it also increases the running
time.
Table 1 the performance of three algorithms
Method Minimum value Average value Success rate
GA 1.67 4.63 98%
PSO 51083.4 5.68 97%
GA-PSO 51005.1 51002.4 100%
Table 2 Calibration results
Method uf vf 0u 0v s
Zhang 832.53 832.53 303.96 206.58 0.20
GA 803.07 803.06 290.10 201.34 0.36
PSO 804.46 804.15 295.93 198.59 0.72
GA-PSO 831.32 831.39 304.04 206.71 0.21
5 Conclusions In order to improve the accuracy of camera
self-calibration, the GA-PSO algorithm which combines GA with PSO
is presented.Firstly, simplified Kruppa equations are obtained by
singular value decomposition of the fundamental matrix. Secondly,
the value of the camera intrinsic parameters are obtained by
optimizing the cost function based on simplified Kruppa equations
with GA-PSO algorithm. The experimental results show that the
accuracy of the proposed method is obviously improved compared with
single optimization methods. Because the calibration method is
simple and high-precision, it has great practical value. The author
will apply this method to real-time sequence images in the later
work.
References [1] Triggs B, Auto-calibration and the absolute
quadric, Proceedings of Computer Vision and Pattern Recognition,
1997:609-614
[2] Lei Cheng,Wu Fu-chao,Hu Zhan-Yi, Kruppa Equations and Camera
Self-calibration, ACTA AUTOMATICA SINICA, 2001,27(5):621-630
[3] Meng Xiao-Qiao, Hu Zhan-Yi, Recent Progress in Camera
Self-Calibration, ACTA AUTOMATICA SINICA, 2003, 29(1):110-
124 [4] Wu Fu-chao, Hu Zhan-Yi, A New Theory
and Algorithm of Linear Camera Self-Calibration, CHINESE J.
COMPUTERS, 2001,24(11):1222-1235
[5] Habed A,Boufama B.Camera self-calibration frombivariate
polynomials derived From Kruppas equations[J]. Pattern Recognition,
2008,41:2484 2492
[6] Faugeras O D,Luong Q T,Maybank S J.Camera
self-calibration:theory and experiments[C]//Proc of European
Conference on Computer Vision.Berlin: Springer,1992:321-334
[7] YANG Guo,ZHENG Fang,XU Xin-he.An analytic solution of a
linear cameraself-calibration[C]//Proc of the 6th world Congress on
IntelligentControl and Automation.2006:9930-9934
[8] Guo Q. y., Lu P. F., et al. Self-calibration for cameras
based on genetic algorithm. Journal of Image and Graphics,
2006,11(11):1712-1715
[9] Hartley R,Zisserman A.Multiple View Geometry in Computer
Vision[M]. Cambridge:Cambridge University Press,2000
[10] Jiang Ze-tao,Wu Wenhuan,Wu Min.Camera Autocalibration from
Kruppas Equations Using Particle Swarm Optimization[C]. 2008
International Conference on Computer Science and Software
Engineering.2008:1032-1034
[11] LOWE D.Distinctive image features from scale-invariant
keypoints[J].International Journal of Computer
Vision,2004,60(2):91-110
