13 November 2013 Birkbeck College, U. London 1
Research Methods
Lecturer: Steve Maybank
Department of Computer Science and Information Systems
[email protected] 2013
Data Research Methods in Computer Vision
13 November 2013 Birkbeck College, U. London 2
Digital Images
A digital image is a rectangular arrayof pixels. Each pixel has a positionand a value.
95 110
40 34
125
108
25 91
158
116
59 112
166
132
101
124
Original colour image from the EfficientContent Based Retrieval Group, Universityof Washington
13 November 2013 Birkbeck College, U. London 3
Size of Images
Digital camera, 5,000x5,000 pixels, 3 bytes/pixel -> 75 MB.
Surveillance camera at 25 f/s ->1875 MB/s.
1000 surveillance cameras -> ~1.9 TB/s.
Not all of these images are useful!
13 November 2013 Birkbeck College, U. London 4
Image Compression Divide the image into blocks, and
compress each block separately, e.g. JPEG uses 8x8 blocks.
Lossfree compression: the original image can be recovered exactly from the compressed image.
Lossy compression: the original image cannot be recovered.
13 November 2013 Birkbeck College, U. London 5
Why is Compression Possible?
Natural image: values ofneighbouring pixels arestrongly correlated.
White noise image: values ofneighbouring pixels are notcorrelated. Compression discardsinformation.
13 November 2013 Birkbeck College, U. London 6
Measurement Space
Each 8x8 block yields a vector in R64. The vectorsfrom natural images tend to lie in a low dimensionalsubspace of R64.
R64
Vectors from 8x8 blocks
13 November 2013 Birkbeck College, U. London 7
Strategy for Compression
Choose a basis for R64 in which the low dimensionalsubspace is spanned by the first few coordinate vectors.Retain these coordinates and discard the rest.
R64 vectors from 8x8 blocks
13 November 2013 Birkbeck College, U. London 8
Discrete Cosine Transform
small. be to tends then large, is If
. and DCT
Then .1, 0,by , vectorsDefine
norm.Euclidean
theis . where,DCTthat
Note . matrix, othogonal 6464certain a is where
DCT
Then block. 88an from obtained vector a be Let
64
1
64
1
64
2/1
64
i
i i
Tii
ij
TT
T
ci
ieUcwiecw
ieijieRie
wUwUwUww
IUUU
Uww
Rw
13 November 2013 Birkbeck College, U. London 9
Basis Images for the DCT
UTe(1) UTe(2) UTe(3) UTe(4)
13 November 2013 Birkbeck College, U. London 10
Example of Compression using DCT
Original image Image constructed from 3 DCTcoefficients in each 8x8 block.
13 November 2013 Birkbeck College, U. London 11
Linear Classification
xx
xx
xx
y yy y
Given two sets X, Y ofmeasurement vectors fromdifferent classes, find a hyperplanethat separates X and Y.
A new vector is assigned to theclass of X or to the class of Y,depending on its position relativeto the hyperplane.
13 November 2013 Birkbeck College, U. London 12
Fisher Linear Discriminant
.
maximises which 0 vector theis FLD The
.1
by covariance class within thedefine and
by covariance classbetween theDefine
.1
,1
,1
define ,,...,,,...,Given
11
1111
11
vSv
vSvvJ
v
yyyyxxxxnm
S
S
zyzynm
nzxzx
nm
mS
S
yxnm
zyn
yxm
x
RyyYRxxX
wT
bT
n
iii
m
iiiw
w
b
b
n
ii
m
ii
n
ii
m
ii
kn
km
13 November 2013 Birkbeck College, U. London 13
Maximisation of J(v)
required. is maximum The . somefor
if 0/therefor
2
.0for maximise and
by Define
2
vSvS
vvJ
vSv
vSvSvvSvSv
v
vJ
vvJ
vSv
vSvvJ
vJ
wb
wT
wbT
bwT
wT
bT
13 November 2013 Birkbeck College, U. London 14
Edge Regions and Regions Without a Peak
3x3 blocks with large Sobel gradients3x3 blocks such that the grey level ofthe central pixel equals the mean greylevel of the 9 pixels
13 November 2013 Birkbeck College, U. London 15
Projections Onto a Random Plane
Sobel edges Regions withouta peak
Superposed plots
13 November 2013 Birkbeck College, U. London 16
Projections Onto a 1-Dimensional FLD
Sobel edges Regions withouta peak
Combinedhistograms
Histogram of a DCT Coefficient
13 November 2013 Birkbeck College, U. London 17
The pdf is leptokurtic, i.e. it has a peak at 0and “fat tails”.
Sparseness of the DCT Coefficients
For a given 8×8 block, only a few DCT coefficients are significantly different from 0.
Given a DCT coefficient of low to moderate frequency, there exist some blocks for which it is large.
13 November 2013 Birkbeck College, U. London 18
Whitening the Data
13 November 2013 Birkbeck College, U. London 19
whitened.also are
1 ,
vectors then thematrix, orthogonalan is If matrix.identity
theis theof covariance The data. whitened theare The
.1 ,
by Define
1
by covariance
theEstimate ts.coefficien DCT of vectorsbe 1 ,Let
2/1
1
niiVz
V
iziz
niiuCiz
iz
iuiun
C
C
niiu
Tn
i
Independence and Correlation
13 November 2013 Birkbeck College, U. London 20
ed.uncorrelat
t thanrequiremenstronger much a is ceindependenbut ,(1)(2)
(2) .for ,,
ift independen are The
(1) .for ,0
ed,uncorrelat are The . of components thebe 1 ,Let
.
by rv thedefine and
of covariance thebe Let .in variablerandom a be Let
2/1
jizpzpzzp
z
jizzE
zzniz
uCz
z
uCRu
jiji
i
ji
ii
n
Independent Components Analysis
Let z be a rv with values in R^n such that cov(z)=I.
Find an orthogonal matrix V such that the components of s =Vz are as independent as possible.
13 November 2013 Birkbeck College, U. London 21
Method
Assume a particular leptokurtic pdf p for the components si, e.g. a two sided Laplace density.
Find the orthogonal matrix V with rows v1,…,vn that maximises the product
p(s1)…p(sn) = p(v1.z)…p(vn.z)
13 November 2013 Birkbeck College, U. London 22
Example Natural Image Statistics: a
probabilistic approach to early computational vision, by A. Hyvarinen, J. Hurri and P.O. Hoyer. Page 161.
ICA applied to image patches yields Gabor type filters similar to those found in the human visual system.
13 November 2013 Birkbeck College, U. London 23
Stereo Pair of Images
13 November 2013 Birkbeck College, U. London 24
UC Berkeley looking westhttp://arch.ced.berkeley.edu/kap/wind/?p=32
Salience A left hand image region is salient if the
correct right hand matching region can be found with a high probability.
H: hypothesis that (v(1),v(2)) is a matching pair.
B: hypothesis that (v(1),v(2)) is a background pair
13 November 2013 Birkbeck College, U. London 25
Formulation using PDFs The region yielding v(1) is salient if
p(v(2)|v(1), H) differs significantly from p(v(2)|v(1), B)
Measurement of difference: the Kullback-Leibler divergence:
13 November 2013 Birkbeck College, U. London 26
kR
dvBvvpHvvpHvvp 2,1|2/,1|2ln,1|2
Illustration of the Kullback-Leibler Divergence
KL divergence = 0.125
13 November 2013 Birkbeck College, U. London 27
KL divergence = 0.659
Preprint on salience available from SJM