Upload
plato
View
28
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Applications of Projection Method of Fourier Transform Inversion in Multimedia Data Treatment. Faculty of Computational Mathematics and Cybernetics, Moscow State University. Andrey S. Krylov. NUS, Singapore, August 14 , 2003. Applications of Projection Method of - PowerPoint PPT Presentation
Citation preview
Faculty of Computational Mathematics and Cybernetics, Moscow State University
NUS, Singapore, August 14NUS, Singapore, August 14, 2003, 2003
Andrey S. KrylovAndrey S. Krylov
Outline:Outline:• Projection Method (Hermite series approach)Projection Method (Hermite series approach)• ApplicationsApplications
1. Image filtering and deblocking by projection filtering
2. Image matching 3. Texture matching4. Low-level methods for audio signal
processing
The proposed methods is based on the features of Hermite functions. An expansion of signal information into a series of these functions enables one to perform information analysis of the signal and its Fourier transform at the same time.
Hermite transformHermite transform
0
4 29
nn
n i ˆ
),(2 L
n
xn
n
xn
n dx
ed
n
ex
)(
!2
)1()(
22 2/
B) They derivate a full orthonormal in system of functions.
The Hermite functions are defined as:
A)
Original image
2D decoded image by 45 Hermite functions at the first pass and 30 Hermite functions at
the second pass
Difference image(+50% intensity)
Original image
2D decoded image by 90 Hermite functions at the first pass and 60 Hermite functions at
the second pass
Difference image(+50% intensity)
Image filtering and deblocking by projection filtering
Original lossy JPEG image
Enhanced image
Difference image (Subtracted high frequency information)
Original image
Zoomed in:
Enhanced image
Scanned image
Enhanced image
Zoomed in:
Scanned image
Enhanced image
Image matching
Information parameterization for image database retrieval
AlgorithmImage normalizingGraphical information parameterization
Parameterized image retrieval_
Information parameterization for image database retrieval
AlgorithmImage normalizingGraphical information parameterization
Parameterized image retrieval
= +HF component LF component Normalized
image
Recovered image with the recovered image plane
Information parameterization for image database retrieval
AlgorithmImage normalizingGraphical information parameterization
Parameterized image retrieval
Database size Initial images formatNormalized images formatNumber of parameterization coefficients Error rate Search time
– 768 images (4.12Gb)– 1600x1200x24bit (5.5Mb)– 512x512x24bit (0.75Mb)– 32x32x3– <0.14%– 4 sec. (for K7-750)
Image matching results
Image matching results
Texture matching
A method of obtaining the A method of obtaining the texture feature vectorstexture feature vectors
0
)()(i
ii xxf
dxxfxii )()(
),()(),(2121
yxyx nnnn
1)(),( xyx nn
Input function
Fourier coefficients
1-D to 2-Dexpansion
A method of obtaining the A method of obtaining the texture feature vectorstexture feature vectors
1-D Hermite functions:1-D Hermite functions:1-D to 2-D expanded 1-D to 2-D expanded
Hermite functions:Hermite functions:
OrientationsOrientations
Localization problemLocalization problem
Decomposition process is optimal, if localization segments of the input function and filtering functions are equal.
Feature vectorsFeature vectors
In this approach to get the feature vectors we consider the functions n(x,y) where n1=0..64, and 6 energy
coefficients are calculated as:E1 = (0)
2 + (1)2 ,
E2 = (2)2+(3)
2 + (4)2,
E3= (5)2 +(6)
2+(7)2+(8)
2,
…E6 = (33)
2+(34)2 + … + (63)
2 +(64)2 ,
f(x,y) is the source image.
Standard codingStandard coding
Feature vectorsFeature vectors
E1 = (0(1))2 + (1
(1))2 ,
E2 = (0(2))2+(1
(2))2 + (2(2))2 +(3
(2))2 ,
…E6 = (0
(6))2+(1(6))2 + … + (62
(6))2 +(63(6))2 ,
dxyxfyxdyA
i
j
i ),(),(1)1(
1,)),(),((),(1
2
)1()(
jdxyxfyxfyxdyA
j
l
li
j
ji
f(x,y) is the source image.
Hierarchical codingHierarchical coding
Feature vectorsFeature vectors
E1 = (0(1))2 + (1
(1))2 ,
E2 = (0(2))2+(1
(2))2 + (2(2))2 +(3
(2))2 ,
…E6 = (0
(6))2+(1(6))2 + … + (62
(6))2 +(63(6))2 ,
dxyxfyxdyA
i
j
i ),(),(1)1(
1,),(),(1)(
jdxyxfyxdyA
i
j
ji
f(x,y) is the source image.
Hierarchical coding without subtractionsHierarchical coding without subtractions
Feature vectorsFeature vectors Texture
sample Standard coding
Hierarchical coding
Hierarchical coding without
subtractions A1
A2
A3
A4
B1
B2
B3
B4
C1
C2
C3
C4
Feature vectorsFeature vectors
a b Standard coding
Hierarchical coding
Hierarchical coding without
subtractions
a1 a2 0.004505 0.005349 0.003739
b1 b2 0.009905 0.008160 0.007742
c1 c2 0.017778 0.011848 0.009946
a3 a4 0.008807 0.006047 0.002422
b3 b4 0.020075 0.010667 0.006275
c3 c4 0.128322 0.101591 0.080176
a1 b1 0.488420 0.492238 0.491653
b1 c1 0.315644 0.304597 0.305442
Image segmentation taskImage segmentation task::Brodatz texturesBrodatz textures
Image segmentation taskImage segmentation task
Image segmentation taskImage segmentation task
Image segmentation taskImage segmentation task
Texture parameterization Texture parameterization using 2-D Hermite functionsusing 2-D Hermite functions
),(),,(),,(),,(),,(),,(),,( 4,35,27,00,31,22,13,0 yxyxyxyxyxyxyx
Quasiperiod’s waveforms
Signal filteringSpeaker indexingSpeaker recognition using databaseSource separation
Areas of Hermite transform application:Areas of Hermite transform application:
Music Speaker 1 Speaker 2 2 speakers
Audio sample
Quasiperiod waveform Hermite histogram
Speaker indexingSpeaker indexing
Speech/music separationSpeech/music separation
Analysis of signal dynamics (distribution of amplitudes in time)
Analysis of base tone distribution in time
Currently implemented criterions:
Result of speech/music separation:
Mix detectionMix detection
One speaker Mix
ReferencesReferences1.1. Krylov A.S., Liakishev A.N.,Krylov A.S., Liakishev A.N., Numerical Projection Method For Numerical Projection Method For
Inverse Fourier Transform and its Application,Inverse Fourier Transform and its Application, Numerical Numerical Functional Analysis and optimization, vol. 21 (2000) p. 205-216.Functional Analysis and optimization, vol. 21 (2000) p. 205-216.
2.2. Krylov A.S., Kortchagine D.N.,Krylov A.S., Kortchagine D.N., Projection filtering in image Projection filtering in image processingprocessing, , Graphicon’2000 Conf. proceedings, p. 42-45, 2000.Graphicon’2000 Conf. proceedings, p. 42-45, 2000.
3.3. Krylov A.S., Kortchagine D.N., Lukin A.S., Krylov A.S., Kortchagine D.N., Lukin A.S., Streaming Waveform Streaming Waveform Data Processing by Hermite Expansion for Text-Independent Data Processing by Hermite Expansion for Text-Independent Speaker Indexing from Continuous SpeechSpeaker Indexing from Continuous Speech, , Graphicon’2002 Conf. Graphicon’2002 Conf. proceedings, p. 91-98, 2002. proceedings, p. 91-98, 2002.
4.4. Krylov A.S., Kutovoi A.V., Wee Kheng Leow, Krylov A.S., Kutovoi A.V., Wee Kheng Leow, Texture Texture Parameterization With Hermite Functions,Parameterization With Hermite Functions, Graphicon’2002 Conf. Graphicon’2002 Conf. proceedings, p. 190-194, 2002.proceedings, p. 190-194, 2002.
5.5. Mohsen Najafi, Andrey Krylov, Danil Kortchagine Mohsen Najafi, Andrey Krylov, Danil Kortchagine Image Image Deblocking With 2-D Hermite Transform, Deblocking With 2-D Hermite Transform, Graphicon’2003 Conf. Graphicon’2003 Conf. proceedings. proceedings.