Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Sampling and Interpolation on Uniform and Non-uniform Grids
Department of Signal Processing
Tampere University of TechnologyTampere University of Technology
Department of Signal Processing
Tampere University of TechnologyTampere University of Technology
Department of Signal Processing
More about motivation: C ti bj t d di t i lContinuous objects and discrete signals
Most of real-world phenomena are continuous• Continuous in scale• Continuous in time• Continuous in frequencyq y
Computers are digitalSampling and acquisition
• Digital photographySAR i i• SAR imaging
• Medical imaging• Digital holography
Reconstruction (interpolation)• High-quality display• Correction of geometrical distortions• Rescaling, zooming, rotation, etc.
Department of Signal Processing
Motivation: DemosMotivation: Demos
Department of Signal Processing
NotationsNotations
Hilbert spacesHilbert spaces• The space of measurable, square-integrable functions of continuous 1-D variable denoted by
:)( );(for :)( 222 RLgRLfRL ;)()(,
dxxgxfgf
dxxfff 2)(,f
)(2 RL
For
• The space of measurable, square-sumable functions of discrete 1-D variable denoted by
Inner product Norm
)(2 Zl
:)(b );(afor :)( 222 RlRlZl ;)()(,
k
kbkagf
k
kaaaa 2)(,For
Inner product Norm
• The space of measurable, square-integrable functions of continuous 2-D variable denoted by
:)( );(for :)( 222222 RLgRLfRL ;),(),(,
dxdyyxgyxfgf
)( 22 RL
Inner product
Department of Signal Processing
dxdyyxfff 2),(,f Norm
Notations (cont.)Notations (cont.)
Convolutions• Continuos convolution
dtvutvu )()())((
kbkb )()())((• Discrete convolution
• Mixed convolution
k
knbkanba )()())((
k
ktukatua )()())((
• Convolution inverse
k
elsewhere00 if1
)())(( 1 nnnbb
Department of Signal Processing
Notations (cont.)Notations (cont.)
Fourier transforms• Fourier transform pair of an (Lebesgue) integrable function
dxexfF xj 2)()(
)()( 1 RLxf
deFxf xj2)()(
• Discrete Fourier transform pair of a discrete sequence
1
0
/2)()(N
n
NknenakA
1
0
/2)()(N
k
NknekAna
Space of band-limited functions (Paley-Wiener space)0n 0k
,supp :)(2 FRLfPV
Department of Signal Processing
Part 1: Sampling Theorems and Shift-Invariant Space Formalism
Department of Signal Processing
Sampling theoremsSampling theorems
Cauchy’s theorem (1841)
1 )(i)(N m
,)(Let 2/)1(
2/)1(
2
N
Nn
jnxnecxf
Classical Sampling theorem (Whittaker-Shannon-Kotelnikov-Someya)
1
0 )/(sin)(sin)1()/()(Then
N
m
m
NmxNxNNmfxf
),()()( , Then,
.120 and 0,Let
nTxsnTfTxfPVf
TT
n
].,[for 1)( and , where )2/(1
SPVs T
n
Department of Signal Processing
Frequency domain interpretation of samplingFrequency domain interpretation of sampling
Th i f tiThe sinc function
• For sake of simplicity assume T = 1
• The sampling function is known as sinc fuction x
xx )sin()(sinc
• Its Fourier transform is rect function
x
otherwise0
21,21for 1)(rect
-1/2 -1/2 x
Department of Signal Processing
Frequency domain interpretation of samplingFrequency domain interpretation of sampling
F d iFrequency domain
• Taking samples at integers is equivalent to multiplying by a series of Diracs (comb
function). It allows representing the function samples as impulse train in continuous
domain. In Fourier, it periodizes the original spectrum
• The convolution between the impulse train and sinc (expressed as product in Fourier
domain) ’washes-up’ the unwanted replicas
),()()()()()()( nxnfnxxfxcombxfxfnn
p
)(sinc)()()( xxcombxfxf 1
Department of Signal Processing
1/2 1 3/2 2
Shift-invariant space interpretationShift invariant space interpretation
The integer translates of the sampling function form an orthonormal
basis for the space 2/1PV
Shanon theorem for non band-limited functions
]; [ :)()(for )(sinc)()( 21
21
Sxsnxnfxfn
dxfxc )(sinc)()(
kkf )(i)()(~
k
kxkcxf )(sinc)()(
sinc(x-) sinc(x-k)f(x) c(x) c(k) )(~ xf
sampling
Department of Signal Processing
A mathematical formalizationA mathematical formalization
Shift i i t f ti d ti b iShift-invariant function space and generating basis• Shift-invariant function space V() being a closed subspace of L2
lcixicxgV 2:)()()()(
• Any function from V() can be represented as a convolution between a discrete set of coefficients and the generating function (continuous)
ilcixicxgV :)()()()(
• Requirements to the generating basis: Riesz basis
22
22
2
2 )()(l
Lil
cBixiccA
• Upon proper choise of the basis functions (t) the sampling problem becomes a problem of finding the model coefficients c(i)
Department of Signal Processing
Orthogonal projection on the given spaceOrthogonal projection on the given space
Approximating a function s(x)L2 by a function from V()
• L2 norm defined as
i
ixicxg )()()(~
• L2 norm defined as
• Minimizing the error
dsssssL
)()(,2
2
• Orthogonal projection
22
22min~
LVgLgsss
• Dual function of (t):
d)()()()()( igigic
;)()()()( 1
ii d)()( iip
Department of Signal Processing
;)()()()( 1
i
ixipx .d)()( iip
Sampling scenarios: ideal vs non-ideal samplingSampling scenarios: ideal vs non ideal sampling
liIdeal sampling
(t) c(x) c(k) )(~ xg
sampling )( kx
Non-ideal sampling
g(t) c(x) c(k) )(xg)(ˆ xt )( kx
p g
( ) (k)
sampling )( kx
g(t) c1(x) c1(k) )(~ xg)( xt )( nx q(n-k)
c2(n)
11 )()()(H)()()(h))(()( Qkkkk
Department of Signal Processing
11 )()()( Hense, .)(),()( where),)(()(
nnnQxkxkakcakq
Part 2: Interpolation, Interpolation Kernels, Error Kernels, B-splines
Department of Signal Processing
Image sampling and reconstructionImage sampling and reconstruction
S li d t ti
ga(x)Image
Sampling and reconstruction• The best way is to sample the
continuous finction with a sampling function (device) being dual to the reconstruction one This will
-4 -3 -2 -1 0 1 2 3 4
ga(x)comb(x)SampledImage
reconstruction one. This will minimize the error between the original and reconstructed function
• In most cases we start directly from discrete data and we have no ga( ) ( )Image
-4 -3 -2 -1 0 1 2 3 4
information about the sampling device
• Interpolation: keep the given samples, i.e. generate such a function which has the same ya(x) = [ ga(x) comb(x) ] * h(x)Reconstucted
Imagefunction, which has the same values at the given coordinates
• Smoothing: fit a smoothing function assuming a certain amount of noise
Department of Signal Processing
-4 -3 -2 -1 0 1 2 3 4amount of noise
Interpolation problemInterpolation problem
DefinitionGi di t d t [k] if id k 0 1 2 t k f t i ti f tiGiven discrete data g[k] on an uniform grid k=0,1,2,…., taken from certain continuous function ga(x).Fit an approximating continuous function ya(x) to the given discrete data and then resample it along a new (finer) grid determined by the smaller sampling interval .
ha(x) ha(x+1)ha(x-1)
ya(x)g(k)
y(l)
l (l+1)1 nl-1
(l-1)(l-2)nl+1nl( )
l l+1
( )( )
Department of Signal Processing
Interpolation problemInterpolation problem
Fitting a continuous model, i.e. choice of the reconstruction function• Classical interpolation
• the given samples g[k] are used as weights of the reconstruction (synthesis) function h(x) – linear model, known also as mixed (digital-continuous) convolution
;)(][)( kxhkgxy
• Interpolation constraint
;)(][)( kkxhkgxy
)()( ][)(][)( 000
kkhkgkkhkgxykkx
1.2
minimax Interpolator, Degree=3 Length=8, Contin
• Example of a classical interpolator
0 6
0.8
1
0.2
0.4
0.6
Department of Signal Processing
0 1 2 3 4 5 6 7 8-0.2
0
Interpolation problemInterpolation problem
Fitting a continuous modelFitting a continuous model• Generalized interpolation: building a linear model of mixed convolution type;
reconstruction function weighted by model coefficients rather than the given samples themselves
kkd )(][)( k
kxkdxy )(][)(
How to obtain the model coefficients• Obey the interpolation constraint
T l th i t l t t th di t d th t ti
][])[(h])[(][
)(][ where),)(()(][)(11
000
kkkkd
kkpkpdkkkdkyk
• To sample the interpolant at the same coordinates we need the reconstruction function, sampled at integers; its convolution-inverse is an IIR digital filter
)(/)()()()()( :domain-zin ][])[(where])[(][ 11
zPzYzDzPzDzYkkppkgpkd
p-1: convolution-inverse of p
Department of Signal Processing
Interpolation problemInterpolation problem
Generalized (some of them non-interpolating) interpolation kernel examples
0.8
1
0.8
1
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
−2 −1 0 1 20
−2 −1 0 1 20
0.6
0.8
0.6
0.7
0.2
0.4
0.6
0.1
0.2
0.3
0.4
0.5
Department of Signal Processing
−2 −1 0 1 20
−2 −1 0 1 20
Interpolation problemInterpolation problem
Fitting a continuous model• Generalized interpolation can be represented as classical interpolation:
kxkdxy )(][)( kkxkdxy )(][)(
)(][)( where)(][)(
)(][][)(])[()(1
11
km
k k n
kxkpxhmxhmgxy
kxnkgnpkxkgpxy
• To regard the interpolation in its explicit form, as sliding kernel being weighted by the given discrete samples, one should use the cardinal function It is infinitely supported due to the recursive part
)(/)()( :domainfrequencyin jePH
function. It is infinitely supported due to the recursive part
Department of Signal Processing
Interpolation problemInterpolation problem
Cardinal interpolation kernel examples
1
0.6
0.8
resp
onse
0.2
0.4
Impu
lse
res
−5 −4 −3 −2 −1 0 1 2 3 4 5
−0.2
0
Time in T
Department of Signal Processing
Time in Tsx
Interpolation problemInterpolation problem
Resampling• Role of the fractional interval (see Fig. 4)
knhkgxy )(][)(
;lll nx ll xn
k lll knhkgxy )(][)(
k lll knkdxy )(][)( or
;lll nx 10 l
k lll nhkngly )()()(
nhkngly )()()( k lll nhkngly )()()( or
Department of Signal Processing
Interpolator’s propertiesInterpolator s properties
Finite supportFinite support• best trade-off between quality and computation complexity
Separability• data to be processed line-by-line, column-by-column
n
Symmetry • to introduce no phase distortions
nn
n
ii Rxxxx
),...,,( )()( 211
xx
Partition of unity• reproduction of the constant
)()( xx
• reproduction of the constant
k
k
kx
kxh
)(1
)(1
Department of Signal Processing
k
Interpolation artifactsInterpolation artifacts
1 . 2m i n i m a x I n t e r p o l a t o r , D e g r e e = 3 L e n g t h = 8 , C o n t i n
Ringing• Caused by the oscillatory reconstruction
kernels, known also as Gibbs effect around sharp transitions
Aliasing (imaging)0
0 . 2
0 . 4
0 . 6
0 . 8
1
Aliasing (imaging)• Appearing of unwanted frequencies resulting
from the repetition of the original spectrum around 2k. Known as Moiré patterns
Blocking−10
0
0 1 2 3 4 5 6 7 8- 0 . 2
g• Caused by short kernels, like nearest neighbor
Blurring • Result of the interpolation function non-ideality
in the pass-band. Non-ideal interpolators −50
−40
−30
−20
Ma
gn
itud
e,
[dB
]
suppress some high frequencies. As a result, the interpolated image appeared with no sharp details
0 0.5 1 1.5 2 2.5 3−80
−70
−60
Normalized frequency
Department of Signal Processing
Normalized frequency
Image artifactsImage artifacts
Result of image processing systems and algorithms, such as
• Image formation channelg• Acquisition (sampling)• Compression• Resizing• DenoisingDenoising
Department of Signal Processing
Image artifactsImage artifacts
Blockiness • Caused by coarse quantization of
transform coefficients of block transforms, such as DCT
• Use of low-order reconstruction kernels, such as nearest neighbor
Department of Signal Processing
Image artifactsImage artifacts
Blurring• Loss of spatial details; reduction
of edge sharpness• Result of
• Low-end optics• Non-ideality of reconstruction
functions in the pass-band• Over-smoothing
Department of Signal Processing
Image artifactsImage artifacts
Ringing• Decaying waves around edges
and contours • Due to Gibb’s phenomenon: finite
terms approximation of continuous operators, mainly continuous Fourier transform
• Appear in convolution-type interpolation or in JPEG2000 likeinterpolation or in JPEG2000-like coding
Department of Signal Processing
Image artifactsImage artifacts
Ali iAliasing• Due to improper resampling
• Appearance of unwanted frequencies
• Manifests as Moiré patterns
Department of Signal Processing
Evaluation of interpolator’s qualityEvaluation of interpolator s quality
T iTwo main error sources• Non-flat magnitude response in the pass-band (blurring)• Non-sufficient suppresion of the periodical replicas in the stop-band (aliasing)
Sampling and reconstruction blur (SR blur) (Park and Schowengerdt, ’82)• Investigate the influence of the phase of sampling
• The error is a periodic function of the phase u
22
2);()()(
Laa uxyuxgu
p pAverage SR blur
• Regarded as the expectation of the random error, depending on the arbitrary sampling phase
k
aSR
kfHfHf
dfffGE
22
222
)()(Re21)(
;)()(
Department of Signal Processing
Evaluation of interpolator’s qualityEvaluation of interpolator s quality
G li d i t l tGeneralized interpolators
k
fj
kf
feP
ffH))(2(
)2()()2()2( 2
• Error kernelk
0
2
)(2
2
22
)())(2(
)()2(1)2(
kkfjfj ePkf
ePff
E i l t f
22
0
222
)(
))(2()2()(
fj
k
fj
eP
kffeP
• Equivalent form
20
22
02
))(2())(2()2(
kk
kfkffj
Department of Signal Processing
))(2(
k
kf
Evaluation of interpolator’s qualityEvaluation of interpolator s quality
G li d i t l t H t t th k lGeneralized interpolators
k
fj
kf
feP
ffH))(2(
)2()()2()2( 2
How to compute the error kernel• Assume symmetrical and finite-lengh (x)• Its sampled version is finite sequence with finite number of cosine
t i F i d i
kxxkp
)()(
• Error kernelk terms in Fourier domain
• Simplified error kernel form
2/
1)2cos()(2)0()( 2
N
kezkfkzP fj
0
2
)(2
2
22
)())(2(
)()2(1)2(
kkfjfj ePkf
ePff
E i l t f• The infinite sum is the Fourier transform of
22
0
222
)(
))(2()2()(
fj
k
fj
eP
kffeP
.)(
)(2()2().(2)()2( 22
2222
2
fjk
fjfj
eP
kffePePf
• Equivalent form
20
22
02
))(2())(2()2(
kk
kfkffj
the function (x) continuously convolved by itself and subsequently sampled
N
kkkfkkf
1
2 )2cos())((2)0)(())(2(
Department of Signal Processing
))(2(
k
kf kk 1
Evaluation of interpolator’s qualityEvaluation of interpolator s quality
Th l ith H t t th k lThe algorithm• Calculate the Fourier transform of the
kernel (x)• Calculate the Fourier transform of the
sampled kernel
How to compute the error kernel• Assume symmetrical and finite-lengh (x)• Its sampled version is finite sequence with finite number of cosine
t i F i d i
kxxkp
)()(
xkp )()( sampled kernel• Calculate the auto-correlation function at
integer coordinates
terms in Fourier domain
• Simplified error kernel form
2/
1)2cos()(2)0()( 2
N
kezkfkzP fj
kxxkp
)()(
))(()( xx kxxaka
)(][
• Find the Fourier transform of the sequence a(k)
• Calculate the error kernel• The infinite sum is the Fourier transform of
.)(
)(2()2().(2)()2( 22
2222
2
fjk
fjfj
eP
kffePePf
the function (x) continuously convolved by itself and subsequently sampled
N
kkkfkkf
1
2 )2cos())((2)0)(())(2(
Department of Signal Processing
kk 1
Approximation theoryApproximation theory
kkd )/()()(Quiatifying the approximation error
• Predictive approximation error
dfffGxyxg
kxkdxy
aLa
k
)()()()()(
);/()()(
2222
2
• Approximation error kernel 2
22
2
)(
)()()(
Zk
ZkZk
kf
kfkff
• Approximation order• Develop the error kernel in McLaurin series• First L-1 would be zero; then we say the
approximation order is L
0
2)2(2
2
)!2()0()2(
n
nn
fn
f
2)2(2
222 )0()2( nn
L ffCf approximation order is L• Neglect the higher order terms (valid for
small , i.e. oversampled signals)• Approximation constant
Predictive approximation error
1 )!2(
)2(Ln
fn
fCf
k
L kL
C2)( )2(
!1
Department of Signal Processing
• Predictive approximation error k
Approximation theoryApproximation theory
Quiatifying the approximation error
• Predictive approximation error
dfffGxyxg
kxkdxy
aLa
kh
)()()()()(
);/()()(
2222
2
• Approximation error kernel 2
22
2
)(
)()()(
Zk
ZkZk
kf
kfkff
• Approximation order• Develop the error kernel in McLaurin series• First L-1 would be zero; then we say the
approximation order is L
0
2)2(2
2
)!2()0()2(
n
nn
fn
f
2)2(2
222 )0()2( nn
L ffCf approximation order is L• Neglect the higher order terms (valid for
small , i.e. oversampled signals)• Approximation constant
Predictive approximation error
1 )!2(
)2(Ln
fn
fCf
k
L kL
C2)( )2(
!1
Department of Signal Processing
• Predictive approximation error k
2)(22222222
)2()(L
LLa
LL gCdffGfCe
Approximation theoryApproximation theory
Factorized approximation error• Predictive approximation error
Strang-Fix conditions• Lth order zeros
1,00)2( ;1)0(
)(
LnZkkn
kxkdxyk
h );/()()(
• Approximation error kernel
• Reproduction of monomials 1,0 0)2( LnZkk
n
nk
nk
xkxc
cLn
)(
1,0
dfffGxyxg aLa
k
)()()()()( 2222
2
22
• Approximation order
• Discrete momentsk
1,0 )()(
lnmkxkxZk
nn
2
2
2
)(
)()()(
Zk
ZkZk
kf
kfkff
2)(2222
)(L
LL gCe
Department of Signal Processing
B-splinesB splines
B spline functions of degree nB-spline functions of degree nPiecewise-polynomial functions of degree nwith n-1 cont. derivatives at the nodes
n
nnni
n ixuixn
x 11 )()(1)1()(
Successive convolutions of zero-degree B-spline
i
ixuixin
x0
22 ),()(!
)(
-for1 110 x
;elsewhere0
for 1)( 220
x
x
1
0 0 0 1 0 ...
n
nn
Frequency characteristics1
)() sin()(
nn
fff
x
Department of Signal Processing
)( f
B-splinesB splines
B-splines form a Riesz basis• Integer shifts of B-spline function of degree n form a Riesz basis for the spline
space of polynomial degree n, that is, all polynomial splines of degree n with knots at integers can be represented as a linear combination of B-spline basis f ti f th d
0.6
0.7
functions of the same degree
Zk
nn kxkdxs )(][)(
0.3
0.4
0.5
0.1
0.2
Department of Signal Processing
−8 −6 −4 −2 0 2 4 6 80
B-splinesB splines
B-spline functions of degree n
- dilated continuous B-spline
.1)(
mx
mx nn
m
m
mm
n
mmmnm uuuuu ]1,....1,1[ ,... 10
1
000
where
- discrete B-spline at integer scale m
- sampled continuous B-spline and corresponding frequency response
; ,1)(
Zk
mk
mkb nn
m
k
nfjnn kfekbf )()()( 2Β k
- m-scale relation
)()(
);()(
1
1
Zkkbukb
xuxnnn
nnm
nm
Department of Signal Processing
. ),()( 1 Zkkbukb mm
B-spline interpolatorsB spline interpolators
Direct B-spline transform (Unser, Aldroubi, Eden ’93) • Given discrete data g[k]. Find a B-spline model gn(x) which ensures gn(x) = g[k] for x = k
;)(][))(()( ixidxdxg nnn
)()()()(][
;)(][)(][][)(
)(][))(()(
1
1
zGDkbkd
ikbidikidkgxg
g
n
i
n
i
nkx
n
i
)()()()()(][
1
11 zB
zDkgbkd nn
n(x)1/Bn(z)
g(k) d(k) gn(x)
Department of Signal Processing
B-spline InterpolatorsB spline Interpolators
210 x
Nearest neighboor
Linear
101
)()( 1 xxxxh
21
21
210
01)()(
xxxxh
;1)(1 zBLinear
Cubic
10
)()(x
xxh
;61
64)( 13 zzzB
2110
)2()2(
)()( 361
221
32
3 xx
xxx
xx
;1)(zB
66
20
)()()( 6
x
0.6
0.8
1
0.6
0.8
1
n(x)1/Bn(z)
g(k) d(k) gn(x)−2 −1 0 1 20
0.2
0.4
−2 −1 0 1 20
0.2
0.4
0.4
0.6
0.8
0.4
0.5
0.6
0.7
Department of Signal Processing
−2 −1 0 1 20
0.2
0.4
−2 −1 0 1 20
0.1
0.2
0.3
B-spline interpolatorsB spline interpolators
Digital pre-filtering• Since the kernel is symmetrical there are pairs of real poles being reciprocal, hence
the IIR filter is not stable. There is a trick to make the filtering stable by applying forward and backward filtering.
E lExamples
)(1)( zBzS nn
32 ),)((61
64)( 113 zzzzzB
f d
)1(1
)1(6
)1)(1(6)(
66
11
zzzzzS
forward
n(x)1/Bn(z)g(k) d(k) gn(x)backward
Department of Signal Processing
Efficient RealizationEfficient Realization
Farrow structure• Piecewise-polynomial form of
the basis function
c(k)
G3(z) G0(z)G1(z)G2(z)
• Example: 3rd degree B-spline
x-mμkmkcgZk
,)()()( where
s(x)
3
0
1
2
2361
030303631331
1)(ccc
g
10141 c
Department of Signal Processing
Geometric transformation of ImagesGeometric transformation of Images
2-D splines by tensor-product basis functions
2
11
21
1)( )(
;)()(],[),(1
1
1
1
n
nnk
kk
nl
ll
nnn
ylxk
lxkxlkdyxg
Mapping from source image coordinates (x,y) to target image coordinates (u,v): (x,y)=T(u,v)
Algorithm:• Pre-compute the spline coefficients d[k,l] (1-D recursive filtering along rows and
columns)• At each target location (u,v) find the corresponding source (x,y) and interpolate g ( , ) p g ( ,y) p
using the neighbor pixels
Department of Signal Processing
Image RotationImage Rotation
• Rotation matrix determines a rotation of the coordinates (x,y) by an angle
cossinsincos
)(R
• Matrix factorization:
cossinsincos
)(
R
• The factorized rotation matrix determines three 1-D translations
102/tan1
1sin01
102/tan1
• Tx=y(-tan(/2)• Ty=xsin• Tx=y(-tan(/2)
Department of Signal Processing
Image RotationImage Rotation
Three 1-D translations (interpolations)
T ( (/2)Tx=y(-tan(/2)Ty=xsinTx=y(-tan(/2)
Department of Signal Processing
Intermediate summaryIntermediate summary
So far in the lecture: How to reconstruct a continuous function from given uniform samples
• Sampling theorems for the class of band-limited functions (Shannon)• An approximation procedure interpreted as projection on the shift-invariantAn approximation procedure interpreted as projection on the shift invariant
subspace of bandlimited functions; only this projection can be reconstructedGeneralization for other shift-invariant spaces
• Uniform splines a very attractive tool InterpolationInterpolation
• When the sampling procedure is unknown: bandlimitedness assumed• Practical efficient interpolation based on generalized sampling kernels (preferably
B-splines)E l ti f i t l ti k l b d th i f d i h t i ti• Evaluation of interpolation kernels based on their frequency domain characteristics
• Strang-Fix conditions to characterize kernels for their capabilities to reproduce polynomials and to provide good approximation error decay
Next question: can we perfectly reconstruct non-bandlimited functions ith fi it b f t (d f f d )?
Department of Signal Processing
with finite number of parameters (degrees of freedom)?
Part 5: Design of Efficient Interpolation KernelsPart 5: Design of Efficient Interpolation Kernels
Department of Signal Processing
Optimized kernels based on B-splinesOptimized kernels based on B splines
Modified B-splinesOptimal Lth order functions Modified B splines• А parametric class of splines
(Egiazarian, Saramäki, Chugurian, and Astola, ’96) obtained by a linear combination of B-splines of different degrees
p• Splines of minimal support (Ron’90)
• MOMS (Blu and Unser’99)
n
nk
xdx )()( degrees.
N
n Zm
nnm mxx
0
mod )()(
k
kk xdx
x0
)()(
Example: 3rd + 1st degree splines
)1()1()()()( 111
111
110
3mod xxxxx
3 – blue;
mod red;
Example: 3 + 1 degree splines
od – red;
1 – green.
Department of Signal Processing
Piecewise-polynomial kernels of minimal supportPiecewise polynomial kernels of minimal support
Piecewise-polynomial (pp) interpolators
pp basis functions of minimal support: A sub class of pp functions
;)()(1
0
N
kk xPxh 1 ][)(
0
kxkxlaxPM
l
lkk
pp basis functions of minimal support: A sub-class of pp functions with the following properties
• Non-interpolating• Symmetric
Mi i l i h l h h i h l i l d• Minimal support, i.e. the lenght N=M+1, where M is the polynomial degree Splines of minimal support (O-MOMS)
• Preserve the order of approximation L=M+1• Minimize the approximation constant 1L
• In frequency domain• After an analytical optimization one gets
LL ffjf )(sinc)2()2( where
1
0
)(L
k
kk zz
)()14(4
)()( 12
2
1 zLzzz LLL
1(z)= 2(z)=112)!2(
!
LL
LCL
Department of Signal Processing
)14(4 L 12)!2( LL
Modified B-splinesModified B splines
Modified B-splines
• Pre-filtering step
N
n Zm
nnm mxx
0
mod )()(
• Example: 3rd + 1st degree splines 3 – blue;
mod – red;
)(1)( modmod zBzS
)1()1()()()( 1113mod xxxxx ;
1 – green.)1()1()()()( 111110 xxxxx
Department of Signal Processing
Modified B-splinesModified B splines
Computational structure
231
mod
1)(
,)()()( x-mμkmkcgZk
where
0
1
2
111110111011
2361
6663666303631331
1)(
ccc
g
c(k)
1
0
111011
111110111011
0616461 c
c(k)
G3(z) G0(z)G1(z)G2(z)
Department of Signal Processing
s(x)
Modified B-splinesModified B splines
Example• combination of fifth-, third-, and first-degree B-splines
))1()1(()(
))1()1(()()()(111
3331
330
5mod
xxx
xxxxx
0 6
0.7
B5B3(x)
• Discrete version in z-domain
. ))2()2((
))1()1(()( 11
12
1110
xx
xxx
0 3
0.4
0.5
0.6 ( )B3(x-1)B3(x+1)B1(x)B1(x-1)B1(x+1)B1(x-2)B1(x+2)
).()(
))(()()()(221
1211
111
10
1331
330
5mod
zzzBzzzBzB
zzzBzBzBzB
0.1
0.2
0.3
Department of Signal Processing
-3 -2 -1 0 1 2 30
Modified 5-3-1 caseModified 5 3 1 case
Computational structure for fifth degree cases: regular and modified
2
3
05203020515101051
cc
2
1
0
1012345120
1
01266626105500505010206020100102002010
)(
cccc
g
05203020515101051
2012666261 c
012020112080202612040806620120802612020112012060120512060120501201206012012050120605
06010601202012012060601202060102020601060802060806020202010
12311131301031303011311231
1211311211301011103011121231
3130313031303131
31303130313031313031
Department of Signal Processing
Modified B-splinesModified B splines
What we get?• Same support• Probably the same approximation order• Same computational complexity for the direct B-spline transform
What we gain?What we gain?• Better representation of different signal details• The adjusting parameters offer more degree of freedom and possibilities
to optimizeWhat we loose?
• Regularity• Slightly more calculations
Department of Signal Processing
Optimization: a filter design approach
Optimization in minimax sense
Optimization: a filter design approach
Optimization in minimax sense• Assumption: most of the signal energy concentrated at a fraction of the passband
• Preserve the signal in this important passband region
A h d i d i i h b d i
,0pF• Attenuate the undesired images in the stopband regions
• Given , and w, specify D( f )’=’1 and W’( f ) ’=’w for f ’Fp and D( f ) ’=’0 and W ( f ) ’=’1for f Fs and find the unknown coefficients mn to minimize
1
,r
s rrF
Ga(2f);G(ej2f);H(2 f)
)()2( )( max fDfjHfWFf
w
Department of Signal Processing
f1/2 1 3/2 2 1- 2 Fs = 1
Optimization problem: ExampleOptimization problem: Example
Fifth-degree piecewise-polynomialg p p y• Cardinal interpolator’s frequency response
)()( 531531 fBffH
46531 i22i)( ffffff 2121110
3130531
sin4cos22cos2
sin2cos2sin)(
ffff
ffffff
fffffB 6/2cos242cos2120/4cos22cos5266)( 3130531
• Unknowns:30, 31, 10, 11, and 12.• DC constant preserving: 30 231 10 211 212 1.
ff 4cos22cos2 121110
• Unconstrained nonlinear optimization: 30 1 231 10 211 212.
Department of Signal Processing
Magnitude responses of optimized kernelsMagnitude responses of optimized kernels
3rd degree kernels 5th degree kernels
Department of Signal Processing
Error approximation kernelError approximation kernel
dfhfefGh
hxyxg
a
Lha
)()()(
)()()(
222
22
2
2
22
2
)(
)()()(
Zk
ZkZk
kf
kfkffe
)( Zkf
Department of Signal Processing
Spline-based interpolators: ConclusionsSpline based interpolators: Conclusions
B iB-spines• Provide separable, symmetric, and compactly supported recosntruction functions. A pre-
filtering step is needed• Provide maximal approximation for a given integer support
M t l f ti ith i d f i ti• Most regular functions with given order of approximationMOMS and modified B-splines
• Preserve all good features of the B-splines except the highest regularity• Allow for additional optimization with well elaborated optimization procedures in Fourier
domain• O-moms are asymptotically optimal approximators• Optimization strategies, based on suffcient suppression of the imaging frequences can
lead to much better interpolation results• The computational complexity remains practically the same, as in the case of regular B-
splines (both classes are in fact piecewise-polynomial functions)
Department of Signal Processing
Part 6: Resampling TechniquesPart 6: Resampling Techniques
Department of Signal Processing
Decimation problemDecimation problem
• Problem: rescale (resample) the given image to a coarser grid, with possible subsequent upscaling (interpolation). Keep as much information about the original image as possible
s(t)Image
-4 -3 -2 -1 0 1 2 3 4
Sampledpossible • By assuming separable processing, the image
(2-D) problem is downgraded to 1-D decimation problem
• Initial grid
pImage
-4 -3 -2 -1 0 1 2 3 4
h =a =b
x(k)
• Initial grid
ba
h
inL
inkkm
10
110
, , , ,...,
τ Re-sampled
Image
h0=a 1 2 3 Lin-1 b
• Target grid plased at the integers: out?=?1; ?=?in is the decimation ratio
-4 -3 -2 -1 0 1 2 3 4y(t) = (c * (t)Reconstucted
Image
Department of Signal Processing
-4 -3 -2 -1 0 1 2 3 4
An intuitive interpolative solutionAn intuitive interpolative solution
Would simple interpolation work?• Fit a continuous model
;)/(][)( kkxkdxy )(][ where][)]([][ 1 kkpikpigkd
• Equivalently
• Resample at integers – aliasing effects are to be encountered since the
;)/(][)( int iixigxy
kkxkpx )(][)()( 1
int
Resample at integers aliasing effects are to be encountered since the reconstruction function in Fourier domain has zeros clustered around multiples of 1/ and 1/> 1
Reconstruction function with nodes at integers• Its frequency response has good antialiasing propertiesIts frequency response has good antialiasing properties• For the generalized model, the filtering is as follows
i
ikigky )(][][ i
ikligkply )(][][)(][ 1
Department of Signal Processing
Resampling via Least Squares at a glanceResampling via Least Squares at a glance
2Shift-invariant space formulation
Continuous least squares (L2 error norm minimization)• Fit an initial continuous model (through interpolation)
ilcitictgV 2:)()()()(
• Fit an initial continuous model (through interpolation)• Involve the reconstruction function and its dual (infinitely supported)• Convolution integrals between functions with different scales to be solved• Solutions involving B-splines (You-Pin Wang’98, Munos et al.’99)
Di t l t (l2 i i i ti )Discrete least squares (l2 error norm minimization)• Solution involving splines (de Boor’76)• Recuire a matrix (Grammian) inversion
A near least squares techniquegΦΦ)(Φc TT 1
• Efficient structure: transposed Farrow structure• Avoid matrix inversion through digital recursive filtering and proper scaling• Comparison with the l2 solution
gΦHPc T1)(
TT ΦHPΦ)(ΦΦE 11 )(
Department of Signal Processing
ΦHPΦ)(ΦΦE )(
Continuous least squaresContinuous least squares
Minimizing L2 norm• a continuous function needed
kK kxKkgxg )(][)(
• minimizing the error norm
• Solving the convolution integral
22
22min~
LKVgLK gggg
Solving the convolution integral
)()(][)()(][
)()()()(][
d)()(][
1
1
dikKkid
dnignaid
igid
Kn
K
.)()(][)()(][
)()(][)()(][
1
1
dnikKkgnaid
dnikKkgnaid
kn
kn
Department of Signal Processing
Continuous least squaresContinuous least squares
• An integral involving functions with different scales
• A linear model
dxKx )()()(
• General scheme
k
kxkgxg )(][)(
( l)
(x/) (x) (a)-1 (x)g[k] d[l] y(x)
X
l (x-l)
gK(x) g(x)
continuous modeling orthogonal projection
Department of Signal Processing
rescaling
Continuous least squaresContinuous least squares
Solution involving splines (Munoz et al., 1999) • Consider an interpolation spline model
i
NN ixibxK )()()()( 1
• An integral involving two splines with different scales • Low-complexity operations: finite differences and running sums
l(t-l)
(bN)-1 (N+1) N+1(x) g[k] d1[k]
X
(b2N+1)-1 bN
d[l] y(l)(N+1)
Department of Signal Processing
Discrete least squaresDiscrete least squares
Minimizing l2 norm• Discrete inner product
1
0)()(,
inL
iii vuvu
• Induced semi-norm and corresponding error norm
• Normal equations
1
0
2 )()(2
inL
iiaial
ggg 22
22min~
laVslaa sggg
Normal equations
1,...,0for
][)(][)()(1
0
1
0
1
0
out
L
kk
L
j
L
kkk
Li
kgijdjiinout in
• Matrix form
gΦΦ)d(Φ TT
gΦΦ)(Φd TT 1
Department of Signal Processing
gΦΦ)(Φd
A near least squares solutionA near least squares solution
A ’hybrid’ model • Modeling the convoluton kernel by a Gaussian
2)( )4/(2xetK )(2lim )4/(
0
2
xe x
• A simpler form of the integral and an hybrid L2:l2 solution
An implemenation based on pp functions
kkxkgxg )(][)(
)(][)()(][ 1 knikgnaidkn
An implemenation based on pp functions m
N
i
N
m
Nm ixicx )]([)(
0 02
1
k
N NmN
mlkilickgxglg 2
1 )]([][)(][
ki m
mlxggg
0 02 )]([][)(][
N
m
N
i
mNkmlx
kilkgicxglg0 0
21 )(][][)(][
Department of Signal Processing
A near least squares solutionA near least squares solution
Transposed Farrow structure
• Fractional interval
N
m
N
i
mNkmlx
kilkgicxglg0 0
21 )(][][)(][
• Preservation of the constant
1for lll kkk x
+ +
x
+
k
g[k]
+
z-1
+
z-1
+
z-1
kl klkglg )(][][
kl kl )(1
• Recursive filteringC0(z) CN(z)C1(z)
g[l]
Department of Signal Processing
+ +g[l]l a-1
Near LS versus discrete LSNear LS versus discrete LS
0.7
Near LS in matrix form
gΦ(DA)d T10.4
0.5
0.6
l2 a
nd L
2:l2
sol
utio
n
Norm of the error matrix
TT Φ(DA)Φ)(ΦΦE 11 0.1
0.2
0.3
LS e
rror b
etwe
en l2
( ))(
0 0.2 0.4 0.6 0.8 10
Decimation ratio
Department of Signal Processing
Frequency-domain analysisFrequency domain analysis
Comparing different schemes• Interpolative scheme
)()2()2( 2int
fjePff
• Scheme minimizing L2 error norm
)()2(
)()2()2(~
222
fjfjL eP
feA
ff
• Near LS scheme
)()2()2(~
2: 22 fjlL eAff
Department of Signal Processing
Frequency-domain analysisFrequency domain analysis
C i diff t k lComparing different kernels• Cubic B-spline
)()( 3 xx
• Modified B-spline
1
11
33
mod )()()(i i ixxx
• MOMS
)()()( 323 xDxxmoms
Department of Signal Processing
Quality assessment of least squares techniquesQuality assessment of least squares techniques
Department of Signal Processing
Near least squares for different kernelsNear least squares for different kernels
Kernels• Regular cubic B-spline• Optimized MOMS (lowest
approximation constant)• Modified B-spline (minimax
optimized)
Department of Signal Processing
Complexity assessmentComplexity assessment
Comparison with the B-spline based L2 methodp p 2
Department of Signal Processing
Part 6: Non-Uniform Sampling and ReconstructionPart 6: Non Uniform Sampling and Reconstruction
Department of Signal Processing
Non-uniform sampling and reconstructionNon uniform sampling and reconstruction
Scenarios• Combination of multiple displaced uniform grids – appear in the framework of
super-resolution• Sampling in biomedical or astronomic imagery (e.g. spiral sampling)• Sampling within the framework of computer tomography
Theoretical fundamentals – Frame theory• Frames are generalization of bases and provide tools for signal expansions• In general frames provide over-complete signal expansionsIn general, frames provide over complete signal expansions• The theory is interested in the question if a set of functions forms a frame for a
certain space• For the case of non-uniform sampling frame theory provide the conditions
ensuring function reconstruction from given samples, in the flavor of the g g p ,Shannon theorem; the corresponding theorems are those of Paley-Wiener, Kadec and Duffin-Schaeffer
Department of Signal Processing
Non-uniform sampling and reconstructionNon uniform sampling and reconstruction
Techniques• The problem is cast as of reconstructing a continuous function belonging to a
function space from its non-uniform samples. Having a function space assumed, the problem is to find the coordinates of the continuous function with
t t th b i ti th trespect to the basis generating that space• Most widely used function spaces are the space of trigonometric polynomials
and splines, as well as those generated by radial basis functions • Iterative methods for solving large systems of equations are employed
Department of Signal Processing
Example: Super-resolution reconstructionExample: Super resolution reconstruction
Real sceneOptically blurred image
by camera lenses Image Acquired byCamera Sensor
Final Image(yk)
Sensor Position
PkMCD 1
Blurring(Ck) Downsampling(Dk) Noise(ηk)Sensor Position
According to Scene (Mk)
PknxMCDy kkkkk ,...,1,
Registered Images Non-uniformly SampledImage Data
Restored Scene
Transformationto desired grid
Non-Uniform to Uniform
Data Resampling
Registration
Department of Signal Processing 83
Reconstruction from non-uniform samples in lispline spaces
Iterative method by Aldroubi and Feichtinger• Provides exact reconstruction of spline function from sufficiently dense samples
by successive oblique projections / interpolations • Available samples g(xi,yi) of function g(x,y) V(φ) are interpolated to the desired
grid using a simple interpolator (e.g. nearest neighbour)
• Then the interpolated function is projected to V(φ)
i iiiii yxhgg ),( ),()())(Q( xxxxx
gg PQ1 • The projected function is sampled at the given non-uniform grid and the error
between the resulting and input samples is calculated. Using this error, the same procedure is repeated until convergence
nnn gggggg
)(PQPQ
1
1
I t l ti ProjectorSampling gn gn+1
g(xi)
-
nnn gggg )(1
Department of Signal Processing
Interpolatior Projectorp gOperator
g(xi)n
Reconstruction from non-uniform samples in lispline spaces
Local least squares method by Gröchenig and Schwab• Fast local least squares reconstruction () method in spline shift-invariant space• Using compact bases, a function can be reconstructed exactly on an arbitrary
interval solely using samples from that interval. Thus, the problem is to solve small size band systems over image segments
• Extendable to 2d case
y2 y
y
φ1 φ2 φ3 φ4 φn-1vφn
x1 x2 x3 x
y1
y2
k1kn
k1
k2
Arbitraty chosen intervals1
yUb *)( kxU jjk
Define and Compute for chosen interval:
)( )(1
* lxkxT j
J
jjjk
Compute
Solve bTd 1-
Department of Signal Processing
2D Non-uniform to uniform resampling: Formal bl t t tproblem statement
Consider P data samples of a certain 2D function, non-uniformly sampled as follows: {g(xi,yi):i=0,1,…,P-1}.
Reconstruct a continuous signal g(x,y) and resample it to uniform 2D grid
lklkd )()(
Assume the function g(.) belongs to shift-invariant space V(2)(φ), spanned by suitable basis φ.
W f h f bl 2D b f d b 1D i i l i l
k k
lykxlkdyxg ),(],[),(
We favor the use of separable 2D bases formed by 1D piecewise-polynomials
1
1 10 0
1( ) ( ) .2
I nm
mi m
nx c i x i
The aim is to obtain unknown discrete sequence d[k,l] which will describe uniquely and entirely g(x,y).There are several available methods in literature that estimate the sequence d[k,l] by different approaches.
Department of Signal Processing 86
Algorithm derivationAlgorithm derivation
The coefficients d[k,l] in desired g(x,y) are given by
Th b i i bi th l(d l) b i f
ddjigjid ш ),(~),(],[ k
kkkk yyxxyxgyxg ),(),(),(ш
)()(~ )2(V )(
, where
The basis is biorthogonal(dual) basis of
The dual basis can be obtained as follows:
)(),(~ )2( Vyx ),( yx
))()(()(~ 1 xax
For a separable basis :
))()(()( xax
diia )()(][
),(~ yx
Substitution of gш (x,y) gives:
qp
ddqjapipagjid )()()()()(),(],[ 11ш
Department of Signal Processing 87
ddqjpiyxayxgqapajid kkp q k
kk )()(),(),()()()()(],[ 11
Algorithm derivationAlgorithm derivation
Using the separable property, we get:
Rearranging the above equation we get:
p q k
kkkk ddpiqjyxayxgqapajid )()()()(),()()()()(],[ 11
Rearranging the above equation we get:
p q k
kkkk dyqjdxpiyxgqapajid )()()()(),()()()()(],[ 11
The latter can be simplified using the replication property of the delta function:
p qqjpigqapajid ),()()()()(],[ 11
where
Sampling uniformly at integers x=s, y = t yields:
k
kkkkk yyxxyxgyxg )()(),(),(
Department of Signal Processing 88
k
kkkkktysx ytxsyxgyxgtsg )]()(][,[|),(],[ ,
Algorithm derivationAlgorithm derivation
4
5
We use compactly supported basis, which is piecewise polynomial of degree n, defined on each interval :
2
3
4
.
I
i
mn
miim inxicx
0 012
1][)(
0 1 2 3 4
0
1
k
n
m
n
i
m
kmk
n
m
n
i
m
kmkkk yinticxinsicyxgtsg0 0
220 0
112 2
2
2
1 1
1
1 21][
21][],[],[
k
Denoting , we get the following form:
12 )()]([][][][ mn n n
mn
yxgicictsg
kk yintv
221
kk xins
121
Department of Signal Processing 89
1 2 2
2
1
1)()](,[][][],[
0 0 02
01 k
m m ikkk
km
imk yxgicictsg
2D version of the transposed Farrow structure
The equation
2D version of the transposed Farrow structure
12 )()]([][][][ mn n n
mn
iit The equation
can be represented by a two dimensional structure based on the Transposed Farrow Structure (TF). We refer to this structure as the 2D Farrow Structure
1
1 2 2
2
2
1
1)()](,[][][],[
0 0 02
01
mk
m m i
mkkk
km
imk yxgicictsg
k
0 ,k k kg x y 1 ,k k kg x y 2 ,k k lg x y 3 ,k k kg x y
X X1[ , ]( )m
k k kg x y X
k
)(0 zC 1( )C z 2 ( )C z 3( )C z
TF TF TF TF +
1z
+
1z
+
1z
+
1z
kblockid
TF =
+++
k +
)(0 zC )(1 zC
+
3( )C z2( )C z
+
Department of Signal Processing 90
,g s t
The 2D Transposed Farrow StructureThe 2D Transposed Farrow Structure
0
k k k k
1 3[ ]( )
1
1 2 2
2
2
1
1)()](,[][][],[
0 0 02
01
mk
n
m
n
m
n
i
mkkk
km
n
imk yxgicictsg
+X
)(0 zC
1z
X
+
)(1 zC
1z
+
3( )C z
1z
0[ , ]( )k k kg x y
+
2 ( )C z
1z
+X
)(0 zC
1z
X
+
)(1 zC
1z
+
3( )C z
1z
+
2 ( )C z
1z
+X
)(0 zC
1z
X
+
)(1 zC
1z
+
3( )C z
1z
+
2 ( )C z
1z
+X
)(0 zC
1z
X
+
)(1 zC
1z
+
3( )C z
1z
+
2 ( )C z
1z
1[ , ]( )k k kg x y 2[ , ]( )k k kg x y 3[ , ]( )k k kg x y
X X X X
2i 2i 2i 2i 2i 2i 2i i 2i 2i 2i i 2i 2i
2 0m 2 1m 2 2m 2 3m 2 0m 2 1m 2 2m 2 3m 2 0m 2 1m 2 2m 2 3m 2 0m 2 1m 2 2m 2 3m
k
2i 2i
kblockid kblockid kblockid kblockid
+
)(0 zC )(1 zC
+
3( )C z
)(0 zC
2 ( )C z
+ +
)(0 zC )(1 zC
+
3( )C z
1( )C z
2 ( )C z
+ +
)(0 zC )(1 zC
+
3( )C z
2 ( )C z
2 ( )C z
+ +
1
+
3( )C z
+
1 0m 1 1m 1 2m 1 3m
2i 2 2 2 2i 2i 2i 2i 2 2i 2i 2i 2 2
1i 1i1i 1i
2 2
+++
k
,g s t
Department of Signal Processing 91
ExperimentsExperiments
We conduct experiments for the applicability of the 2D resampling method to the super-resolution reconstruction from registered images for arbitrary sampling.
The reconstruction results are compared with results from Aldroubi-Feichtinger’s iterativereconstruction method and Gröchenig Schwab’s fast local reconstruction for translational andreconstruction method and Gröchenig-Schwab s fast local reconstruction for translational androtational motions as follows:
• Non-aliased LR images with pure translational motion, and noise added to the coordinates.• Non-aliased LR images with both translational and rotational motion, and noise is added to the
coordinatescoordinatesThese test cases are quite practical as they simulates the real case when we attempt to reconstruct
a SR image from blurred LR images after estimating the motion parameters by registration.Registration is never 100% accurate and by adding small amount of noise to the coordinates we
take into account such registration error. The noise variance has been chosen in accordance with reported results on state-of-the-art image registration methods
Department of Signal Processing 92
Experimental resultsExperimental results
Image MethodAliased Non Aliased
No Noise[dB] Noise[dB] No Noise[dB] Noise[dB]
Lena
Aldroubi >50 (6) 37.07(1) >50 (6) 39.23(1)
Schwab >50 40.51 >50 42.82
Farrow 41.20 41.56 42.61 43.09
Aldroubi >50 (6) 31 16 (1) >50 (6) 35 69 (1)
Barbara
Aldroubi >50 (6) 31.16 (1) >50 (6) 35.69 (1)
Schwab >50 34.45 >50 38.96
Farrow 41.20 37.39 40.47 40.65
Aldroubi >50 (6) 30.24(1) >50 (6) >50 (1)
Camera Schwab >50 35.47 >50 38.82
Farrow 38.71 38.83 40.17 40.39
Department of Signal Processing 93
Average PSNR results from 10 trials for Monte-Carlo simulation of random sampling experiment.
Experimental resultsExperimental results
Aldroubi-Feichtinger Gröchenig- 2D resampling g/(iteration)[dB]
gSchwab[dB]
p gmethod[dB]
Barbara 45.36 (6) 50.97 44.27
Lena 45.35 (7) 50.87 46.79
( )Cameraman 42.53 (6) 47.86 42.96
PSNR results for reconstructed SR images using non-aliased LR images with pure translational motion and noise added to the coordinates of the non-uniform samples
Aldroubi-Feichtinger /(iteration)[dB]
Gröchenig-Schwab[dB]
2D resampling method[dB]
Barbara 45.36 (6) 50.97 44.27
Lena 45 35 (7) 50 87 46 79Lena 45.35 (7) 50.87 46.79
Cameraman 42.53 (6) 47.86 42.96
PSNR results for reconstructed SR images using non-aliased LR images with translational and rotational motionand noise added to the coordinates of the non-uniform samples.
Department of Signal Processing 94
p
Experimental resultsExperimental results
Cameraman reconstruction results for the aliased random sampling experiment with noise added to the coordinatesnoise added to the coordinates
2D Transposed Farrow Structure Method Aldroubi & Feichtinger Method
(a) (b)
2D Transposed Farrow Structure Method(a)
Aldroubi & Feichtinger Method(b)
(c) (d)( ) ( )
Department of Signal Processing 95
Original ImageSchwab & Gröchenig Method(c)
2D Transposed Farrow Structure MethodWith IIR Filtering
(d)
2D non-uniform resampling: Conclusions2D non uniform resampling: Conclusions
The proposed 2D near least squares resampling method performs well for the non-p p q p g puniform to uniform resampling problem
A very important advantage of the proposed method is that all the processing can bedone by digital filtering and does not require any matrix inversions and hence it iscomputationally very efficient
From the PSNR values this method also seems to be quite immune to typicalregistration errors
The drawback is that large gaps in the non-uniform sampling cannot be handledThis method is very fast and memory requirements are moderatey y qIt gives good, smooth reconstruction results after the 2D resampling structure. The
final reconstruction result after IIR filtering and sampling is sharper, but in somecases it decreases the PSNR, as it is somehow mimicking the true Grammian with abanded matrix
Based on the two sets of experiments, we can conclude, that in cases of perfectalignment, the proposed near least squares 2D resampling technique gives somehowmoderate but still acceptable results compared with state-of-the-art methods
In the case of coordinate noise it is highly competitive, if not better than the other twoh d
Department of Signal Processing 96
methods