1-D KernelDensity Estimation

For ImageProcessing

Pi19404

September 21, 2013

Contents

Contents

1-D Kernel Density Estimation For Image Processing 3

0.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30.2 Non Parametric Methods . . . . . . . . . . . . . . . . . . . . . . . . . 30.3 Kernel Density Estimation . . . . . . . . . . . . . . . . . . . . . . . . 3

0.3.1 Kernel Density Estimation . . . . . . . . . . . . . . . . . . . 5Parzen window technique . . . . . . . . . . . . . . . . . . . 5

0.3.2 Rectangular windows . . . . . . . . . . . . . . . . . . . . . . . 60.3.3 Gaussian Windwos . . . . . . . . . . . . . . . . . . . . . . . . . 7

0.4 Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 | 9

1-D Kernel Density Estimation For Image Processing

1-D Kernel Density Estimation

For Image Processing

0.1 Introduction

In the article we will look at the basics methods for Kernel Density

Estimation.

0.2 Non Parametric Methods

The idea of the nonâASparametric approach is to avoid restrictive

assumptions about the form of f(x) and to estimate this directly

from the data rather than assuming some parametric form for the

distribution eg gaussian,expotential,mixture of gaussian etc.

0.3 Kernel Density Estimation

kernel density estimation (KDE) is a non-parametric way to estimate

the probability density function where the estimation about the pop-

ulation/PDF is performed using a finite data sample.

A general expression for non parametric density estimation is

p(x) =k

NV

� where k is number of examples inside V

� V is the volume surrounding x

� N is total number of examples

Histograms are most simplest form of non-parametric method to

estimate the PDF .

To construct a histogram, we divide the interval covered by the

data values into equal sub-intervals, known as bins. Every time, a

data value falls into a particular sub-interval/bin the count associated

3 | 9

1-D Kernel Density Estimation For Image Processing

with bin is incremented by 1.

For histogram V can be defined WxH where W is bin width and

H is unbounded

In the figure 1 the hue histogram of rectangular region of im-

(a) original image

(b) Hue histogram bin width 6 (c) bin width 1

Figure 1: Object model

age is shown.

Histograms are described by bin width and range of values. In the

above the range of Hue values is 0� 180 and the number of bins are

4 | 9

1-D Kernel Density Estimation For Image Processing

30

We can see that histograms are discontinuous ,which may not nec-

essarily be due to underlying discontinuity of underlying PDF but also

due to discretization due to bins and Inaccuracies may also exist in

the histogram due to binning . Histograms are not smooth and de-

pend on endpoints and width of the bins This can be seen in figure 1 b.

Typically estimate becomes better as we increase the number of

points and shrink the bin width and this is true in case of general

non parametric estimation as seen in figure 1 c.

In practice the number of samples are finite,thus we not observe

samples for all possible values,in such case if the bin width is small,we

may observe that bin does no enclose any samples and estimate will

exhibit large discontinuties. For histogram we group adajcent sample

values into a bin.

0.3.1 Kernel Density Estimation

Kernel density estimation provides another method to arrive at es-

timate of PDF under small sample size.The density of samples about

a given point is proportional to its probability. It approximate the

probability density by estimating the local density of points.

Parzen window technique

Parzen-window density estimation is essentially a data-interpolation

technique and provide a general framework for kernel density esti-

mation.

Given an instance of the random sample, x, Parzen-windowing esti-

mates the PDF P (X) from which the sample was derived It essentially

superposes kernel functions placed at each observation so that each

observation xi contributes to the PDF estimate.

Suppose that we want to estimate the value of the PDF P (X) at

point x. Then, we can place a window function at x and determine

how many observations xi fall within our window or, rather, what is

the contribution of each observation xi to this windowing

The PDF value P (x) is then the sum total of the contributions from

5 | 9

1-D Kernel Density Estimation For Image Processing

the observations to this window

Let (x1; x2; : : : ; xn) be an iid sample drawn from some distribution with

an unknown density f. We are interested in estimating the proba-

bility distribution f. Its parzen window estimate is defined as

fh(x) =1

n

nXi=1

Kh(x� xi) =1

nh

nXi=1

K�x� xi

h

�

Where K is called the kernel,h is called its bandwidth,kh is called a scaled

kernel

Kernel density estimates are related to histograms,but possess prop-

erties like smoothness or smoothness by using a suitable kernel.

Commonly used kernel functions are uniform,gaussian,Epanechnikov

etc

Superposition of kernels centered at each data point is equivalent

to convolving the data points with the kernel.we are smoothing the

histogram by performing convolution with a kernel. Different ker-

nels will produce different effects.

0.3.2 Rectangular windows

For univariate case the rectangular windows encloses k examples

about a region of width h centered about x on the histogram.

To find the number of examples that fall within this region ,the

kernel function is defined as

k(x) =

(1; if jxj < h

0; otherwise

hence total number of bins of histogram be 180,hence bin width

is 1.Let us apply a window function with bandwidth 6,12,18 etc and ob-

serve the effect on histogram

The kernel density estimate using parzen window of bandwidth 6,12

and 18 are shown in figure 3.

6 | 9

1-D Kernel Density Estimation For Image Processing

(a) bandwidth 6 (b) bin width 12 (c) bin width 18

Figure 2: rectangular window

0.3.3 Gaussian Windwos

The kernel function for the gaussian window is defined as

k(x) = C � exp��

x2

2 � �2

�Instead of a parze rectangular window let us apply a gaussian window

of width 6,12 and 18 and observe the effects on the histogram It

(a) bandwidth 6 (b) bin width 12 (c) bin width 18

Figure 3: Gaussian window

can be seen that estimate of PDF is smooth,however the bandwidth

plays an important role in the estimated PDF.A small bandwidth of 6

estimates a bimodal PDF width peaks well seperated. A bandwidth of

12,still is bimodal however the peaks are no longer seperated. A larger

bandwidth of 16 estimates a unimodal PDF.

The bandwidth of the kernel is a free parameter which exhibits a

strong influence on estimate of the PDF.Selecting bandwidth is a

tradeoff between accuracy and generality.

7 | 9

1-D Kernel Density Estimation For Image Processing

0.4 Code

The class Histogram contains methods to perform kernel density

estimation for 1D histogram using rectangular and gaussian win-

dows.The definition for Histogram class can be found in files His-

togram.cpp and Histogram.hpp.The code can be found at https://

github.com/pi19404/m19404/tree/master/OpenVision/ImgProc The file to

test the kernel density estimation is kde_demo:cpp and can be found

in https://github.com/pi19404/m19404/tree/master/OpenVision/demo To

compile the code for kde_demo run command

make -f MakeDemo kde_demo

8 | 9

Bibliography

Bibliography

[1] �An introduction to kernel density estimation�. In: (). url: http://www.mvstat.net/tduong/research/seminars/seminar-2001-05/.

9 | 9