IMAGE PROCESSING>AUTOMATIC THRESHOLDING

UTRECHT UNIVERSITYRONALD POPPE

OUTLINEHistogram-based thresholding

Local adaptive thresholding

HISTOGRAM-BASED THRESHOLDING

AUTOMATIC THRESHOLDINGWe have discussed operations on binary images

• What we need is a way to convert grayscale to binary

Requires setting a threshold

• Domain- and image-dependent: there is no single best threshold• Manual setting is not ideal (takes time, is subjective)

We focus on determining the threshold automatically

• Information used only from the grayscale image

AUTOMATIC THRESHOLDING2

Two options:

• Global thresholding: single optimal threshold• Local thresholding: threshold per pixel

GLOBAL THRESHOLDING

GLOBAL THRESHOLDINGGoal: Find a single threshold q that is applied to each pixel of grayscale image I:

• Divides the image into two disjoint sets C0 (background) and C1 (foreground)

•

GLOBAL THRESHOLDING2

Essential information present in histogram

We can identify two types of global thresholding methods:

• Shape-based: look for peaks, valleys, etc.• Statistical: use means, variance, etc.

GLOBAL THRESHOLDING3

We consider histograms h(g) of grayscale image I with N pixels and Kpossible intensity values 0 ≤ 𝑔𝑔 ≤ 𝐾𝐾

We can calculate the mean and variance from the histogram:

• Mean:

• Variance:

GLOBAL THRESHOLDING4

If we apply threshold q, we obtain two disjoint sets with number of pixels:

•

Each of these sets is a histogram and we can again calculate their means:

•

•

Relation to overall mean:

GLOBAL THRESHOLDING5

Variance per set can also be easily calculated

• No trivial relation to overall variance

We want to find a threshold q to split the values into background and foreground

To set threshold q, we can simply:

• Set q = mean(I)• Set q = median(I)• Set q to be the average of the minimum and maximum:

GLOBAL THRESHOLDING6

These measures are typically suboptimal:

• For median, equal number of pixels in foreground and background are assumed• For the mid-range technique, extreme pixel values largely affect the result

GLOBAL THRESHOLDING7

Mean, median, mid-range:

GLOBAL THRESHOLDING8

If we know in advance which percentage b of the pixels is background, we can determine q as:

•

• Typically, we don’t know this percentage

We now discuss specific threshold finding algorithms

ISODATA ALGORITHMAssumptions:

• Image consists of two distributions: background and foreground• Variances of the two distributions are assumed equal

Initially, q is set to the mean or median of histogram h(g)

Iterative steps:

• The means of the foreground and background are calculated• q is repositioned to the average• Repeat until convergence (q doesn’t change between iterations)

ISODATA ALGORITHM2

Simple algorithm, but suffers when number of pixels between classes is biased

Example:

OTSU’S METHODAssumption:

• Image consists of two distributions: background and foreground

Goal is to find q such that:

1. The variances of each distribution are minimal (within)2. The distance between the means is maximal (between)

OTSU’S METHOD2

The within-class variance:

•

• With P0(q) and P1(q) the class probabilities:

OTSU’S METHOD2

Between-class variance:

•

The total variance is the sum of within- and between-class variance:

•

Since total variance is constant for a given image, we can either minimize within or maximize between:

• Maximizing between is easier (means are easier to calculate than variances)

OTSU’S METHOD3

Construct table of between-class variances as a function of q

• Use a for-loop to find optimal value for q

In red, between-class variance

MAXIMUM ENTROPY

MAXIMUM ENTROPYWe define the probability of an intensity value g to occur as:

• p(g) = p(I(u,v) = g)• Termed prior (a priori) probabilities

Probability distribution is the vector of probabilities for all possible values:

• (p(0), p(1), … , p(K-1))

Estimate of the probability distribution comes from the histogram:

•

MAXIMUM ENTROPY2

We define the cumulative distribution function as:

•

• Here, P(0) = p(0) and P(K – 1) = 1

We define entropy as:

•

• logb(x) is the logarithm of x to the base b

g=1

K-1

MAXIMUM ENTROPY3

Entropy is a measure of the “surprise” of the values

When there is only one intensity value:

• p(g) = 0 for all g expect for one value• H(I) = 0 so entropy is minimal (because logb(1)=0)

When all intensities have the same probability (1/K):

• Entropy is maximum, H(I) = log(K)

Entropy is therefore always in range [0, log(K)]

MAXIMUM ENTROPY4

Entropy can be used as a criterion for threshold selection

Given threshold q, the probability distributions for C0 and C1 are:

•

• with

• P0(q) and P1(q) are the cumulative probabilities for the pixels in the background and foreground

MAXIMUM ENTROPY5

Given a threshold q, we can calculate the entropy within each partition:

•

•

The overall entropy is the sum of both: H01(q) = H0(q) + H1(q)

Since we aim for the maximum entropy, we search for q that maximizes H01

• Algorithm for determining q is presented in the book

MAXIMUM ENTROPY6

Background H0 (green), foreground H1 (blue) and overall H01 (red) entropy

MAXIMUM ENTROPY7

Maximum entropy algorithm is relatively fast O(K)

• Calculations based on image histogram

Extensions make use of local structure of the image (see later algorithms)

QUESTIONS?

MINIMUM ERROR THRESHOLDINGGoal of minimum error thresholding is to optimally fit Gaussians to a probability distribution

• Derived from image histogram

MINIMUM ERROR THRESHOLDING2

Idea: each pixel originates from either foreground (C1) or background (C0) class

• Both classes are modeled as Gaussian distribution (with μ and σ2)

We now need to determine for each pixel value x whether it belongs to C0 or C1

Probability that value x belongs to background is p(x | C0)

• Probability that value x is observed given that it is background• Requires that we know what is background

MINIMUM ERROR THRESHOLDING3

We are interested in the reverse problem: P(C0 | x) and P(C1 | x)

• For each x, select the class with highest probability: foreground or background

We can calculate these using Bayes’ theorem:

•

• 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 = 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙∗𝑝𝑝𝑝𝑝𝑙𝑙𝑙𝑙𝑝𝑝𝑙𝑙𝑒𝑒𝑙𝑙𝑙𝑙𝑙𝑙𝑒𝑒𝑒𝑒𝑙𝑙

• Posterior: probability that we are interested in• Prior: general knowledge how often Cj occurs• Likelihood: conditional probability• Evidence: what we can observe only “scales” result

MINIMUM ERROR THRESHOLDING4

𝑝𝑝 𝐴𝐴 𝐵𝐵 =𝑝𝑝 𝐵𝐵 𝐴𝐴 𝑝𝑝(𝐴𝐴)

𝑝𝑝(𝐵𝐵)

Example:

• A = it is raining (3 days per week)• B = streets are wet (4 days per week)• 𝑝𝑝 𝐵𝐵 𝐴𝐴 = 0.9, 90% chance on wet streets when it is raining• Calculate the probability it was raining when the streets are wet:

• 𝑝𝑝 𝐴𝐴 𝐵𝐵 = 𝑝𝑝 𝐵𝐵 𝐴𝐴 𝑝𝑝(𝐴𝐴)𝑝𝑝(𝐵𝐵)

=0.9∗3747

= 2740

MINIMUM ERROR THRESHOLDING5

We can thus formulate the Bayes’ decision rule:

•

• Also called minimum error criterion

If we model P(x | C0) and P(x | C0) as Gaussian distributions, the (scaled) probability distribution becomes:

•

MINIMUM ERROR THRESHOLDING6

With some work, we can derive the measure of potential error of classifying value x as class Cj:

•

We can thus make a decision for value x as follows:

•

Assumption is that classes are distributed normally, and parameters are well estimated

MINIMUM ERROR THRESHOLDING7

When we apply a threshold q, all values g ≤ q are considered background

Criterion function e(q) of this classification is:

•

MINIMUM ERROR THRESHOLDING8

Our goal is now to find the q that minimizes e(q)

• Requires the prior probabilities and the means and variances of both classes

Prior probabilities are obtained by “counting values”:

•

• Means and variances also calculated from histogram

MINIMUM ERROR THRESHOLDING9

Again, algorithm is relatively fast O(K) due to calculations on image histogram

QUESTIONS?

LOCAL ADAPTIVE THRESHOLDING

LOCAL ADAPTIVE THRESHOLDINGOften, a single threshold is not optimal due to local changes in intensity in the image

Local methods take into account the pixel location (u,v) and have varying thresholds Q(u,v) per pixel

• We discuss two methods that differ in how to derive Q from an input image

BERNSENS’ METHODFind a dynamic threshold based on the minimum and maximum pixel value of a neighborhood R(u,v) around (u,v)

•

The threshold is then determined to be the mid-range value:

•• Only when Imax and Imin differ sufficiently• When (Imax(u,v) – Imin(u,v)) < cmin, all pixels are assigned to the background

BERNSENS’ METHOD2

BERNSENS’ METHOD3

Less predictable results with smoothly varying backgrounds

BERNSENS’ METHOD4

Clear relation with morphological filters:

• Min and max filter: erosion and dilation

NIBLACK’S METHODNiblack’s method estimates Q(u,v) as a function of local intensity average μR(u,v) and standard deviation σR(u,v):

•• Q(u,v) is set to the mean intensity plus K times the standard deviation

NIBLACK’S METHOD2

In even areas, σR(u,v) is small and the threshold is close to the mean intensity

• Can be solved by adding a constant:

Original formulation assumes dark background

• Inverse formulation used when background is lighter

NIBLACK’S METHOD3

NIBLACK’S METHOD4

QUESTIONS?

ASSIGNMENT 3

ASSIGNMENT 3Object detection/recognition

• Pick an object class in a given context• Search at least 10 images with variation and 10 images with distractions

Two/three phases:

1. Pre-processing2. Object detection3. Refinement (only phase in which you are allowed to use color!)

Hand in report, code and images

NEXT LECTURE

NEXT LECTURENext lecture is about:

• Comparing Images (Book II, Chapter 11)• Wednesday October 24, 15:15 - 17:00 (RUPPERT-PAARS)

Deadline for Assignment 3 (Shape detection):

• November 11, 23:00• Walk-in session Friday 26-10 and 2-11, 9:00-10:45

CONTENTS OF THIS LECTURE

CONTENTS OF THIS LECTUREAdvanced Methods (book III)

• Chapter 2: Automatic Thresholding (not 2.3 and 2.4)