Image enhancement and sharpening

Lecture 6-7February 19-26, 2008

Procedures of image processing

Preprocessing Radiometric correction is concerned with improving the accuracy of surface spectral reflectance, emittance, or back-scattered measurements obtained using a remote sensing system. Detector error correction, Atmospheric and topographic correctionsGeometric correction is concerned with placing the above measurements or derivativeproducts in their proper locations.

Information enhancementPoint operations change the value of each individual pixel independent of all other pixelsLocal operations change the value of individual pixels in the context of the values of neighboring pixels.Information enhancement includes image reduction, image magnification, transect extraction, contrast adjustments (linear and non-linear), band ratioing, spatial filtering, fourier transformations, principle components analysis, image sharpening, and texture transformations

Information extractionPost-classificationInformation output

Image or enhanced image itself, thematic map, vector map, spatail database, summary statistics and graphs

1. Image reduction

2x reduction

1. Sampling every other row or columnor Nearest Neighbor in ENVI 2. Pixel aggregate (average of 4 pixels)

2. Image magnification

2x magnification

1. Duplicate every row and columeor Nearest Neighbor in ENVI

2. Bilinear resampling 3. Cubic resampling

A common interface in ENVI

For Spatial • Subsetting• Reduction• Magnification

3. Transects (spatial profiles)

4. Spectral profiles

5. Contrast Enhancement (stretch)

Materials or objects reflect or emit similar amounts of radiant flux (so similar pixel value)Low-contrast imagery with pixel range less than the designed radiometric range

20-100 for TM less than the designed 0-255To improve the contrast:

Linear techniqueMinimum-maximum contrast stretchPercentage linear contrast stretchStandard deviation contrast stretchPiecewise linear contrast stretch

Non-linear technique Histogram equalization

Contrast enhancement is only intended to improve the visual quality of a displayed image by increasing the range (spreading or stretching) of data values to occupy the available image display range (usually 0-255). It does not change the pixel values, unless save it as a new image. It is not goodpractice to use saved image for classification and change detection.

Minimum-maximum contrast stretch

kkk

kinout quantBVBV ⎟⎟

⎠

⎞⎜⎜⎝

⎛−−

=minmaxmin

where:- BVin is the original input brightness value - quantk is the range of the brightness values that can be

displayed on the CRT (e.g., 255),- mink is the minimum value in the image,- maxk is the maximum value in the image, and- BVout is the output brightness value

where:- BVin is the original input brightness value - quantk is the range of the brightness values that can be

displayed on the CRT (e.g., 255),- mink is the minimum value in the image,- maxk is the maximum value in the image, and- BVout is the output brightness value

Percentage linear and standard deviation contrast stretch

X percentage (say 5%) top or low values of the image will be set to 0 or 255, rest of values will be linearly stretched to 0 to 255ENVI has a default of a 2% linear stretch applied to each image band, meaning the bottom and top 2% of image values are excluded by positioning the range bars at the appropriate points. Low 2% and top 2% will be saturated to 0 and 255, respectively. The values between the range bars are then stretched linearly between 0 and 255 resulting in a new image. If the percentage coincides with a standard deviation percentage, then it is called a standard deviation contrast stretch. For a normal distribution, 68%, 95.4%, 99.73% values lie in ±1σ, ±2 σ, ±3 σ. So 16% linear contrast stretch is the ±1σ contrast stretch.

original Saturating the waterStretching the land

Saturating the landStretching the water

Special linear contrast stretchOr Stretch on demand

Piecewise linear contrast stretch

When the histogram of an image is not Gaussian (bimodal, trimodal, …), it is possible to apply a piecewise linear contrast stretch.But you better to know what each mode in the histogram represents in the real world.

Stretch both land and water

Nonlinear contrast stretch: histogram equalization

It automatically reduces the contrast in the very light or dark parts of the image associated with the tails of a normally distributed histogram. Some pixels that originally have different values are now assigned the same value (perhaps loss information), while other value that were once very close together are now spread out, increasing the contrast between them

Scan the p272 and 274

Li

(n=7)

Histogram equalized

Original

6. Band Ratioing

Sometimes differences in brightness values from identical surface materials are causedby topographic slope and aspect, shadows, atmospheric constitutional change, or seasonschanges in sun angle and intensity. Band ratio can be applied to reduce the effects of such environmental conditions. In addition, band ratio also help to discriminatebetween soils and vegetation

lji

kjiratioji BV

BVBV

,,

,,,, =

where:- BVi,j,k is the original input brightness value in band k- BVi,j,l is the original input brightness value in band l- BVi j ratio is the ratio output brightness value

where:- BVi,j,k is the original input brightness value in band k- BVi,j,l is the original input brightness value in band l- BVi,j,ratio is the ratio output brightness value

None of the single band relation passed the Durbin-Watson statistical test.

relative phycocyanin content (PC)

Source: Vincent et al., 2004

Based on Equation 4 Based on Equation 5

Applying best equations derived from Landsat 7 to Landsat 5:

Indicating the best spectral ratio model is more robust than the best model derived from singleBands, with regard to changes in sun angle (season), atmospheric transmission, and instrumentSettings between Landsat 5 and 7

Applied to Sep 27, 2000Applied to July 1, 2000

7. Spatial filtering to enhance low, high frequency detail, and edges

Spatial frequency is the number of changes in brightness value per unit distance for any particular part of an image. If there are very few changes in brightness value over a given area in an image, this is referred to as a low-frequency area. Conversely, if the brightness values change dramatically over short distances, this is an area of high-frequency detail. To see the spatial frequency, we look at the local (neighboring) pixel value changes.The spatial frequency may be enhanced or subdued using:

Spatial convolution filtering: based primarily on the use of convolution masksFourier analysis: mathematically separates an image into its spatial frequency components

7.1 Spatial convolution filtering

A linear spatial filter is a filter for which the brightness value (BVi,j,out) at location i,j in the output image is a function of some weighted average (linear combination) of brightness values located in a particular spatial pattern around the i,j location in the input image.

The process of evaluating the weighted neighboringpixel values is called two-dimensional convolution filtering.

A linear spatial filter is a filter for which the brightness value (BVi,j,out) at location i,j in the output image is a function of some weighted average (linear combination) of brightness values located in a particular spatial pattern around the i,j location in the input image.

The process of evaluating the weighted neighboringpixel values is called two-dimensional convolution filtering.

The size of the neighborhood convolution mask or kernel (n) is usually 3 x 3, 5 x 5, 7 x 7, or 9 x 9.

We will constrain our discussion to 3 x 3 convolution masks with nine coefficients, ci, defined at the following locations:

c1 c2 c3Mask template = c4 c5 c6

c7 c8 c9

The size of the neighborhood The size of the neighborhood convolution maskconvolution mask or or kernel kernel ((nn) is ) is usually 3 x 3, 5 x 5, 7 x 7, or 9 x 9. usually 3 x 3, 5 x 5, 7 x 7, or 9 x 9.

We will constrain our discussion to We will constrain our discussion to 3 x 33 x 3 convolution masks convolution masks with with ninenine coefficients, coefficients, ccii, defined at the following locations:, defined at the following locations:

cc1 1 cc2 2 cc33Mask templateMask template = = cc4 4 cc5 5 cc66

cc7 7 cc8 8 cc99

The coefficients, c1, in the mask are multiplied by the following individual brightness values (BVi) in the input image:

c1 x BV1 c2 x BV2 c3 x BV3 Mask template = c4 x BV4 c5 x BV5 c6 x BV6

c7 x BV7 c8 x BV8 c9 x BV9

The primary input pixel under investigation at any one time is BV5 = BVi,j

The coefficients, c1, in the mask are multiplied by the following individual brightness values (BVi) in the input image:

c1 x BV1 c2 x BV2 c3 x BV3 Mask template = c4 x BV4 c5 x BV5 c6 x BV6

c7 x BV7 c8 x BV8 c9 x BV9

The primary input pixel under investigation at any one time is BV5 = BVi,j

7.1.1 Low frequency (pass) filter: block the high spatial frequency detail, left the low-frequency

A kernel with small positive values with the same or a little large central value

7.1.2 High frequency (pass) filter: remove the slowly varying components and enhance the high-frequency local variations.

a kernel with a high central value, typically surrounded by negative weights

11 11 1111 11 1111 1111

.25.25 .5.5 .25.25

.5.5 11 .5.5

.25.25 .25.25.5.5

11 11 1111 22 1111 1111

a b c

--11 --11 --11--11 99 --11--11 --11--11

11 --22 11--22 55 --2211 11--22

ed

0 1 3 4 7

2 5 4 9 7

8 3 1 6 4⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

=∑=

n

BVxcFF

ii

i

out

9

1,5 int

3 4 5 3 0 2

1 1 3 2 -2 2

b e

da

This will result in 2 lines and 2 columns smaller than the original image. Software can deal withthis problem in different ways (refer to book p277)

7.1.3 Linear edge enhancement

Directional Laplacian

Highlight points, lines, edges, suppress uniform and smooth regions

-1 0 1

-1 0 1

-1 0 1

-1 -1 -1

0 0 0

1 0 1

0 1 1

-1 0 1

-1 -1 0

1 1 0

1 0 -1

0 -1 -1

Vertical edges Horizontal edge NW-SE NE-SW

directional

0 -1 0

-1 4 -1

0 -1 0

-1 -1 -1

-1 8 -1

-1 -1 -1

1 -2 1

-2 4 -2

1 -2 1

Laplacian

7.1.4 Nonlinear edge enhancement

Sobel

Robert

( ) ( )( ) ( )987321

741963

22,5

2222

BVBVBVBVBVBVYBVBVBVBVBVBVX

whereYXSobel out

++−++=++−++=

+=

14

7 8

26

9

3

order

5X = X = Y = Y =

-1 0 1

-2 0 2

-1 0 1

1 2 1

0 0 0

-1 -2 -1

86

95

,5

BVBVY

BVBVXwhere

YXRoberts out

−=

−=

+=

X = X = Y = Y =

0 0 0

0 1 0

0 0 -1

0 0 0

0 0 1

0 -1 0

7.2 Fourier transform

Spatial domain to frequency domainFrequency image contains all information found in the original imageFrequency image is useful for image restoration (noise remove), filtering (low, high frequency, and edge detection) , radiometric correction, image to image registrationFor noise removed in frequency domain, it is much easier to identify and remove than in spatial domain.

When stationaryperiodic noise is a single-frequency sinusoidal function in the spatial domain, its Fourier transform is bright points. A line connecting the points is always perpendicular to the orientation of the noise lines in the original image

Spatial domain to frequency domain

Frequency domain back to spatial domain after removed the noises

After user-defined cut filter, transform back to spatial domain.

Low-pass filter block high frequency

High-pass filter block low frequency

A solution to get G is to put the convolution mask at the center of a zero-value image that has the same size of F.

8. Principle Components Analysis (PCA)

There are large correlations among remote sensing bands. PCA will result in another uncorrelated datasets: principal component images (PCs). PC1 contains the largest variation, PC2 contains the second largest variation, …The first two or three components (PCs) contain over 90% of information from the original many bands. The rest of bands contains the noises. It is a great compress operation, and reducing the dimensionality of the original dataThe new principal component images that may be more interpretable than the original data.

Using Covariance matrixIs unstandardized PCA

Using correlation matrixIs standardized PCA

E

Or using the Correlation matrix replace the Cov(covariance matrix)

where EVT is the transpose of the eigenvector matrix, EV, and E is a diagonal covariance matrix whose elements, λi,i, called eigenvalues, are the variances of the pth principal components, where p = 1 to n

components. The largest λi,j will relate to the PC1, then…

where EVT is the transpose of the eigenvector matrix, EV, and E is a diagonal covariance matrix whose elements, λi,i, called eigenvalues, are the variances of the pth principal components, where p = 1 to n

components. The largest λi,j will relate to the PC1, then…

Get the Eigenvalues

Get the Eigenvectors

Correlation, Rk,p, Between Band k and Each Principal Component pCorrelation, Correlation, RRk,pk,p, Between Band , Between Band kk and Each Principal Component and Each Principal Component pp

where:ak,p = eigenvector for band k and component pλp = pth eigenvalueVark = variance of band k in the

covariance matrix

where:ak,p = eigenvector for band k and component pλp = pth eigenvalueVark = variance of band k in the

covariance matrix

Computer pixel values for the new PCs

∑=

=n

kkjikppji BVanewBV

1,,,,

New pixel value for new PCsSee an example in the book p299-301

where αkp = eigenvectors, BVi,j,k = brightness value in band k for the pixel at row i, column j, and n = number of bands.

where αkp = eigenvectors, BVi,j,k = brightness value in band k for the pixel at row i, column j, and n = number of bands.

Inverse PC rotation

Similar as the inverse FFT, to transfer from frequency domain back to the spatial domain, the inverse PC rotation is to transfer the PC images back into their original data space, without including the noise bands, using the same statistics files saved from the forward PC rotation. Be sure to use the same covariance matrix or correlation matrix you used for forward PC rotation.

9. Minimum Noise Fraction Transform (MNF)

It is determine the inherent dimensionality of image data, to segregate noise in the data, and to reduce the computational requirements for subsequent processing

The first rotation uses the principal components of the noise covariance matrix to decorrelate and rescale the noise in the data (noise whitening), resulting in transformed data in which the noise has unit variance and no band-to-band correlationsThe second rotation uses the PCs derived from the original image data after they have been noise-whitened by the first rotation and rescaled by the noise standard deviation. You can divide the images into two parts: large eigenvalues and coherent eigenimages and near-unity eigenvalues and noise-dominated images. Only use the first part for furthering processing.Use estimate noise statistics from data

Shift diff subset (optional), selecting a subset that is spectrally uniform, less variationBands with larger eigenvalues (>1) contain data, others contain noises. Visually examine the bands along with their eigenvalues and find those bands with noises

Inverse MNF use only the first part of the MNF bands to remove the noise from the images.

10. Image sharpening or data fusionImage sharpening: automatically merge a low-resolution color image with a high-resolution grayscale imageData fusion: quantitatively combining or merging data from multiple sources to maximize the extractible information from the data. It is used to merge remote sensing data of different spatial and/or spectral resolutions to create a higher spatial and/or spectral resolution image(s).

HSV (hue-saturation-value), IHS (intensity-hue-saturation), HLS (hue-lightness-saturation) color transformsBrovey transformGram-Schmidt spectral sharpening PC spectral sharpeningCN spectral sharpening

Data fusion can be used toLow spatial resolution with high spatial resolution image to increase spatial resolutionLow spatial and high spectral with high spatial and high spectral to increase both spatial and spectral resolutionTwo complementary but completely different types of remote sensing data (e..g. radar and optical)

HSV

Transform an RGB image to HSV (hue, saturation, value) color space. Replace the value band with the high-resolution image, automatically resample the hue and saturation bands to the high resolution pixel size using a nearest neighbor, bilinear, or cubic convolution tech, and finally transform the image back to RGB color space.

Xie and Keller, 2006

Color normalization (brovey)

Using a mathematical combination of the color image and high resolution data. Each band in the color image is multiplied by a ratio of the high resolution data divided by the sum of the color bands. The function automatically resamples the three color bands to the high resolution pixel size using a nearest neighbor, bilinear, or cubic convolution tech, and finally transform the image back to RGB color space.

Gram-Schmidt spectral sharpening

First, a panchromatic band is simulated from the lower spatial resolution spectral bands. Second, a Gram-Schmidt transformation is performed on the simulated panchromatic band and the spectral bands, where the simulated panchromatic band is employed as the first band. Third, the high spatial resolution panchromatic band is swapped with the first Gram-Schmidt band. Finally, the inverse Gram-Schmidt transform is then applied to form the pan-sharpened spectral bands.

PC spectral sharpening

A principal components transformation is performed on the multispectral data. The PC band 1 is replaced with the high resolution band, which is scaled to match the PC band 1 so no distortion of the spectral information occurs. Then, an inverse transform is performed. The multispectral data is automatically resampled to the high resolution pixel size using a nearest neighbor, bilinear, or cubic convolution technique.

The assumption behind this is that the first PC carries the information common to all bands and is approximately equal to the high space resolution image.