6.869 Problem Set 3: Statistical models of

images

Nikhil Naik

naik@mit.edu

1 Retinex: improving the input for the simple

visual system

The Retinex algorithm is applied as follows-

1. Use the logarithmic representation of mean of RGB image as input i.e. foran RGB image I, Ilog = log(mean(I))

2. Calculate the x and y derivatives of Ilog and apply a lower threshold onthese images to eliminate the low amplitude values which correspond to thegradient changes in illumination. For instance, the thresholded gradientimage in x direction, dIx is given by,

dIx =

{∂Ilog∂x if

∂Ilog∂x > th

0 else(1)

3. The output image is obtained by deconvolving dIx and dIy with the deriva-tive kernels.

Following code-snippet implements this algorithm

1 img = mean(imread('.\images\simpleworld.jpg'),3);2 imgLog=img;3 imgLog(img==0)=1;4 imgLog=log(imgLog);5 % Derivation filters6 clear fn7 fn(:,:,1) = [0 0 0; -1 1 0; 0 0 0];8 fn(:,:,2) = [0 0 0; -1 1 0; 0 0 0]';9 %%

10 % Derivate11 out = convFn(imgLog, fn);12 %Apply threshold13 th=0.1;14 out(abs(out)<th)=0;

1

Figure 1: Input and output images of Retinex algorithm with thresholded gra-dients in x and y direction

2

Figure 2: Two column pro�les after applying Retinex algorithm. The gradientchanges due to illumination disappear as seen from the two plots

15

16 % Pseudo-inverse (using the trick from Weiss, ICCV 2001; equations 5-7)17 im=deconvFn(out,fn);

.Figure 1 shows the input and output images along with the thresholded

gradient images.

1.1 Removing the gradient due to illumination

According to Retinex algorithm, small gradients are classi�ed as illuminationwhile large gradients are classi�ed as re�ectance. So by setting small gradientsto zero, the e�ect of illumination is removed. This is shown in Figure 2 byplotting two columns of the image from both input and output images. As itcan be seen, the white area which belongs to background, and is only a�ected byslowly varying illumination, becomes �at after applying the retinex algorithm.

3

2 Wiener Filtering

120 images were selected by doing a random crawl on google images. They werenatural images with di�erent categories such as nature, buildings, vehicles andso on. An average amplitude spectrum was calculated as used as a prior for theWiener �ltering process. The following code-snippet performs Wiener �ltering

1 function imgFilt=img_wiener(img,ps,sd)2 % img=Image corrupted with zero mean Gaussian noise3 % ps= the power spectrum of mean amplitude spectrum...4 % genereated from the dataset5 %sd= standard deviation of Gaussian noise.6

7 imgFFT=fft2(img); % FFT of noisy image8 imgWF=1./(1+((numel(img)*(sd^2))./ps)); %define Wiener filter9 imgFilt=ifft2(imgWF.*imgFFT); % Apply filter and get inverse FFT

10 end

.The following code-snipper performs Gaussian �ltering

1 function imgFilt=img_gaussian(img,sd)2 %img: input image with added zero mean Gaussian noise3 %sd: standard deviation for the Gaussian Filter4

5 gFilt=fspecial('gaussian',size(img,1),sd); % define filter kernel6 imgFFT=fftshift(fft2(img));7 imgFilt=ifft2(ifftshift(gFilt.*imgFFT)); %perform Gaussian Filtering8 end

.Figure 3 and 4 show two images added with zero mean Gaussian noise and

their Wiener �ltered and Gaussian �ltered results.

3 Histogram equalization

For an image with histogram h(I),we calculate the CDF given by

c(I) =1

N

I∑i=0

h(i) = c(I − 1) +1

Nh(I) (2)

The CDF is applied as compensation transfer function(CTF) i.e. f(I) =c(I).

The main script

1 %% Read image and convert to luminance2 img = imread('.\images\office_6.jpg');3 img=im2double(img);4

4

Figure 3: Image added with Gaussian noise with σn = 10 and �ltered withWiener �lter as well as Gaussian �lter with σs = 100. As it can be seen inthe zoomed-in version on the right, the noise removal is similar but the highfrequncy content such as hair and eyes is preserved better by the Wiener �lter.Please see �gures generated from the code for details

Figure 4: Image added with Gaussian noise with σn = 20 and �ltered withWiener �lter as well as Gaussian �lter with σs = 100. As it can be seen inthe zoomed-in version on the right, the noise removal is similar but the highfrequency content is similarly degraded in both cases due to the high σn of noise.So Wiener �lter does not perform signi�cantly better as compared to Gaussian�lter in this case due to high σn of noise content.

5

5 % provide fractional value for alpha for...6 %better results in case of saturated colors7 alpha=1;8

9 %Calculate Luminance image10 imgLuma=sqrt(img(:,:,1).^2+img(:,:,2).^2+img(:,:,3).^2);11 %Get Normalized RGB image12 rgbNorm=calcNormRGB(img,imgLuma);13

14

15 %% Calculate histogram, CDF and compensation transfer fn (CTF)16 intLuma=uint8(255*(imgLuma/max(max(imgLuma))));17 imgHeq=calcHisteq(intLuma,alpha); %Equalize luminance histogram18 eps=10^-3;19 % Ratio for individual color channel adjustment...20 % obtained from histogram equalization of luminance21 imgScale=double(imgHeq)./(double(intLuma)+eps);22

23 %Perform RGB adjustment using ratio values24 rgbHeq=adjustRGB(rgbNorm,imgScale,imgLuma);25 rgbHeq=uint8(255*rgbHeq);

.Following function performs the histogram equalization

1 function imgHeq=calcHisteq(img,alpha)2 %% Calculate Histogram,CDF and CTF3 ch=size(img,3);4 imgHeq=uint8(zeros(size(img)));5 tempHeq=uint8(zeros(size(img(:,:,1))));6

7 for i=1:ch8 data(i).imgHist=imhist(img(:,:,i)); % calculate histogram9 data(i).imgCDF=zeros(size(data(i).imgHist));

10 data(i).imgCDF(1)=data(i).imgHist(1); %initialize CDF11

12 % Calculate CDF13 for j=2:length(data(i).imgHist)14 data(i).imgCDF(j)=data(i).imgCDF(j-1)+data(i).imgHist(j);15 end;16

17 % Calculate CTF18

19 idnTF=0:255; %Identity transform20 idnTF=idnTF(:);21

22 %CTF=alpha*CDF+(1-alpha)*Identity23 data(i).imgCTF=uint8((alpha*255*(data(i).imgCDF/data(i).imgCDF(255)))...24 +(1-alpha)*idnTF);25

26

27 %% Calculate the histogram-equalized image28 % Apply the CTF curve on the image29 for x=0:25530 tempHeq(img(:,:,i)==x)=data(i).imgCTF(x+1);31 end;

6

Figure 5: Luminance channel, equalized luminance channel and histogram-equalized RGB image

32 imgHeq(:,:,i)=tempHeq;33 clear tempHeq;34 end;35 end

.Figure 6 shows the result obtained for a washed-out color imageThe RGB equalization function is as follows-

1 function adjImg=adjustRGB(img,ratio,amp)2 %% Calculate adjusted RGB images3 % amp: Amplitude/Luminance image;4 % ratio: the scale obtained from histogram equalization of luminance5

6 adjImg=zeros(size(img));7

8 adjImg(:,:,1)=ratio.*amp.*img(:,:,1);9 adjImg(:,:,2)=ratio.*amp.*img(:,:,2);

10 adjImg(:,:,3)=ratio.*amp.*img(:,:,3);11 end

7

Figure 6: The toys in the original image are saturated. In second image, withdirect histogram equalization i.e.α = 1, they get unnatural colors. In the lastimage, using α = 0.3, we get more natural looking colors, with histogram equal-ized to some extent.

.

3.1 Saturated color values

Color values that are clipped in the original image, i.e., have one or more sat-

urated color channels, may appear unnatural when remapped to a non-clipped

value. Extend your algorithm to handle this case in some useful way.

This case can be handled by only partially compensating for the histogramunevenness. This is done by using a mapping function as described by thefunction

f(I) = αc(I) + (1− α)I (3)

where I is an identity transform. The saturated colors look more natural byusing a fractional value of α as seen in Figure 6

8