A simulated annealing restoration algorithm for restoring

ARTICLE IN PRESS

0923-5965/$ - se

doi:10.1016/j.im

�Correspondfax: +8522362

E-mail addr

Signal Processing: Image Communication 21 (2006) 280–292

www.elsevier.com/locate/image

A simulated annealing restoration algorithm for restoringhalftoned color-quantized images

Yik-Hing Fung, Yuk-Hee Chan�

Centre for Multimedia Signal Processing, Department of Electronic and Information Engineering,

The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong

Received 15 June 2004; received in revised form 17 September 2005; accepted 18 November 2005

Abstract

Restoration of color-quantized images is rarely addressed in the literature especially when the images are color-

quantized with halftoning. Most existing restoration algorithms are generally inadequate to deal with this problem as they

were proposed for restoring noisy blurred images. In this paper, a restoration algorithm based on simulated annealing is

proposed to solve the problem. This algorithm makes a good use of the available color palette and the mechanism of a

halftoning process to derive useful a priori information for restoration. Simulation results show that it can improve the

quality of a halftoned color-quantized image remarkably in terms of both SNR and CIELAB color difference metric. The

subjective quality of the restored images can also be improved.

r 2005 Elsevier B.V. All rights reserved.

Keywords: Image restoration; Color-quantization; Halftoning; Simulated annealing

1. Introduction

Color quantization is a process of reducing thenumber of colors in a digital image by replacingthem with some representative colors selected froma palette [1]. It is widely used because it can savetransmission bandwidth and data storage require-ment in many multimedia applications. When colorquantization is performed, certain types of degrada-tion are introduced due to the limited number ofcolors used to produce the output image. The mostcommon artifact is false contour. False contouroccurs when the available palette colors are not

e front matter r 2005 Elsevier B.V. All rights reserved

age.2005.11.003

ing author. Tel.: +852 27666264;

8439.

ess: [email protected] (Y.-H. Chan).

enough to represent a gradually changing region.Another common artifact is color shift. In general,the smaller the color palette size, the more severe thedefects are.

Digital halftoning [2–6] would be helpful toeliminate these defects by making use of the factthat human eyes act as spatial low-pass filters.During color quantization, the quantization error ofa pixel is diffused to neighboring pixels so as to hidethe defects and to achieve a more faithful reproduc-tion of colors. At the moment, the most popularhalftoning method is error diffusion and severalwell-known error diffusion filters such as Floyd–Steinberg, Stucki, and Jarvice–Judice–Ninke [2] aregenerally used to achieve the goal.

Sometimes it is necessary to recover the originalimage from its color-quantized output. However,

.

www.elsevier.com/locate/image

ARTICLE IN PRESS

Fig. 1. Color quantization with halftoning.

Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 281

though there are a lot of reported works on therestoration of noisy and blurred color images [7–14],little effort has been seen in the literature forrestoring halftoned color-quantized images. Ob-viously, the degradation models of the two casesare completely different and hence direct adoptionof conventional restoration algorithms does notwork effectively.

Error diffusion is a nonlinear process. Conse-quently, conventional gradient-oriented optimiza-tion algorithms are not suitable to solve theaddressed problem. Simulated annealing (SA) [15]is an adaptive search algorithm that works very wellon discrete optimization problems. The basic idea ofSA is to simulate an annealing process in a system.As compared with some other conventional meth-ods, it accepts solution with deteriorated cost to alimited extent. This feature gives SA the capabilityto escape from the local minima. This paper isdevoted to formulating the process of colorquantization when error diffusion is involved anddeveloping a SA restoration algorithm to restorecorresponding degraded images.

The organization of this paper is as follows.Section 2 describes the basic concept of colorquantization with halftoning and its correspondingdegradation model. Section 3 presents the derivationof a restoration algorithm for handling halftonedcolor-quantized images. In Section 4, simulationresults and comparative study are provided toevaluate the performance of the proposed algorithm.Finally, conclusions are given in Section 5.

2. Image degradation in color quantization with

halftoning

A color image X generally consists of three colorplanes, say, Xr, Xg and Xb, which represents the red,the green and the blue color planes of the image,respectively. Accordingly, the (i, j)th color pixel of a24-bit full color image of size N�N consists ofthree color components. The intensity valuesof these three components form a 3D vector

X!

ði; jÞ ¼ Xði; jÞr;Xði; jÞg;Xði; jÞb

� �, where Xði; jÞc 2 ½0; 1� is

the intensity value of the cth color component of the(i,j)th pixel. Here, we assume that the maximum andthe minimum intensity values of a pixel are,respectively, 1 and 0.

Fig. 1 shows the system which performs colorquantization with error diffusion. The input imageX is scanned in a row-by-row fashion from top to

bottom and left to right. The relationship betweenthe original image X and the encoded image Y isdescribed by

Uði; jÞc¼Xði; jÞc �Xðk;lÞ2O

Hðk;lÞcEði�k; j�lÞc for c 2 fr; g; bg,

(1)

Y!

ði; jÞ ¼ Qc½U!

ði; jÞ� (2)

and

E!

ði; jÞ ¼ Y!

ði; jÞ �U!

ði; jÞ ¼ Qc½U!

ði; jÞ� �U!

ði; jÞ, (3)

where Uði; jÞ ¼ ðUði; jÞr;Uði; jÞg; Uði; jÞbÞ is a state vector

of the system, E!

ði; jÞ is the quantization error of the

pixel at position (i, j) andHðk;lÞc is a coefficient of the

error diffusion filter for the cth color component. Ois the corresponding causal support region of Hðk;lÞc.

The operator Qc½�� performs a 3D vectorquantization. Specifically, the 3D vector U

!

ði; jÞ iscompared with a set of representative color vectorsstored in a previously generated color paletteC ¼ fv̂i : i ¼ 1; 2; . . . Ncg. The best-matched vectorin the palette is selected based on the minimumEuclidean distance criterion. In other words, a state

vector U!

ði; jÞ is represented by color v̂k if and only if

jjU!

ði; jÞ � v̂kjjpjjU!

ði; jÞ � v̂jjj for all j ¼ 1; 2 . . .Nc.

Once the best-matched vector is selected from thecolor palette, its index is recorded and the quantiza-

tion error E!

ði; jÞ ¼ v̂k �U!

ði; jÞ is diffused to pixel

(i, j)’s neighborhood as described in Eq. (1). Note

that, to handle the boundary pixels, E!

ði; jÞ is defined

to be zero when (i, j) falls outside the image.Without loss of generality, in this paper, we use atypical Floyd–Steinberg error diffusion kernel asHðk;lÞc to perform halftoning. The recorded indices

will be used in the future to reconstruct the color-quantized image with the same color palette.

ARTICLE IN PRESSY.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292282

3. Formulation of restoration algorithm

Let S be the output image of the restoration.Obviously, when the restored image S is color-quantized with error diffusion, the output should beequal to Y. In formulation, we should have

Y ¼ Qch½S�, (4)

where Qch½�� denotes the operator which performscolor quantization with halftoning as shown inFig. 1. Based on this criterion, the cost function of arestored image is defined as

E ¼ S Y�Qch½S��

, (5)

where S½I� denotes the total number of nonzeroelements in image I. Note that E ¼ 0 implies thatboth the restoration result S and the original imageX provide the same color quantization results.Without additional priori knowledge about X, costfunction (5) would be a good measure to judge if Sis a good estimate of X based on the fact thatY ¼ Qch½X�.

Since an image usually contains a number ofpixels and color quantization is a non-linearoperation, finding an image to minimize (5) is alarge combinational optimization problem. Exhaus-tive search is impractical and it is easy to be tappedin a local minimum when a gradient-orientedoptimization algorithm is exploited. In our ap-proach, S is searched with SA to minimize costfunction E. Without loss of generality, SA is adouble-loop iterative algorithm that simulates anannealing process at a given temperature T. DuringSA, temperature T is reduced in a controlledmanner as given by

Tkþ1 ¼ aTk, (6)

where Tk is the temperature at stage k and a is aconstant used to achieve cooling. At a particulartemperature Tk, the amount of time spent inannealing is gradually adjusted by

Mkþ1 ¼ bMk, (7)

where Mk is the number of iterations performed attemperature Tk and b is a constant used to do theadjustment. The algorithm is terminated at tem-perature Tm when

Pmk¼0Mk is larger than a

predefined threshold tmax. Here, we assume thatthe SA process starts at its initial stage k ¼ 0.

Let Scur be the current estimate of the restoredimage at a particular iteration at temperature Tk

and Ecur be its corresponding cost. The new estimate

of the restored image is given by

Snew ¼ Scur þ g Y�Qch Scur½ ��

, (8)

where g is a controlling parameter used to controlthe amount of perturbation applied to the Scur.

The cost of Snew, say, Enew, is then evaluated withEq. (5). When EnewoEcur happens, Scur is updatedto be Snew. Furthermore, if EnewoEbest happens,where Ebest is the cost of the best estimate sofar (Sbest), then Sbest will be replaced by Snew.In formulation, we have

Scur ¼Snew if EnewoEcur;

Scur otherwise

((9)

and

Sbest ¼Snew if EnewoEbest;

Sbest otherwise:

((10)

When EnewXEcur happens, Scur will be updated to beSnew only if roeðEcur�EnewÞ=KBT , where r is a randomlygenerated value which is uniformly distributedbetween 0 and 1, T denotes the current temperatureand KB is the Boltzmann constant. This criterion foraccepting the new solution is known as the Metro-polis criterion. At the beginning, temperature T ishigh. This permits many uphill moves and provideschances for the solution to leave a local minimum. Astemperature T is reduced gradually, fewer and feweruphill moves are permitted and only downhill movesare allowed eventually.

4. Performance evaluation and comparative study

Simulation has been carried out to evaluate theperformance of the proposed algorithm. In oursimulation, a number of de facto standard 24-bitfull-color images of size 256� 256 each were used.These test images were color-quantized to produceY’s. Color palettes of different size were used forquantization and they were generated with differentpalette generation algorithms such as median-cutalgorithm [3] and octree algorithm [20]. In colorquantization, halftoning was performed with errordiffusion and Floyd–Steinberg diffusion filter [2] wasused. The proposed restoration algorithm was usedto restore the halftoned color-quantized images (Y’s).

In the realization of the proposed algorithm, boththe initial estimate of S and Sbestwere initialized tobe the filtered output of the observed image Y.Specifically, a 3� 3 Gaussian filter was used togenerate the initial estimate of S and Sbest. Initial

ARTICLE IN PRESS

Fig. 2. SNR Improvements achieved with various combinations

of g and M in restoring halftoned color-quantized (a) ‘‘Lenna’’,

(b) ‘‘Peppers’’ and (c) ‘‘Couple’’.


temperture T0 was selected to be jES0� ES00 j=

ðKB logeð0:95ÞÞ, where ES0and ES00 were, respec-

tively, the cost of S0 and S00. Here, S0 denotes theinitial estimate of S and S00 is the first estimateobtained with Eq. (8) based on S0. In formulation,we have S00 ¼ S0 þ g Y�Qch½S0�

� �. This allows

reasonable amount of uphill move at the beginning.Parameter a was selected to be 0.9, the middle valueof the selection range suggested in [15]. To simplifythe algorithm, we selected b to be 1 such that we hadMk ¼M for all kX0.

Different combinations of M and g were evalu-ated to study their impact to the restorationperformance. Fig. 2 shows some typical results ofthe study. Specifically, it shows the Signal-to-NoiseRatio Improvement (SNRI) that was achieved whenthe proposed algorithm was used to restore color-quantized ‘‘Lenna’’, ‘‘Peppers’’ and ‘‘Couple’’. Thecolor quantization was realized with a 128-colorpalette generated with median-cut algorithm [3].Here, SNRI is defined as

SNRI ¼ 10 log

Pði; jÞjjX

!

ði; jÞ � Y!

ði; jÞjj2

Pði; jÞjjX

!

ði; jÞ � S!

bestði; jÞjj2, (11)

where X!

ði; jÞ, Y!

ði; jÞ and S!

bestði; jÞ are, respectively, the(i, j)th pixels of the original, the halftoned color-quantized and the restored images. In Fig. 2, thebroken lines correspond to the cases of M ¼ 10while the solid lines correspond to the cases ofM ¼ 50 and 100. Note the lines for M ¼ 50 and 100overlap with each other for any particular g. Forany g the value of which is larger than 0.002, it isdifficult to discriminate the line for M ¼ 10 and thelines for M ¼ 50 and 100. It was also found that,when MX50, the smaller the value of g, the moreiterations were required for the estimate to convergebut the better the converged output could beachieved in terms of SNRI.

Based on the evaluation results of the study, itcan be found that, the combination of M ¼ 50 andg ¼ 0:002 is a reasonable choice for restoringhalftoned color-quantized images. With this combi-nation, the best SNRI performance can be achievedin around 1000 iterations. It is good enough for theestimates of the iterative algorithm to converge toits restoration result. If complexity is a criticalconcern, one may select a combination of M ¼ 50and g ¼ 0:05, which can achieve a SNRI close to theone obtained with the previous selection in lessthan 100 iterations. In this paper, the presented

ARTICLE IN PRESS

Fig. 3. Training images used to realize Mese’s algorithm [17].

Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292284

results of the proposed algorithm were ob-tained with a ¼ 0:9, b ¼ 1, T0 ¼ jES0

� ES00 j=ðKB logeð0:95ÞÞ, tmax ¼ 1000, M ¼ 50 and g ¼ 0:002.

Some other restoration algorithms were alsoevaluated for comparison. Among them, Galatsa-nos’s algorithm [9] is based on the constrained leastsquare approach and Hunt’s algorithm [10] is basedon Wiener filtering. There are two algorithmspresented in Altunbasak’s work [11]. The first onemakes use of the correlation among the colorcomponents of a pixel while the other one doesnot. They are, respectively, referred to as KL andIND in [11]. All the aforementioned algorithmswere originally proposed for handling noisy blurredcolor images. Fung’s algorithm [16] adopts a POCS-based approach. Unlike [9–11], it was proposed forrestoring color-quantized images. However, in [16],it is assumed that there is no halftoning involved incolor quantization. Mese’s algorithm [17] was

originally proposed for inverse halftoning. It makesuse of some training images to pre-train a linearprediction filter for filtering binary halftones. It wasmodified here to handle halftoned color-quantizedimages.

In realizing Galatsanos’s algorithm [9], the noisepower of each channel was estimated with theoriginal full-color image. In realizing Hunt’s algo-rithm [10], three separate Wiener filters were used inthree different channels and, during the design of thefilters, the noise spectrum of each channel wasestimated with the original full-color image. Simi-larly, the original full-image was used to estimate thepower spectra of different channels in realizingAltunbasak’s algorithms [11]. In a practical situa-tion, no original image is available and hence allmust be estimated from the degraded image. In otherwords, in practice, the restoration results of [9–11]may not be as good as those presented in this paper.

ARTICLE IN PRESS

Table 1

SNR improvements of various algorithms in restoring halftoned color-quantized images

SNR improvement (dB)

Proposed Fung

[16]

Altunbasak-IND

[11]

Altunbasak-KL

[11]

Mese

[17]

Galatsanos

[9]

Hunt

[10]

Palette size: 256

Lenna 7.2292 5.4038 4.0755 3.9394 4.1646 1.6259 3.1495

Baboon 2.7570 2.6836 1.9198 1.7546 2.5360 0.5088 1.2670

Boat 3.9082 3.4746 3.2372 3.1287 3.4953 0.8375 2.0733

Peppers 8.5542 5.0088 5.3847 5.2902 4.1263 1.1409 3.8154

Fruits 7.9054 4.4207 4.0514 3.6858 3.9884 1.4248 3.0521

Couple 5.9121 4.6150 4.0953 3.9070 3.7473 2.1920 2.8439

Girl 6.8524 4.4647 4.4029 4.2232 3.8023 2.0865 2.8951

Average 6.1598 4.2959 3.8810 3.7041 3.6943 1.4023 2.7280

Palette size: 128

Lenna 8.5110 5.8254 4.9392 4.5433 5.7123 2.4357 3.8346

Baboon 4.3958 3.7212 2.6872 2.4976 3.6064 0.8323 1.9535

Boat 5.6739 3.7710 4.0782 3.8174 4.9277 1.4246 2.7164

Peppers 10.2060 6.1409 6.2363 6.1193 5.6587 1.7904 4.5946

Fruits 10.1360 3.8925 5.1414 4.6937 5.6356 2.2694 3.8296

Couple 7.6694 5.2696 4.9311 4.5735 5.3854 3.4066 3.8429

Girl 8.6239 4.4373 5.3435 5.0748 5.1745 3.2783 3.8128

Average 7.8880 4.7226 4.7653 4.4742 5.1572 2.2053 3.5121

Palette size: 64

Lenna 9.8515 6.0192 5.6204 5.2254 7.0742 3.2720 4.5439

Baboon 5.6862 4.2475 3.4326 3.1365 4.7706 0.9225 2.9029

Boat 6.6932 4.4319 4.8928 4.5457 6.0329 2.3850 3.4323

Peppers 11.2850 5.6258 6.7896 6.6543 7.0382 2.5847 5.2137

Fruits 11.0170 3.9601 5.6140 5.1302 6.8376 2.9100 4.3907

Couple 9.8891 4.7026 5.8604 5.4856 7.0296 4.9702 4.5849

Girl 10.0790 4.9431 5.9710 5.7091 6.4191 4.1484 4.5656

Average 9.2144 4.8472 5.4544 5.1267 6.4575 3.0275 4.2334

Palette size: 32

Lenna 11.7410 5.8347 5.9938 5.7719 8.9195 5.1395 4.9865

Baboon 5.8750 3.5332 3.9324 3.5293 5.5499 1.2983 3.1871

Boat 8.3006 4.4603 5.7094 5.3401 7.6022 3.3472 4.0502

Peppers 10.9850 4.7372 6.6413 6.5218 7.9197 3.4330 5.2593

Fruits 11.9420 3.0951 5.9704 5.6623 8.4185 4.2259 4.9070

Couple 9.6600 4.8138 5.5600 5.2710 8.1417 6.2369 4.7835

Girl 10.5450 4.7718 6.2691 6.0062 7.3840 5.6861 5.0606

Average 9.8641 4.4637 5.7252 5.4432 7.7051 4.1953 4.6049

Palette size: 16

Lenna 12.1150 6.4483 6.4987 6.3982 10.059 4.6921 5.2783

Baboon 7.8493 3.3234 4.9719 4.5367 7.1757 1.2976 4.3003

Boat 9.6222 4.2405 5.7226 5.5335 8.3951 2.5125 4.4839

Peppers 7.5616 3.5723 4.9149 4.8189 6.8783 1.7858 4.1941

Fruits 5.5229 2.3384 3.0841 2.9029 4.8328 2.6197 2.8355

Couple 10.7470 4.1525 5.8154 5.7268 9.2980 6.6020 4.9114

Girl 8.5940 3.3463 5.4382 5.3473 7.0026 4.4366 4.6745

Average 8.8589 3.9174 5.2065 5.0378 7.6631 3.4209 4.3826


ARTICLE IN PRESS

Table 2

CIELAB difference DE measurement of the outputs of various algorithms in restoring halftoned color-quantized images

Average of CIELAB difference DE

Observed Y Proposed Fung

[16]

Altunbasak-IND

[11]

Altunbasak-KL

[11]

Mese

[17]

Galatsanos

[9]

Hunt

[10]

Palette size: 256

Lenna 3.2825 2.0043 2.2942 2.4992 2.5408 2.6592 2.9921 2.6074

Baboon 4.5157 3.4646 3.7660 3.8570 3.8893 3.8170 4.4015 4.0216

Boat 4.3246 2.9612 3.3129 3.4187 3.4453 3.5574 4.1808 3.6966

Peppers 4.8757 3.0295 3.7024 3.9070 3.9413 4.0404 4.7539 4.2931

Fruits 3.4039 2.1027 2.6086 2.6524 2.7085 2.8116 3.2263 2.8190

Couple 9.3832 4.7604 5.5639 6.8130 6.9349 7.5452 7.7575 7.0757

Girl 7.1182 3.9498 4.8083 5.1771 5.2650 5.6876 6.6614 5.7108

Average 5.2720 3.1818 3.7223 4.0463 4.1036 4.3026 4.8534 4.3177

Palette size: 128

Lenna 3.9581 2.3189 2.7375 2.8777 3.0124 2.9825 3.4767 3.0619

Baboon 5.7313 3.9406 4.3301 4.5950 4.6499 4.4859 5.4871 4.7857

Boat 5.1483 3.3552 4.0071 3.9664 4.0272 3.9847 4.8933 4.3653

Peppers 6.2261 3.7152 4.5155 4.7408 4.8001 4.8953 5.8101 5.2532

Fruits 4.4127 2.5316 3.4507 3.2907 3.3936 3.3813 4.0053 3.5722

Couple 10.0050 5.0681 6.3255 7.3146 7.5623 7.4200 7.8051 7.7401

Girl 8.4566 4.5556 6.2587 6.0465 6.2002 6.3329 7.5471 6.8269

Average 6.2769 3.6407 4.5179 4.6902 4.8065 4.7832 5.5750 5.0865

Palette size: 64

Lenna 4.6383 2.7927 3.3420 3.3728 3.5928 3.3759 4.0210 3.6044

Baboon 7.2079 4.5986 5.3470 5.4557 5.5802 5.2686 6.8561 5.6664

Boat 6.3857 3.9120 4.8146 4.6909 4.7850 4.6646 5.7469 5.2097

Peppers 7.2352 4.2868 5.3879 5.5308 5.6176 5.4084 6.5681 6.0995

Fruits 5.4861 3.1426 4.3304 4.0054 4.1481 4.0384 4.8940 4.3324

Couple 11.8820 5.6279 8.3154 8.5876 9.0331 8.0450 8.4878 9.3496

Girl 11.1570 5.6124 7.8426 7.4886 7.7466 7.7247 9.4510 8.5264

Average 7.7132 4.2819 5.6257 5.5903 5.7862 5.5037 6.5750 6.1126

Palette size: 32

Lenna 5.7174 3.2258 4.2656 4.1548 4.3888 3.8653 4.5761 4.4620

Baboon 8.1388 5.5644 6.4859 6.1560 6.3694 6.0041 7.7539 6.4359

Boat 8.0848 4.6100 6.0734 5.6439 5.8113 5.4462 7.0438 6.1993

Peppers 10.1800 6.3679 8.0440 7.7957 7.9312 7.5899 8.9396 8.4388

Fruits 7.5845 4.1855 6.2373 5.3853 5.5915 5.2303 6.4358 5.8271

Couple 10.9610 6.3518 7.8506 8.6510 9.1970 7.5988 8.5580 8.9274

Girl 13.3290 6.8493 9.4531 8.9461 9.3961 8.8682 10.728 10.122

Average 9.1422 5.3078 6.9157 6.6761 6.9550 6.3718 7.7193 7.2018

Palette size: 16

Lenna 7.3126 3.7922 5.1302 5.0374 5.1990 4.4485 5.9601 5.4103

Baboon 10.0690 6.6696 8.5034 7.4306 7.7479 7.1575 9.7593 7.7710

Boat 9.8613 6.3887 7.8429 7.4625 7.6573 7.0723 9.9600 8.0856

Peppers 13.2580 9.8104 11.2280 10.6720 10.7180 10.3110 12.4990 11.3620

Fruits 10.8120 8.4798 9.6897 8.9921 9.0418 8.9801 10.1900 9.1836

Couple 13.0560 7.5965 10.4450 10.4730 11.0520 8.6742 10.8320 10.5120

Girl 14.2730 9.5449 11.8480 10.6720 11.0010 10.6960 12.6920 11.3500

Average 11.2350 7.4689 9.2410 8.6771 8.9167 8.1914 10.2700 9.0964


ARTICLE IN PRESS

Table 3

Percentage of pixels whose CIELAB difference is not detectable (DEo3) after restoration

% of pixels whose CIELAB DE o3

Observed Y Proposed Fung

[16]

Altunbasak-IND

[11]

Altunbasak-KL

[11]

Mese

[17]

Galatsanos

[9]

Hunt

[10]

Palette size: 256

Lenna 53.71 81.50 74.58 69.98 69.05 65.76 58.15 68.02

Baboon 36.55 51.01 45.92 43.96 43.44 44.90 37.07 42.19

Boat 45.43 60.81 56.17 55.08 54.75 53.26 46.84 52.29

Peppers 37.36 68.47 54.90 53.51 52.94 47.67 38.28 45.78

Fruits 63.48 81.08 73.81 73.60 72.72 70.84 65.67 71.34

Couple 14.79 35.54 29.21 24.33 23.78 21.01 19.26 20.65

Girl 21.72 45.58 36.69 33.02 32.30 29.14 26.45 28.25

Average 39.01 60.57 53.04 50.50 49.85 47.51 41.67 46.93

Palette size: 128

Lenna 41.76 76.32 65.04 62.18 59.46 58.85 48.48 58.32

Baboon 24.37 42.48 37.18 32.81 32.29 34.77 25.58 31.75

Boat 37.33 56.05 48.12 48.33 47.57 47.86 39.72 44.61

Peppers 26.59 60.97 46.38 43.92 43.11 39.97 30.13 35.93

Fruits 50.42 76.20 63.20 64.79 63.17 63.38 55.27 60.46

Couple 8.57 30.74 22.22 17.38 16.41 16.57 14.51 14.60

Girl 13.39 37.71 24.62 24.72 23.72 22.40 19.35 19.81

Average 28.92 54.35 43.82 42.02 40.82 40.54 33.29 37.93

Palette size: 64

Lenna 31.41 69.27 53.55 51.52 48.11 50.32 38.41 47.00

Baboon 15.15 34.89 27.17 23.50 22.89 26.36 16.40 22.90

Boat 29.25 50.01 40.69 41.18 40.22 41.50 34.55 37.03

Peppers 16.78 52.18 34.07 33.82 32.74 32.23 20.72 26.77

Fruits 38.88 70.80 53.87 56.28 54.02 56.94 43.95 51.01

Couple 3.89 24.28 10.10 10.10 9.21 10.55 8.58 7.78

Girl 4.95 30.32 13.95 15.26 14.52 13.44 9.60 12.13

Average 20.04 47.39 33.34 33.10 31.67 33.05 24.60 29.23

Palette size: 32

Lenna 19.26 60.10 39.65 38.51 35.06 41.94 28.42 32.62

Baboon 12.30 26.16 19.00 18.79 17.66 21.11 14.06 17.24

Boat 23.37 44.30 33.06 34.89 33.80 38.34 29.65 30.28

Peppers 8.51 37.56 20.04 21.42 20.30 21.88 11.90 16.11

Fruits 24.26 58.35 35.00 39.43 37.80 43.74 31.28 33.97

Couple 4.72 20.69 11.92 8.89 7.86 12.34 9.04 7.46

Girl 3.51 24.07 9.64 11.03 10.01 10.82 6.81 8.60

Average 13.70 38.75 24.04 24.71 23.21 27.17 18.74 20.90

Palette size: 16

Lenna 9.73 48.90 27.62 27.74 25.36 33.06 15.97 22.93

Baboon 6.09 18.78 9.60 11.25 10.45 14.62 6.83 10.58

Boat 14.68 26.87 20.25 18.78 18.77 22.15 18.02 17.05

Peppers 3.03 10.02 5.84 7.11 7.15 7.67 3.84 6.02

Fruits 15.16 40.91 26.71 31.59 30.84 30.76 18.38 29.37

Couple 1.96 12.56 4.53 4.86 4.31 7.54 4.95 4.30

Girl 4.40 16.51 8.72 9.60 9.23 10.92 7.38 8.05

Average 7.86 24.94 14.75 15.85 15.16 18.10 10.77 14.04


ARTICLE IN PRESS

Fig. 4. Restoration results of color-quantized ‘‘Fruits’’ (palette was generated with median-cut algorithm [3] and is of size 128).


Mese’s algorithm trains the linear prediction filterwith best linear estimator [17]. In the training phase,the four standard images shown in Fig. 3 and theircorresponding halftoned color-quantized outputswere used to train the filter. As for Fung’s algorithm[16] and the proposed algorithm, no original full-color image is required to extract information andno training images are required to pre-train a linearprediction filter. Hence, additional credit should beadded to the simulation results of these twoalgorithms.

Table 1 shows the SNRI performance achieved byvarious algorithms when the involved palettes wereobtained with median-cut algorithm [3]. One can seethat the proposed algorithm is superior to any otheralgorithms whatever color palette size is concerned.On average, the proposed algorithm achieved,respectively, a SNRI of 6.16, 7.89, 9.21, 9.86 and

8.86dB when the involved palette is of size 256, 128,64, 32 and 16.

Tables 2 and 3 show the performance of theevaluated algorithms in terms of the CIELAB colordifference ðDEÞ metric. The CIELAB color differenceðDEÞ metric is defined as the Euclidean distancebetween the original color of a pixel and itsreproduction in CIELAB color metric space [18].It is well accepted that color error is visuallydetectable when DE43 [11,19]. Table 2 shows theaverage of the DE values of all pixels in a restorationoutput and Table 3 shows the percentage of pixelswhose color error is visually undetectable in arestoration output. Again, one can see that theproposed algorithm is superior to the others.

Figs. 4 and 5 show the restoration results ofdifferent algorithms for visual evaluation. Figs. 4band 5b were, respectively, obtained with palettes of

ARTICLE IN PRESS

Fig. 4. (Continued)


sizes 128 and 16. Though halftoning can removefalse contour and color shift to a certain extent,these artifacts still appear in the color-quantizedoutputs as the palette size is too small. Besides,

pepper noise was introduced by halftoning. Afterrestoration, one can see that the proposed algorithmcan remove most of the artifacts while the others arecomparatively inferior in this aspect.

ARTICLE IN PRESS

Fig. 5. Restoration results of color-quantized ‘‘Lenna’’ (palette was generated with median-cut algorithm [3] and is of size 16).


Table 4 shows the average performance ofvarious algorithms when the palettes were generatedwith octree algorithm [20]. The same set of standardtest images used to obtain the results presented inTables 1–3 were used for evaluation. The figures inthe table are the average values obtained with therestoration results of the color-quantized images.One can see that the proposed algorithm is stillsuperior to the others. This simulation result showsthat the proposed algorithm works with differentpalettes generated with different palette generationalgorithms.

5. Conclusions

Very little research has been carried out toaddress the restoration of halftoned color-quantizedimages. Although there are some restoration algo-rithms for restoring blurred and noisy color images,

they are generally not adequate to handle halftonedcolor-quantized images. The noise introduced bycolor quantization and halftoning is basically signaldependent and is not white, which violates theassumptions adopted in most current multichannelrestoration algorithms.

In this paper, we proposed a dedicated restorationalgorithm for restoring halftoned color-quantizedimages. This algorithm makes a good use of theavailable color palette and the halftoning process toderive useful a priori information for restoration.Simulation results and comparative study demon-strates that the proposed algorithm can achieve aremarkable improvement in the quality of a half-toned color-quantized image in terms of both SNRand CIELAB color difference DE metric. Theproposed algorithm can remove most of the artifactsintroduced by color quantization and halftoning andhence can improve the restoration result subjectively.

ARTICLE IN PRESS

Fig. 5. (Continued)


Acknowledgements

This work was supported by a grant fromthe Research Grants Council of the Hong Kong

Special Adminstrative Region, China (PolyU 5217/03E) and a grant from Centre for MultimediaSignal Processing, The Hong Kong PolytechnicUniversity.

ARTICLE IN PRESS

Table 4

Average performance of various algorithms in restoring halftoned color-quantized images in various aspects (palette was generated with

octree algorithm [20])

Palette size Observed (Y) Restored (X0)

Proposed Fung

[16]

Altunbasak-IND

[11]

Altunbasak-KL

[11]

Mese

[17]

Galatsanos

[9]

Hunt

[10]

(a) Average of (SNR improvement (dB))

256 — 6.5653 4.0159 4.0097 3.6977 3.7508 1.5941 2.8789

128 — 7.3868 4.0430 4.7119 4.3549 4.8697 2.2585 3.5493

64 — 8.8699 3.6709 5.4454 5.0258 6.6426 2.9054 4.2723

32 — 8.7839 3.0231 5.4257 5.0174 7.5855 3.4035 4.4434

16 — 6.1545 2.1742 4.0367 3.7613 5.8446 3.2980 3.5478

(b) Average of (average of CIELAB difference DE)

256 4.7305 3.3756 3.8534 3.9139 3.9739 4.0520 4.4882 4.2346

128 5.8678 4.1237 4.8119 4.7375 4.8371 4.8130 5.4207 5.1096

64 7.3602 5.1048 6.2701 5.8456 5.9957 5.7979 6.7306 6.2729

32 9.3714 6.5337 8.1267 7.3447 7.5776 7.1194 8.5231 7.8172

16 11.6090 8.9606 10.5540 9.4940 9.7033 9.3656 10.6690 9.8516

(c) Average of (% of pixels whose CIELAB DEo3)

256 39.46 57.95 50.75 49.68 48.88 47.06 41.56 46.55

128 29.91 49.97 41.08 40.93 39.96 40.02 33.10 37.40

64 22.77 42.94 31.53 32.42 31.44 33.60 25.94 28.72

32 12.79 32.06 19.04 22.15 20.99 24.54 15.60 18.67

16 7.72 19.93 11.63 15.43 14.50 15.68 9.79 13.28


References

[1] M.T. Orchard, C.A. Bouman, Color quantization of images,

IEEE Trans. Signal Process. 39 (12) (1991) 2677–2690.

[2] R. Ulichney, Digital Halftoning, MIT Press, Cambridge,

MA, 1987.

[3] P. Heckbert, Color image quantization for frame buffer

displays, Comput. Graph. 16 (4) (1982) 297–307.

[4] R.S. Gentile, E. Walowit, J.P. Allebach, Quantization and

multi-level halftoning of color images for near original image

quality, Proc. SPIE 1249 (1990) 249–259.

[5] S.S. Dixit, Quantization of color images for display/printing

on limited color output devices, Comput. Graph. 15 (4)

(1991) 561–567.

[6] X. Wu, Color quantization by dynamic programming and

principal analysis, ACM Trans. Graph. 11 (4) (1992)

372–384.

[7] M. Barni, V. Cappellini, L. Mirri, Multichannel m-filtering

for color image restoration, ICIP’2000, Vancouver, Canada,

September 2000, pp. 529–532.

[8] G. Angelopoulosnd I. Pitas, Multichannel wiener filters in

color image restoration, IEEE Trans. CASVT 4 (1) (1994)

83–87.

[9] N.P. Galatsanos, A.K. Katsaggelos, R.T. Chin, A.D.

Hillery, Least squares restoration of multichannel images,

IEEE Trans. Signal Process. 39 (10) (1991) 2222–2236.

[10] B.R. Hunt, O. Kubler, Karhunen–Loeve multispectral image

restoration, Part 1: theory, IEEE Trans. Acoust. Speech

Signal Process. 32 (3) (1984) 592–600.

[11] H. Altunbasak, H.J. Trussell, Colorimetric restoration of

digital images, IEEE Trans. Image Process. 10 (3) (2001)

393–402.

[12] K.J. Boo, N.K. Bose, Multispectral image restoration with

multisensor, IEEE Trans. Geosci. Remote Sensing 35 (5)

(1997) 1160–1170.

[13] N.P. Galatsanos, R.T. Chin, Digital restoration of multi-

channel images, IEEE Trans. Acoust. Speech Signal Process.

37 (1989) 415–421.

[14] N.P. Galatsanos, R.T. Chin, Restoration of color images by

multichannel Kalman filtering, IEEE Trans. Signal Process.

39 (10) (1991) 2237–2252.

[15] S. Kirkpatrick, C.D. Gelatt, M.P. Vecchi, Optimation by

Simulated Annealing, Science 220 (1983) 671–680.

[16] Y.H. Fung, Y.H. Chan, POCS-based algorithm for restoring

colour-quantised images, IEE Proc.—Vision Image Signal

Process. 151 (2) (2004) 119–127.

[17] M. Mese, P.P. Vaidyanathan, Look-Up Table (LUT)

method for inverse halftoning, IEEE Trans. Image Process.

10 (10) (2001) 1566–1578.

[18] C.I.E., Recommendations on uniform color spaces, color

difference equations, psychometric color terms, Supplement

No. 2 to CIE publication No. 15 (E.-1.3.1) 1971/(TC-1.3.), 1978.

[19] C. Connolly, T.W.W. Leung, J. Nobbs, Colour measure-

ment by video camera, J. Soc. Dyers Colour 111 (1995)

373–375.

[20] M. Gervautz, W. Purgathofer, A simple method for color

quantization: Octree quantization, Graphics Gems, Aca-

demic Press, New York, 1990, pp. 287–293.

Documents

A simulated annealing restoration algorithm for restoring