Contrast Enhancement for Low-light Image Enhancement: A … Park et al.: Contrast Enhancement for Low-light Image Enhancement: A Survey 38 images in the sense of intensity value by

IEIE Transactions on Smart Processing and Computing, vol. 7, no. 1, February 2018 https://doi.org/10.5573/IEIESPC.2018.7.1.036 36

IEIE Transactions on Smart Processing and Computing

Contrast Enhancement for Low-light Image Enhancement: A Survey

Seonhee Park1, Kiyeon Kim2, Soohwan Yu1, and Joonki Paik1*

1 Image Processing and Intelligent Systems Laboratory, Graduate School of Advanced Imaging Science, Multimedia, and Film, Chung-Ang University / Seoul, 06974, Korea tjsgml1463, shyu, [email protected] 2 Department of Integrative Engineering, Chung-Ang University / Seoul, 06974, Korea [email protected]

* Corresponding Author: Joonki Paik, [email protected]

Received October 27, 2017; Accepted December 8, 2017; Published February 28, 2018

* Regular Paper

* Review Paper: This paper reviews the recent progress, possibly including previous works in a particular research topic, and has been accepted by the editorial board through the regular reviewing process.

Abstract: In this paper, various contrast and low-light image enhancement methods are described and classified into three categories: i) histogram-based, ii) transmission map–based, and iii) retinex-based. The performance of the image enhancement algorithms is evaluated by comparing the resulting images using low-contrast and low-light images acquired under different illumination conditions. Various image enhancement algorithms are analyzed for both performance and efficiency. The described image enhancement algorithms can be applied to various visual surveillance or video analytics applications with low-light, low-contrast video input.

Keywords: Contrast Enhancement, Low-Light Image, Histogram Equalization, Retinex, Transmission Map 1. Introduction

Digital images acquired by digital imaging sensors have been widely used in various fields, such as surveillance systems, remote sensing, and advanced driver assistance systems. However, the quality of the observed images is degraded degrades under low-light conditions because of limited sensitivity in the imaging sensors. To restore a low-light image, the input low-intensity values can be simply mapped to higher intensity values using gamma or logarithm transformations [1, 2]. Although these methods improve the brightness of a low-light input image using a simple, closed formula, it compresses the range of output intensity and provides a resulting image with a low dynamic range. On the other hand, histogram-based methods were proposed to redistribute the histogram bins using a cumulative distribution function (CDF) of the input image [3-13]. However, histogram-based methods provide a low-quality image because of over-saturation in bright regions.

Transmission map–based methods were developed based on the observation that an inverted low-light image looks visually similar to a hazy image [14-20]. These methods invert the low-light image and enhance brightness

by removing haze-like components using a degradation model of the hazy image. The dehazed image is then inverted to finally obtain an enhanced image. However, it cannot avoid color distortion, over-enhancement, and noise amplification due to an inaccurately estimated transmission map and atmospheric light. On the other hand,

Table 1. Abbreviations.

HE Histogram equalization BBHE Brightness preserving bi-histogram equalizationDSIHE Dualistic sub-image histogram equalization

MMBEBHE Minimum mean brightness error bi-histogram equalization

RMSHE Recursive mean separate histogram equalizationAHE Adaptive histogram equalization

GCCHE Gain-controllable clipped histogram equalization

AGCWD Adaptive gamma correction with weighting distribution

SSR Single-scale retinex MSR Multi-scale retinex

MSRCR Multiscale retinex with color restoration

IEIE Transactions on Smart Processing and Computing, vol. 7, no. 1, February 2018

37

1l - and 2l -norm minimization–based variational models have been proposed to simultaneously enhance brightness and reduce amplified noise [21].

Other approaches to enhancing a low-light image, called retinex-based methods, were proposed [22, 23]. The input image, which can be formulated by a multiplication of reflectance and illumination components, can be enhanced by eliminating or reducing the illumination component. Since center/surround-based retinex algorithms generally estimate the illumination component using a Gaussian low-pass filter, the resulting image exhibits low contrast with a halo effect near the edge [24-26]. To solve this problem, the 2l -norm–based variational retinex model was proposed [27-29]. Although these methods estimate the optimal illumination component by minimizing the energy functional, they cannot suppress noise amplification without blurring artifacts near the edges. To solve this problem, the 1l -norm–based variational retinex model was proposed [30-33]. Since this method estimates the reflectance component using 1l -norm minimization, it can perform noise reduction without blurring artifacts near the edges [32].

This paper is organized as follows. Section 2 describes the low-light image enhancement methods in the literature. Section 3 shows experimental results from both objective and subjective assessments, and Section 4 concludes the paper.

2. Contrast Enhancement Methods

2.1 Basic Image Enhancement Methods Gamma correction is a simple brightness enhancement

method using a non-linear intensity transfer function to restore a low-contrast image [1]. The enhanced image can be obtained as follows:

1G

gfL

⎛ ⎞= ⎜ ⎟−⎝ ⎠

γ

(1)

where Gf represents the enhanced image, g is the input low-contrast image, L is the intensity range of the input image, and γ is the adjusting parameter. The dynamic range of the input image can be extended and compressed according to the adjusting parameter. Similarly, the input intensity values can be enhanced using a logarithmic transformation: ( )loˆ g 1Lf gμ= + (2)

where Lf represents the enhanced image, and μ is a constant value. These methods improve the contrast by transforming the intensity value at each pixel.

Fig. 1 shows intensity transfer functions in (1) and (2) with four different adjusting parameters. As shown in Fig. 1, when the adjusting parameter is lower than 1, all

the intensity values are mapped to higher ones, and the dynamic range of a dark region is extended.

On the other hand, the histogram-based methods enhance a low-contrast image using the CDF. The histogram is defined as

( ) , for= = … −knp k k L

N (3)

where N represents the total number of pixels in the input image, and kn is the number of pixels with gray level k . The CDF can be obtained by integrating (3) as follows:

( ) ( )0=

= ∑k

j

c k p j (4)

where ( )⋅c represents the CDF. Histogram equalization (HE) enhances the contrast using the transfer function, which is generated as ( ) ( )( )max min min= − +T k c k g g g (5)

where ( )⋅T represents the transfer function, and maxg and

ming are the maximum and minimum intensity values, respectively, of the input image. The contrast of an image can be improved by uniformly distributing the histogram bins using the transfer function. However, when the histogram bins are concentrated in a dark or bright area, the performance of HE is limited because of saturation and color distortion.

2.2 Histogram-based Image Enhancement Methods

In a low-contrast image, the histogram is concentrated near the mean brightness value, and it results in a drastic change in the intensity transfer function. For that reason, HE cannot preserve the mean brightness value of an input image and the details, because of under- and over-enhancement in the dark and bright regions. To solve this problem, Kim proposed a contrast enhancement method performing histogram equalization in two sub-histograms [3]. This method divides the input image into two sub-

Fig. 1. Various intensity transfer functions.

Park et al.: Contrast Enhancement for Low-light Image Enhancement: A Survey

38

images in the sense of intensity value by the mean brightness value of the input image, and separately redistributes the histogram bins of each sub-image. The sub-images are defined as

| = ≤l

k k gg g g m (6)

and h

k k gg g g m= (7)

where gm is the mean brightness value of the input image, lg and hg are the sub-images that respectively include

lower and higher intensity values than gm . The CDFs of each sub-image are obtained as follows. For 0, , gk m= …

( ) ( )0

k

l l

j

c k p j=

= ∑ (8)

and for 1, , 1,= + … −gk m L

( ) ( )1g

kh h

j m

c k p j= +

= ∑ (9)

where lp and hp represent the probability distribution functions (PDF) of lg and hg , respectively, and lc and

hc the CDFs of lp and hp , respectively. The input low-contrast image can be enhanced via intensity transformation using the estimated CDFs. Although this method can improve contrast while preserving the mean brightness value of an input image, it cannot avoid saturation in a bright region.

Wang et al. showed that the entropy of an image is maximized using a uniform PDF [4]. They proposed the dualistic sub-image segmentation method based on entropy, and adjusted the HE into two divided sub-images (DSIHE). Brightness preserving bi-histogram equalization (BBHE) divides an input image into two sub-images according to the mean brightness value. On the other hand, this method separates an input image into two sub-images when the Shannon entropy of an input image is maximized. The Shannon entropy is defined as

( )( ) ( ) ( )

( )( ) ( )( )log

1 log 1

= −

− − −

ES c k c k c k

c k c k (10)

where ES represents the segmented entropy. ES is maximized when CDF ( )c k is 0.5 , which implies that the divided sub-images respectively represent the dark and bright regions having the same number of pixels. Although this method enhanced contrast, preserving both mean brightness value and image information, it cannot avoid saturation in a bright region.

Chen and Ramli proposed minimum mean brightness

error bi-histogram equalization (MMBEBHE), an efficient contrast enhancement method that iteratively reduces the mean brightness error between the input and contrast enhanced image using the BBHE [5]. To enhance contrast while preserving the mean brightness value, this method performed BBHE n times according to n threshold values and estimated the absolute mean brightness error (AMBE) between the mean brightness values of the input and the n -th enhanced images as follows: n n

m g fE m m= − (11)

where n represents the number of threshold values, nmE is

the n -th AMBE, and nfm is the n -th mean brightness

value of the enhanced image. Since this method estimates the optimal threshold value by minimizing the AMBE, it can provide a better enhanced result, preserving the mean brightness value of the input image. Meanwhile, since this method estimates the resulting image in an iterative manner, the resulting image is obtained at a high computational cost. Chen and Ramli also proposed recursive mean-separate histogram equalization (RMSHE) [6]. This method recursively separates the histogram according to the mean brightness value of each divided histogram. However, this method shows poor performance according to the number of separations.

Pizer proposed the adaptive histogram equalization (AHE) method by clipping the histogram bins higher than a threshold value and redistributing the clipped histogram bins in the local window [8]. The modified histogram by the clipped histogram is defined according to threshold P, as follows:

( ) ,if,if

+ <⎧= ⎨ ≥⎩

k k

C k

n U n PH k

l n P (12)

where ( )H k represents the modified histogram, U is the number of uniformly redistributed pixels, P is the clipping level, and Cl is the actual clipping level. The modified histogram is estimated by decreasing the number of pixels that have a higher distribution by clipping the histogram, and increasing the number of pixels with a lower distribution. The intensity of the input image is transformed using the CDF of the modified histogram. Zuiderveld proposed an improved version of Pizer’s method, called contrast limited AHE (CLAHE) [9]. This method reduces noise amplification by adaptively adjusting the clipping limits in a homogeneous region. Fig. 2 shows a comparison of the input and modified histograms using AHE.

To adjust the clipping level according to the input image, Kim and Paik used the pixel intensity value k when CDF ( )c k is 0.5 to estimate the clipping rate [10]. The clipping rate is computed as 0.5100 *0.4cr k= − (13)


39

where cr represents the clipping rate, and 0.5k is the intensity value when ( )c k is 0.5 . This method enhances the contrast of an image in global and local gain-controllable manners. The global gain represents the histograms that are uniformly added to the clipped histogram, defined as

( ) GG C

gH k H

L= + (14)

where CH represents the clipped histogram, Gg is the global gain, and ( )GH k is the enhanced histogram by the global gain. The local gain plays a role in preventing under- and over-enhancement in the dark and bright regions, respectively. The local gain of a dark region is modified as follows:

( )( )

( )

1

1

1 1

for

1 otherwise

⎧ ≥⎪⎪ ⎛ ⎞= −⎨

− +⎜ ⎟⎪ ⎜ ⎟⎪ ⎝ ⎠⎩

L

G L

T GG

L L

H k

H k k k

g g kH kk k

(15)

where ( )

1LH k represents the modified histogram of the dark region, Tg is the total gain representing the number of clipped histogram bins, and

1Lk is the threshold value of the dark region.

In the bright region, the histogram is modified as follows:

( )( )

( )( )( )

( )

2

1 2

2

1

2

2

, for

otherwise1

⎧ ≤⎪⎪ − −= ⎨ +⎪

− −⎪⎩

L

L L

T G L

L

L

H k

H k k k

g g k kH k

L k

(16)

where ( )

2LH k represents the modified histogram of the bright region, and

2Lk is the threshold value to set the local gain of the bright region. Since local gain makes the CDF linear in both dark and bright regions, Kim and Paik’s

method can prevent brightness saturation in the bright region.

Chiu, Cheng, and Huang proposed adaptive gamma correction with weighting distribution (AGCWD) [12, 13]. To enhance brightness without undesired artifacts, this method lowered the slope of the CDF by estimating the weighted PDF, and is computed as

( ) ( ) minw max

max min

p k pp k p

p p

α⎛ ⎞−

= ⎜ ⎟⎜ ⎟−⎝ ⎠ (17)

where ( )wp k represents the weighted PDF of an input image, and α is the adjusting parameter. In their method, the input low-contrast image is enhanced by gamma correction using adjusting parameter ( ) ,r k which is

computed with the CDF of ( )wp k and is defined as ( ) ( )1 wk c kγ = − (18)

where ( )wc k represents the CDF of ( )wp k . This method enhances contrast by performing gamma correction using the adaptive adjusting parameter at each pixel. Since ( )r k is close to 0 at the brightest pixels, their method can prevent brightness saturation in the bright region.

2.3 Transmission Map-based Image Enhancement Methods

Recently, the transmission map–based low-light image enhancement method was proposed based on the degradation model of a hazy image [14], calculated as ( )1g ft A t= + − (19)

where t represents the transmission map, and A is the atmospheric light.

Dong et al. applied the haze removal method on inverted low-light images by analyzing histograms of the inverted low-light images [15]. Since these methods assumed that the inverted low-light image can be regarded as a hazy image, the inverted low-light image can be enhanced with the degradation model of a hazy image:

invinv

g Af A

t−

= + (20)

where invg represents the inverted low-light image, and

invf is the dehazed image. This method refines the transmission map to better improve the low-light region to prevent saturation in the bright region. Although this method can enhance brightness using the haze-removal method, performance is limited due to the inaccurately estimated transmission.

To estimate the transmission accurately, Jiang et al. proposed an improved dark channel prior–based low-light

Fig. 2. Comparison of the original and modifiedhistograms using AHE (a) the input histogram, (b) modified histogram ( )H k .


40

image enhancement method [16]. This method estimates a coarse dark channel map in a pixel-wise manner by selecting the minimum intensity value from among red (R), green (G), and blue (B). The refined dark channel is obtained by performing smoothing on the top layer of a Gaussian pyramid, which is generated using the coarse dark channel map [16]. However, it cannot avoid color distortion and noise amplification, since their method does not consider the additive noise in the degradation model of Eq. (19).

To consider additive noise, Ko et al. proposed a modified degradation model using an inverted transmission map [21]:

( )1C C

inv invg f K AK η= − + + (21)

where

1K t= − (22)

and η represents the additive noise. In addition, Ko et al. performed variational optimization-based low-light image enhancement using 1l - and 2l -norm minimization [21]. The energy functional is defined as

( )

2

2,

22 01 2 321 2

min 1

λ λ λ

⎡ ⎤− − +⎣ ⎦

+ ∇ + ∇ + −

cinv

C Cinv inv

f T

Cinv

arg g f K AK

f K K K (23)

where R,G,B ,∈c ( )2

2 1⎡ ⎤− − +⎣ ⎦

C Cinv invg f K AK represents

the data-fidelity term, 1

∇ Cinvf is the smoothness term on

the inverted low-light image, 2

2∇K is the smoothness

term on the inverted transmission map, and 20

2−K K is

the data-fidelity term between the inverted transmission map and the dark channel to estimate the optimal transmission map. 1λ , 2λ , and 3λ are the regularization parameters. Since this method estimates the inverted low-light image using 1l -norm minimization, it can significantly reduce the amplified noise, preserving the edge.

2.4 Retinex-based Image Enhancement Methods

The human visual system (HVS) is more sensitive to reflected light from an object than to direct illumination from the light source [22, 23]. In retinex-based image enhancement methods, the formation of the image is defined as multiplication of the reflectance and illumination components:

R Lg f f= (24)

where Rf and Lf , respectively, represent the reflectance

and illumination components. The contrast of an input image can be improved by eliminating the illumination component estimated by Gaussian low-pass filtering and logarithmic transformation on an input image.

Jobson et al. proposed the concept of the center/surround retinex method called single-scale retinex (SSR), which estimates the reflectance component by subtracting the illumination component from the input image [24] as follows:

( )log log *Rf g g G= − (25)

where G represents the Gaussian low-pass filter, and * is the convolution operator. Fig. 3 shows a block diagram of SSR. Although the higher standard deviation of a Gaussian low-pass filter provides better contrast enhancement, the resulting image shows the halo effect around the edges. In addition, SSR makes the RGB color image look grayish, because contrast enhancement is performed in each color channel.

To solve this problem, Rahman et al. estimated multiple illumination components using different Gaussian filters with different standard deviations: multi-scale retinex (MSR) [25]. The resulting image is obtained by weighted summation using the estimated reflectance components:

( ) 1

log log *i

R n nn

f w g g G=

= −∑ (26)

where nw represents the weight value to the n -th reflectance components, and i is the number of Gaussian filters. However, this method cannot avoid color distortion in the resulting image, as shown in SSR. Jobson and Rahman proposed MSR with color restoration (MSRCR) using a color restoration function that is estimated from the ratio of each color channel [26]. The color restoration function is defined as

( )

( ), ,

,log

,c

ccc R G B

I x yR

I x yβ α

∈

⎡ ⎤⎧ ⎫⎪ ⎪⎢ ⎥= ⎨ ⎬⎢ ⎥⎪ ⎪⎩ ⎭⎣ ⎦∑ (27)

where α represents a parameter that controls the nonlinearity, β is the gain, and cR is the color restoration function. MSRCR performs multiplication of the color restoration function and the reflectance component, which is estimated using multiple Gaussian filters:

Fig. 3. The block diagram of SSR.


41

c c Rf R f= (28)

where cf represents the color-corrected resulting image of the reflectance component. Although this method compensates for the color component using the estimated color restoration function, the resulting image shows the halo effect and noise amplification.

To solve this problem, a variational retinex model–based low-light image enhancement method was proposed. Kimmel et al. proposed a variational retinex model that consists of constraints on the illumination and reflectance components [27]:

( )(

( ) )

2

log

2

min log log log

log log

α

β

Ω

∇ + −

+ ∇ −

∫L

L Lf

L

arg f f g

f g (29)

where α and β represent the regularization parameters,

2log Lf∇ is the smoothness term on the illumination

component, ( )log logLf g− is the data fidelity term, and

( ) 2log logLf g∇ − is the smoothness term on the

reflectance component. Since the illumination component varies smoothly in the entire image, this method allows higher weight for the constraint on the illumination component. Though this method can separate the

illumination component by minimizing the energy function using 2l -norm minimization, it cannot successfully reduce noise amplification and the halo effect near the edge.

To suppress the halo effect, Fu et al. proposed a variational retinex model using the bright channel prior [29]: 2 2 2 2

2 2 2 2,min α β γ− + ∇ + ∇ + −

R LR L L R L bf f

arg f f g f f f g

(30)

where α, β , and γ represent the regularization parameters,

bg is the bright channel of the input image, 2

2−R Lf f g is

the data fidelity term, 2

2∇ Lf and 2

2∇ Rf are the

smoothness terms on the illumination and reflectance components, and 2

2−L bf g is the constraint on the

illumination component to minimize the difference between the bright channel and the illumination component.

The bright channel, which consists of the brightest value from among the R, G, and B channels, is defined as follows [29, 34]:

( )( )

( )( )Ωmax maxb cc y x

g x g y∈

= (31)

Fig. 4. A set of ideal images used to simulate the low-light images.

Fig. 5. Performance comparison of various image enhancement methods (a) the input image, (b) standardhistogram equalization, (c) BBHE, (d) DSIHE, (e) MMBEBHE, (f) RMSHE, (g) AHE, (h) GCCHE, (i) AGCWD, (j) Dong et al.’s method, (k) Jiang et al.’s method, (l) Ko et al.’s method, (m) SSR, (n) MSR, (o) MSRCR, (p) Kimmel et al.’s method, (q) Fu et al.’s method, (r) Park et al.’s method.


42

where R,G,Bc∈ , ( )Ω x represents a patch centered at

x , and ( )bg x is the bright channel of the input image. Since the bright channel of an ideal image converges at

1, according to its statistical property, this method constrains the illumination component using the bright channel prior. The bright channel plays a role in estimating the optimal illumination component by giving the illumination component discontinuity near the edges. However, since the total energy of the reflectance component is minimized using 2l -norm minimization, the blurring artifact is generated near the edges with higher regularization parameter β .

On the other hand, Park et al. presented a variational retinex model using 1l -norm minimization [32]:

22 2

1 2 32 2 1 2,mi ˆn λ λ λ− + ∇ + ∇ + −

R LR L L R L bf f

arg f f g f f f g

(32)

where 1,λ 2 ,λ and 3λ represent the regularization parameters, and ˆbg is the enhanced bright channel. Since Fu et al.’s method estimates the patch-wise bright channel, it cannot avoid the blocking artifacts in the resulting image.

On the other hand, Park et al.’s method estimates the pixel-wise bright channel and applies the bilateral filter to correct the error by additive noise [35].

Moreover, since the reflectance component, which contains the high-frequency components, is minimized using 1l -norm minimization, this method can simultaneously perform brightness enhancement and noise

Fig. 6. Performance comparison of various image enhancement methods (a) the input image, (b) standard histogram equalization, (c) BBHE, (d) DSIHE, (e) MMBEBHE, (f) RMSHE, (g) AHE, (h) GCCHE, (i) AGCWD, (j) Dong et al.’s method, (k) Jiang et al.’s method, (l) Ko et al.’s method, (m) SSR, (n) MSR, (o) MSRCR, (p) Kimmel et al.’s method, (q) Fu et al.’s method, (r) Park et al.’s method.



43

reduction, preserving the edge. This method proposed an enhanced variational retinex model using combined 1l - and 2l -norm minimization to estimate the optimal reflectance and illumination components to suppress the halo effect and noise amplification.

3. Experimental Results

In this section, the performance of various image enhancement methods is evaluated using both simulated and real low-light images for subjective and objective assessment. Fig. 4 shows a set of ideal images used for simulating low-light image degradation [36, 37]. The simulation was performed by decreasing the brightness of the standard test image using gamma correction and adding Gaussian noise with standard deviation σ=3.

Figs. 5-9 show the resulting images from various contrast enhancement methods using simulated and multiple real low-contrast, low-light images acquired under different illumination conditions. The objective assessments are summarized in Tables 2-4 using peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM) [38], and average entropy (AE) [39]. The executable codes can be downloaded online [40].

3.1 Comparison of Histogram-based Methods

In this subsection, the performance of histogram-based image enhancement methods is evaluated using both simulated and real low-contrast images. As shown in Figs. 5-7, the histogram-based methods, which divide the histogram according to the mean brightness value, provided the enhanced result at a low computational cost.

Fig. 8. Performance comparison of various image enhancement methods (a) the input image, (b) standard histogram equalization, (c) BBHE, (d) DSIHE, (e) MMBEBHE, (f) RMSHE, (g) AHE, (h) GCCHE, (i) AGCWD, (j) Dong et al.’s method, (k) Jiang et al.’s method, (l) Ko et al.’s method, (m) SSR, (n) MSR, (o) MSRCR, (p) Kimmel et al.’s method, (q) Fu et al.’s method, (r) Park et al.’s method.



44

However, since the distribution of histogram bins is concentrated on the bright and dark regions in backlit and shadowy images, the performance of histogram-based methods is limited due to low contrast and saturation, as shown in Figs. 8 and 9. In addition, RMSHE provides an insufficiently enhanced result due to an inappropriate number for separating the original histogram. However, since MMBEBHE enhances a low-contrast image by iteratively minimizing the error between the mean brightness values in (11), it provides the enhanced result at the cost of increased computational load. The histogram clipping-based methods can provide robustly enhanced results under various illumination conditions. However, it is difficult to select a proper clipping level and gain ratio based on the input image. On the other hand, since Chiu, Cheng, and Huang’s method uses the weighted CDF of the input image as the adjusting parameter for gamma correction, it can provide a naturally enhanced image with reduced saturation in the bright region.

3.2 Comparison of Transmission Map-based Methods

The transmission map–based image enhancement

methods can enhance the low contrast of a low-light input image better than the histogram-based methods, as shown in Figs. 5-9. Under low-light conditions, the amount of light is so small that it results in unbalanced amplification of brightness in the color channels. In addition, since these methods use the same transmission map in each color channel, they exhibit color distortion.

As shown in Figs. 5(j)-9(j), Dong et al.’s method provides the resulting images with a black-halo effect near the edge due to the patch-wise transmission map. To suppress the halo effect, Jiang et al.’s method estimates the transmission map in a pixel-wise manner. However, the resulting image shows noise amplification, as shown in Figs. 5(k)-9(k), On the other hand, Ko et al.’s method can successfully suppress the noise amplification using 1l -norm minimization, preserving the edge, as shown in Figs. 5(l)-9(l).

3.3 Comparison of Retinex-based Methods

Retinex-based methods can provide an enhanced image from different illumination conditions by eliminating the

Table 2. Comparison of objective quality assessment using PSNR.

[1] [3] [4] [5] [6] [8] [10] [13] [15] [16] [21] [24] [25] [26] [27] [29] [32]Fig. 4(a) 19.53 13.11 13.89 19.57 12.09 20.33 20.02 15.66 17.12 16.97 13.56 12.74 13.83 17.38 18.04 15.52 18.73Fig. 4(b) 14.45 13.42 13.53 14.49 11.98 14.64 14.58 16.96 18.11 15.43 12.17 15.35 15.47 19.83 19.03 18.25 20.77Fig. 4(c) 16.35 15.46 16.27 14.99 14.78 16.53 15.88 18.08 18.82 18.32 14.20 11.53 13.06 15.64 19.80 14.48 18.42Fig. 4(d) 14.88 14.26 15.27 14.77 12.89 15.34 14.84 15.80 17.85 16.20 12.96 11.91 12.80 16.71 18.33 16.84 18.29Fig. 4(e) 15.45 13.31 14.80 14.33 12.31 15.54 15.43 15.51 17.68 16.02 12.66 14.96 16.00 19.17 18.70 16.87 18.48Fig. 4(f) 15.34 13.71 15.41 15.12 12.60 15.72 15.35 15.65 15.02 16.95 12.52 13.57 14.54 19.24 18.09 16.46 18.02

Avg. 16.00 13.87 14.86 15.54 12.77 16.35 16.01 16.27 17.43 16.64 13.01 13.34 14.28 17.99 18.66 16.40 18.78

Table 3. Comparison of objective quality assessment using SSIM [36].


Avg. 0.471 0.472 0.425 0.502 0.480 0.491 0.475 0.648 0.687 0.658 0.499 0.595 0.653 0.749 0.777 0.735 0.815

Table 4. Comparison of objective quality assessment using AE [37].


Avg. 5.829 6.754 6.797 7.043 6.604 7.088 7.049 6.933 7.509 7.620 7.152 6.889 6.934 7.202 7.369 7.662 7.585


45

illumination components. Although SSR provides the enhanced result by estimating the illumination component using a Gaussian low-pass filter, it shows the halo effect near the edge, as shown in Figs. 5(m)-9(m). On the other hand, MSR shows better enhanced results than the SSR method because it estimates the illumination component using multiple Gaussian low-pass filters. However, both SSR and MSR methods show grayish results, as shown in Figs. 5(n)-9(n).

MSRCR restores the color component using a color restoration function. However, the resulting image shows low contrast with noise amplification. On the other hand, the variational retinex model–based methods provide high-quality images, since these methods can estimate the optimal illumination and reflectance components by minimizing the energy functional within the constraints. Figs. 5(r)-9(r) show the results of Park’s method with reduced noise amplification and a wider dynamic range than other enhancement methods.

4. Conclusion

This paper surveyed various contrast and low-light image–enhancement methods, which are categorized into three groups: i) histogram-based, ii) transmission map–based, and iii) retinex-based. In addition, we described the algorithms based on objective assessment of the image enhancement, and compared enhancement performance using different low-contrast, low-light images. This paper provides an understanding of various image enhancement algorithms and the best criterion on which to choose the best enhancement algorithm for a specific application, such as satellite imaging, surveillance systems, and self-driving–vehicle systems.

Appendix: Solving the 1l -norm–based variational Retinex model

The 1l -norm–based variational model proposed by Park et al. is convex but non-differential. To solve the energy functional in Eq. (32), Rf∇ is replaced with d, based on the split Bregman method [41], as follows:

2 2

1 22 2 1,

2

3 2

min

such thatˆ ,

λ λ

λ

− + ∇ + +

− ∇ =

R LR L Lf f

L b R

arg f f g f d

f g f d (33)

where d represents the auxiliary variable. The constrained problem in (33) can be converted to an unconstrained problem as follows:

2 2

1 22 2 1,

22

4 32 2ˆ

min λ λ

λ λ

− + ∇ +

+ −∇ − + −

R LR L Lf f

R L b

arg f f g f d

d f b f g (34)

where b represents the Bregman variable. The

unconstrained problem in (34) is separated into Rf -, Lf -, d -, and b -related sub-problems to obtain the optimum

Rf and Lf in an iterative manner. The Rf -related sub-problem with a fixed Lf is defined

as 2 2

42 2min λ− + −∇ −

RR L Rf

arg f f g d f b (35)

Since (35) is convex and quadratic, it can be solved

using the Euler-Lagrange equation:

( ) ( )T4 41 T

RL

gf d bf

λ λ+ ∇ ∇ = + ∇ − (36)

Since (36) is a linear equation in Ax b= , the solution

is obtained using fast Fourier transform (FFT):

( )

( )

T4

1

41L

R T

g d bf

fλ

λ−

⎡ ⎤⎧ ⎫+ ∇ −⎢ ⎥⎨ ⎬

⎩ ⎭⎢ ⎥= ⎢ ⎥+ ∇ ∇⎢ ⎥⎢ ⎥⎣ ⎦

FF

F (37)

where F and 1−F represent FFT and inverse FFT operators, respectively.

Next, the auxiliary variable d is updated using the shrinkage operator:

2

4

*max ,0RR

R

f bd f b

f bλλ

⎛ ⎞∇ += ∇ + −⎜ ⎟∇ + ⎝ ⎠

(38)

The Bregman variable is updated:

Rb b f d= +∇ − (39)

Optimal solution Lf is obtained by solving the Lf -

related sub-problem with Rf fixed, as follows: 22 2

1 32 2 2min ˆλ λ− + ∇ + −

LR L L L bf

arg f f g f f g (40)

Since (40) is quadratic and convex, it can be solved

using the Euler-Lagrange equation:

( )3 1 3 ˆ1 TL b

R

gf gf

λ λ λ+ + ∇ ∇ = + (41)

And (41) can be solved using FFT:

( )

31

3 11

ˆbL

L T

g gf

fλ

λ λ−

⎡ ⎤⎧ ⎫+⎢ ⎥⎨ ⎬

⎩ ⎭⎢ ⎥= ⎢ ⎥+ + ∇ ∇⎢ ⎥⎢ ⎥⎣ ⎦

FF

F (42)


46

Acknowledgement

This research was supported by the Ministry of Science and ICT (MSIT), Korea, under the Information Technology Research Center (ITRC) support program (IITP-2017-2014-0-00636) supervised by the Institute for Information & communications Technology Promotion (IITP), the Korea Aerospace Research Institute (NRF-2017M1A3A4A07028434), and an IITP grant funded by the Korea government (MSIT) (2017-0-00250, Intelligent Defense Boundary Surveillance Technology Using Collaborative Reinforced Learning of Embedded Edge Camera and Image Analysis).

References [1] R. C. Gonzalez and R. E. Woods, “Digital Image

Processing,” 3rd edition, Prentice-Hall, 2007. Article (CrossRef Link)

[2] Y. B. Rao and L. T. Chen, “A Survey of Video Enhancement Techniques,” Journal of Information Hiding and Multimedia Signal Processing, vol. 3, pp. 71-99, Jan. 2012. Article (CrossRef Link)

[3] Y. T. Kim, "Contrast Enhancement Using Brightness Preserving Bi-histogram Equalization," IEEE Trans. Consum. Electron., vol. 43, no. 1, pp. 1-8, Feb. 1997. Article (CrossRef Link)

[4] Y. Wang, Q. Chen, and B. Zhang, "Image Enhancement Based on Equal Area Dualistic Sub-Image Histogram Equalization Method," IEEE Trans. Consum. Electron., vol. 45, no. 1, pp. 68-75, Feb. 1999. Article (CrossRef Link)

[5] S. D. Chen and A. R. Ramli, "Minimum Mean Brightness Error Bi-histogram Equalization in Contrast Enhancement," IEEE Trans. Consum. Electron., vol. 49, no. 4, pp. 1310-1319, Nov. 2003. Article (CrossRef Link)

[6] S. D. Chen and A. R. Ramli, “Contrast Enhancement Using Recursive Mean Separate Histogram Equalization for Scalable Brightness Preservation,” IEEE Trans. Consum. Electron., vol. 49, no. 4, pp. 1301-1309, Nov. 2003. Article (CrossRef Link)

[7] Q. Tian and J. Duan, "Local Histogram Modification Based Contrast Enhancement," in Proc. of Int. Conf. Audio, Language, Image Process., pp. 1-6, 2012. Article (CrossRef Link)

[8] S. M. Pizer, “Adaptive Histogram Equalization and Its Variations,” Comput. Vis. Graph. Image Process. 39, 355-368, Oct. 1987. Article (CrossRef Link)

[9] K. Zuiderveld, “Contrast Limited Adaptive Histogram Equalization,” Graphic Gems IV. San Diego: Academic Press Professional, pp. 474–485, 1994. Article (CrossRef Link)

[10] T. Kim and J. Paik, "Adaptive Contrast Enhancement using Gain-Controllable Clipped Histogram Equalization," IEEE Trans. Consum. Electron., vol. 54, no. 4, pp. 1803-1810, Nov. 2008. Article (CrossRef Link)

[11] Y. Li and H. Zhang, "Modified Clipped Histogram Equalization for Contrast Enhancement," in Proc. of Int. Conf. Parallel Distrib. Comput., Appl. Technol.,

pp. 653-658, 2012. Article (CrossRef Link) [12] Y. S. Chiu, F. C. Cheng, and S. C. Huang, "Efficient

Contrast Enhancement Using Adaptive Gamma Correction and Cumulative Intensity Distribution," in Proc. of IEEE Int. Conf. Systems, Man, and Cybernetics, Anchorage, pp. 2946-2950, 2011. Article (CrossRef Link)

[13] S. C. Huang, F. C. Cheng, and Y. S. Chiu, "Efficient Contrast Enhancement Using Adaptive Gamma Correction with Weighting Distribution," IEEE Trans. Image Process., vol. 22, no. 3, pp. 1032-1041, Mar. 2013. Article (CrossRef Link)

[14] K. He, J. Sun, and X. Tang, "Single Image Haze Removal Using Dark Channel Prior," IEEE Trans. Patter. Anal. Mach. Intell., vol. 33, no. 12, pp. 2341-2353, Dec. 2011. Article (CrossRef Link)

[15] X. Dong, G. Wang, Y. Pang, W. Li, J. Wen, W. Meng, and Y. Lu, "Fast Efficient Algorithm for Enhancement of Low Lighting Video," in Proc. of IEEE Int. Conf. Multimedia and Expo, pp. 1-6, 2011. Article (CrossRef Link)

[16] X. Jiang, H. Yao, S. Zhang, X. Lu, and W. Zeng, "Night Video Enhancement using Improved Dark Channel Prior," in Proc. of IEEE Int. Conf. Image Process., pp. 553-557, 2013. Article (CrossRef Link)

[17] Y. Hu, Y. Shang, X. Fu, and H. Ding, "A Low Illumination Video Enhancement Algorithm Based on the Atmospheric Physical Model," International Congress on Image and Signal Processing, pp. 119-124, 2015. Article (CrossRef Link)

[18] L. Li, R. Wang, W. Wang, and W. Gao, "A Low-Light Image Enhancement Method for Both Denoising and Contrast Enlarging," in Proc. of IEEE Int. Conf. Image Process., pp. 3730-3734, 2015. Article (CrossRef Link)

[19] P. Shen, L. Zhang, J. Song, X. Peng, G. Zhu, Y. Zhang, L. Zhi, and K. Yi, "Parallel Implementation of Low Light Level Image Enhancement using CUDA," in Proc. of IEEE Int. Conf. Inform. Autom., pp. 673-677, 2015. Article (CrossRef Link)

[20] J. Pang, S. Zhang, and W. Bai, "A Novel Framework for Enhancement of the Low Lighting Video," IEEE Symposium on Computers and Communications, pp. 1366-1371, 2017. Article (CrossRef Link)

[21] S. Ko, S. Yu, S. Park, B. Moon, W. Kang, and J. Paik, “Variational Framework for Low-light Image Enhancement using Optimal Transmission Map and Combined 1l and 2l -minimization,” Signal Processing: Image Communication, vol. 58, pp. 99-110, 2017. Article (CrossRef Link)

[22] E. H. Land. “The Retinex,” American Scientist, vol. 52, no. 2, pp. 247–264, 1964. Article (CrossRef Link)

[23] E. H. Land and J. J. McCann, “Lightness and Retinex Theory,” J. Opt. Soc. Am., vol. 61, no. 1, pp. 1-11, 1971. Article (CrossRef Link)

[24] D. J. Jobson, Z. Rahman, and G. A. Woodell, “Properties and Performance of a Center/Surround Retinex,” IEEE Trans. Image Process., vol. 6, no. 3, pp. 451–462, Mar. 1997. Article (CrossRef Link)

[25] Z. Rahman, D. J. Jobson, and G. A. Woodell, "Multi-scale Retinex for Color Image Enhancement," in Proc.

http://www.imageprocessingplace.com/

http://www.imageprocessingplace.com/

http://www.jihmsp.org/~jihmsp/2012/vol3/JIH-MSP-2012-01-007.pdf

https://doi.org/10.1109/30.580378

https://doi.org/10.1109/30.754419

https://doi.org/10.1109/TCE.2003.1261234

https://doi.org/10.1109/TCE.2003.1261233

https://doi.org/10.1109/ICALIP.2012.6376577

https://doi.org/10.1016/S0734-189X(87)80186-X

https://doi.org/10.1016/B978-0-12-336156-1.50061-6

https://doi.org/10.1109/TCE.2008.4711238

https://doi.org/10.1109/TCE.2008.4711238

https://doi.org/10.1109/PDCAT.2012.97

https://doi.org/10.1109/ICSMC.2011.6084119

https://doi.org/10.1109/TIP.2012.2226047

https://doi.org/10.1109/TPAMI.2010.168

https://doi.org/10.1109/ICME.2011.6012107

https://doi.org/10.1109/ICIP.2013.6738114

https://doi.org/10.1109/CISP.2015.7407861


https://doi.org/10.1109/ICInfA.2015.7279371

https://doi.org/10.1109/ISCC.2017.8024714

https://doi.org/10.1016/j.image.2017.06.016

https://www.jstor.org/stable/27838994?seq=1#page_scan_tab_contents

https://doi.org/10.1364/JOSA.61.000001

https://doi.org/10.1109/83.557356


47

of IEEE Int. Conf. Image Process., vol.3, pp. 1003-1006, 1996. Article (CrossRef Link)

[26] D. J. Jobson and Z, Rahman, “A Multiscale Retinex for Bridging the Gap Between Color Images and the Human Observation of Scenes,” IEEE Trans. Image Process., vol. 6, no. 7, pp. 965-976, Jul. 1997. Article (CrossRef Link)

[27] R. Kimmel, M. Elad, D. Shaked, R. Keshet, and I. Sobel, “A Variational Framework for Retinex,” Int. J. Comput. Vis., vol. 52, no. 1, pp. 7–23, 2003. Article (CrossRef Link)

[28] D. Zosso, G. Tran, and S. Osher, “A Unifying Retinex Model Based on Non-local Differential Operators,” IS&T/ SPIE Electronic Imaging, 865702-865702, 2013. Article (CrossRef Link)

[29] X. Fu, D. Zeng, Y. Huang, X. Ding, and X. P. Zhang, "A Variational Framework for Single Low Light Image Enhancement Using Bright Channel Prior," in Proc. of IEEE Global Conf. Signal Inform. Process., pp. 1085-1088, 2013. Article (CrossRef Link)

[30] W. Ma, J. M. Morel, S. Osher, and A. Chien, “An 1l -based Variational Model for Retinex Theory and Its Application to Medical Images,” in Proc. IEEE Conf. on Comput. Vis. Pattern Recog., pp. 153-160, Jun. 2011. Article (CrossRef Link)

[31] X. Fu, Y. Liao, D. Zeng, Y. Huang, X. P. Zhang, and X. Ding, "A Probabilistic Method for Image Enhancement With Simultaneous Illumination and Reflectance Estimation," IEEE Trans. Image Process., vol. 24, no. 12, pp. 4965-4977, Dec. 2015. Article (CrossRef Link)

[32] S. Park, B. Moon, S. Ko, S. Yu, and J. Paik, "Low-light Image Restoration Using Bright Channel Prior-Based Variational Retinex Model," EURASIP Journal on Image and Video Processing, vol. 2017, no. 44, pp. 1-11, Jun. 2017. Article (CrossRef Link)

[33] S. Park, S. Yu, B. Moon, S. Ko, and J. Paik, "Low-light Image Enhancement Using Variational Optimization-based Retinex Model," IEEE Trans. Consum. Electron., vol. 63, no. 2, pp. 178-184, May 2017. Article (CrossRef Link)

[34] X. Fu, Q. Lin, W. Guo, X. Ding, and Y. Huang, “Single Image De-haze Under Non-uniform Illumination using Bright Channel Prior,” Journal of Theoretical and Applied Information Technology, 2013. Article (CrossRef Link)

[35] C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” in Proc. IEEE Int. Conf. Comput. Vis., pp. 836–846, Jan. 1998. Article (CrossRef Link)

[36] H. Hase, M. Yoneda, and M. Sakai, “Evaluation of Handprinting Variation of Characters using Variation Entropy,” IEICE Transaction, No. 6, pp. 1048-1056, 1988.

[37] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, 2004. Article (CrossRef Link)

[38] https://dragon.larc.nasa.gov/retinex/pao/news/ [39] http://r0k.us/graphics/kodak/

[40] https://github.com/tjsgml1463/Low-light_en [41] T. Goldstein and S. Osher, “The Split Bregman

Method for 1l -regularized Problems,” SIAM Journal on Imaging Sciences, vol. 2, no. 2, pp. 323-343, Apr. 2009. Article (CrossRef Link)

Seonhee Park was born in Busan, South Korea, in 1993. She received her BSc in Integrative Engineering from Chung-Ang University, South Korea, in 2016. Currently, she is pursuing an MSc in Digital Imaging Engineering at Chung-Ang University. Her research interests include super-resolution,

remote sensing images, denoising, and image enhancement and restoration for display processing.

Kiyeon Kim was born in Incheon, South Korea, in 1996. She is pursuing a BSc in Digital Imaging Engineering from Chung-Ang University, South Korea. Her research interests include super-resolution, image enhancement, remote sensing images, and denoising and restoration for display processing.

Soohwan Yu was born in Incheon, Korea, in 1988. He received a BSc in Information and Communication Engineering from Suwon University, Korea, in 2013. He received an MSc in Image Engineering from Chung-Ang University, Korea, in 2016, where he is currently pursuing his PhD in Image

Engineering. His research interests include image enhancement, super-resolution, and image restoration.

Joonki Paik was born in Seoul, Korea, in 1960. He received a BSc in Control and Instrumentation Engineering from Seoul National University in 1984. He received an MSc and a PhD in Electrical Engineering and Computer Science from Northwestern University in 1987 and 1990, respectively. From

1990 to 1993, he joined Samsung Electronics, where he designed the image stabilization chip sets for consumer camcorders. Since 1993, he has been with the faculty at Chung-Ang University, Seoul, Korea, where he is currently a Professor in the Graduate School of Advanced Imaging Science, Multimedia, and Film. From 1999 to 2002, he was a visiting professor in the Department of Electrical and Computer Engineering at the University of


https://doi.org/10.1109/83.597272

https://doi.org/10.1109/83.597272

https://doi.org/10.1023/A:1022314423998

https://doi.org/10.1023/A:1022314423998

http://dx.doi.org/10.1117/12.2008839

https://doi.org/10.1109/GlobalSIP.2013.6737082

https://doi.org/10.1109/CVPR.2011.5995422

https://doi.org/10.1109/TIP.2015.2474701

https://doi.org/10.1109/TIP.2015.2474701

https://doi.org/10.1186/s13640-017-0192-3

https://doi.org/10.1109/TCE.2017.014847

https://www.semanticscholar.org/paper/Single-Image-De-haze-under-Non-uniform-Illuminatio-Fu-Lin/73f81d12523ee777f629a03317b3d09fc93de5f5

https://doi.org/10.1109/ICCV.1998.710815

https://doi.org/10.1109/ICCV.1998.710815

https://doi.org/10.1109/TIP.2003.819861

https://doi.org/10.1109/TIP.2003.819861

https://doi.org/10.1137/080725891


48

Tennessee, Knoxville. Dr. Paik was a recipient of the Chester-Sall Award from the IEEE Consumer Electronics Society, received an Academic Award from the Institute of Electronic Engineers of Korea, and a Best Research Professor Award from Chung-Ang University. He has served the Consumer Electronics Society of IEEE as a member of the Editorial Board. Since 2005, he has been the head of the National Research Laboratory in the field of image processing and intelligent systems. In 2008, he worked as a full-time technical consultant for the System LSI Division of Samsung Electronics where he developed various computational photographic techniques, including an extended depth of field (EDoF) system. From 2005 to 2007, he served as Dean of the Graduate School of Advanced Imaging Science, Multimedia, and Film. From 2005 to 2007, he was Director of the Seoul Future Contents Convergence (SFCC) Cluster established by the Seoul Research and Business Development (R&BD) Program. Dr. Paik is currently serving as a member of the Presidential Advisory Board for Scientific/Technical Policy with the Korean Government and is a technical consultant for the Korean Supreme Prosecutor's Office for computational forensics.

Copyrights © 2018 The Institute of Electronics and Information Engineers

Documents

Contrast Enhancement for Low-light Image Enhancement: A … Park et al.: Contrast Enhancement for Low-light Image Enhancement: A Survey 38 images in the sense of intensity value by