IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. ?, NO. ?, ? 200? 1

Globally Optimized Linear WindowedTone-Mapping

Qi Shan, Student Member, IEEE, Jiaya Jia, Member, IEEE, and Michael S. Brown, Member, IEEE

Abstract—This paper introduces a new tone-mapping operator that performs local linear adjustments on small overlapping windowsover the entire input image. While each window applies a local linear adjustment that preserves the monotonicity of the radiance values,the problem is implicitly cast as one of global optimization that satisfies the local constraints defined on each of the overlapping windows.Local constraints take the form of a guidance map that can be used to effectively suppress local high contrast while preserving details.Using this method, image structures can be preserved even in challenging high dynamic range (HDR) images that contain either abruptradiance change, or relatively smooth but salient transitions. Another benefit of our formulation is that it can be used to synthesize HDRimages from low dynamic range (LDR) images.

Index Terms—High dynamic range image, tone mapping, display algorithms, image enhancement, filtering.

✦

1 INTRODUCTION

HIGH dynamic range compression – or tone map-ping – embodies techniques that map HDR images

to low dynamic range displays that only have moderatecontrast. While a global linear scaling can be used tocompress the dynamic range, e.g. from 10000:1 to 200:1,the result will be undesirable since image structures arelinearly flattened, causing the loss of the visual infor-mation mostly in highlights and shadows. To avoid thistrivial solution, most tone mapping techniques employsome type of non-linear mapping functions. Several tonemapping algorithms have been proposed that eitheradjust the global tone response curve [14] or locallyreproduce tonal values [7] [9] [16]. It is also generallynoticed [9] [16] that local operators, which reproducethe tonal values in a spatially variant manner, performmore satisfactorily than global operators in terms ofdetail preservation and compression ratio of the dynamicrange.

The objective in this paper is to develop a new tonereproduction operator. We observe that HDR in an im-age can typically be categorized into two types – 1)regions exhibiting significant high dynamic range butwith smooth radiance transition (Figures 1(a)) and 2)regions exhibiting sharp and significant local radiancechange among neighboring pixels (Figure 1(b)). Our tonereproduction operator provides a unified framework toeffectively address these two types of HDR. Our ap-proach maintains local structures, including sharp edges

• Q. Shan and J. Jia are with the Department of Computer Science andEngineering, the Chinese University of Hong Kong, Hong Kong.E-mail: {qshan, leojia}@cse.cuhk.edu.hk

• M. S. Brown is with the Department of Computer Science, NationalUniversity of Singapore.E-mail: [email protected]

Fig. 1. HDR regions exhibiting abrupt and significant ra-diance change (upper right) and smoother local transition(lower right).

and smooth color transitions at a perceptual level whilenot introducing visible artifacts such as halos.

Our approach uses a window-based tone mappingmethod in which a global optimization problem is solvedthat satisfies local constraints. Specifically, our methodoperates on windows; in each window a linear functionis used to constrain the tone reproduction in order tonaturally suppress strong edges while retaining weakones. The high dynamic range is compressed by solvingan image-level optimization problem that integrates allwindow-based constraints.

While our method is regarded as a local operator, itdoes not involve scale decomposition, layer separation,or image segmentation, and is therefore less susceptiblefrom the associated artifacts known with these previousprocedures. Within our global optimization framework,the overall tone mapping effect is non-linear and spa-tially variant, and adapts to rich and diverse structuresof images. Our method can be expressed with a closed-form solution that reaches a global optimum. Owing tothe global optimality, any sufficient local tone adjust-ment in our method gives rise to a global effect whereerrors are minimized and distributed across the entire


image. Moreover, our method provides flexibility forimage quality adjustments with only a few parameters,adaptive to various user requirements pertaining to therange compression ratio and the detail preserving ratio.

In addition to high dynamic range compression, ourmethod also contributes a unified framework for toneenhancement of ordinary images by solving the sameoptimization problem. An application of synthesizing anHDR image from a single low dynamic range (LDR)image is presented to simulate the high contrast envi-ronment in indoor and outdoor scenes containing shinylight sources or obscure shadows. The effectiveness ofour HDR synthesis is evaluated by comparisons againstground truth images.

The rest of our paper is organized as follows. Section 2reviews related work. Section 3 presents our algorithmof HDR compression. Section 4 describes the parametersetting and shows our HDR compression results. Ouralgorithm is applied to ordinary image enhancement andHDR image synthesis as described in Section 5. Section 6discusses the proposed approach and summarizes ourwork.

2 PREVIOUS WORK

Tone mapping operators have attracted broad interestsrecently. Surveys of these methods can be found in[5] [25] [21]. We review HDR compression methods intwo categories: global operators and local operators.The global tone mapping operators map the radiancevalues in a spatially invariant manner. Histogram basedand sigmoid transfer function based algorithms, suchas gamma correction, are two main categories of globaloperators. For global operators, the key problem is to es-tablish a one-to-one or onto mapping from high dynamicradiance values to the low dynamic ones. Sigmoidalcompression is to redistribute radiance values in anHDR image using a sigmoid curve. This results in lowcompression at the low and high end of the radiancerange, with more compression in the mid-range. Sigmoidcompression is a reasonable way to compress manyHDR images. Other similar global functions with similarproperties are discussed in detail in Chapters 6 and7 of [21]. In [14] [6], non-linear global operators wereproposed where global tone reproduction curves withdifferent measures of image qualities are estimated toreduce the dynamic range in a computationally efficientway. These methods do not need manual tuning andare capable of producing visually satisfactory resultsfor a large set of examples. However, these approachesmay fail for HDR images where strong local contrastis present (i.e. a local region contains both high andlow radiance values). Further, finding a perfect tonereproduction curve is difficult [4] for several cases.

Inspired by how the human visual system adapts tolocal regions of scene luminance [21], more sophisticatedlocal tone reproduction methods have been proposed.Many of these approaches involve decomposing, com-pressing, and recombining layers in the HDR images.

Krawczyk et al. [13] first segmented an HDR image, andthen applied different tone reproduction curves to thesegments to achieve spatially variant HDR compression.With appropriate algorithm design and implementation,no seams will be present in the tone mapping results.

In [24], an image is decomposed into a reflectanceimage R and an illuminance image L, according to theobservation that the high dynamic range is generallycaused by the illuminance, while reflectance is unlikelyto produce high contrast. However, accurately separat-ing the two layers from a single image is non trivial. Tosimplify the intrinsic image decomposition, it is assumedthat L contains low frequency information while R mayhave spatially abrupt change in values [11]. Assumingthe surface reflectance and illumination properties areknown, Tumblin et al. [25] compressed high dynamicrange only in the illuminance image. In [7], an imageis decomposed into a base layer and a detail layer. Thebase layer is obtained using a bilateral filter, and thedetail layer is computed by subtracting the base layerfrom the input image. A linear scaling in the logarithmdomain of the base layer is performed to compress thedynamic range and the final result is the combined detaillayer and the compressed base layer.

Other scale decomposition based techniques decom-pose the HDR images into several layers [12] [19] [26],imitating human vision adaptation to local changes. In[16], a multiscale approach using symmetrical analysis-synthesis filter banks and automatic gain control wasproposed to compress the HDR images in subbands.This method, along with those in [20] [2], incorpo-rated schemes to avoid so haloing artifacts. This typeof artifacts, as explained in [16], can be considered assignal distortions that occurs when combining multiplecompressed subband layers. It can often be reduced withcareful selection of the parameter values.

Recently, in [8], a new edge-preserving multi-scaleimage decomposition method was proposed. Resultsdemonstrate that this image decomposition method out-performs bilateral filtering in producing range com-pressed images.

Gradient domain dynamic range compression [9] doesnot directly process the image radiance. Instead, the tonemapping operator attenuates the image gradients withlarge magnitude while magnifying the small ones. Thefinal result is computed by solving a Poisson equation.Lischinski et al. [17] presented an interactive method tolocally adjust the tonal values and other visual param-eters. In their method, the user uses a set of brushes toimpose constraints on the image. Influence functions arecomputed to confine the modifications of the tonal valuein image space.

Different from these methods, our local operator doesnot require multiscale decomposition or segmentation ofthe images into binary or fractional maps. This avoidsproblems associated with layer decomposition, includingthe decomposition itself as well as layer recombination.We directly process the radiance map where a global op-


timization framework is adopted to naturally constrain therespective pixel values in a global sense. In addition, ourapproach does not require any smoothness constraints tobe imposed on the final output.

Inferring an HDR image from an LDR images isknown as “inverse tone mapping” [3]. Single imagemethods can only achieve an approximation of the solu-tion due to the information loss. In [3], high luminanceareas are estimated from an LDR image. An inversetone mapping operator, which reverses the techniquein [20] is then applied to enhance luminance in thedetected areas. Meylan et al. [18] proposed detecting thespecular highlights in the LDR images and boosting theintensity in these regions. The LDR images generatedin the tone mapping stage produce small errors in thedynamic range expansion stage. In [1], psychophysicalstudies showed that linear contrast scaling works wellin most cases for expanding the dynamic range of someordinary images. Rempel et al. [22] proposed a real-timetechnique to synthesize an HDR images by enhancingthe brightness gradation in the saturated regions. In [16],Li et al. addressed a special HDR compression-expansionproblem (a process called “companding”, for example, toturn a 12 bit/channel image into an 8 bit/channel TIFF,and later convert it back to a good approximation ofthe original 12-bit image) by adopting a feedback-loopscheme. This method reconstructs an HDR image froman LDR image that was first tone-mapped with their sub-band technique. This approach, however, cannot be usedto expand the dynamic range of an arbitrary LDR image.

Comparing our work with previous inverse tone map-ping methods, our algorithm not only attempts to makethe results look natural by enhancing pixel intensitymore significantly in highlights, but also aims to produceresults with radiance similar to that of the ground truthHDR images. Synthesizing an HDR image in our systemcan be regarded as a partial inverse of HDR compression.It is naturally solved in our unified framework withoutrequiring large modifications of the system.

3 ALGORITHM AND IMPLEMENTATION

Given an input HDR image with radiance map Ih, thehigh dynamic range compression operator f(·) computesthe radiance map in a low dynamic range I l = f(Ih).An intuitive approach to designing a tone mappingoperator is to decrease the large radiance values and in-crease small ones. To correctly retain the local structures,however, the monotonic mapping of radiance should besatisfied regarding each pixel’s neighborhood in a smalllocal region, similar to the monotone characteristic of theglobal tone mapping curves. Therefore, if we zoom intothe square window wi containing only a few pixels withpixel i in center, the linear function

I l(j) = piIh(j) + qi, j ∈ wi, (1)

provides a simple yet complete representation satisfyingthe local monotonic constraint. The coefficients pi and

p

q

p

q

(a) (b)

Ih

Ih

Il

Il

Ih

Ih

Il

Il

Fig. 2. Illustration of the local window operator. (a) and(b) show two configurations that Ih undergoes linearmappings (represented as the slanted dashed lines) withdifferent coefficients p and q to produce Il. q controls thebase radiance level and p manipulates the local contrastin the window. Setting large p makes the local detailsin Il be enhanced, as shown in (a), whereas a small psuppresses local high contrast, as shown in (b).

qi, within the local window, control the mapping in twoways. The value of q determines the base radiance levelwhile p, representing the slope of the linear function.the terms q and p directly control the local contrast asshown in Figure 2. If p > 1, as shown in Figure 2(a), the local contrast is enhanced, making the resultenhance details in dark regions. Figure 2 (b) showsanother example where small positive p reduces imagecontrast and improves the visibility of structures inbright regions. Note that we do not add regularizationterms, such as smoothness constraints to the overallequations. This is because pi’s and qi’s are computedin overlapping windows that contain multiple pixels. Inthis case, a smoothness-like constraint across pixels isnaturally enforced in flat regions of an HDR image, whilein regions with strong structures, such as edges, contrastis reasonably maintained.

Combining all local linear equations defined on in-dividual windows, we can reconstruct an image byminimizing

∑i

∑j∈wi

(I l(j) − piIh(j) − qi)2, (2)

where i sums over all pixels in the HDR image. However,directly minimizing (2) cannot compress the high dy-namic range because a trivial solution exist. For example,pi = 1 and qi = 0 for all i results in Il = Ih whichis one solution that produces zero error in minimizing(2). As a result, additional constraints are needed. Notethat pi in (1) is the parameter directly controlling thechange of local contrast. We propose guiding its valuein each window to reduce the global contrast whilemaintaining the local visual information. We express the


(a) (b) (c)

(d) (e) (f)

Fig. 3. Tone mapping results with window sizes (a) 3 ×3, (b) 7 × 7, and (c) 15 × 15. (d)-(f) Close-ups of (a)-(c)respectively.

final objective function to be minimized as

f =∑

i

(∑j∈wi

(I l (j) − piI

h (j) − qi

)2

+ εc−2i (pi − ci)

2

), (3)

where ci is a pre-set positive value to guide the modifi-cation of local contrast. The collective ci in the imagespace form a guidance map. By appropriately settingthe values in this map, we can suppress strong localcontrasts and elevate weak ones. A detailed descriptionof the guidance map construction is given in Section 4.The term c−2

i (pi − ci)2 in (3) is the squared relative error

from the guidance map and c−2i is used to normalize the

cost introduced by the difference between pi and ci. Theterm ε is a weight to balance the two terms in (3). Weset its value to 0.1 in all our experiments.

The benefit of introducing the guidance map c istwofold. First, although the value of ci influences localcontrast, the configuration of ci does not require highaccuracy compared to direct modification of the radiancevalues in Ih. This is because in our optimization frame-work, ε is only a small weight, making the guidanceci act as a soft constraint. The linear mapping term∑

j∈wi(I l(j)− piI

h(j)− qi)2 also constrains modificationof radiance at each pixel to be similar to the changeon the neighboring pixels. This helps preserving imagestructure. Second, by modifying the guidance map c, oursystem can readily be applied to other tone managementapplications, such as ordinary image enhancement andHDR image synthesis. Examples are shown in Section 5.

3.1 Optimization and Implementation

Directly minimizing f in (3) is not trivial given thelarge set of unknowns. Methods such as gradient decentcannot be used to the global optimum because Il, pi

and qi are highly coupled and the energy function is not

convex. However, as each pair of the linear coefficients[pi, qi] is only defined in a single window, minimizing(3) can be expressed as

arg minp,q,Il

f = argminIl

∑i

arg minpi,qi

fi, where

fi =

⎛⎝∑

j∈wi

(I l (j) − piI

h (j) − qi

)2+ εc−2

i (pi − ci)2

⎞⎠ . (4)

Using the above expression, we first compute (p∗i , q∗i ) ,

i.e. the optimal solution of (pi, qi), by setting the partialderivatives of function fi with respect to pi and qi tozero and then computing the optimal I l by solving alinear system. The computation process is described inthe Appendix. We discuss in Section 4 the configurationof the guidance map c for naturally compressing the highdynamic range.

The local window size about each pixel can be ad-justed in our system. In Figure 3, we show tone mappingresults computed by setting window sizes from 3× 3 to15× 15. For all examples shown in this paper, we foundthat window size 3×3 is suitable in terms of maintainingedge sharpness and computational efficiency. However,other sizes can also be used.

The above operations process the image’s luminancechannel. After obtaining the tone mapped radiance mapI l, the RGB channels are reconstructed using the methodof Schlick [23]:

I lk(i) = (Ih

k (i)/Ih(i))s × I l(i), k ∈ {r, g, b} (5)

where I lk is one of the RGB color channels in the image

result and Ihk is one of the color channels in the input

HDR image. The term s represents the saturation factor.Larger s produces a more saturated result. The under-lying idea of introducing (5) is to preserve the ratiosbetween the luminance and color channels so that thehue of the range compressed image can be similar to thatof the input HDR image. In our experiments, s ∈ [0.4, 0.6]produces chromatically natural image results. The skele-ton of our algorithm is given in Table 1.

1. Input: An HDR image with radiance map Ih.2. Main Steps:

2.1 Generate the guidance map c according to (6)in Section 4.

2.2 Construct matrix S and B according to (18) inthe Appendix.

2.3 Compute I l by solving the linear system de-fined in (17) in the Appendix.

2.3 Restore the RGB channels Ilk in the tone

mapped result by (5).3. Output: An ordinary image with color channels I l

k.

TABLE 1Skeleton of our tone mapping algorithm.


(a) (b)

(c) (d)

Fig. 4. “Window” example. (a) The input HDR image (im-age courtesy of Shree Nayar). (b) The truncated variancemap of Ih on all local windows w. (c) The truncated mapof mean values of Ih. (d) The truncated radiance map Ih.

4 GUIDANCE MAP CONFIGURATION IN HDRCOMPRESSION

In the linear optimization framework introduced in Sec-tion 3, the definition of the guidance map c principallydetermines the quality of HDR compression. In thissection, we describe how to obtain a suitable guidancemap c.

Recall that the value pi controls the compression ratioof the dynamic range. To preserve the visual informationin the tone-mapped image, we reduce the value of pi inregions where the local contrast is large, and increaseit when the local contrast is small. A measure of localcontrast in each window wi is the standard deviationσi of Ih. However, setting ci inversely proportionalto σi cannot produce a satisfying result. An exampleis illustrated in Figure 4 where (a) shows the inputHDR image and (b) shows the variance map of Ih.The variance map is overly sensitive to the textures,edges, and even the noise in the HDR image. The HDRcompression result illustrated in Figure 5 (a) is producedby setting ci = (σβ2

i +κ)−1, where β2 = 0.75 to attenuatethe variance, and κ is a small constant value. Althoughthe high dynamic range of the original image in Figure4(a) is reduced in the result shown in Figure 5(a), thereare noticeable blemishes in the result especially in thehighlighted region. These artifacts are caused by thesusceptibility of the variance to the local structures.

To improve the visual quality of the result, we com-pute σi on the Gaussian filtered images to reduce noise.

(a) (c)(b) (d)

Fig. 5. Illustration of the guidance map configuration.(a) The tone mapping result when ci is set inverselyproportional to the local standard deviation of Ih in eachwindow. There are visible artifacts on the wall. (b) Mag-nified guidance map c. The noise is amplified. (c) Thetone mapping result when ci is configured according to(6). (d) Magnified guidance map. Note that the noise issuppressed.

Two other visual factors the local mean μi and the imageradiance Ih(i), are incorporated into the guidance mapwith the following considerations. First, μi and Ih(i)represent the image radiance information. Their mapsglobally reflect image structures. By taking μi and Ih(i)into the guidance formation, the over-exaggeration ofstructures using only local contrast can be preventedand the tonal value adjustment depends more on theinput radiance in a global manner. Second, we observedthat the contrast of the radiance within a local windowdepends largely on the absolute variance values. For in-stance, in Figure 4(a), the variance of the radiance insidethe dark window is as low as 0.1 whereas the radiancevariance in the bright windowsill is approximately 104.These observations indicate that incorporating μi andIh(i) into the guidance map construction helps suppresscontrast in bright regions.

The maps of μi and Ih(i) are illustrated in Figures 4(c)and 4(d) which emphasize the salient radiance changebetween the bright and dark regions while not makingit highly sensitive to the isolated noise. We define ci as

ci = (μβ1i σβ2

i Ih(i)β3 + κ)−1, (6)

where κ is a small weight set to 0.05 in all our ex-periments to prevent ci from being divided by zero.Normally β1 ∈ [0.4 − 0.9], β2 ∈ [0.1 − 0.4], and β3 isfixed to 0.1. We show an HDR compression result inFigure 5(c) using the guidance map constructed using(6). Comparing the magnified guidance maps shown inFigures 5(b) and 5(d), it is apparent that (d) containsmuch less visual artifacts.

Figure 6 shows the terms we introduced to constructthe guidance map c. Figure 6(a) shows an input HDRimage. The sky is bright and the hill is relatively dark.Figures 6(b) and 6(c) show the image brightness and themean radiance in all 3× 3 windows. Figure 6 (d) showsour measure of local contrast by the standard deviation


(a) (b) (c) (d) (e) (f)

Fig. 6. Illustration of the guidance map configuration. (a) An input HDR image. (b) Original radiance map Ih(i). (c)Map of mean radiance μi. (d) Map of standard deviation σi. (e) The visualized guidance map c. (f) The tone mappingresult using the guidance map shown in (e).

(a) (b) (c) (d)

(e) (f) (g) (h)

Fig. 7. Lamp desk. (a) An input HDR image. (b) The guidance map c generated from (a) using our method. (c) Atone-mapping result by our method. (d) The guidance map c generated from (c) using the same parameters. It isstructurally flat comparing to the map shown in (b). (e)-(h) Close-ups of (a)-(d).

(a) (b) β1 = 0.8, β2 = 0.0 (c) β1 = 0.7, β2 = 0.1 (d) β1 = 0.6, β2 = 0.2 (e) β1 = 0.5, β2 = 0.3

(f) (g) (h) (i)

Fig. 8. HDR compression with different parameter settings. (a) shows the original HDR image, displayed with linearscaling. (b)-(e) The tone mapping results with different parameter settings. Close-ups are shown in (f)-(i).


(a) (b) (c)

(d)β1 = 0.8, β2 = 0 (e)β1 = 0.7, β2 = 0.1 (f)β1 = 0.6, β2 = 0.2

(g) (h) (i)

(j) (k) (l)

Fig. 9. HDR compression with different settings of β1 and β2. (a) The input HDR image. (b) Visualization of the maplog2(μ + 1). (c) Visualization of the map log2(σ + 1). (d)-(f) The tone mapping results with different parameter settings.We magnify two regions and compare them in (g)-(l). Local contrast is enhanced more significantly using larger β2.

(a) Input (b) Input (c) β1 = 0.2 (d) β1 = 0.4 (e) β1 = 0.6 (f) β1 = 0.8

Fig. 10. HDR compression with different β1’s. (a) shows the HDR images under low exposures. (b) shows the HDRimages under high exposures. (c)-(f) show our HDR compression results. In all these results, we fix β2 = 0.1. Sinceβ1 + β2 controls the overall compression ratio, larger β1 + β2 makes the result exhibit more details in both bright anddark regions.

of radiance σi (images are scaled to aid visualization).This emphasizes strong gradients or edges. The termβ2 can be used to control the compression ratio onstrong edges, which in turn makes it possible to vacatemore space for revealing fine local details. We show ourtone mapping result in Figure 6(f), computed using theguidance map shown in Figure 6(e).

In order to preserve the details in both bright and darkregions, our algorithm balances local contrast and radi-ance values in the entire image. Intuitively, the guidancemap constructed from a medium to low dynamic rangeimage should be flatter in structure than that from anHDR image. To validate this, we show one example inFigure 7. The guidance map of an input HDR image (a)

is shown in (b). For comparison, the guidance map of theHDR compressed image (c) is shown in (d) constructedusing the same parameters. The guidance maps arelinearly scaled for illustration. It can be observed that theguidance values in (d) have much smaller variations.

4.1 Parameter Settings

The parameters β1 and β2 can be tuned to producedifferent tone mapping results. We assign the valuesbased on two principles. First, the sum of the threeβ’s influences global dynamic range compression. Largervalue makes the result visually flatter. Second, increasingβ2 enhances compression on strong edges, thus leavingmore space for less salient structures.


It is notable that, in our algorithm, small modificationsto β1 and β2 do not largely affect the result. The defaultvalues β1 = 0.6 and β2 = 0.2 are already capable ofproducing visually satisfactory results for most HDRimages we have tested. To illustrate the effect of differentparameter settings, we show an example in Figure 8.Image (a) shows the original HDR image. Our HDRcompression results are shown in (b)-(e) using differentparameter settings. In all these images, the structures onthe lamp and the desk corner are augmented. Close-upsshow that the text on the book is better enhanced witha larger β2.

Figure 9 shows another example. Image (a) shows aninput HDR image. The maps of μ and σ are shown in(b) and (c). The HDR compression results using differentparameters are shown from (d) to (f). In all these images,details are preserved. We show the magnified regions in(g)-(l) extracted from (d)-(f). The local contrast is betterpreserved with larger β2.

Figure 10 shows two examples where β1 is set differ-ently. The input HDR images under certain exposuresare shown in (a) and (b). We fix β2 = 0.1 and onlymodify β1. Images (c)-(f) respectively show our HDRcompression results with the increased sum of all β’s.It can be noticed that the original high dynamic rangeis compressed with gradually increased ratio, and moredetails are maintained in both bright and dark regionsusing larger β1.

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

x 106

0

5

10

15

20

25

30

35

40

45

Number of pixels in an HDR image

Run

ning

tim

e (s

)

Running time of our algorithm

Fig. 11. Plot of running time of our tone mapping algo-rithm with respect to different image resolutions.

4.2 Running Time

Our algorithm is implemented using Matlab on a desk-top PC with an Intel Core2Duo 2.4GHz CPU. The run-ning time to process different images is closely related tothe image resolution and is plotted in Figure 11. TakingFigure 12 as an example, the results can be obtainedwithin 2 seconds given the image containing 512 × 768pixels and with window size 3×3. We have found usinglarger window size for w needs more computation dueto solving larger linear systems. To produce the resultsshown in Figure 3(a-c), our method spends 3, 25, and150 seconds respectively in computation.

(a) (b) (c) (d)

(e) (f) (g) (h)

Fig. 12. Memorial Church. (a) Tone mapping result ofDurand and Dorsey [7]. (b) Tone mapping result of Fattalet al. [9]. (c) Result by the multiscale method using oneaggregated gain map [16]. (d) Our result with parametersβ1 = 0.7, β2 = 0.2, β3 = 0.1. (e)-(h) Magnified regionsfrom (a)-(d) respectively. HDR image courtesy of PaulDebevec.

(a) (b)

Fig. 13. Chairs. (a) Our tone mapping result with param-eters β1 = 0.6, β2 = 0.2, and β3 = 0.1. (b) The tonemapping result published in the project web page of [9].HDR image courtesy of Shree Nayar.

4.3 More Results and Comparison

In this section, we show more HDR compression resultsand compare them with other state-of-the-art techniques.

In Figure 12, the tone mapping results of the HDRimage “memorial church” are shown. The images in (a),(b), and (c) are published in [7], [9], and [16] respectively.Our result is illustrated in (d) with parameters β1 =0.7, β2 = 0.2, β3 = 0.1. The radiance of the dark regions,for instance the upper left corner of image, is elevated.The brightness of the stained glass window and theclerestory is reduced, revealing clear structural details.The textures and paintings on the wall and ceiling arealso preserved. The magnified local regions are shownfrom (e) to (h).

HDR compression results of the “chairs” example


Fig. 14. Street. HDR images c©Industrial Light & Magic. The top row shows the input HDR image under differentexposures. The bottom row shows our HDR compressed image reuslt and its close-ups, with parameters β1 = 0.6,β2 = 0.2, and β3 = 0.1. The radiance of the sun is largely reduced.

Fig. 15. More high dynamic range compression results by our method. The parameters that produce these results areidentical: β1 = 0.5, β2 = 0.2, and β3 = 0.1.

by our method and the gradient-based method [9] areshown in Figures 13(a) and 13(b) respectively. Moreresults using our optimization method are shown inFigures 14 and 15. The dynamic range is compressedfrom about 105 : 1 to 255 : 1. All structures, especiallythose originally buried in shadow and highlights, arenaturally enhanced. In Figure 16, we show a comparisonwith the result of Farbman et al. [8].

5 APPLICATIONS

5.1 Ordinary image enhancement

Our method can be extended to perform automatic en-hancement of ordinary LDR images in order to improvethe visibility of dark and heavily saturated regions con-taining structural information. The configuration of ci, inthis case, is similar to that introduced in Section 4. Sincethe ordinary image has much lower contrast, the valuesof β1, β2 and β3 are set smaller. In our experiments, thedefault values β1 = 0.4, β1 = 0.2, and β3 = 0.05 workwell for most examples.

Two examples are shown in Figure 17 using the default


(a) (b)

Fig. 16. Comparison with the result of Farbman et al. [8].(a) The tone mapped image shown in [8]. (b) Our tonemapping result.

parameter setting. In the input image (a), the mountainand its reflection in water are hardly seen. After ourimage enhancement, the trees and the green patternson the mountain together with the distorted reflectionare visible in (b). Images in (c) and (d) show anotherexample. The lichen on the tree in the original image isalmost unidentifiable while our result clearly enhancesits details.

(a) (b)

(c) (d)

Fig. 17. Ordinary image enhancement. (a) and (c) Theoriginal images. (b) and (d) The automatically enhancedimages showing more balanced local details and globalluminance.

5.2 HDR image synthesis

In this section, we present a method for synthesizing anHDR image from a single LDR image. In our optimiza-tion framework, this process can be regarded as spatiallyvariant dynamic range expansion. Note that in our HDRcompression model, we minimize the following least

square energy according to the local constraint in eachwindow i

fi =∑j∈wi

(I l(j) − piI

h(j) − qi

)2+ εc−2

i (pi − ci)2. (7)

Similarly, in the LDR expansion, we take Il as input andcompute Ih by simulating the global contrast expansionusing the same model. We rewrite (7) as

p2i

⎛⎝∑

j∈wi

(Ih(j) − 1

piI l(j) +

qi

pi

)2

+ εc−2i (1 − ci

pi)2

⎞⎠ (8)

by taking p2i out of the outermost parenthesis. Note that

there does not exist a closed-form solution in minimizing(8) because pi and Ih(j) are both unknowns and cannotbe separated in the proposed two-stage optimization. Forcomputation simplicity, we approximate the weight p2

i in(8) by c2

i . As a result, the HDR synthesis can be achievedby minimizing the energy

E′=∑

i

c2i

(∑j∈wi

(Ih(j) − 1

piI l(j) +

qi

pi

)2

+ εc−2i (1 − ci

pi)2)

=∑

i

c2i

(∑j∈wi

(Ih(j) − 1

piI l(j) +

qi

pi

)2

+ ε(c−1i − pi

−1)2)

.(9)

Denoting p′i = 1/pi, q′i = −qi/pi, and c′i = 1/ci, (9) canbe written as

f ′ = minp,q,Ih

∑i

c′−2i

(∑j∈wi

(Ih(j) − p′

iIl(j) − q′i

)2

+

ε(p′i − c′i)

2) , (10)

in a form similar to (3) defined for HDR compression.Thus, the HDR synthesis can be dealt with within ourframework by similarly employing the linear optimiza-tion.

Similar to that of HDR compression, we constructa guidance map c′ incorporating the image color andstructure information to appropriately constrain the LDRexpansion in (10). Specifically, we use the local contrastmeasure σ′

i in each window wi of I l, the local mean μ′i of

I l, and the image radiance I l(i) to construct the guidancemap, encouraging the exaggeration of local structures inappropriate degrees. The new guidance map for (10) isexpressed as

c′i = (μ′β1i σ′β2

i I l(i)β3 + κ),

where κ = 1 to maintain or expand the image contrast.The values of β1, β2, and β3 are defined similar to thosein Section 4.1. In our experiments, β3 is fixed to 0.1, β1 ∈[0.4 − 0.9] and β2 ∈ [0.1 − 0.4].

Equation (10) can be optimized by the algorithm de-scribed in the Appendix. Since (10) is slightly differentfrom (3), there exist variable differences on the expansionof Δi and S introduced in (14) and (18). Specifically, in


(a) Input (b) β1 = 0.2, β2 = 0.1 (c) β1 = 0.3, β2 = 0.1 (d) β1 = 0.4, β2 = 0.2 (e) β1 = 0.5, β2 = 0.2

Fig. 18. Image enhancement with different parameter settings. We fix β3 = 0.05 in this example. Larger parametervalues enhance more structural details.

(a)

(b)

(c)

(d)

(e)

Fig. 19. HDR image synthesis. (a) The input LDR image Il tone mapped from an HDR image Ih displayed in (b).(b) The original HDR image Ih displayed under different exposures. (c) The re-synthesized HDR image Ih′

by ouralgorithm (displayed under different exposures). (d) The synthesized HDR image using the LDR2HDR method [22].(e) The synthesized HDR image using linear scaling.

the HDR synthesis, we have new Δi and skj defined as

Δi=σ2i +

(ε

mi

),

skj =∑

i|{k,j}⊂wi

c′−2i (δkj − 1

miΔi((Ih(k) − μi)(Ih(j) − μi) + Δi)).

All other variables are unchanged. The radiance expan-sion can be computed by solving the linear system of(17) similarly.

We denote the HDR synthesis result as Ih′from the

LDR image Il. To evaluate the efficacy of our algorithm,

we assume that the I l is originally tone mapped from anHDR image Ih. By comparing Ih and Ih′

, we are ableto compute the ground truth expansion errors. In ourexperiments, we use Peak Signal-to-Noise Ratio (PSNR)to measure the ground truth error between the originalHDR image Ih and re-synthesized image Ih′

.

Two examples are shown in Figures 19 and 20. InFigure 19, the input LDR image I l shown in (a) iscompressed in dynamic range from an HDR image Ih

illustrated in row (b) under different exposures. Wesynthesize HDR image Ih′

from I l. The parameters are


(a)

(b)

(c)

Fig. 20. Indoor example. (a) The input LDR image Il compressed from an HDR image. (b) The original HDR imageIh shown under different exposures. (c) The re-synthesized HDR image Ih′

from I l shown under different exposures.

(a) (b) (c) (d)

Fig. 21. Night view example. (a) The input LDR image. (b)-(d) Our synthesized HDR scene, shown under differentexposures.

set as β1 = 0.5, β2 = 0.1, β3 = 0.1 for the example shownin Figure 19, and β1 = 0.6, β2 = 0.2, β3 = 0.1 for theexamples shown in Figures 20 and 21. For comparison,in rows (d) and (e) of Figure 19, we show the HDRimage results produced by the LDR2HDR method [22]and simple linear scaling. The PSNRs of the results inFigures 19 and 20 are 58.2873 and 57.1476 respectively ,with respect to the radiance of the original HDR images.

Figure 21 shows examples of our HDR image synthesisfrom ordinary 8-bit LDR images. Figure 21(a) showsthe input images. Images (b)-(d) are the synthesizedHDR images shown under different exposures. The high-contrast illumination on the buildings and the fountainsis naturally synthesized. The dynamic range is increasedfrom 200:1 to 106:1 which is not possible with a singleshot of a standard commodity camera. These examplesdescribe dynamic scenes with moving persons or foun-tains, which are difficult to be generated using tradi-tional methods involving multiple camera shots under

different exposures.

6 DISCUSSION AND CONCLUSION

This paper has introduced a novel high dynamic rangecompression method which effectively suppresses theglobal contrast while preserving local image structuredetails. We proposed a globally non-linear method thatuses overlapping window-based linear functions to re-construct the image radiance. From a global perspective,the overlapping of the windows makes the modificationof a pixel value be confined within a certain range largelydepending on its neighborhood. Range compression isachieved in our method using guided linear models.Our method can also be applied to ordinary imageenhancement and LDR image expanding in the sameframework. The parameter adjustment is easy where arange of values are suitable for tonal adjustment.

Our method is different from multiscales approachesthat require layer separation and combination steps. In


our approach, each p and q locally influence a group ofpixels which in turn globally guide the optimization. Wenote that although our method focuses on the automaticdynamic range compression, it is not difficult to add userinteractions for further interactive refinement. To modifythe local radiance, the user can use stroke brushes tospecify the desired values. The corresponding elementsare then fixed in solving (3). The user can also modifyci locally to adjust the level of details in the results.

APPENDIXAll the terms in the energy definition in (4) are quadraticmaking fi a continuous convex function. We thus com-pute (p∗i , q

∗i ), the optimal solution of (pi, qi), by taking the

partial derivatives of function fi with respect to pi andqi and setting them to zeros, similar to the derivationsin [15]:

∂fi

∂pi|pi=p∗

i ,qi=q∗i = 2εc−2i (p∗

i − ci) +∑j∈wi

2(I l (j) − p∗

i Ih (j) − q∗i

)·(−Ih (j)

)= 0, (11)

∂fi

∂qi|pi=p∗

i ,qi=q∗i =∑j∈wi

−2(I l (j) − p∗

i Ih (j) − q∗i)

= 0.(12)

(11) and (12) are linear equations with respect to pi andqi, thus can be written into a matrix form

Hi · [p∗i q∗i ]T = ηi, (13)

where Hi =[

εc−2i +

∑j∈wi

Ih(j)2∑

j∈wiIh(j)∑

j∈wiIh(j)

∑j∈wi

1

]and

ηi =[

εc−1i +

∑j∈wi

Ih(j) · I l(j)∑j∈wi

I l(j)

]. By solving (13), we

obtain

[p∗i q∗i ]T = H−1

i · ηi =1

mi · Δi

[1 −μi

−μi Δi + μ2i

]· ηi (14)

where

mi =∑j∈wi

1, μi =1

mi

∑j∈wi

Ih(j),

Δi = σ2i +

(εc−2

i

mi

).

After estimating pi and qi, we similarly compute I l, theoptimal solution of (3), by taking the partial derivativesof function f with respect to Il(k) and setting them tozeros:

∂f

∂I l(k)|Il=Il =

∑i|k∈wi

2(I l(k) − p∗

i Ih(k) − q∗i)

= 0. (15)

Since f is also a continuous convex function in differen-tiability class C∞ with respect to Il, the optimal solutionof I l by solving (15) can be obtained. Combining (14) and(15), we get

∂f

∂I l(k)|Il=Il =

∑i|k∈wi

(I l(k) − 1

miΔi(εc−1

i Ih(k) + Ih(k)

∑j∈wi

Ih(j)I l(j) − μiIh(k)

∑j∈wi

I l(j) − μiεc−1i −

μi

∑j∈wi

Ih(j)I l(j) + (Δi + μ2i )∑j∈wi

I l(j))),

= 0. (16)

3

i 3

3

......

3

Fig. 22. Illustration of the region containing pixel i. Allwindows w with size 3× 3 and containing i must be withinthe yellow region. There are at most 24 pixels in relationto i by local windows w. Since there are N pixels inan image, the total number of the non-zero elements inmatrix S will be less than 25 × N .

It can be observed that (16) is a linear combination ofI l’s. We can rewrite it in a form of a large linear system

S · I l = B, (17)

where

skj =∂2f

∂I l(k)∂I l(j)

=∑

i|{k,j}⊂wi

(δkj − 1

miΔi((Ih(k) − μi)(I

h(j) − μi) + Δi)),

bk =∑

i|k∈wi

ε

miΔici

(Ih(k) − μi

), (18)

where δkj is the Kronecker delta. The optimal solutionof (3) can be obtained by solving a linear system (17).

The matrix S in (17) is symmetric and sparse. In eachrow i of S, the number of nonzero elements is up toa defined region size according to (17). Given an inputimage containing N pixels and the size of the windowwi set to k × k, S has less than (2k − 1)2 × N non-zero elements (a k = 3 example is shown in Figure 22).Mathematically, the minimum possible window size is2×2, as there are three unknowns in each local windowand at least three linear constraints are needed to makethe problem well-posed.

If the image has moderate resolution, it is still possibleto directly apply the symmetric LQ or generalized mini-mum residual methods to solving the linear system (17)in Matlab. However, to process an image with resolutionhigher than 1000 × 1000, the above methods will takeseveral minutes to converge. To accelerate, a multigridmethod [10], same as the one used in [15], is employedwhich produces a satisfactory I l in only seconds in allour experiments.

ACKNOWLEDGMENTS

The authors would like to thank the associate editorand all reviewers for their constructive comments toimprove the manuscript. This work was supported bya grant from the Research Grants Council of the Hong


Kong Special Administrative Region, China (Project No.412708), a grant from SHIAE (No. 8115016), and an AcRFTier 1 Singapore grant (Project No. R-252-000-333-133).

REFERENCES

[1] A. O. Akyuz, R. Fleming, B. E. Riecke, E. Reinhard, and H. H.Bulthoff. Do HDR displays support LDR content? a psychophysicalevaluation. In SIGGRAPH, page 38, 2007.

[2] M. Ashikhmin. A tone mapping algorithm for high contrast im-ages. In Proceedings of the 13th Eurographics workshop on Rendering,pages 145–156, 2002.

[3] F. Banterle, P. Ledda, K. Debattista, and A. Chalmers. Inverse tonemapping. In GRAPHITE, pages 349–356, 2006.

[4] K. Chiu, M. Herf, P. Shirley, S. Swamy, C. Wang, and K. Zimmer-man. Spatially nonuniform scaling functions for hight contrastimages. In Graphics Interface, pages 245–253, 1993.

[5] K. Devlin. A review of tone reproduction techniques. TechnicalReport CSTR-02-005, 2002.

[6] J. Duan, G. Qiu, and M. Chen. Comprehensive fast tone mappingfor high dynamic range image visualization. In Pacific Graphics,2005.

[7] F. Durand and J. Dorsey. Fast bilateral filtering for the display ofhigh-dynamic-range images. In SIGGRAPH, pages 257–266, 2002.

[8] Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski. Edge-preserving decompositions for multi-scale tone and detail manip-ulation. In SIGGRAPH, 2008.

[9] R. Fattal, D. Lischinski, and M. Werman. Gradient domain highdynamic range compression. In SIGGRAPH, pages 249–256, 2002.

[10] W. W. Hager, S.-J. Huang, P. M. Pardalos, and O. A. Prokopyev.Multiscale Optimization Methods and Applications (Nonconvex Opti-mization and Its Applications). Springer, 2005.

[11] B. K. P. Horn. Determining lightness from an image. ComputerGraphics and Image Processing, 3:277–299, 1974.

[12] D. J. Jobson, Z. Rahman, and G. A. Woodell. A multiscaleretinex for bridging the gap between color images and the humanobservation of scenes. IEEE Transaction of Image Processing, 6(7):965–976, 1997.

[13] G. Krawczyk, K. Myszkowski, and H.-P. Seidel. Computationalmodel of lightness perception in high dynamic range imaging. InIS&T/SPIE’s Human Vision and Electronic Imaging, 2006.

[14] G. W. Larson, H. Rushmeier, and C. Piatko. A visibility matchingtone reproduction operator for high dynamic range scenes. IEEETransactions on Visualization and Computer Graphics, 3(4):291–306,1997.

[15] A. Levin, D. Lischinski, and Y. Weiss. A closed form solution tonatural image matting. In CVPR, 2006.

[16] Y. Li, L. Sharan, and E. H. Adelson. Compressing and compand-ing high dynamic range images with subband architectures. InSIGGRAPH, pages 836–844, 2005.

[17] D. Lischinski, Z. Farbman, M. Uyttendaele, and R. Szeliski. In-teractive local adjustment of tonal values. In SIGGRAPH, pages646–653, 2006.

[18] L. Meylan, S. Daly, and S. Susstrunk. The reproduction of specularhighlights on high dynamic range displays. In Color ImagingConference, 2006.

[19] S. N. Pattanaik, J. A. Ferwerda, M. D. Fairchild, and D. P.Greenberg. A multiscale model of adaptation and spatial visionfor realistic image display. In SIGGRAPH, pages 287–298, 1998.

[20] E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda. Photographictone reproduction for digital images. SIGGRAPH, 21(3):267–276,2002.

[21] E. Reinhard, G. Ward, S. Pattanaik, and P. Debevec. High DynamicRange Imaging. Morgan Kaufman, 2005.

[22] A. G. Rempel, M. Trentacoste, H. Seetzen, H. D. Young, W. Hei-drich, L. Whitehead, and G. Ward. Ldr2hdr: on-the-fly reversetone mapping of legacy video and photographs. In SIGGRAPH,page 39, 2007.

[23] C. Schlick. A customizable reflectance model for everyday ren-dering. In Fourth Eurographics Workshop on Rendering, pages 73–83,1993.

[24] T. G. Stockham. Image processing in the context of a visual model.Proceedings of the IEEE, 60(7):828–842, 1972.

[25] J. Tumblin. Three methods of detail-preserving contrast reduction fordisplayed images. PhD thesis, Georgia Institute of Technology, 1999.

[26] J. Tumblin and G. Turk. LCIS: a boundary hierarchy for detail-preserving contrast reduction. In SIGGRAPH, pages 83–90, 1999.

Qi Shan received the joint Bachelor degree fromFudan University in China and University Col-lege Dublin in Ireland in 2006 and the MPhildegree in computer science from the ChineseUniversity of Hong Kong in 2008. Currently, heis a PhD student in the Department of Com-puter Science and Engineering in the Universityof Washington, Seattle. His research interestsinclude computer graphics and vision. He is anIEEE student member.

Jiaya Jia received the PhD degree in computerscience from the Hong Kong University of Sci-ence and Technology in 2004. He joined the De-partment of Computer Science and Engineering,Chinese University of Hong Kong, in September2004, where he is currently an assistant pro-fessor. His research interests include vision ge-ometry, image/video editing and enhancement,and motion deblurring. He has served on theprogram committees of ICCV, CVPR, ECCV, andACCV. He served as co-chair of the interactive

computer vision workshop 2007 (in conjunction with ICCV07). He is amember of IEEE.

Michael S. Brown obtained his BS and PhD inComputer Science from the University of Ken-tucky in 1995 and 2001 respectively. He is cur-rently the Sung Kah Kay Assistant Professor inthe School of Computing at the National Univer-sity of Singapore. His research interests includeComputer Vision, Image Processing and Com-puter Graphics. He served as the general co-chair for the 5th ACM/IEEE Projector-Camera-Systems (PROCAMS08) workshop co-locatedwith SIGGRAPH08 and is an organizer for the

eHeritage09 Workshop co-located with ICCV09. He was an Area Chairfor CVPR’09. He is a member of IEEE.

Documents

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL