Seeing Mt. Rainier: Lucky Imaging forMulti-Image Denoising, Sharpening, and Haze Removal

Neel Joshi and Michael F. CohenMicrosoft Research

[neel,mcohen]@microsoft.com

AbstractPhotographing distant objects is challenging for a num-

ber of reasons. Even on a clear day, atmospheric haze oftenrepresents the majority of light received by a camera. Un-fortunately, dehazing alone cannot create a clean image.The combination of shot noise and quantization noise isexacerbated when the contrast is expanded after haze re-moval. Dust on the sensor that may be unnoticeable in theoriginal images creates serious artifacts. Multiple imagescan be averaged to overcome the noise, but the combina-tion of long lenses and small camera motion as well as timevarying atmospheric refraction results in large global andlocal shifts of the images on the sensor.

An iconic example of a distant object is Mount Rainier,when viewed from Seattle, which is 90 kilometers away.This paper demonstrates a methodology to pull out a cleanimage of Mount Rainier from a series of images. Rigidand non-rigid alignment steps brings individual pixels intoalignment. A novel local weighted averaging method basedon ideas from “lucky imaging” minimizes blur, resamplingand alignment errors, as well as effects of sensor dust, tomaintain the sharpness of the original pixel grid. Finally,dehazing and contrast expansion results in a sharp cleanimage.

1. IntroductionDistant objects present difficulties to photograph well.

Seeing detail obviously requires lenses with a very long fo-cal length, thus even small motions of the camera duringexposure cause significant blur. But the most vexing prob-lem is atmospheric haze which often leads to the majorityof photons arriving from scattering in the intervening mediarather than from the object itself. Even if the haze is fullyremoved, there are only a few bits of signal remaining, thusquantization noise becomes a significant problem. Othernoise characteristics of the sensor are also increased in thecontrast expansion following haze removal. Variations inthe density of air also cause refraction thus photons cannotbe counted on to travel in straight lines. Finally, small dustparticles on the sensor that cause invisible artifacts on theoriginal images can become prominent after haze removal.

Figure 1. Multi-Image Dehazing of Mount Rainier: Given multipleinput images, a sequence of rigid and non-rigid alignment and per-pixel weighted averaging, minimizes blur, resampling, and align-ment errors. Dehazing and contrast expansion then results in asharp clean image.

One such distant subject often photographed is MountRainier when viewed from Seattle, approximately 90 kilo-meters distant. For those who live in or visit Seattle, seeingthe mountain on a clear day is an exhilarating experience.One can just make out the glaciers which pour down fromits 14,411 foot peak rising from the sea. Unfortunately, inmost amateur photographs the mountain seems to simplydisappear. Even with a long lens and tripod on a clear day,the haze precludes creating a clean image of the mountain.

This paper demonstrates a methodology to create a cleanshot of a distant scene from a temporal series of images.Care is taken to align the images due to global camera mo-tion as well as considerable local time varying atmosphericrefraction. Noise reduction is achieved through a novelweighted image averaging that avoids sacrificing sharpness.Our main technical contribution is in the weight determi-nation. Significant loss of sharpness can occur due to theinteraction of the pixel grid with strong edges in the sceneas well as resampling due to sub-pixel alignment. We over-come this loss of sharpness through a novel weighted aver-aging scheme by extending ideas related to lucky imagingdeveloped in the astronomy literature.

Figure 2. Imaging Mount Rainier: Several processes occur that introduce errors in the captured images. The atmosphere absorbs, scatters,refracts, and blurs light rays, while the camera adds artifacts due to motion, defocus blur, dust, noise, and discrete sensor sampling. Ourmethod compensates for these multiple sources of error.

2. Related Work

There are three bodies of previous work that have themost influence on our current problem. Those are the liter-atures on denoising, image alignment and optical flow, anddehazing. We’ll discuss the most relevant work.

Denoising: Image denoising methods have been re-ported in a very wide and deep body of literature [11, 18,12, 13]. Most methods address the problem of denoising asingle image. In general, for each pixel, a weighted aver-aging is performed over a local neighborhood. The weightscan be as simple as a radially symmetric Gaussian (sim-ple smoothing), may be determined by their similarity tothe pixel being smoothed as in Bilateral Filters [18], or arebased on higher order local statistics [16]. If one has mul-tiple exact copies of an image, with each pixel corruptedindependently by Gaussian noise, the temporal stack of cor-responding pixels from each image can simply be averagedto remove the noise. Video denoising operates in a similarmanner. Typically, an alignment phase is first performedto align the spatial neighborhoods in each frame. Then aweight is determined for pixels in the aligned spatiotempo-ral neighborhood. The weights may be based on the confi-dence in the alignment [3], temporal similarity, not unlikespatial bilateral filtering, to avoid averaging over movingobjects for example [1], and/or other local statistics. In ourcase, we perform a weighted averaging of the pixel stacks,where the weights are determined from the local (spatial andtemporal) statistics as well as a model to avoid spatial re-sampling of pixel values due to sub-pixel alignment. Sincewe have a deep stack to choose from, we can highly weightonly a small percentage of the pixels and still achieve a gooddenoising. We extend ideas from lucky imaging [9, 6] in theastronomy domain for this purpose.

Alignment and Flow: Our task involves both perform-ing a rigid alignment of images caused by small rotations ofthe camera as well as local alignment of pixels due to time-varying air turbulence. Szeliski [17] gives a nice tutorial ofalignment and stitching methodologies. Similarly, there isa very rich literature on optical flow [5] for tracking pixels

that move small amounts from frame to frame. Our case isrelatively simple compared to finding general flow since themotion is spatially smooth, with no occlusions, and smallenough to use a simple patch based SSD search after theglobal alignment.

Dehazing: There has been considerable work on remov-ing haze from photographs. Haze removal is challengingbecause the haze is dependent on the scene depth which is,in general, unknown. Many methods use multiple images,such as a pair with and without a polarizing filter [14] ortaken under different weather conditions [10]. The differ-ences between the images are then used to estimate depthand the airlight color for dehazing. In some cases, depthcan be derived from external sources by geo-registering theimage to known 3D models[8]. Recently, single image hazeremoval [4, 7] has made progress by using a strong prior.Fattal [4] assumes the transmission and surface shading arelocally uncorrelated to derive the effects of haze. He etal. [7] propose an interesting dark channel prior. They ob-serve that for outdoor scenes, in any local region of a hazefree image, there is at least one channel of one pixel thatis dark. The presence and quantity of haze is therefore de-rived from the darkest pixel channel in some local region.We will use a variation of this work in our processing.

None of the above methods address the issue of noisewhen dehazing very distant objects obscured by a lot ofhaze. Most show results where they visual quality of dis-tant regions is improved by “adding back a bit of haze”.

3. Imaging The MountainTo create a clean image of Mount Rainier we will work

from a temporal series of images. For each of these images,It, we observe at each pixel, p, the following:It(p) =D(p)[B(p+ ∆t(p))⊗ [J(p+ ∆t(p))α(p+ ∆t(p))

+A(1− α(p+ ∆t(p))]] +Nt(p) (1)

where J(p) will represent a measure (after tone-mapping)of the true radiance reflecting from the mountain in a givendirection. ∆t(p) expresses the pixel’s offset due to the shiftsof the camera’s orientation and the air turbulence that may

have refracted the light to arrive from different direction.α(p+∆t(p)) expresses the attenuation of light due to atmo-spheric scattering, and A is the airlight. The total radiancerecorded at a pixel due to airlight goes up just as the trueradiance from the mountain is attenuated. B(p + ∆t(p))captures any blurring that may occur due to atmosphericscattering and in-camera defocus resulting in a point spreadon the image. D(p) is another attenuation factor due to duston the sensor. Finally Nt(p) is a zero mean additive noiseas a result of both quantization and shot noise. An exampleof one observation is shown in the upper half of Figure 1and in Figure 4(a).

Our goal is to extract an image which is as close as pos-sible to J(p) using a temporal series of such observations.Thus we must attempt to undo the spatial shifts ∆tp, as wellas remove the airlight and minimize the corruption due toblur, noise, and sensor dust. An example result is shown inthe bottom half of Figure 1 and in Figure 4(f).

3.1. Input and System Overview

We create a final image of Mount Rainier from a se-quence of 124 individual images shot at approximately 1 persecond on a Canon 1Ds Mark III camera at ISO 100 with a400mm lens. The aperture and exposure were at f/14 and1/200th second, respectively. The mountain only occupiedabout one quarter of the frame, so we cropped out a 2560 by1440 portion of the center of the frame for further process-ing. We also down-sampled the image to half-resolution,as we ran into memory limitations when processing at theoriginal image resolution.

The camera was mounted on a tripod but the shutter re-lease was operated manually. The images were recordedwith as JPEGs. Although the camera’s automated sensorcleaning was activated, as will be seen, small dust particlesbecome apparent.

We create our final image of Mount Rainier with the fol-lowing steps:

• Perform a global translational alignment of each im-age to a single image and average over the resultingimages.

• Compute pixel-wise optical flow to the globallyaligned average image, initialized by the global align-ment result for each image.

• For each pixel location, determine a pixel-wise weightfor each corresponding pixel in each image. Createda weighted average image from the set of normalizedweights.

• Dehaze the result.

We will describe each of these steps in more detail belowand provide intermediate results.

3.2. Image Alignment

The images of Mount Rainier are misaligned due to cam-era motion and temporally varying warping due to atmo-spheric refraction. Fortunately, while the misalignments arequite large, several aspects of our setup simplify the align-ment process significantly: 1) images taken from 90 kmaway with a long focal length are well modeled by an or-thographic camera model, 2) the scene is mostly static, thusall misalignment is due to the camera and atmosphere, 3)the lighting on the mountain is effectively static over therelatively short time the images were taken, and finally 4)sensor noise is reasonably low during the daylight shootingconditions.

Given these properties, a straightforward combination ofa global translation and local block-based flow allows us tocreate very well aligned images. In fact, we found moresophisticated methods, such as Black and Anandan’s well-known method [2], to perform worse, as the regularizationintended to handle the complexities of general scenes, (suchas occlusions and parallax, scene motion, noise, lightingchanges, etc.), led to overly smooth flow that did not alignthe small local features in our images.

Our alignment process proceeds in four steps. First, weperform a global translational alignment of each image toa single reference image using a full-frame alignment [15].Both the camera x, y translation and yaw and pitch rotationare modeled by translation, due to the orthographic projec-tion model. The remaining z translation is irrelevant alsodue to the orthographic projection. Any camera roll is han-dled in the next step.

Next, we average these globally aligned frames to pro-duce a reference frame for the local alignment process. Foreach pixel in each image, we compute the sum-of-squared-differences (SSD) between the 5 × 5 neighborhood aroundthe pixel and a corresponding translated window on the av-eraged image. The per pixel flow is chosen as the minimumSSD over a 1/2 pixel discrete sampling within [−5, 5] pix-els translation in x and y. This flow vector captures both thecamera roll and atmospheric warping. Lastly, the global andlocal translations are added to determine the offset, ∆t(p),for each pixel. These offsets are used to warp each inputimage, It using bilinear interpolation to produce a warpedresult, I ′t, such that I ′t(p) = It(p+ ∆t(p)).

It should be noted that all computations are done in float-ing point to avoid further quantization errors. Figures 4 and5 illustrate the affect of the image alignment process.

3.3. Determining Weights for Averaging

Once the images are aligned, they can be temporally av-eraged, (i.e., across a stack of pixels), to reduce both sensorand quantization noise. Unfortunately, a simple averagingof these pixels (Figure 4(c) and 5(g)) does not produce aresult with very high visual quality, due to the errors intro-

duced into the capture process as discussed in Section 3.Residual mis-alignments after flow warping, interpolationduring bilinear resampling, dust on the sensor, and vary-ing atmospheric blur all lead to artifacts when using only asimple average. To overcome these issues we developed anovel per-pixel weighting scheme that is a function of localsharpness.

There are two main properties we believe to be idealfor overcoming errors due to the atmosphere and alignmentprocess. Specifically, our weighting scheme is designedwith these two goals in mind:

1. To maximally suppress noise it is best to average overas many samples as possible, and

2. to maximize image sharpness it is best to only averageover a few well-aligned, sharp pixels.

It may seem that these goals are contradictory, and theyare in some sense – as the number of samples in the av-erage increase, if any of those samples are mis-aligned orblurred, the sharpness of the resulting image will decrease.Our approach to merging these goals is to break-down theper-pixel weight as a combination of sharpness weight anda “selectivity” parameter that governs how many samplesare averaged. For both of these aspects we drew partly onideas from from “lucky imaging”.

Lucky imaging is used in earth-based astronomic pho-tography to overcome warping and blurring due to the atmo-sphere. There are many similarities between the approach’sgoals and ours. Mackay et al. [9] compensate for atmo-spheric shifts and blurs by first ranking each image by asharpness measure which, in the domain of images of stars,is simply the maximum pixel value in the image. Then thetop N% (often 1% to 10%) of the images, ranked by sharp-ness, are aligned by computing a global translation – thisrepresents the “selectivity” of the averaging process. Theresulting images are averaged. Harmeling et al. [6] proposean online method that extracts signal from each image byestimating the PSF to update a final result.

We will use a combination of three weights and a selec-tivity measure to determine the final weight given to eachpixel. The weights measure local sharpness, resampling er-ror, and the presence of dust. The selectivity is based on ameasure of local variance to promote more noise reductionin smooth areas.

Sharpness Weight: In contrast with the astronomy do-main, simple intensity is not a meaningful sharpness mea-sure and, as we have densely textured images, a full framemetric is not appropriate. Instead, we compute a per-pixelweight that is a function of a local sharpness measure. Weuse the discrete Laplacian of the image as the local sharp-ness measure and set our sharpness weight proportional tothe magnitude of the Laplacian.

Specifically, consider L′t to be the convolution of anwarped input image I ′t with a 3×3 discrete Laplacian filter,and Lµ to be the Laplacian of the un-weighted mean image:

µ(p) =1

N

N∑t=1

I ′t(p), (2)

where p is a pixel and there are t = [1...N ] images. Theuse of Lµ is discussed later in this section. The sharpnessweight for a pixel is then:

w∗tex(p) = |L′t(p)|. (3)

We create a normalized weight, wtex(p), by linearly re-mapping the output range of the absolute value of the Lapla-cian to the range of [0, 1] .

Resampling: In addition, we consider that smoothingcan be introduced during global and local alignment, as theprocess requires pixels’ values to be estimated by an inter-polation of the original input pixel values. If an edge fallsacross integer pixel coordinates, it is known that the sub-pixel interpolation of that edge will be smoothed. To re-duce this type of smoothing, we have also derived a “re-sampling” weight that down-weights pixels interpolated atfractional pixel locations as a quadratic function of distanceof the fractional location from the nearest integer location.Specifically,

fsamp(p)=1−√frac(∆t(p)x)2 + frac(∆t(p)y)2 (4)

w∗samp(p)=fsamp(p)2. (5)

∆t(p) is the total alignment translation of pixel p, and the“frac” function returns the fractional distance to the nearestinteger, i.e., frac(x) = min(mod(x, 1), 1 − mod(x, 1)).We create a normalized resampling weight, wsamp(p) bylinearly re-mapping the output range of w∗samp(p) to therange of [ε, 1] . We map to a minimum value of ε insteadof 0 as we have observed qualitatively better results whenallowing the interpolated pixels have some small non-zeroweight. We have found ε = 0.1 to work well.

Selectivity: As discussed above, it is important to weighpixels not only by sharpness, but to also have a selectiv-ity parameter. The more selective, i.e., fewer pixels aver-aged, the sharper the result. One might think that being ex-tremely selective is ideal, which is the approach lucky imag-ing takes. However, this has as a downside, as with fewersamples, noise is not well suppressed. When averaging afixed number of images, an equal amount of denoising oc-curs across the entire image. In our image of Mount Rainier,this has an undesired affect: being less selective, (i.e., usingmany samples), denoised the sky well, but softens featureson the mountain, while fewer samples resulted in the moun-tain being sharp, but the sky containing unacceptable noise.

Thus, just as we found a full frame sharpness measureto be unsuitable for our images, we found a fixed selectiv-ity measure non-ideal. We developed a per-pixel selectively

measurement that is more selective in areas of high localtexture, (i.e. the mountain), and averages over more sam-ples in areas of low local texture, (i.e., the sky). Specifically,we implement this selectivity parameter as an exponent onour per-pixel weights which lie in [0, λ] for some large valueλ: wsharp(p) = (wsamp(p) ∗ wtex(p))γ(p). (6)

The exponent is calculated by first computing:γ∗(p) = |Lµ(p)|, (7)

and then we compute the exponent values γ(p) by linearlyre-mapping the output range of γ∗(p) to the range of [0, λ]for some large value λ. We have found λ = 10 to work wellin practice.

Dust Removal: Lastly, we also consider sensor dust. Tominimize the effect of sensor dust on the final image we canleverage the fact that the alignment shifts the dust aroundfrom image to image. We hand-mark dust spots (a singleimage of the clear sky can be used to automate this step)on a single initial input frame to create a binary dust mask,where a value of 1 indicated the presence of dust. We thenwarp this mask using the computed global alignment. Thedust weight is then: wdust(p) = 1− dust(p).

Only the global alignment is performed on the dust maskand the corresponding pixels in the input image, since thedust texture itself is not part of the true texture. The globalalignment shifts the dust over a big enough range so that forany output pixel there will be choices in the pixel stack thatdo not have dust covering them. This effectively removesthe dust spots from the final result.

Putting it all together: The final per-pixel weight in-cludes the dust mask simply as an additional multiplier todown-weight dust spots:

w(p) = wdust ∗ (wsamp(p) ∗ wtex(p))γ(p). (8)

Finally, we recover a denoised image as the weighted sumof warped images:

J(p) =

∑Nt=1 w(p)I ′t(p)∑Nt=1 w(p)

−A(1− α(p)). (9)

3.4. Dehazing and Constrast Expansion

Once we have a denoised image from the per-pixelweighted sum of aligned images, the final step is to de-haze the image. We adapt the dark channel method of He etal. [7] to model the haze and airlight color by surmising thatin any local region of a haze free image, there is at least onechannel of one pixel that is dark. The presence and magni-tude of the haze is derived from the darkest pixel channel insome local region.

The local region model of He et al. is appropriate formany natural scenes as they often have local dark regionsdue to shadows or high-frequency textures, e.g., as imagesof trees, urban environments, etc. However, in our image

Figure 3. Computing the Airlight Component using the DarkChannel Prior: (left) The initial estimate of the dark channel foreach pixel is the darkest value per horizontal scanline. The dashedline shows where we set the airlight contribution equal to that forthe top of the mountain, which dehazes the sky region up to thedepth of the top of the mountain. (right) Finally, as the dark-channel values are noisy across scanlines, we smooth the values.

of Mount Rainier there are many large local areas with nodark values, such as the large white glaciers. Thus the localmodel is not appropriate.

Instead, we note that as the haze amount and thus thedark channel value is proportional to depth; any neighbor-hood which captures a constant depth and has dark regionscan be used to measure the dark channel value. As anyonewho has flown into a metropolitan area has witnessed, theair quality and color often takes on a layered appearance.Due to the relatively conical shape of the volcano as wellas the haze’s relationship with altitude, we assume that thehaze is effectively constant per scan-line. In contrast withprevious work, we do not assume a single global airlightcolor [4, 7]. Instead the airlight color can vary per-scanline.We have found this necessary for our images where theairlight color appears quite different towards the bottom ofthe mountain (see Figure 3).

We estimate the dark channel value as the darkest valueper horizontal scanline:

[A(1− α(p))] = minWx=1I(p), (10)

where p is pixel and W is the image width.We process the per-scanline minimum in two ways. The

dark channel value is somewhat meaningless in the sky re-gion, as this region is completely airlight. In previous work,pure sky regions were often simply ignored or masked out.We instead choose to set the airlight color for the sky abovethe mountain top to be equal to that at the top of the moun-tain. This effectively dehazes the sky region up to the depthof the mountain. Furthermore, as the dark-channel val-ues can be somewhat noisy from scanline-to-scanline, wesmooth the dark channel image in the vertical direction us-ing a broad 1D Gaussian filter. Figure 3 shows plots of theper-scanline dark channel values.

The final dehazed image is computed as I(p) − [A(1 −α(p))], for an image I . This dehazing operation is not onlyvalid for our final weighted mean. In the result section, wewill show dehazing applied at various stages of our process-ing pipeline to illustrates the affect of each stage.

Finally, we stretch the contrast by a linear remapping ofthe luminance to the full image range of [0, 1]. We color bal-ance the final image using the gray granite of the mountainand white glaciers as as a gray and white point.

4. ResultsWe demonstrate the results through a series of images

and detail crops. The image in Figure 4(a) shows a sin-gle input image, It(p). All further images demonstrateintermediate results of the pipeline after dehazing (Beforedehazing the differences are almost imperceptible.) Fig-ure 4(b) shows the same input image after haze removal,It(p)−A(1−α(p)). The effects of noise and dust becomesapparent. Before performing any processing on the images,we crop our full-frame 21 MP images to include on the rel-evant sections of the mountain and remove the gamma cor-rection factor of 1.24, which we calibrated by imaging thegray panels on a Macbeth Color Checker, from the JPEGimages. We re-apply a gamma correction of 1.45 for dis-playing our results.

We also show the effect of simply averaging the temporalsamples. Figure 4(c) represents∑N

t=1 It(p)

N−A(1− α(p)) (11)

after averaging and dehazing. The averaging removes thenoise but also blurs considerably due to camera motion andair turbulence.

Removing the global motion of each image and averag-ing removes much of the blur as can be seen in Figure 4(d).Adding the local flow into the pixel motion further refinesthe image (Figure 4(e)):∑N

t=1 I′t(p))

N−A(1− α(p)). (12)

Finally, by weighting each sample as described in Sec-tion 3.3 we achieve our final result:∑N

t=1 w(p)I ′t(p)∑Nt=1 w(p)

−A(1− α(p)), (13)

which can be seen in Figure 4(f).Figure 5 shows zoomed-in side-by-side comparisons of

two regions on the mountain for each of the results pre-sented above. Each results shows progressively increasingimage quality, as a function of decreasing noise and increas-ing sharpness. Our final result, that uses full alignment andour novel per-pixel weights is significantly sharper than anyof the other results.

5. DiscussionWe have shown that with careful registration and select-

ing the most reliable pixels, multiple images can providea sharp clean signal. The key contribution of this work is

the concept that such an image can be captured through 90kilometers of hazy air. The main technical contribution isin the choice of weights based on local sharpness measuresand resampling. While we have used these weights for de-noising images for input to a dehazing process, we believeour weighting methodology would improve general multi-image denoising algorithms.

A second contribution is the use of spatially-varying (inour case, per scanline) airlight color when performing de-hazing. We have found this necessary for the scene we con-sider, and it is likely important for dehazing any large andvery distant outdoor object.

One might consider which parts of the process can beachieved on the sensor. If the camera is static and there isno air turbulence, the main problem becomes one of remov-ing the airlight before it saturates the pixels. A sensor couldopen an electron drain per pixel or small patch. The draincould be set equivalent to the electron gain from a large per-centage of the minimum incoming radiance over the patch.This would minimize the quantization noise by allowing alonger exposure to spread the signal over more bits in thesensor range. The final image plus the “drain” image, whichwould approximate the airlight, would need to be recordedto recover the full image. The effect would be similar to thatoutlined in the Gradient Camera [19]. It is less clear howany blur due to camera motion and air turbulence could beminimized.

Our work demonstrates overcoming one specific difficultimaging scenario. We hope this paper inspires further workin capturing difficult to image scenes.

References[1] E. P. Bennett and L. McMillan. Video enhancement using

per-pixel virtual exposures. In SIGGRAPH ’05, pages 845–852, New York, NY, USA, 2005. ACM.

[2] M. J. Black and P. Anandan. A framework for the robustestimation of optical flow. In Computer Vision, 1993, pages231–236, 1993.

[3] J. Chen and C.-K. Tang. Spatio-temporal markov randomfield for video denoising. In CVPR ’07, pages 1–8, June2007.

[4] R. Fattal. Single image dehazing. In SIGGRAPH ’08, pages1–9, New York, NY, USA, 2008. ACM.

[5] D. Flett and Y. Weiss. Optical flow estimation. In N. Para-gios, Y. Chen, and O. Faugeras, editors, Handbook of Math-ematical Models in Computer Vision, chapter 15, pages 239–258. Springer, 2005.

[6] S. Harmeling, M. Hirsch, S. Sra, and B. Scholkopf. Onlineblind deconvolution for astronomical imaging. In ICCV ’09,May 2009.

[7] K. He, J. Sun, and X. Tang. Single image haze removal usingdark channel prior. pages 1956–1963, 2009.

[8] J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or,O. Deussen, M. Uyttendaele, and D. Lischinski. Deep photo:

(a) Original Single Input Image (b) Dehazed Single Input Image

(c) Dehazed Mean of Input Images (d) Dehazed Mean of Globally Aligned Images

(e) Dehazed Mean of Globally + Locally Aligned Images (f) Dehazed Weighted Mean of Globally + Locally Aligned Images

Figure 4. Dehazing Results: (a) A single input image. (b) The dehazed single image is very noisy and does not show very much detail. (c)Due to camera movement and local shifts due to atmospheric refraction, taking the mean across the input images results in a very blurryresult. (d) Global alignment improves the result, while (e) adding local alignment leads to an even sharper result. (e) In our final result,per-pixel weights lead to increased sharpness on the mountain, while smooth regions such as the sky are denoised successfully. Dust spotsare also removed. Note: We have not shown the pre-dehazing images for results (b)–(e) as before dehazing there is almost no perceptualdifference when compared to image (a). Only after dehazing and contrast expansion are the differences are very apparent.

Model-based photograph enhancement and viewing. ACMTransactions on Graphics (SIGGRAPH Asia), 27(5):116:1–116:10, 2008.

[9] C. D. Mackay, J. Baldwin, N. Law, and P. Warner. High-resolution imaging in the visible from the ground withoutadaptive optics: new techniques and results. volume 5492,pages 128–135. SPIE, 2004.

[10] S. Narasimhan and S. Nayar. Removing weather effects from

monochrome images. In CVPR ’01, pages II:186–193, 2001.[11] P. Perona and J. Malik. Scale-space and edge detection using

anisotropic diffusion. PAMI, 12(7):629–639, 1990.[12] J. Portilla, V. Strela, M. Wainwright, and E. Simoncelli.

Image denoising using scale mixtures of gaussians in thewavelet domain. IEEE TIP, 12(11):1338–1351, 2003.

[13] S. Roth and M. J. Black. Fields of experts: A framework forlearning image priors. In CVPR ’05, pages 860–867, 2005.

(f) Single Image (g) Mean (h) Global (i) Global+Local (j) Weighted

Figure 5. Detailed Dehazing Results: Cropped zoom-in regions, as indicated by yellow boxes on our final result (top), show the progressingincrease in image quality, as a function of decreasing noise and increasing sharpness, from single to multi-image dehazing with variousstages of alignment and weighting of the images. Our final result, with full alignment and our novel per-pixel weights, is significantlysharper than any of the other results.

[14] Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar. In-stant dehazing of images using polarization. In CVPR ’01,volume 1, pages 325 – 332, June 2001.

[15] H.-Y. Shum and R. Szeliski. Construction of panoramic im-age mosaics with global and local alignment. Int. J. Comput.Vision, 36(2):101–130, 2000.

[16] E. Simoncelli and E. Adelson. Noise removal via bayesianwavelet coring. Image Processing, 1996, 1:379–382 vol.1,

1996.[17] R. Szeliski. Image alignment and stitching: a tutorial. Found.

Trends. Comput. Graph. Vis., 2(1):1–104, 2006.[18] C. Tomasi and R. Manduchi. Bilateral filtering for gray and

color images. Computer Vision, 1998, pages 839–846, 1998.[19] J. Tumblin, A. Agrawal, and R. Raskar. Why i want a gradi-

ent camera. In CVPR ’05, pages 103–110, Washington, DC,USA, 2005. IEEE Computer Society.