1

A

PROJECT REPORT

By

Pratyush Pandab1

Under The Guidance Of

Prof. B. Majhi

Professor and Head of Dept. Computer Science and Engineering, NIT Rourkela

Department of Computer Science Engineering

NATIONAL INSTITUTE OF TECHNOLOGY ROURKELA

ROURKELA-769008 (ORISSA)

1 Pratyush Pandab is a 4th year B. Tech student from Dept. of Computer Science and Engineering at CET, BBSR..

HIGH DYNAMIC RANGE IMAGING

2

This is to certify that the work in this Project entitled “High Dynamic Range

Imaging” by Pratyush Pandab has been carried out under my supervision during

the Summer Internship 2009 at NIT RKL in the Department of Computer

Science and Engineering.

Place: Rourkela Prof. Banshidhar Majhi

Date: July 15, 2009 Professor and Head of Dept. of CSE

NIT, Rourkela.

Certificate

National Institute of Technology Rourkela

3

I hereby declare that the project report entitled “High Dynamic Range Imaging”

is a record of bonafide project work carried out by me under the guidance of

Prof. B. Majhi of NIT- Rourkela, Orissa.

Pratyush Pandab

Dept. of CSE, CET

Bhubaneswar.

Declaration by the Candidate

4

Abstract ......................................................................................... 5 1. Introduction to HDRI ........................................................ 1.1. A Brief History of HDRI 1.2. Traditional Techniques used 1.3. Recent Technology 1.4. Tone Mapping

6 6 7 7 8

2. Problem Statement ..............................................................

9

3. Sample Example ..................................................................

10

4. Background Research .......................................................... 11 4.1. Recovering High Dynamic Range Radiance Map from

Photographs 11-12

4.1.1. The Algorithm 12 4.1.1.1. Film Response Recovery 13-16 4.1.1.2. Construction of High Dynamic Range Radiance

Map. 17

4.1.2. Implementation 17-18 4.2. A Simple Spatial Tone Mapping Operator for High

Dynamic Range Images. 19

4.2.1. The Need for Tone Mapping. 19 4.2.2. The Algorithm 19-20 4.2.2.1. Cases of Interest 20-21 4.2.3. Implementation

21-22

5. Implementation Results ......................................................

23-26

6. Application of HDRI ..........................................................

27

7. Conclusions ..........................................................................

28

8. References ............................................................................

28.

Table of Contents

5

In image processing, computer graphics, and photography, High Dynamic Range Imaging

(HDRI) is a set of techniques that allows a greater dynamic range of luminance between light

and dark areas of a scene than normal digital imaging techniques. The intention of HDRI is to

accurately represent the wide range of intensity levels found in real scenes ranging from

direct sunlight to shadows.

Information stored in high dynamic range images usually corresponds to the physical values

of luminance or radiance that can be observed in the real world. This is different from traditional digital images, which represent colors that should appear on a monitor or a paper print.

One problem with HDR has always been in viewing the images. Typical computer monitors

(CRTs, LCDs), prints, and other methods of displaying images only have a limited dynamic

range. Various methods of converting HDR images into a viewable format have been developed, generally called Tone Mapping.

Recent methods have attempted to compress the dynamic range into one reproducible by

the intended display device. The more complex methods tap into research on how the

human eye and visual cortex perceive a scene, trying to show the whole dynamic range while retaining realistic color and contrast.

Figure 1: Here the dynamic range of the image is demonstrated by adjusting the "exposure" when tone -mapping the HDR image into an LDR one for display.

In this project, it is considered that nearly any camera can actually capture a vast dynamic

range-- just not in a single photo. By varying the shutter speed alone, most digital cameras can change how much light they let in by a factor of 50,000 or more. This project attempts to combine these captured images into a single High Dynamic Range Image by utilizing the different exposures characterized by these images composed of multiple exposures. The difficulties that accompany HDRI are Halo effects and artifacts. Care has been taken to avoid these pitfalls. The complete project was simulated using Matlab 7.6 software. The simulation results are shown as figures. Results have shown very high quality rendering of

images by this technique.

Abstract

6

High Dynamic Range Imaging (HDRI) is a set of techniques that allows a greater dynamic

range of luminance between light and dark areas of a scene than normal digital imaging techniques. High dynamic range (HDR) images enable photographers to record a greater range of tonal detail than a given camera could capture in a single photo. This opens up a whole new set of lighting possibilities which one might have previously avoided. The intention of HDRI is to accurately represent the wide range of intensity levels found in real scenes ranging from direct sunlight to shadows.

In the past, photographers dealt with the limitation of dynamic range by experimenting

with media. Ansel Adams was perhaps the first to systematically measure the sensitivity range of all of the equipment he used to precisely indicate what the photograph would display depending on the length of time the shutter was open. Before HDR was put into

practice, photographers and film makers needed to find ways around the limitations of modern digital imaging which fail to capture the total perceptual display that the human eye

is capable of seeing. The problem is that real-world scenes contain light ranges that exceed a 50,000:1 dynamic range2 while digital equipment is limited to around 300:1 dynamic

range.

Also, through Adams' work it was discovered that one way to get really great dynamic

range with colour photography is to use black-and-white film and colour filters. Although this was a practical solution, as technology advanced so did the digital imaging market, thus creating HDR.

Images tend to result in the subject seen and the surrounding blanked out.

Figure 2: In the first image the shelf, the table and the sofa set are the subject, whereas the window is the surrounding. Not much is seen outside the window, but the details on shelf and the table are seen pretty

clearly. In the second image the surrounding is clear enough but nothing is visible inside the room. This is the problem caused due to the dynamic range of the capturing equipment.

2 Dynamic Range: The dynamic range is the ratio between the maximum and minimum values of a physical

measurement. Its definition depends on what the dynamic range refers to.

1. Introduction to HDRI

1.1. A Brief History of HDRI

7

Most photographic systems struggle on high contrasts scenes. With the entire modern

technology one would be wrong to think a digital camera would resolve the issues related to dynamic range. The reason for this is simple, current film and image sensors are developed to record enough light that can be reproduced on paper or compute r screen and neither are currently capable of displaying a dynamic range that we can see with our eyes. Until recently, to overcome this, digital photographers have either been using the old approach of ND graduated filters at the taking stage or combining the best bits of two different exposures on the computer, much like the dodge & burn3 technique used by experienced darkroom users.

New technology is on the way to record and display at higher dynamic ranges, but for now

there are a range of software solutions.

The job of HDR processing is to allow photographers to produce images with a much larger

dynamic range. The photographer takes a series of different exposures of the same scene and then uses HDR software to automatically merge the resulting images into one image, referred to as exposure blending. This produces a file with the entire desired dynamic range, known as a radiance map. Ideally the fewer images used the better, because it's less

memory intensive and there's less chance of subject been out of alignment on each shot as a result of camera or subject movement.

HDR uses 16 to 32 bits values. This jump in resolution allows for a much greater range of

colours, greater than the monitor can display at once. As a result, to view these images, the computer breaks the images into levels of brightness and darkness. The brightest part of the image are viewed with the brightest colours the monitor can display the darkest are viewed

with the darkest the screen can display. This allows for each image to display details, bright highlights and dark shadows.

3Dodging and Burning are terms used in photography for a technique used during the printing process , to

manipulate the exposure of a selected area(s) on a photographic print, deviating from the rest of the image's exposure. Dodging decreases the exposure for areas of the print that the photographer wishes to be lighter, while burning increases the exposure to areas of the print that should be darker.

1. 2 Traditional Techniques Used

1. 3 Recent Technology

8

The above mentioned merged photographs are saved as 32-bit. This means there's very

little one can do to edit the photos, they cannot be saved as JPG without reducing the bit-depth4 and one cannot view the superb dynamic range on a conventional monitor. In such cases, the whole dynamic range of the scene (e.g. 100,000:1 or higher) has to be reproduced on a medium with less than 100:1 contrast ratio. Tone mapping deals with the issue of reproducing the dynamic range captured. This is why it becomes necessary to use techniques that scale the dynamic range down while preserving the appearance of the original image captured. One way around this is to apply tone mapping which converts the HDR image into a balanced exposure.

Tone mapping operators are divided into two broad categories, Global and Local

Global operators are simple and fast. They map each pixel based on its intensity and global

image characteristics, regardless of the pixel's spatial location. Global operators are OK for mapping 12-bit sensor data but usually don't work well with HDR images. An example of a

global type of tone mapping is a tonal curve.

Local operators take into account the pixel's surroundings for mapping it. This means that a

pixel of a given intensity will be mapped to a different value depending on whether it is located in a dark or bright area. This makes local operators slower but tends to produce more pleasing results, given that our eyes react locally to contrast.

4 Bit depth of a capturing or displaying device gives an indication of i ts dynamic range capacity, i .e. the highest dynamic

range that the device would be capable of reproducing.

1. 4 Tone Mapping

9

Here is the problem in a nutshell:

a. Real-world scenes contain light ranges that exceed a 50,000:1 dynamic range.

b. Media has been limited to around a 300:1 dynamic range.

c. So we have a mapping issue: how do we represent light values in a scene using a

much more limited set of light values for a particular media?

d. What if the set of images with different exposures have unknown noise in it? Can we

get the HDR image sans the noise (noise gets added, so it has to be removed.).

2. Problem Statement

10

Figure 3 : Sample input set of images with different exposure time.

Figure 4: Output; tone mapped HDR image. (Note the details present in this single image.)

3. Sample Example

11

The method of recovering high dynamic range radiance maps from photographs taken with

conventional imaging equipments [𝐼] is presented in this project work. The proposed algorithm by Debevec & Malik [𝐼] uses these differently exposed photographs to recover the response function of the imaging process, up to factor of scale, using the assumption of reciprocity. With the known response function, the algorithm can fuse the multiple photographs into a single, high dynamic range radiance map whose pixel values are proportional to the true radiance values in the scene.

Besides this, the project also implements a simple and effective tone mapping operator

proposed by Biswas & Pattanaik [𝐼𝐼], that preserves visibility and contrast impression of high dynamic range images. It use a s-function type operator which takes into account both

the global average of the image, as well as local luminance in the immediate neighbourhood of each pixel. The local luminance is computed using a median filter. It is seen that the

resulting low dynamic range image preserves fine details, and avoids common artefacts such as halos, gradient reversals or loss of local contrast.

Digitized photographs are becoming increasingly important in computer graphics. When

one photographs a scene, either with film or an electronic imaging array, and digitizes the

photograph to obtain a two dimensional array of brightness5 values, these values are rarely true measurements of relative radiance in the scene. For example, if one pixel has twice the

value of another, it is unlikely that it observed twice the radiance6. Instead, there is usually an unknown, nonlinear mapping that determines how radiance in the scene becomes pixel

values in the image. This nonlinear mapping is hard to know beforehand because it is actually the composition of several nonlinear mappings that occur in the photographic process.

In a conventional camera (see Fig.5), the film is first exposed to light to form a latent image.

The film is then developed to change this latent image into variations in transparency, or

density, on the film. The film can then be digitized using a film scanner, which projects light through the film onto an electronic light-sensitive array, converting the image to electrical voltages. These voltages are digitized, and then manipulated before finally being written to the storage medium. If prints of the film are scanned rather than the film itself, then the printing process can also introduce nonlinear mappings.

5 Brightness values represent the gray levels of the image.

6 Radiance is the i rradiance/luminance value of the object at the si te of the scene.

4. Background Research

4.1. Recovering High Dynamic Range Radiance Maps from Photographs [𝐼]

12

In the first stage of the process, the film response to variations in exposure 𝑿7 is a non-

linear function, called the “characteristic curve” of the film. The development, scanning and digitization processes usually introduce their own nonlinearities which compose to give the

aggregate nonlinear relationship between the image pixel exposures 𝑿 and their values 𝒁8.

As with film, the most the most significant nonlinearity in the response curve is at its

saturation point, where any pixel with radiance above a certain level is mapped to the same maximum image value. The most obvious difficulty is that of limited dynamic range—one

has to choose the range of radiance values that are of interest and determine the exposure time suitably. Sunlit scenes, and scenes with shiny materials and artificial light sources,

often have extreme differences in radiance values that are impossible to capture without either under-exposing or saturating the film. To cover the full dynamic range in such a

scene, one can take a series of photographs with different exposures. This then poses a problem: how can we combine these separate images into a composite radiance map? Here the fact that the mapping from scene radiance to pixel values is unknown and nonlinearity begins to haunt us.

A simple technique for recovering this response function, up to a scale factor, using

nothing more than a set of photographs taken with varying, known exposure durations has been the proposed work by Debevec & Malik [𝐼] and is simulated in this project. With this

mapping, the pixel values from all the available photographs are used to construct an accurate map of the radiance in the scene, up to a factor of scale. This radiance map will

cover the entire dynamic range captured by the original photographs.

This section presents the algorithm cited in [𝐼], for recovering the film response function,

and then presents the method of reconstructing the high dynamic range radiance image from the multiple photographs. The algorithm assumes gray scale images, but can be extended to colour images with proper conversions9. 7 𝑿 = 𝑬𝜟𝒕 ; i .e. 𝑿 is the product of the i rradiance 𝑬 the film receives and the exposure time 𝜟𝒕.

8 𝒁 is the final digi tal value after processing. Refer fig. 5.

9 We can convert RGB images to YCbCr images and consider only the Y (luminance) component or else we can consider

each component individually and plot the mapping.

Figure 5 : Image Acquisition Pipeline: shows how scene radiance becomes pixel values for both film and digital cameras. Unknown nonlinear mappings can occur during exposure, development, scanning, digitization, and remapping.

Lens Shutter Film Development CCD ADC Remapping

Digital camera

𝑿 𝜡 𝜠

Scene radiance

Sensor irradiance

Sensor exposure Latent image Film density Analog voltage

Digital values

Final digital values

4.1.1. The Algorithm

13

This algorithm is based on exploiting a physical property of imaging systems, both

photochemical and electronic, known as reciprocity.

Let us consider photographic film first. The response of a film to variations in exposure is

summarized by the characteristic curve (or Hurter-Driffield curve) [𝐼]. This is a graph of the optical density 𝑫 of the processed film against the logarithm of the exposure 𝑿 to which it

has been subjected. The exposure 𝑿 is defined as the product of the irradiance 𝑬 at the film and exposure time, 𝜟𝒕, so that its units are 𝑱𝒎−2. Key to the very concept of the

characteristic curve is the assumption that only the product 𝑬𝜟𝒕 is important, and that halving 𝑬 and doubling 𝜟𝒕 will not change the resulting optical density 𝑫. Under extreme conditions (very large or very low 𝜟𝒕 ), the reciprocity assumption can break down, a

situation described as reciprocity failure. In typical print films, reciprocity holds to within 1

3

stop10 for exposure times of 10 seconds to 1

10000 of a second.

After the development, scanning and digitization processes, we obtain a digital number 𝒁,

which is a nonlinear function of the original exposure 𝑿 at the pixel. Let us call this function 𝒇, which is the composition of the characteristic curve of the film as well as all the nonlinearities introduced by the later processing steps. The first goal is to recover this function 𝒇. Once we have that, we can compute the exposure 𝑿 at each pixel, as

𝑿 = 𝒇−𝟏(𝒁). A reasonable assumption made is that the function 𝒇 is monotonically

increasing [𝐼], so its inverse 𝒇−𝟏 is well defined. Knowing the exposure 𝑿 and the exposure time 𝜟𝒕, the irradiance 𝑬 is recovered as 𝑬 = 𝑿/𝜟𝒕, which will be taken to be

proportional to the radiance 𝑳 in the scene.

The input to the algorithm is a number of digitized photographs taken from the same vantage point with different known exposure durations 𝜟𝒕𝒋.

11 It is assumed that the scene is

static and that this process is completed quickly enough that lighting changes can be safely ignored. It can then be assumed that the film irradiance values 𝑬𝒊 for each pixel 𝒊 are constant. Pixel values are denoted by 𝒁𝒊𝒋 where 𝒊 is a spatial index over pixels and 𝒋 indexes

over exposure times 𝜟𝒕𝒋. Thus the film reciprocity equation is:

Since it is assumed 𝒇 is monotonic, it is invertible, and so we can rewrite (𝟏) as:

Taking the natural logarithm of both sides of (𝟐) , we have:

10 1 stop is a photographic term for a factor of two ;

1

3 s top is thus 2

13 .

11 The actual exposure times are varied by powers of two between s tops (1

64,

1

32,

1

16,

1

8,

1

4,

1

2, 1, 2, 4, 8, 16, 32), rather

than the rounded numbers displayed on the camera readout (1

60,

1

30,

1

15,

1

8,

1

4,

1

2, 1, 2, 4, 8, 15, 30).

4.1.1.1. Film Response Recovery

𝒁𝒊𝒋 = 𝒇(𝑬𝒊𝜟𝒕𝒋) (𝟏)

𝒇−𝟏(𝒁𝒊𝒋) = 𝑬𝒊𝜟𝒕𝒋 (𝟐)

𝐥𝐧 𝒇−𝟏(𝒁𝒊𝒋) = 𝐥𝐧𝑬𝒊 + 𝐥𝐧𝜟𝒕𝒋 (𝟑)

14

To simplify notation, let us define a function, 𝒈 = 𝐥𝐧 𝒇−𝟏 . We then have the set of

equations: where 𝒊 ranges over pixels and 𝒋 ranges over exposure durations. In this set of equations, the 𝒁𝒊𝒋 are known, along with their corresponding 𝜟𝒕𝒋. The unknowns are the irradiances 𝑬𝒊

as well as the function 𝒈, although it is assumed that 𝒈 is smooth and monotonic.

Our aim now is to recover the function 𝒈 and the irradiances 𝑬𝒊 that best satisfy the set of

equations arising from Equation 𝟒 in a least-squared error sense. It can be verified that recovering 𝒈 only requires recovering the finite number of values that 𝒈(𝒛) can take since the domain of 𝒁, pixel brightness values, is finite. Let 𝒁𝒎𝒊𝒏 and 𝒁𝒎𝒂𝒙 be the least and greatest pixel values (integers), 𝑵 be the number of pixel locations, and

𝑷 be the number of photographs.

Thus the existing problem can be formulated as one of finding the (𝒁𝒎𝒊𝒏 − 𝒁𝒎𝒂𝒙 + 𝟏)

values of 𝒈(𝒁) and the 𝑵 values of 𝐥𝐧𝑬𝒊 that minimize the following quadratic objective

function:

The first term ensures that the solution satisfies the set of equations arising from Equation

(𝟒) in a least squares sense. The second term is a smoothness term on the sum of squared values of the second derivative of 𝒈 to ensure that the function 𝒈 is smooth. In this discrete

setting 𝒈′′ 𝒛 = 𝒈 𝒛− 𝟏 − 𝟐𝒈 𝒛 + 𝒈(𝒛 + 𝟏) is used. This smoothness term is essential to the formulation in that it provides coupling between the values 𝒈(𝒛) in the minimization.

The scalar 𝝀 weights the smoothness term relative to the data fitting term, and should be chosen appropriately for the amount of noise expected in the 𝒁𝒊𝒋 measurements.

Because it is quadratic in the 𝑬𝒊’s and 𝒈(𝒛)’s, minimizing 𝓞 is a straightforward linear least

squares problem. The over-determined system of linear equations is robustly solved using the singular value decomposition (SVD) method.

Along with this equation some addition points need to be considered.

First, the solution for the 𝒈(𝒛) and 𝑬𝒊 values can only be up to a single factor 𝜶 ;[𝐼]. If each

log irradiance value 𝐥𝐧 𝑬𝒊 were replaced by 𝐥𝐧 𝑬𝒊 + 𝜶, and the function 𝒈 replaced by

𝒈 + 𝜶, the system of equation (4) and also the objective function 𝓞 would remain unchanged. To establish a scale factor we [𝐼] introduced the additional constraint

𝒁𝒎𝒊𝒅 = 𝟎, where 𝒁𝒎𝒊𝒅 =𝟏

𝟐(𝒁𝒎𝒊𝒏 + 𝒁𝒎𝒂𝒙), simply by adding this as an equation in the

𝒈(𝒁𝒊𝒋) = 𝐥𝐧𝑬𝒊 + 𝐥𝐧𝜟𝒕𝒋 (𝟒)

𝓞 = 𝒈 𝒁𝒊𝒋 − 𝐥𝐧𝑬𝒊 − 𝐥𝐧𝜟𝒕𝒋 𝟐

+ 𝝀

𝑷

𝒋=𝟏

𝑵

𝒊=𝟏

𝒈′′

𝒛=𝒁𝒎𝒂𝒙−𝟏

𝒛=𝒁𝒎𝒊𝒏+𝟏

𝒛 𝟐

(𝟓)

15

linear system. The meaning of this constraint is that a pixel with value midway between 𝒁𝒎𝒊𝒅 and 𝒁𝒎𝒊𝒅 will be assumed to have unit exposure.

Second, the solution can be made to have a much better fit by anticipating the basic shape

of the response function. Since 𝒈(𝒛) will typically have a steep slope near 𝒁𝒎𝒊𝒏 𝒂𝒏𝒅 𝒁𝒎𝒂𝒙 we should expect that 𝒈(𝒛) will be less smooth and will fit the data more poorly near the extremes;[𝐼]. To recognise this, we need to introduce a weight function 𝒘(𝒛) to emphasize the smoothness and fitting terms towards the middle of the curve. A sensible choice of 𝒘 is a simple hat function: Thus equation (𝟓) now becomes:

𝓞 = 𝒘(𝒁𝒊𝒋) 𝒈 𝒁𝒊𝒋 − 𝐥𝐧𝑬𝒊 − 𝐥𝐧𝜟𝒕𝒋 𝟐

+ 𝝀

𝑷

𝒋=𝟏

𝑵

𝒊=𝟏

𝒘 𝒛 𝒈′′ (𝒛) 𝟐

𝒛=𝒁𝒎𝒂𝒙−𝟏

𝒛=𝒁𝒎𝒊𝒏+𝟏

Finally, we need not use every available pixel site in this solution procedure. Given

measurements of 𝑵 pixels in 𝑷 photographs, we have to solve for 𝑵 values of 𝐥𝐧𝑬𝒊 and (𝒁𝒎𝒂𝒙 − 𝒁𝒎𝒊𝒏 ) samples of 𝒈. To ensure a sufficiently over-determined system, we want 𝑵(𝑷 − 𝟏) > (𝒁𝒎𝒂𝒙 − 𝒁𝒎𝒊𝒏 ). For the pixel value range (𝒁𝒎𝒂𝒙 − 𝒁𝒎𝒊𝒏 ) = 𝟐𝟓𝟓,𝑷 = 𝟏𝟏

photographs, a choice of 𝑵 on the order of 50 pixels is more than adequate. Since the size of the system of linear equations arising from Equation (𝟓) is on the order of

𝑵 × 𝑷 + (𝒁𝒎𝒂𝒙 − 𝒁𝒎𝒊𝒏 ), computational complexity considerations make it impractical to use every pixel location in this algorithm. The pixels are best sampled from regions of the

image with low intensity variance so that radiance can be assumed to be constant across the area of the pixel, and the effect of optical blur of the imaging system is minimized.

A Matlab routine is given to show the simplicity of the evaluation of the equation (𝟒).

We can represent this idea in a graphical manner as below:

𝒘 𝒛 = 𝒛 − 𝒁𝒎𝒊𝒏 𝑓𝑜𝑟 𝒛 ≤

𝟏

𝟐(𝒁𝒎𝒊𝒏 + 𝒁𝒎𝒂𝒙)

𝒁𝒎𝒂𝒙 − 𝒛 𝑓𝑜𝑟 𝒛 >𝟏

𝟐(𝒁𝒎𝒊𝒏 + 𝒁𝒎𝒂𝒙)

(𝟔)

16

Figure 6: In the figure on the 𝑡𝑜𝑝 𝑙𝑒𝑓𝑡, the × symbols represent samples of the 𝒈 curve derived from the digital values

at one pixel for 5 different known exposures using Equation(𝟒). The + and ο symbols show samples of 𝒈 curve

segments derived by consideration of two other pixels. (𝑏𝑜𝑡𝑡𝑜𝑚 𝑓𝑖𝑔𝑢𝑟𝑒)The unknown log irradiance 𝒍𝒏𝑬𝒊 has been arbitrarily assumed to be 0. It’s worth noting that the shape

of the 𝒈 curve is correct, though its position on the vertical scale is arbitrary corresponding to the unknown 𝒍𝒏𝑬𝒊.; again

the vertical position of each segment is arbitrary. Essentially, what we want to achieve in the optimization process is to slide the 3 sampled curve segments up and down (by adjusting their 𝒍𝒏𝑬𝒊’s) until they “line up” into a single smooth, monotonic curve, as shown in the 𝑏𝑜𝑡𝑡𝑜𝑚 𝑓𝑖𝑔𝑢𝑟𝑒. The vertical position of the composite curve will remain arbitrary.

17

Once the response curve 𝒈 is recovered, it can be used to quickly convert pixel values to relative radiance values, assuming the exposure 𝜟𝒕𝒋 is known.

From equation (𝟒), we obtain:

𝐥𝐧𝑬𝒊 = 𝒈 𝒁𝒊𝒋 − 𝐥𝐧𝜟𝒕𝒋

For robustness, and to recover high dynamic range radiance values, we should use all the

available exposures for a particular pixel to compute its radiance. For this, we reuse the weighting function in Equation (𝟔) to give higher weight to exposures in which the pixel’s value is closer to the middle of the response function:

𝐥𝐧𝑬𝒊 = 𝒘 𝒁𝒊𝒋 (𝒈 𝒁𝒊𝒋 − 𝐥𝐧𝜟𝒕𝒋)

𝑷𝒋=𝟏

𝒘 𝒁𝒊𝒋 𝑷𝒋=𝟏

(𝟖)

Combining the multiple exposures has the effect of reducing noise in the recovered

radiance values. It also reduces the effects of imaging artefacts such as film grain. Since the weighting function ignores saturated pixel values, blooming 12 artefacts have little impact on the reconstructed radiance values.

The heart of the algorithm is the optimization routine that needs to be implemented

effectively. The implementation is given in [𝐼]. The Matlab routine is give below:

12

Blooming occurs when charge or light at highly saturated si tes on the imaging surface spills over and affects values at neighboring si tes.

4.1.1.2. Construction of High Dynamic Range Radiance Map

(𝟕)

4.1.2. Implementation

18

19

After obtaining the HDR scene values, the next task was to tone map these values onto a

LDR matrix for being able to see it on the screen. This project presents a simple and effective tone mapping operator, which preserves visibility and contrast impression of high dynamic range images. The method is conceptually simple, and easy to use. It uses a 𝑠 − 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 type operator which takes into account both the global average of the image, as well as local luminance in the immediate neighbourhood of each pixel. The local luminance is computed using a median filter. It is seen that the resulting low dynamic range image preserves fine details, and avoids common artefacts such as halos, gradient reversals or loss of local contrast.

The real world scenes often have a very high range of luminance values. While digital

imaging technology now enables us to capture full dynamic range of the real world scene, still we are limited by the low dynamic range displays. Thus the scene can be visualized on a display monitor only after the captured high dynamic range is compressed to available range of the display device. This has been referred to as the tone mapping problem in the literature and a great deal of work has been done in this area by using a mapping that varies spatially depending on the neighbourhood of a pixel, often at multiple scales [𝐼𝐼𝐼];[𝐼𝑉]; 𝑉 . In this paper[𝐼𝐼] the authors have proposed a simple tone mapping operator

which allows us to preserve the visual content of the real-world scene without the user having to manually set a number of parameters. We show that by using a log of relative

luminance at a pixel with respect to its local luminance in a small neighbourhood, the standard 𝑠 − 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 can be modified to yield visually pleasing results. This project

computes the local luminance using a median filter, which provides a stronger central indicator than the mean filter.

The global contrast helps us to differentiate between various regions of the HDR image,

which can be loosely classified as dark, dim, lighted,, bright etc. Within each region objects become distinguishable due to local contrast against the background− either the object is darker than the background or it is brighter than the background.

If the HDR image consisted of only regions of uniform illuminations, the following

𝑠 − 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 would compress the range of illumination across the image, to displayable luminance 𝒀𝑫 in range 0 − 1.

4.2. A Simple Spatial Tone Mapping Operator for High Dynamic Range Images [II]

4.2.1. The Need for Tone Mapping

4.2.2. The Algorithm

20

𝒀𝑫(𝒙,𝒚) = 𝒀(𝒙,𝒚)/[𝒀(𝒙,𝒚) + 𝑮𝑪] (𝟗)

where 𝑮𝑪 is the global contrast factor computed through

𝑮𝑪 = 𝒄 𝒀𝑨 (𝟏𝟎)

where 𝒀𝑨 is the average luminance value of the whole image and 𝒄 is a multiplying factor. This would have the effect of bringing the high luminance value closer to 1, while the low luminance values would be a fraction closer to zero. The choice of factor 𝒄 has an effect on bridging the gap between the two ends of the luminance value.

While equation (𝟗) is able to provide a global contrast reduction, it does not attempt to

preserve local luminance variations within a region, causing many details to be lost.

A detail in the image can result from two possible reasons− either the pixel is brighter than

its surrounding pixels or the pixel is darker than its surrounding pixels. If luminance 𝒀 at a pixel is more than 𝒀𝑳, the local luminance in its immediate neighbourhood, we would like to increase the displayable value 𝒀𝑫 at that point, so that it appears brighter against its neighbourhood. On the other hand, if the luminance 𝒀 at a point is less than the local luminance 𝒀𝑳 in its immediate neighborhood, we would like to reduce the displayable value 𝒀𝑫, so that it appears darker than its neighbourhood. This has been achieve by modifying the 𝑠 − 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 as follows:

𝒀𝑫(𝒙,𝒚) = 𝒀(𝒙,𝒚)/[𝒀(𝒙,𝒚) + 𝑪𝑳(𝒙,𝒚)] (𝟏𝟏)

where 𝑪𝑳 is the contrast luminance at a point (𝑥, 𝑦) obtained by modifying the global

contrast factor 𝑮𝑪 with a term involving the logarithm of ratio of local low-pass filtered value 𝒀𝑳 to original luminance value,

𝑪𝑳 = 𝒀𝑳[𝒍𝒐𝒈(𝜹 + 𝒀𝑳/𝒀)] + 𝑮𝑪 (𝟏𝟐)

where 𝜹 is a very small value and is included to avoid singularity while doing the 𝑙𝑜𝑔

operation.

Also 𝒀𝑳 is computed using a median filter in a [3 × 3] neighbourhood rather than a mean

filter. The median is a stronger central indicator than the average.

𝑪𝒂𝒔𝒆 𝟏:

Pixel (𝑥, 𝑦) belongs to a uniform region, i.e. the Luminance 𝒀 and the local luminance 𝒀𝑳

have same magnitude. In this case the log value is going to be close to zero and thus the contribution of the first term is negligible. 𝑪𝑳 will be very close to 𝑮𝑪, and the new 𝒀𝑫

value using equation (𝟏𝟏), is almost same as computed by equation (𝟗).

4.2.2.1. Cases of Interest

21

𝑪𝒂𝒔𝒆 𝟐:

Pixel (𝑥, 𝑦) is darker than its immediate surrounding neighbourhood, i.e. the magnitude of

pixel luminance 𝒀 is less than the local luminance 𝒀𝑳. In this case, the first term in equation (𝟏𝟐) is positive. The contribution from 𝒀𝑳 is moderated by a factor which depends on how relatively high 𝒀𝑳 is with respect to 𝒀 itself at that pixel. If the difference is large, the first term in equation (𝟏𝟐), is significantly large. In any case a positive term adds on to 𝑮𝑪, the

global contrast value, resulting in an overall reduction in 𝒀𝑫 using equation (𝟏𝟏). (𝑐𝑜𝑚𝑝𝑎𝑟𝑒𝑑 𝑡𝑜 𝑪𝒂𝒔𝒆 𝟏).

𝑪𝒂𝒔𝒆 𝟑:

Pixel (𝑥, 𝑦) is brighter than its immediate neighborhood, i.e. the magnitude of 𝒀 is more

than the local luminance 𝒀𝑳. Since the denominator is higher in 𝒍𝒐𝒈(𝒀𝑳/𝒀), the first term is going to provide a negative contribution, reducing the value of 𝑪𝑳 and consequently

resulting in a higher value of 𝒀𝑫 (𝑐𝑜𝑚𝑝𝑎𝑟𝑒𝑑 𝑡𝑜 𝑪𝒂𝒔𝒆 𝟏).

For most HDR images, the computed YD can be displayed directly.

The luminance values are obtained from the RGB inputs with the formula:

𝒀 = 𝟎.𝟐𝟗𝟗 × 𝑹 + 𝟎. 𝟓𝟖𝟕 × 𝑮 + 𝟎. 𝟏𝟏𝟒 × 𝑩

After computing the YD, the displayable luminance, the new R, G, B values are computed

using the formula[𝑉𝐼]:

𝑹𝑫 = (𝑹/𝒀)𝜸 × 𝒀𝑫;

𝑮𝑫 = (𝑮/𝒀)𝜸 × 𝒀𝑫;

𝑩𝑫 = (𝑩/𝒀)𝜸 × 𝒀𝑫;

where 𝜸controls the display colour on the monitor. 𝜸 = 𝟎. 𝟒 is used in this implementation.

The implemented Matlab code is given below.

4.2.3. Implementation

22

23

Figure 7: The input of the HDRI program. 𝑰𝒏𝒑𝒖𝒕 𝒊𝒎𝒂𝒈𝒆 −> 𝑐𝑕𝑎𝑖𝑟. Note the variations in the details of each image. The

exposure times are in decreasing order. i.e. the darkest image has the least exposure time while the brightest has the maximum.

Figure 8: The HDR radiance map. 𝒙 −𝒂𝒙𝒊𝒔 Represents “𝒍𝒐𝒈 𝑬𝒙𝒑𝒐𝒔𝒖𝒓𝒆 (𝑿)” while 𝒚− 𝒂𝒙𝒊𝒔 represents ”𝒑𝒊𝒙𝒆𝒍 𝒗𝒂𝒍𝒖𝒆 𝒁”. The three curves represent the Red, Green, and Blue components of the images.

5. 1. Implementation Results

24

Figure 9: The HDR image obtained from the radiance map.

Figure 10: The final tone mapped HDR image (LDR).

25

Figure 11: The input of the HDRI program. 𝑰𝒏𝒑𝒖𝒕 𝒊𝒎𝒂𝒈𝒆 −> 𝑠𝑢𝑛𝑠𝑒𝑡. It is obvious from the images that the first image

is overexposed (high exposure time) while the second one is underexposed (less exposure time).

Figure 12: The HDR radiance map. 𝒙 − 𝒂𝒙𝒊𝒔 Represents “𝒍𝒐𝒈 𝑬𝒙𝒑𝒐𝒔𝒖𝒓𝒆 (𝑿)” while 𝒚− 𝒂𝒙𝒊𝒔 represents ”𝒑𝒊𝒙𝒆𝒍 𝒗𝒂𝒍𝒖𝒆 𝒁”. The three curves represent the Red, Green, and Blue components of the images.

5. 2. Implementation Results

26

Figure 13: (𝒍𝒆𝒇𝒕 𝒊𝒎𝒂𝒈𝒆) HDR image obtained from the radiance map. (𝒓𝒊𝒈𝒉𝒕 𝒊𝒎𝒂𝒈𝒆) The final tone mapped HDR image(LDR).

27

Today, the main users of HDR imaging devices are specialized professionals working in the

film, animation and VR industries. Some applications are listed below.

Film - Tools such as HDRShop enables one to convert a series of photographs into a light

probe - a special image that represents the lighting environment in a room. One can then use the light probe to light virtual objects, so that the virtual objects actually appear to be lit

by the light from the room. This technique is especially useful for compositing computer graphic objects into images of real scenes. Hollywood films use light maps extensively to

blend CGI into a scene.

Panoramas - Another use for HDR is in panoramic images. Panoramas often have a wide

dynamic range, e.g. one part of the panorama may contain the sun, and another part may be in deep shadow. Online web panoramas constructed from HDR images look much better

than non-HDR equivalents.

Games - A third use for HDR is in computer games. Recent computer graphics cards support

HDR texture maps. With HDR texture maps, you can render objects using light probes, in real time, yielding much more dynamic and interesting lighting effects. "High Dynamic Range Lighting Effects" are used in many new high-end games.

By determining the response functions of the imaging device, the method presented here

allows one to correctly fuse pixel data from photographs taken at different exposure settings. As a result, one can properly photograph outdoor areas with short exposures, and indoor areas with longer exposures, without creating inconsistencies in the data set. Furthermore, knowing the response functions can be helpful in merging photographs taken with different imaging systems, such as video cameras, digital cameras, and film cameras with various film stocks and digitization processes.

Most image processing operations, such as blurring, edge detection, colour correction, and

image correspondence, expect pixel values to be proportional to the scene radiance. Because of nonlinear image response, especially at the point of saturation, these operations

can produce incorrect results for conventional images. In computer graphics, one common image processing operation is the application of synthetic motion blur to images . It can be

shown that by using true radiance maps one can produce significantly more realistic motion blur effects for high dynamic range scenes.

5. Applications of HDRI

28

In this project a simple implementation of High Dynamic Range Imaging along with tone

mapping has been shown. The results achieved are pleasing to the eye and the images are

comparable to any of the standard HDRI toolbox available in the market.

[𝐼] Debevec, P. E., & Malik, J. (n.d.). Recovering High Dynamic Range Radiance Maps from

Photographs.

[𝐼𝐼] Biswas, K. K., & Pattanaik, S. (n.d.). A Simple Spatial Tone Mapping Operator for High Dynamic.

[𝐼𝐼𝐼] Pattanaik, S. N., Ferwerda, J. A., Fairchild, M. D., & Greenberg, D. P. (1998). A multiscale model

of adaptation and spatial vision for realistic images display. SIGGRAPH,Annual Conference

Series,ACM , 267-276.

[𝐼𝑉] Fattal, R., Lischinski, D., & Werman, M. (2002). Gradient Domain High Dynamic range

compression. SIGGRAPH,Annual Conference series,ACM , 246-249.

[𝑉] Reinhard, E., Stark, M., Shirley, P., & Ferwerda, J. (2002). Photographic Tone reproduction for

digital images. SIGGRAPH,Annual Conf. Series,ACM , 267-276.

[𝑉𝐼] Duan, J., & Qiu, G. (2004). Fast Tone Mapping for high dynamic range images. International

Conference on Pattern Recognition, ICPR , 847-850.

7. References

6. Conclusion