Lecture 2: The Human Visual System (HVS)agcggs680.pbworks.com/f/AGC_Lec_2_Spring2011.pdf · 2 Cones Cones are located in the fovea and are sensitive to color. • Recent research

1

Lecture 2: The Human Visual System

(HVS)

(chapter 2)

GGS 680 1 Spring 2011

Human visual perception

  Based on: • Human intuition • Subjective visual

judgment

  But … • Robust • Fast • Reliable (?)


The Human Eye   Diameter: 20 mm

  3 membranes enclose the eye •  Cornea & sclera •  Choroid •  Retina


The Human Eye   The choroid contains blood vessels

for eye nutrition and is heavily pigmented to reduce extraneous light entrance and backscatter.

  The ciliary body and the iris, which controls the amount of light that enters the eye •  (diameter varies - 2 mm ~ 8 mm).


The Lens   The lens is made up of concentric layers of

fibrous cells and is suspended by fibers that attach it to the ciliary body.

  It is slightly yellow and absorbs approx. 8% of the visible light spectrum.

  The “focal length” is controlled by the radius of the lens (a flexible lens!)


The Retina   The retina lines the entire

posterior portion.

  Discrete light receptors are distributed over the surface of the retina:

•  cones (6-7 million per eye)

•  rods (75-150 million per eye)


2

Cones

  Cones are located in the fovea and are sensitive to color. • Recent research has indicated the existance of 3

types of cones: “R” cones, “G” cones, “B” cones.

  Each one is connected to its own nerve end.

  Cones are responsible for photopic vision (or bright-light vision).


Rods

  Rods are giving a general, overall picture of the field of view and are not involved in color vision.

  Several rods are connected to a single nerve and are sensitive to low levels of illumination

  Rods are responsible for scotopic vision (dim-light vision).


Cones and Rods


Receptor Distribution

  The distribution of receptors is radially symmetric about the fovea.

  Cones are most dense in the center of the fovea while rods increase in density from the center out to approximately 20% off axis and then decrease.


Receptor Distribution


The Fovea   The fovea is circular (1.5 mm in diameter) but can be

assumed to be a square sensor array (1.5 mm x 1.5 mm).

  The density of cones: 150,000 elements/mm2 ~ 337,000 for the fovea.

  A CCD imaging chip of medium resolution requires approximately 5 mm by 5 mm for this number of elements …


3

Image formation in the eye


Image Formation in the Eye

  Distance between the center of the lens and the retina (focal length): • varies from 17 mm to 14 mm (refractive power

of lens goes from minimum to maximum).

  Objects farther than 3 m use minimum refractive lens powers (and vice versa).


Image Formation in the Eye   Calculation of retinal image of an object and the visual angle

1710015 x

= mmx 55.2=

€

A = tan−1 15100⎛

⎝ ⎜

⎞

⎠ ⎟ = 8.53deg


Image Formation in the Eye

  Perception takes place by the relative excitation of light receptors.

  These receptors transform radiant energy into electrical impulses that are ultimately decoded by the brain.


Brightness Adaptation & Discrimination

  The HVS can adapt to a wide range of light intensity levels - on the order of 1010.

Daylight 105

Indoors light 102 moonlight 10-1

starlight 10-4


Dealing with a high dynamic range   Subjective brightness (i.e. intensity as

perceived by the HVS) is a logarithmic function of the light intensity incident on the eye.


4


  The HVS cannot operate over such a range simultaneously.

  For any given set of conditions, the current sensitivity level of HVS is called the brightness adaptation level (Ba).



  The eye also discriminates between changes in brightness at any specific adaptation level.

Where: ΔIc is the increment of illumination discriminable 50% of the time, and I is the background illumination



  Small values of Weber ratio mean good brightness discrimination (and vice versa).

  At low levels of illumination brightness discrimination is poor (rods) and it improves significantly as background illumination increases (cones).



  The typical observer can discern one to two dozen different intensity changes

•  i.e. the number of different intensities a person can detect at any given point in a monochrome image

• As the eye roams around, different conditions are formed and different set of intensities become apparent.



  Overall intensity discrimination is broad due to different set of incremental changes to be detected at each new adaptation level.

  Perceived brightness is not a simple function of intensity •  Perceived brightness •  Simultaneous contrast


Perceived Brightness


5

Simultaneous Contrast


Image Acquisition

(Chapter 2, cont.)


The image acquisition process


The spectrum


Characteristics of light sources   Light sources:

•  Radiance: The total amount of energy that flows from a source.

•  Luminance: The amount of intensity that is observed from a light source or a surface.

•  Brightness: a subjective measure of light perception.

  Luminance is an objective term.

  Brightness is a subjective term.


Luminance

  Let L(λ) be an EM energy source   Let V(λ) be the sensor sensitivity

function   Then the Luminance is:

I[lumen]=∫λL(λ)V(λ)dλ

energy

λ

L(λ) V(λ)


6

Principal Sensor arrangements


Source: Fossum, 1998

(pixel size 7.9 x 7.9 micron)

Sensors: CCD   A CCD has sensor elements (sels),

arranged in a 2D array.

  Each sel is comprised of a photodiode that converts light into charge (electrons) and a charge holding region

  After the sensor is exposed, each row of sensors moves its charge to a register. Charges are then amplified and converted to voltage.

  Pros: •  A very mature technology (~25 years) •  Low noise

  Cons: •  A more complex reading mechanism & control •  Higher power consumption

Camera control

Source: Photonics spectra (2001)


Sensors: CMOS   A CMOS has sensor elements (sels),

arranged in a 2D array. Each sel is comprised of a photodiode that converts light into charge (electrons) and a charge-to-voltage converter.

  Each sel is indexed, sels are read through a row and column decoders

  Pros: •  A CMOS can be easily integrated and

fitted on a single silicon chip •  Lower power consumption

  Cons: •  CMOS has only begun mass production •  More difficult to “pack” sels

Camera control

Source: Photonics spectra (2001)

Camera control


Source: Fossum, 1998

Super CCD (Fuji)


Super CCD (Fuji)


Capturing images in 2D/3D

  Microdensitometers   Sensor strips (linear, circular)   Sensor arrays   Mirror scanning


7

A Simple Image Model

  Image: a 2-D light-intensity function f(x,y)

  The value of f at (x,y) the intensity (“brightness”) of the image at that point

  0 < f(x,y) < ∞



  Nature of f(x,y):

• The amount of source light incident on the scene being viewed

• The amount of light reflected by the objects in the scene



  Given the illumination & reflectance components: •  Illumination: i(x,y) •  Reflectance / absorption: r(x,y)

  The image f(x,y) is given by:

f(x,y) = i(x,y) ⋅ r(x,y) where: 0 < i(x,y) < ∞ and 0 < r(x,y) < 1 (total reflectance) (total absorption)


Illumination & Reflectance

  Sample values of r(x,y): • 0.01: black velvet • 0.65: stainless steel • 0.93: snow

  Sample values of i(x,y): • 10000 lm/m2 : cloudy day • 1000 lm/m2: office illumination • 0.1 lm/m2 : full moon


A Simple Image Model (cont.)   Intensity of a monochrome image f at (x0,y0): l=f(x0, y0)

  Lmin ≤ l ≤ Lmax

Where Lmin is positive and Lmax is finite

  In practice: •  Lmin = imin rmin •  Lmax = imax rmax

  For example, in an indoor image: •  Lmin ≈ 10 Lmax ≈ 1000

  We will consider the range [Lmin, Lmax] as gray levels:

•  Often shifted to [0,L-1] l=0: “black” and l=L-1: “white”


Sampling & Quantization

  The spatial and amplitude digitization of f(x,y) is called:

•  image sampling when it refers to spatial coordinates (x,y) and

• gray-level quantization when it refers to the amplitude of f(x,y).


8





Sampling & Quantization: Representing digital images

  x and y should not be confused with the values of physical coordinates when the image was sampled


Sampling & Quantization: Representing digital images

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−−−

−

−

=

)1,1(...)1,1()0,1(............

)1,1(......)0,1()1,0(...)1,0()0,0(

),(

MNfNfNf

MffMfff

yxf

Digital Image Picture Elements (Pixels)



  Important terms for future discussion:

• Z: set of real integers

• R: set of real numbers


Sampling & Quantization   Sampling: partitioning xy plane into a grid

•  the coordinate of the center of each grid is a pair of elements from the Cartesian product Z x Z (Z2)

  Z2 is the set of all ordered pairs of elements (a,b) with a and b being integers from Z.


9


  f(x,y) is a digital image if:

•  (x,y) are integers from Z2 and •  f is a function that assigns a gray-level value

(from R) to each distinct pair of coordinates (x,y) [quantization]

  But … gray levels are usually integers •  then Z replaces R



  The digitization process requires to determine:

• The values of N and M

and

• The number of discrete gray levels allowed for each pixel.



  Usually, in DIP these quantities are integer powers of two:

N=2n M=2m and L=2k

(number of gray levels)

  Another assumption is that the discrete levels are equally spaced between 0 and L-1 in the gray scale.


Spatial and gray level resolution

  An image is characterized by (M,N,L). What is the effect of each of these values?

  Resolution – the smallest discernible detail an image can hold:

•  Space - Spatial resolution •  Spectrum - Spectral (gray level) resolution •  Time - Temporal resolution


Spring 2011

Spatial and Temporal Resolution

GGS 680 60

Source: Longley et al., 2006

Spatial and Spectral Resolution

Spectral Resolution Spatial Resolution

House House House


10

Keeping pixel size constant by sub-sampling

Spatial resolution – example1


Keeping image size constant by re-sampling

Spatial resolution – example1


256 128

64 32

16 8

4 2

Spectral resolution - False contouring


Resolution and memory   If b is the number of bits required to store a

digitized image and k is the number of bits per pixel then:

•  b = N x M x k (if M=N, then b=N2k)


Memory / file size (Gray scale imagery)


Measuring Resolution: example   Spatial:

•  Using line pairs: •  Each pair consists of a line-space pair •  The width of each pair is 2W

•  Measured in lines per millimeter •  The imaging system resolution is a combination of the lens

resolution and the film/sensor array resolution.

  Spectral •  For gray levels - 2k •  For RGB images – 2k for each channel.


11

Example: Resolution targets

  USAF 1951 test target: •  Five groups (-2,-1,0,1) •  Each group contains six targets

•  Three vertical lines •  Three horizontal lines

•  The number of lines per mm doubles with every 6th target

•  The resolution of each target is given by:

•  Where k is the group number and n is the target number


Determining the resolution of an imaging system (film)

  The actual resolution is computed by placing the imaging system in front of a target

  Capturing an image   Identifying the finest resolution target   Using the nominal target resolution, compute the imaging system

resolution (lines/mm)


ISO 12233


Optimizing Sampling & Quantization   How many samples and gray levels are

required for a good approximation?

•  The resolution of an image depends N,M and k.

•  The more these parameters are increased, the closer the digitized array approximates the original image.


Optimizing Sampling & Quantization   Is there an optimal N,M, and k values?   What is the relation between the spatial

and radiometric resolution?   One possible way to check this:

• Generate different versions (images) of the same image:

• Varying N, M • Varying k (number of bits per pixel) • Varying both


Optimizing Sampling & Quantization

  Isopreference curves (in the N-k plane)

• Each point: image having values of N and k equal to the coordinates of this point

• Points lying on an isopreference curve correspond to images of equal subjective quality.


12

Optimizing Sampling & Quantization


N N N

k k k

(a) (b) (c)


  As the level of detail increases, the curves become more vertical

  The curves has a general tendency to shift right and upwards.

Isopreference Curves


Optimizing Sampling & Quantization   Conclusions:

•  Generally, the subjective quality increases as N and k increases

•  For images with small amounts of detail : as N increases k tends to decrease (higher contrast)

•  For images with medium amounts of detail : the curves become more vertical (lower dependency on k)

•  For images with large amounts of detail: the quality is nearly independent of k. • Only few gray levels are needed!


Documents

Lecture 2: The Human Visual System (HVS)agcggs680.pbworks.com/f/AGC_Lec_2_Spring2011.pdf · 2 Cones Cones are located in the fovea and are sensitive to color. • Recent research