Embed Size (px)
Citation preview
S.V.Shojaedini
“One picture is worth more than ten thousand words”Anonymous
“Digital Image Processing”, Rafael C. Gonzalez & Richard E. Woods,
Addison-Wesley, 2002óMuch of the material that follows is taken from this book
“Machine Vision: Automated Visual Inspection and Robot Vision”, David
Vernon, Prentice Hall, 1991óAvailable online at:
homepages.inf.ed.ac.uk/rbf/BOOKS/VERNON/
This lecture will cover:ó What is a digital image?ó What is digital image processing?ó History of digital image processingó State of the art examples of digital image processingó Key stages in digital image processing
f(x,y) = reflectance(x,y) * illumination(x,y)Reflectance in [0,1], illumination in [0,inf]
We can think of an image as a function, f, from R2 to R:ó f( x, y ) gives the intensity at position ( x, y ) ó Realistically, we expect the image only to be defined over a rectangle, with a
finite range:ó f: [a,b]x[c,d] à [0,1]
A color image is just three functions pasted together. We can write this as a “vector-valued” function:
( , )( , ) ( , )
( , )
r x yf x y g x y
b x y
=
We usually operate on digital (discrete) images:ó Sample the 2D space on a regular gridó Quantize each sample (round to nearest integer)
If our samples are ∆ apart, we can write this as:f[i ,j] = Quantize{ f(i ∆, j ∆) }
The image can now be represented as a matrix of integer values
Pixel values typically represent gray levels, colours, heights, opacities etcRemember digitization implies that a digital image is an approximation of a real scene
1 pixel
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Pixel values typically represent gray levels,
X: 217 Y: 79RGB: 223, 211, 215
50 100 150 200 250 300 350 400
50
100
150
200
250
X: 227 Y: 92RGB: 242, 236, 236
120 140 160 180 200 220 240 260 280 300 320
80
100
120
140
160
180
Common image formats include:ó 1 sample per point (B&W or Grayscale)ó 3 samples per point (Red, Green, and Blue)ó 4 samples per point (Red, Green, Blue, and “Alpha”, a.k.a.
Opacity)
For most of this course we will focus on grey-scale images
Digital image processing focuses on two major tasksó Improvement of pictorial information for human
interpretationó Processing of image data for storage, transmission and
representation for autonomous machine perceptionSome argument about where image processing ends and fields such as image analysis and computer vision start
The continuum from image processing to computer vision can be broken up into low-, mid- and high-level processes
Low Level ProcessInput: ImageOutput: Image
Examples: Noise removal, image sharpening
Mid Level ProcessInput: Image Output: Attributes
Examples: Object recognition, segmentation
High Level ProcessInput: Attributes Output: Understanding
Examples: Scene understanding, autonomous navigation
In this course we will stop here
Early 1920s: One of the first applications of digital imaging was in the news-
paper industryó The Bartlane cable picture
transmission serviceó Images were transferred by submarine cable between
London and New Yorkó Pictures were coded for cable transfer and reconstructed
at the receiving end on a telegraph printer
Early digital imageImag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Mid to late 1920s: Improvements to the Bartlane system resulted in higher quality images
ó New reproduction processes based on photographic techniques
ó Increased number of tones in reproduced images
Improved digital image
Early 15 tone digital image
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
1960s: Improvements in computing technology and theonset of the space race led to a surge of work in digitalimage processingó 1964: Computers used to
images improve the quality of of the moon taken by the Ranger 7 probe
ó Such techniques were usedin other space missions including the Apollo landings
A picture of the moon taken by the Ranger 7 probe minutes before landing
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
1970s: Digital image processing begins to be used inmedical applicationsó 1979: Sir Godfrey N.
Hounsfield & Prof. Allan M. Cormack share the Nobel Prize in medicine for the invention of tomography, the technology behind Computerised Axial Tomography (CAT) scans
Typical head slice CAT image
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
1980s - Today: The use of digital image processingtechniques has exploded and they are now used for allkinds of tasks in all kinds of areasó Image enhancement/restorationó Artistic effectsó Medical visualisationó Industrial inspectionó Law enforcementó Human computer interfaces
One of the most common uses of DIP techniques:improve quality, remove noise etc
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Launched in 1990 the Hubble telescope can take images of very distant objectsHowever, an incorrect mirror made many of Hubble’s images useless
Image processing techniques were
used to fix this
Artistic effects are usedto make images morevisually appealing, to addspecial effects and tomake composite images
Take slice from MRI scan of canine heart, and findboundaries between types of tissueó Image with gray levels representing tissue densityó Use a suitable filter to highlight edges
Original MRI Image of a Dog Heart Edge Detection ImageImag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Geographic Information Systemsó Digital image processing techniques are used extensively
to manipulate satellite imageryó Terrain classificationó Meteorology
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Night-Time Lights of theWorld data setó Global inventory of
human settlementó Not hard to imagine the
kind of analysis that mightbe done using this data
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Human operators are expensive,slow andunreliableMake machines do thejob insteadIndustrial vision systems are used in all kinds of industriesCan we trust them?
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Printed Circuit Board (PCB) inspectionó Machine inspection is used to determine that all
components are present and that all solder joints are acceptable
ó Both conventional imaging and x-ray imaging are used
Image processing techniques are used extensively by law enforcersó Number plate recognition
for speed cameras/automated toll systems
ó Fingerprint recognitionó Enhancement of CCTV
images
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Try to make human computer interfaces more naturaló Face recognitionó Gesture recognition
Does anyone remember the user interface from “Minority Report”?These tasks can be extremely difficult
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Representation & Description
Image Enhancement
Object Recognition
Problem Domain
Colour Image Processing
Image Compression
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Representation & Description
Image Enhancement
Object Recognition
Problem Domain
Colour Image Processing
Image Compression
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Representation & Description
Image Enhancement
Object Recognition
Problem Domain
Colour Image Processing
Image Compression
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Representation & Description
Image Enhancement
Object Recognition
Problem Domain
Colour Image Processing
Image Compression
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Simulate Blur and Noise
Restoration of Blurred, Noisy Image Using Estimated NSR
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Representation & Description
Image Enhancement
Object Recognition
Problem Domain
Colour Image Processing
Image Compression
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Representation & Description
Image Enhancement
Object Recognition
Problem Domain
Colour Image Processing
Image Compression
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Representation & Description
Image Enhancement
Object Recognition
Problem Domain
Colour Image Processing
Image Compression
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Representation & Description
Image Enhancement
Object Recognition
Problem Domain
Colour Image Processing
Image Compression
Imag
es ta
ken
from
Gon
zale
z &
Woo
ds, D
igita
l Im
age
Proc
essi
ng (2
002)
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Representation & Description
Image Enhancement
Object Recognition
Problem Domain
Colour Image Processing
Image Compression
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Representation & Description
Image Enhancement
Object Recognition
Problem Domain
Colour Image Processing
Image Compression
ó Early detection of symptoms (fungi) or of the initial presence of a bioagressor isa key-point in the context.
ó The detection of biological objects as small as such insects (dimensions areabout 2 mm) is a real challenge, especially when considering greenhousesdimensions (10 to 100 meters long).
ó Traditionally, visual observations are made each week by human experts, oftenon colored sticky traps.
ó it is difficult or even not possible to perform a continuous (typically daily)control and to examine every leaf in the greenhouse.
ó We need to reduce the 3D volume of canopy to be investigated
ó Firstly a model “crop × bioagressor” was chosen .rose, an ornamental crop, waschosen because it attracts various bioagressors and it requires high levelstandard quality for flowers as well as leaves.
ó white fly was chosen because this bioagressor requires early detection andtreatment to prevent durable infestation.
ó Eggs and larvae identification and counting by vision techniques are difficultbecause of critical dimension (eggs) and weak contrast between object andimage background (larvae). For these reasons we decided to focus first onadults.
ó Since adults may fly away, we chose to scan the leaves when flies were not veryactive.
ó The agrosystem was a 256 m2 plastic twin-tunnel greenhouse planted withroses and equipped with an opening roof, heating and a fog generator
ó They are known for their different resistance to powdery mildew andattractive powers to insects.
ó As human biologists do, our system has to analyze raw images and to label interesting regions that correspond to objects of interest (e.g., insects).
ó To recognize a region as an insect, a human expert relies on (biological and contextual) domain knowledge about insects (e.g., species features, life cycle, host plant) as well as visual data that can be extracted from images (color, texture, shape, and size).
ó A software system must take into account both kinds of knowledge. To separate the different types of knowledge and the different reasoning strategies, we propose an architecture based on specialized modules as shown in Figure bellow.
ó It consists of two knowledge-based systems (KBS), a set of image processing(IP) algorithms, and an initial learning module.
ó The classification KBS (Figure bottom) aims at selecting interestingregions in images. To this end it triggers image processing requests andinterprets the numerical results into higher level concepts, i.e. (parts of)objects of interest. It retains only the regions corresponding to targetinsects and returns their number to the user.
ó The supervision KBS (Figure bottom) is used to monitor the execution ofimage processing requests. It selects and plans the best programs with thebest parameter values for each image. From raw images provided by theend-user, the goal is to extract numerical values needed by theclassification KBS.
ó
ó Illustration of the influence of parameter setting: a segmentationalgorithm, Hysteresis thresholding, is tuned with two different values forits two control thresh-old parameters
ó Image processing flow for white fly extraction. From left to right: inputó image, (a) background subtraction, (b) filtering result, (c) segmentation
result.
False Negative Rate (FNR) and False Positive Rate (FPR) for test images with nowhite flies (class C1), at least one white fly (class C2) and for the whole test set.
Example of an ambiguous situation leading to a wrong interpretation result.
Segmentation Results of Method Using XYZ Plane
We have looked at:ó What is a digital image?ó What is digital image processing?ó History of digital image processingó State of the art examples of digital image processingó Key stages in digital image processing
Thanks
