Embed Size (px)
Citation preview
1
Image Processing.
1
Image Processing and Graphics
Gabriel Brostow & Simon Arridge
TA: Laura Panagiotaki
COMP 3079 / GC31
Image Processing.
2
Image Processing.
3
Motivation and Goals
• Grounding in image processing techniques
• Concentrate on algorithms used in machine
vision, graphics, medical imaging
• Best sensors ever!
Image Processing.
4
Motivation and Goals
• Grounding in image processing techniques
• Concentrate on algorithms used in machine
vision, graphics, medical imaging
• Best sensors ever! With a few problems…
2
Transmission interference
5
Compression artefacts
6
Spilling
7
Scratches, Sensor noise
8
3
Bad contrast
9
Resolution Super resolution?
10
Super resolution
11
Removing motion blur
Original image
Cropped subwindow
After motion blur removal
[Images from Amit Agrawal]12
4
Removing motion blur
13 14
Removing motion blur
15
Image Processing.
16
Syllabus (for 1st half: IP)
1. The digital image
2. Image segmentation*
3. Image transformations
4. Morphological operations*
5. Image Filtering
*= Homework will be assigned
5
Syllabus for 2nd half
Promotional photo, Disney & Pixar
Image Processing.
17
Image Processing.
20
Why Now?
• Why are these sectors paying more attention?
•Medicine• Automatic or assisted diagnosis
• Image-guided surgery
•Agriculture
•Film and television
•Surveillance and police work
•Military
•http://www.cs.ubc.ca/spider/lowe/vision.html
Image Processing.
21
Course content
• Lots of material!
• Some mathematics• Calculus (light)
• Geometry and matrix algebra
• Probability and statistics (light)
• Some programming – Matlab
22
Lectures and notes
• http://www.cs.ucl.ac.uk/staff/G.Brostow/classes/IPG2010/
• TA: Laura Panagiotaki: E.Panagiotaki at cs.ucl.ac.uk
• Mondays 11:00-12:00 (Foster Court 220)
• Wednesdays 09:00-11:00 (Foster Court 216)
• Lab sessions
– Tuesday 11-12 (Malet Place Eng 4.06)
– Tuesday 12-13 (Malet Place Eng 4.06)
• subject: “join” to [email protected]
6
Image Processing.
25
Assessment
• Exam 80%
• Four Courseworks 20%.• IP: Implement and test algorithms in Matlab
• Honor System
Image Processing.
28
Unassessed CW Assignment
• Matlab introduction
• Start matlab:
% matlab or
% /opt/matlab7/bin/matlab
• Download any simple image
• Load it into matlab:
>> I = imread(‘foo.jpg’);
29
Unassessed CW Assignment
• Display the image in Matlab:
>> imshow(I);
• Print the image data array:
>> I
(Ha! It’s a trap! – use Ctrl-C to make it stop)
• Print the size of the image array and create a subimage:
>> size( I )
>> Isubwindow = I(72:92, 62:82);
>> imshow(Isubwindow);
Image Processing.
30
Unassessed CW Assignment
• Start the Matlab help tool (Help menu).
• In the “Contents” pane to the left of the
window. Click on MATLAB.
• Go through the “Getting Started” section.
• Continue to the “Using MATLAB” section
when you have time.
7
IP is Only Part of the Picture
• See Machine Vision
and/or Computational
Photography + Capture
(GV15 / M085 / 3085)
• Why?
– To work on fun projects!
31
MRI of GJB A Computational Investigation into
the Human Representation and
Processing of Visual Information
• 3D Gesture Interfaces
(Xbox 360)
• Build Your Own 3D Scanner: Optical Triangulation for
Beginners (Lanman + Taubin)
Image Processing.
32
Image Processing.
33
Developing Drosophila eye (30 hours)With Franck PichaudEpithelial Morphogenesis & Cell PolarityLMCB, Cell Biology Unit, MRC, UCLNeeded Innovations:
- Locate & track branching structures
- Propagate confidence to neighbors
Stop Motion Animation
Image Processing.
34
Super Lego Mario (Level 1)
http://www.youtube.com/watch?v=hmWWfnKVbYY
Video Annotation, Navigation, and Composition
UIST2008, http://www.danbgoldman.com/
8
Next Time: The Digital Image
Image Processing.
35
Image Processing.
37
The Digital Image
Image Processing.
38
Outline
• What is an image?
• What is a pixel?
• How do we store them?
Image Processing.
39
What is an image?
9
Image as 2D signal
• Signal: function depending on some variable with physical meaning
• Image: continuous function2 variables: xy - coordinates3 variables: xy + time (video)
• Brightness is usually the value of the function
• But can be other physical values too: temperature, pressure, depth …
40
Example 2d images
ultrasound temperature
camera image CT41
Image Processing.
46
Image?
>> t=rand(256,256);
>> imshow(t)
What is a pix-el?
x
y
(x,y)
f(x,y)
(0,0)
49
10
Not a little square!
• A Pixel Is Not A Little
Square, A Pixel Is Not A
Little Square, A Pixel Is
Not A Little Square! (And
a Voxel is Not a Little
Cube),
– Alvy Ray Smith,
MS Tech Memo 6, Jul 17, 1995
Image Processing.
50
Not a little square!
Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995
Image Processing.
51
Gaussian reconstruction filter
Not a little square!
Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995
Image Processing.
52
Cubic reconstruction filter
Not a little square!
Graphics: Dick Lyon, 2006
Image Processing.
53
11
Image Processing.
54
Where do images come from?
• Digital cameras
• MRI scanners
• Computer graphics packages
• Body scanners
• Laser range finders
• Many more…
Image Processing.
55
Where do images come from?
• Digital cameras
• MRI scanners
• Computer graphics packages
• Body scanners
• Laser range finders
• Many more…
Image Processing.
56
The digital camera
• A Charge Coupled Device (CCD).
Lens
Sensor
array
ADC
Image array
Image Processing.
57From: Lecture Notes – EAAE
and/or Science “Nuggets” 2000
Capturing photons
12
Image Processing.
58
http://www.astro.virginia.edu/class/oconnell/astr121/im/CCD-fullframearc-FSU.jpg
Image Processing.
60
The sensor array
• Can be < 1cm2.
• An array of photosites.
• Each photosite is a bucket of electrical charge.
• They contain charge proportional to the incident light intensity during exposure.
Image Processing.
61
Analog to Digital Conversion
• The ADC measures the
charge and digitizes the
result.
• Conversion happens line
by line.
• The charges in each
photosite move down
through the sensor array. ADC
RAM
Image Processing.
62
ADC
RAM
13
Image Processing.
63
Blooming
• The buckets have finite capacity
• Photosite saturation causes blooming
Image Processing.
64
Dark Current
• CCDs produce thermally-generated charge.
• They give non-zero output even in darkness.
• Partly, this is the dark current.
• Fluctuates randomly.
• How can we reduce dark current? From: Lecture Notes - EAAE
Yohkoh satellite, 9 years apart ..
Image Processing.
65
Dark Current
• CCDs produce thermally-generated charge.
• They give non-zero output even in darkness.
• Partly, this is the dark current.
• Fluctuates randomly.
• How can we reduce dark current? From: Lecture Notes - EAAE
Image Processing.
66
14
Sampling 1D
Sampling in 1D takes a function, and returns a vector whose elements are
values of that function at the sample points.
69
Sampling 2D
Sampling in 2D takes a function and
returns an array; we allow the
array to be infinite dimensional and to
have negative as well as positive indices.
70
Image Processing.
71
Greyscale digital image Nyquist Frequency
• Half the sampling frequency of a discrete
signal processing system
Image Processing.
72
15
Sampling grids
73
Retina-like sensors
74
Quantization
• Real valued function will get digital values –
integer values
• Quantization is lossy!!
– After quantization, the original signal cannot be
reconstructed anymore
• This is in contrast to sampling, as a sampled but
not quantized signal can be reconstructed.
• Simple quantization uses equally spaced levels
with k intervals
bk 275
Quantization
00
01
10
11
76
16
Quantization
00
01
10
11
77
Usual quantization intervals
• Grayvalue image
8 bit = 2^8 = 256 grayvalues
• Color image RGB (3 channels)
8 bit/channel = 2^24 = 16.7Mio colors
• 12bit or 16bit from some sensors
• Nonlinear, for example log-scale
78
Image Processing.
79Photo: Paulo Barcellos Jr.
Winding Down
Image Processing.
80
17
Properties
• Image resolution
• Geometric resolution: How many pixel per area
• Radiometric resolution: How many bits per pixel
81
Image resolution
1024x1024
512x512
512x1024
82
Image Processing.
83
Geometric resolution
144x144 72x72 36x36
18x18 9x9 4x4
Image Processing.
84
Radiometric resolution
256 128 64 32
16 8 4 2
18
Lossless vs. Lossy
• Name some formats?
Image Processing.
85
Image Processing.
86
Aliasing and SNR
• What is the disadvantage of low sampling
resolution?
• What is the disadvantage of high sampling
resolution?
Finish
Next week:
Image segmentation
Image Processing.
87
Image Processing.
94
Unassessed Assignment
Use matlab to change the geometric and
radiometric quantization resolution in one
of your images. For each level of sampling
and quantization, plot the image function, as
in slides 67 & 68, and compare the
approximations to the true intensity function
that you get at each level.
