DTU Compute
Medical Image Analysis
Plenty of slides adapted from Thomas Moeslunds lectures
Rasmus R. PaulsenDTU Compute
http://www.compute.dtu.dk/courses/02512
mailto:[email protected]
DTU Compute
2020Image Analysis2 DTU Compute, Technical University of Denmark
Lecture 8 – Geometric Transformation
DTU Compute
2020Image Analysis3 DTU Compute, Technical University of Denmark
What can you do after today? Compute the translation of a point Compute the rotation of a point Compute the scaling of a point Compute the shearing of a point Construct a translation, rotation, scaling, and shearing transformation
matrix Use transformation matrices to perform point transformations Describe the difference between forward and backward mapping Use bilinear interpolation to compute the interpolated value at a point Transform an image using backward mapping and bilinear interpolation Compute a line profile using bilinear interpolation Describe the homography Describe how the coefficients of a homography can be computed using
four corresponding points
DTU Compute
2020Image Analysis4 DTU Compute, Technical University of Denmark
Geometric transformation Moving and changing the dimensions of images Why do we need it?
DTU Compute
2020Image Analysis5 DTU Compute, Technical University of Denmark
Change detection Patient imaged before and after surgery What are the changes in the operated organ? Patient can not be placed in the exact same position in the
scanner
Before surgery After surgery
DTU Compute
2020Image Analysis6 DTU Compute, Technical University of Denmark
Image Registration Change one of the images so it fits with the other Formally
– Template image– Reference image– Template is moved to fit the reference
Template Reference
Compute the difference
DTU Compute
2020Image Analysis7 DTU Compute, Technical University of Denmark
Camera Calibration
2D projective transformations (homographies). Christiano Gava, Gabriele Bleser
DTU Compute
2020Image Analysis8 DTU Compute, Technical University of Denmark
Geometric Transform The pixel intensities are not changed The “pixel values” just change positions
Same value
Different place
DTU Compute
2020Image Analysis9 DTU Compute, Technical University of Denmark
Different transformations Translation Rotation Scaling Shearing
Homographies Advanced transformations
DTU Compute
2020Image Analysis10 DTU Compute, Technical University of Denmark
Translation The image is shifted – both vertically and
horizontally
DTU Compute
2020Image Analysis11 DTU Compute, Technical University of Denmark
Rotation The image is rotated around the centre or the upper left corner Remember to use degrees and radians correctly
– Matlab uses radians – Degrees easier for us humans
DTU Compute
2020Image Analysis12 DTU Compute, Technical University of Denmark
Rotation coordinate system
y
x
DTU Compute
2020Image Analysis13 DTU Compute, Technical University of Denmark
Rigid body transformation Translation and rotation Rigid body = stift legeme Angles and distances are kept
DTU Compute
2020Image Analysis14 DTU Compute, Technical University of Denmark
Scaling The size of the image is changed Scale factors
– X-scale factor – Y-scale factor
Uniform scaling:
SxSy
Sx = Sy
DTU Compute
2020Image Analysis15 DTU Compute, Technical University of Denmark
Similarity transformation Translation, rotation, and uniform scaling Angles are kept Distances change
DTU Compute
2020Image Analysis16 DTU Compute, Technical University of Denmark
Shearing Pixel shifted horizontally or/and vertically Shearing factors
– X-shear factor – Y-shear factor
Is less used than translation, rotation, and scaling
BxBy
DTU Compute
2020Image Analysis17 DTU Compute, Technical University of Denmark
Transformation Mathematics Transformation of positions Structure found at position (x,y)
in the input image f Now at position (x’,y’) in output
image g A mapping function is needed
f
g
(x,y)
(x’,y’)
DTU Compute
2020Image Analysis18 DTU Compute, Technical University of Denmark
Translation mathematics The image is shifted – both vertically and
horizontally
(x,y)(x’,y’)
DTU Compute
2020Image Analysis19 DTU Compute, Technical University of Denmark
Matrix notation Coordinates in column matrix format
DTU Compute
2020Image Analysis20 DTU Compute, Technical University of Denmark
Translation mathematics in matrix notation The image is shifted – both vertically and
horizontally
DTU Compute
2020Image Analysis21 DTU Compute, Technical University of Denmark
Scaling The size of the image is changed Scale factors
– X-scale factor – Y-scale factor
Uniform scaling:
SxSy
Sx = Sy𝑆𝑆𝑥𝑥 = 1.2
𝑆𝑆𝑦𝑦 = 0.9
DTU Compute
2020Image Analysis22 DTU Compute, Technical University of Denmark
Matrix multiplication details
Is equal to:
DTU Compute
2020Image Analysis23 DTU Compute, Technical University of Denmark
Transformation matrix
Can be written as
Where
is a transformation matrix
DTU Compute
2020Image Analysis24 DTU Compute, Technical University of Denmark
Rotation A rotation matrix is used y
x
DTU Compute
2020Image Analysis25 DTU Compute, Technical University of Denmark
Shearing Pixel shifted horizontally or/and vertically
New x value depends on x and y
DTU Compute
2020Image Analysis26 DTU Compute, Technical University of Denmark
Affine transformation The collinearity relation between points, i.e., three
points which lie on a line continue to be collinear after the transformation
DTU Compute
2020Image Analysis27 DTU Compute, Technical University of Denmark
Combining transformations Suppose you first want to
rotate by 5 degrees and then scale by 10%
Scaling
Rotation
How do we combine the transformations?
DTU Compute
2020Image Analysis28 DTU Compute, Technical University of Denmark
Combining transformations Combination is done by matrix multiplication
Scaling
Rotation
Combined
DTU Compute
2020Image Analysis29 DTU Compute, Technical University of Denmark
Combining transformations Compact notation
Scaling
Rotation
Combined
DTU Compute
2020Image Analysis30 DTU Compute, Technical University of Denmark
Combining transformsA) 1B) 2C) 3D) 4E) 5
31
1 2 0 0
A B C D E
The point (x,y)=(5,6) is transformed. First with:
and then with:
The result is:
Chart1
A
B
C
D
E
31
1
2
0
0
Sheet1
A31
B1
C2
D0
E0
DTU Compute
2020Image Analysis31 DTU Compute, Technical University of Denmark
What do we have now? We can pick a position in the input image f and find
it in the output image g
f g
We can transfer one pixel –what about the whole image?
DTU Compute
2020Image Analysis32 DTU Compute, Technical University of Denmark
Solution 1 : Input-to-output
f
Scaling example
0 1 2 3
0
1
2
3
g
• Run through all pixel in input image
• Find position in output image and set output pixel value
0 1 2 3
0
1
2
3
x
y
x
y
Missing value!
23 216 120
DTU Compute
2020Image Analysis33 DTU Compute, Technical University of Denmark
Input-to-Output The input to output transform is not good! It creates holes and other nasty looking stuff What do we do now?
DTU Compute
2020Image Analysis34 DTU Compute, Technical University of Denmark
Some observations We want to fill all the pixels in the output image
– Not just the pixels that are “hit” by the pixels in the input image
Run through all pixels in the output image?– Pick the relevant pixels in the input image?
g
0 1 2 3
0
1
2
3
x
y
We need to go “backwards”
From the output to the input
DTU Compute
2020Image Analysis35 DTU Compute, Technical University of Denmark
Inverse transformation We want to go from the output to the input
f
Scaling example
0 1 2 3
0
1
2
3
g
0 1 2 3
0
1
2
3
x
y
x
y
?
inverse
DTU Compute
2020Image Analysis36 DTU Compute, Technical University of Denmark
Output-to-input transformationBackward mapping
f
0 1 2 3
0
1
2
3
g
0 1 2 3
0
1
2
3
x
y
x
y
• Run through all pixel in output image
• Find position in input image and get the value
What is the value here?
DTU Compute
2020Image Analysis37 DTU Compute, Technical University of Denmark
Bilinear Interpolation The value is calculated from
4 neighbours The value is based on the
distance to the neighbours
Value?
Lots of 89!
Not so much 189
DTU Compute
2020Image Analysis38 DTU Compute, Technical University of Denmark
Linear Interpolation
a b
V=?
1
t
𝑣𝑣 = 𝑡𝑡𝑡𝑡 + 1 − 𝑡𝑡 𝑎𝑎
DTU Compute
2020Image Analysis39 DTU Compute, Technical University of Denmark
Bilinear interpolation
DTU Compute
2020Image Analysis40 DTU Compute, Technical University of Denmark
Bilinearinterpolation
A) 1B) 2C) 3D) 4E) 5
5
28
0 2 0
A B C D E
Bilinear interpolation is used to create a line profile from an image. In a given point (x,y) = (173.1, 57.8), the four nearest pixels are:
What is the interpolated value in the point:
Chart1
A
B
C
D
E
5
28
0
2
0
Sheet1
A5
B28
C0
D2
E0
DTU Compute
2020Image Analysis41 DTU Compute, Technical University of Denmark
Output-to-input transformationBackward mapping Run through all the pixel in the output image Use the inverse transformation to find the position in
the input image Use bilinear interpolation to calculate the value Put the value in the output image
DTU Compute
2020Image Analysis42 DTU Compute, Technical University of Denmark
Inverse transformation We can calculate the inverse
transformation for the scaling
What about the others?Scaling
Inverse
DTU Compute
2020Image Analysis43 DTU Compute, Technical University of Denmark
General inverse transformation The transformation is
expressed as a transformation matrix A
The matrix inverse of A gives the inverse transformation
Affine transformation
Inverse transformation
Where
DTU Compute
2020Image Analysis44 DTU Compute, Technical University of Denmark
General inverse transformation
You can create an arbitrary sequence of rotation, scaling, and shearing
Scaling ScalingRotation
Inverse:
Combined:
DTU Compute
2020Image Analysis45 DTU Compute, Technical University of Denmark
Transformation
A) 1B) 2C) 3D) 4E) 5
26
1 1 03
A B C D E
The point (x,y) = (45, 23) is transformed using:
And the result is translated with (-15, 20). The result is:
Chart1
A
B
C
D
E
26
1
1
0
3
Sheet1
A26
B1
C1
D0
E3
DTU Compute
2020Image Analysis46 DTU Compute, Technical University of Denmark
Profile Analysis Sample the pixel values
under the profile From start point to end
point
DTU Compute
2020Image Analysis47 DTU Compute, Technical University of Denmark
Profile Analysis Length of profile
Unit vector
Point on profile
ps
pe
DTU Compute
2020Image Analysis48 DTU Compute, Technical University of Denmark
Profile Analysis Point on profile
Sample all points on profile using bi-linear interpolation
ps
pe
DTU Compute
2020Image Analysis49 DTU Compute, Technical University of Denmark
Profile Analysis ps
pe ps pe
DTU Compute
2020Image Analysis50 DTU Compute, Technical University of Denmark
Homography
DTU Compute
2020Image Analysis51 DTU Compute, Technical University of Denmark
Homography Maps from one plane to
another plane Projective transformation
Mapping from coordinates (x,y) to (x’, y’)
DTU Compute
2020Image Analysis52 DTU Compute, Technical University of Denmark
Virtual Billboard Example
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
DTU Compute
2020Image Analysis53 DTU Compute, Technical University of Denmark
Virtual Billboard Example
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
DTU Compute
2020Image Analysis54 DTU Compute, Technical University of Denmark
Virtual Billboard Example
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
We want to estimate:
One point in each image gives:
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
DTU Compute
2020Image Analysis55 DTU Compute, Technical University of Denmark
Virtual Billboard Example
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
Eight unknowns:
Requires four corresponding points
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
DTU Compute
2020Image Analysis56 DTU Compute, Technical University of Denmark
Virtual Billboard Example
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
Equation system:
Can be solved using linear algebra (SVD):
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
DTU Compute
2020Image Analysis57 DTU Compute, Technical University of Denmark
Virtual Billboard Example
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
• For each pixel in the output image
• Compute the position in the billboard image using the homography
• If position is within the billboard
• Get the billboard pixel value and put it in the output image
https://www.learnopencv.com/homography-examples-using-opencv-python-c/
DTU Compute
2020Image Analysis71 DTU Compute, Technical University of Denmark
Next week – Image Registration
Medical Image AnalysisLecture 8 – Geometric TransformationWhat can you do after today?Geometric transformationChange detectionImage RegistrationCamera CalibrationGeometric TransformDifferent transformationsTranslationRotationRotation coordinate systemRigid body transformationScalingSimilarity transformationShearingTransformation MathematicsTranslation mathematicsMatrix notationTranslation mathematics in matrix notationScalingMatrix multiplication detailsTransformation matrixRotationShearingAffine transformationCombining transformationsCombining transformationsCombining transformationsCombining transformsWhat do we have now?Solution 1 : Input-to-outputInput-to-OutputSome observationsInverse transformationOutput-to-input transformation�Backward mappingBilinear InterpolationLinear InterpolationBilinear interpolationBilinear�interpolationOutput-to-input transformation�Backward mappingInverse transformationGeneral inverse transformationGeneral inverse transformationTransformationProfile AnalysisProfile AnalysisProfile AnalysisProfile AnalysisHomographyHomographyVirtual Billboard ExampleVirtual Billboard ExampleVirtual Billboard ExampleVirtual Billboard ExampleVirtual Billboard ExampleVirtual Billboard ExampleNext week – Image Registration