Automatic Landmark Detection using Statistical Shape Modelling
and Template Matching
Authors Habib Baluwala, Duane Malcolm, Jess Jor,
Poul Nielsen, Martyn Nash
Introduction
● Research Problem: Develop biomechanical models of the human breast
● Construction of biomechanical model of the torso skin surface
● Objective : – Align the mean mesh of the skin surface with new image
data– Match the mesh to the edges of the skin surface of the new
image
● Wide intensity range ● Variability of breast shapes ● Variability of torso shapes
Challenges in Mesh Alignment
Statistical Shape Modelling (SSM)
● Most structures of clinical interest have a characteristic shape and anatomical location relative to other structures
● Across the normal population the shape varies statistically
● Provides prior knowledge of shape
Training Data
● Thirty 2D MRI training images ● The outline of the torso skin surface is represented by 24
manually labelled points● We add another 6 anatomical landmarks:
– Sternum centre
– Aorta centre
– Spinal cord centre
– Vertebra centre – Left and right nipple
● Total : 30 landmarks
Modelling Shape using PCA
● Compute the mean of the data ● Compute the covariance of the data ● Compute the eigenvectors and eigenvalues of the
covariance matrix, sorted in decreasing order of eigenvalue size
● Remove the small eigenvalues, retaining most of the variation
● is the mean shape, is a set of orthogonal modes of variation and defines a set of components of deformable model.
Torso Shape Model
+
++
Template Matching● Move a template over an image and calculate
the similarity between the template and image patch
● Similarity measures– Cross Correlation (CC)
– Normalised Cross correlation (NCC)
– Sum of Squared Differences (SSD)
– Normalised Sum of Squared Differences (NSSD)
Average Template
+ + +
=
…..(30 images)
Template Matching
Vertebra centre template
Test image
Template matching result (correlation maps)
Template Matching
Aorta centre template
Test image
Template matching result (correlation maps)
Combining SSM and Template matching
● Vary the mode weights for the first three shape components
● Calculate the new shape and landmark positions
● Move the correlation map to its respective landmark location in the SSM shape model
● Multiply the correlation maps
SSM + Template matching
..
.27 more landmark template matching results
Shape Model
SSM + Template Matching Results(Shape Predicted Landmarks)
Manually selected landmarks and skin surface
Shape predicted landmarks
Local Maxima Search ● Crop the correlation map around the shape
predicted landmark (120 mm x 120 mm) ● Move the shape predicted landmark to the
local maximum of the correlation map
Shape predicted landmark Cropped correlation map Shape predicted landmark + local maxima
search
SSM + Template Matching Results+ Local Maxima Search
Move the shape predicted landmark to a local maximum using correlation maps for individual landmarks
Manually selected landmarks and skin surface
Shape predicted landmarks + local maxima search
ResultsSeries of leave-one-out experiments performed on thirty
2D MRI images
Conclusion
● SSM + Template Matching + Local Maxima search provides a robust detection of landmark points on skin surface
● Average error = 3.4mm 2.1 mm
Future Work● Extend the algorithm to 3D● Incorporate active appearance models
Questions !!!