Computerized Craniofacial Reconstruction using CT-derived Implicit Surface Representations

D. Vandermeulen, P. Claes, D. Loeckx, P. SuetensMedical Image Computing (ESAT-Radiology)

S. De Greef, G. WillemsCentre of Forensic Odontology

K.U.Leuven, Faculties of Medicine and Engineering

“Louvre” Seminar March 29 2006

Cranio-facial reconstruction: What?

Manual Craniofacial Reconstruction

• subjective/ artistic talentLots of expertise, both explicit

(documented) and implicit

• Errors by inconsistencies in application

Misalignment of LM on the skull

• time consuming• Only few reconstructions

possible

TAYLOR, K. T. 2001. Forensic Art and Illustration. CRC Press LLC.

Introduction

•Manual

•Computer

•Bias

•Dense LM

Data

Method

Results

Discussion

Conclusion

Computer-based reconstructions

• Small number of tissue thickness measurement landmarks (LMs)

• Independent soft tissue thickness and facial surfaces

• Template-bias: face interpolation with a single facial template (either generic or gender/ancestry/age matched)

http://www.cs.ubc.ca/nest/imager/contributions/katrinaa/recon.html

Introduction

•Manual

•Computer

•Bias

•Dense LM

Data

Method

Results

Discussion

Conclusion

Removing template bias

• Using statistical facial templates to remove bias (Claes et al.)

• Using combined statistical model of facial template and soft tissue thicknesses (Claes et al.)

Fitting Algorithm

. . .

Database

Introduction

•Manual

•Computer

•Bias

•Dense LM

Data

Method

Results

Discussion

Conclusion

Surface and sparse Landmark-based Craniofacial Reconstruction

Dense Landmark model

Warp W

Warp W

target

Reference skull

Reference skin Warped skin

Warped skull

Introduction

•Manual

•Computer

•Bias

•Dense LM

Data

Method

Results

Discussion

Conclusion

Statistical dense LM model

…..

…..

Template database

target

Introduction

•Manual

•Computer

•Bias

•Dense LM

Data

Method

Results

Discussion

Conclusion

CT scan+ Simultaneous visualisation hard & soft tissues- Using ionising radiation

• PM-CT can be used as golden standard?dehydration!

• in-vivo CT on control population? only by lowering radiation dose!

Volumetric Template Data : CTIntroduction

Data

•MR

•CT

•LD-CT

Method

Results

Discussion

Conclusion

Volumetric Template Data : MRI

MRI scan+ Excellent visualisation of soft tissues

- Bone details lost

Introduction

Data

•MR

•CT

•LD-CT

Method

Results

Discussion

Conclusion

Low-Dose CT

• Decrease radiation dose to acceptable (?) level while retaining sufficient quality for diagnosis, therapy or image-based measurements

• Starting from clinical multi-slice spiral CT protocol (Siemens Sensation 16 (Erlangen, Germany)) by lowering the X-ray source current and voltage and increasing the pitch.

• Measured effective radiation dose: 0.18 mSv i.o. 1.5 mSv

• Measuring image quality: thickness differences smaller than a voxel (<0.5 mm).

Introduction

Data

•MR

•CT

•LD-CT

Method

Results

Discussion

Conclusion

CT image preprocessing

• Acquisition and conversion from DICOM to Analyze

• Noise reduction using edge preserving filtering

• Metal artifact removal

• Segmentation of skin and bone surfaces by (hysteresis) thresholding and mathematical morphology

• Implicit Surface representation by signed Distance Transformation

Introduction

Data

Method

Results

Discussion

Conclusion

Implicit Functions for Object Representations and

Transformations:a tentative tutorial

Dirk Vandermeulen

Medical Image Computing

Seminar January 17, 2003

3-D example

Copyright FarField Technology Ltd.

… Shape Morphing

Alternative: interpolate the smooth implicit functions!Example: f(x) = signed distanceG. Turk and J. F. O’Brien, Shape Transformation using Variational Implicit Functions, Siggraph 99

f1(x)>0 f2(x)>0t.f1(x)+(1-t).f2(x)>0

0t1

Shape Transformation Using Variational Implicit Functions, Greg Turk James F. O’Brien, ACM Siggraph99

Implicit Surface Representation

Signed Distance Transform (sDT)Introduction

Data

Method

•MAR

•sDT

•Warping

•Reconstruction

Results

Discussion

Conclusion

Craniofacial reconstruction: method

Warp W

target

Reference skull Warped skull

Warp W

Reference skin Warped skin

Introduction

Data

Method

•MAR

•sDT

•Warping

•Reconstruction

Results

Discussion

Conclusion

Warping method (D.Loeckx et al.)

• Represent warping by tensor-product B-Spline Free Form Deformation (FFD)Introduction

Data

Method

•MAR

•sDT

•Warping

•Reconstruction

Results

Discussion

Conclusion

Warping method (D.Loeckx et al.)

Regularization of FFD by Volume-preserving penalty

Example: template skull to target skull warping

Example: extrapolation to template skin warping

=?

=?

Example: extrapolation to template skin warping

Skin Surface Reconstruction

• Construct (weighted) average of warped skin sDT’s

Introduction

Data

Method

•MAR

•sDT

•Warping

•Reconstruction

Results

Discussion

Conclusion

Example

Validation

• Given only small-sized database (N=20), how to separate into test and validation subsets?

• N-fold Cross-Validation or Leave-one-out CV:– For i=1:NrSubjects

• Reconstruct Subject i from all other subjects in Database• Compare Result to ground truth of i• Evaluate Error

}0/))(ˆ(|{ˆ N

iH

it NxDxS

Qualitative Validation

• Qualitative: 3D reconstructions vs subset of database (face pool comparisons)

Introduction

Data

Method

Results

Discussion

Conclusion

Face pool comparisons

Quantitative Validation I

• Calculate distances between reconstructed and ground truth surface

Introduction

Data

Method

Results

Discussion

Conclusion

|d| = 1.6 ± 1.2 mm

Quantitative Validation I

• Gather error statistics over all subjects• Define M ( 500) test points on a reference head surface

• Find corresponding points on all surfaces by non-rigid surface-based warping (Claes et al.)

• Evaluate error (distance from reconstructed surface to real surface) at test points: mean, std, etc…

Quantitative Validation I: results

Average (1.9mm) Std (1.7mm)

Quantitative Validation II

• Not reconstruction accuracy but recognition accuracy• Based on similarity measure between surfaces or, in this case,

M reference points pi (same as before) on the surfaces S

• Use coordinate-system free representation (invariant to translation/rotation) of surface SEuclidean Distance Matrix ES: ES (i,j) = ||pi-pj||

Quantitative Validation II

• How to measure similarity between two surfaces S1 and S2?

• Compare ES1 to ES2: e.g. – Sum of Squared Differences:

||ES1-ES2|| = i,j>i ( ES1 (i,j) – ES2 (i,j) )2/L >= 0

– (normalized) Cross-Correlation

• Invariance to scaling/size by normalizing EDM with size factor, e.g. geometric mean

NS (i,j) = ES (i,j) / (ES), with (ES) = (ij ES (i,j) )1/L

Quantitative Validation II• Generate Classification Rank Matrix

1 7 20 11 5 4 16 13 3 12 6 15 19 17 8 9 2 18 14 102 6 15 12 19 16 3 5 17 7 11 4 8 20 1 13 18 10 14 93 5 4 16 11 12 7 6 13 15 17 19 2 20 8 1 14 9 10 1813 4 3 14 9 8 16 12 17 5 10 1 11 7 2 15 6 20 19 187 3 11 5 20 1 16 4 6 19 2 12 15 13 17 18 8 9 14 106 15 19 11 2 7 16 12 3 20 5 1 17 13 4 18 8 14 10 95 1 13 6 9 3 14 7 4 16 2 8 11 17 12 20 10 15 19 188 10 14 4 13 17 12 3 9 7 16 15 6 2 1 5 11 20 19 189 7 4 14 8 13 10 3 1 17 5 12 6 16 11 20 15 2 19 1810 8 14 4 13 17 12 7 9 3 16 15 6 1 2 11 5 20 19 1811 5 6 3 20 1 16 19 12 4 15 13 2 7 17 18 8 14 10 912 15 3 16 19 6 2 17 4 20 11 8 5 13 1 7 10 18 14 94 13 14 7 17 8 3 16 10 1 11 15 12 6 9 5 20 2 19 1814 13 8 4 10 9 7 17 3 16 12 1 6 15 11 5 2 20 19 186 12 2 3 15 20 11 16 13 5 17 4 1 8 19 7 10 18 14 916 3 19 12 6 11 4 5 13 17 2 1 15 7 20 8 10 14 18 917 13 12 3 8 4 16 10 2 14 15 6 19 7 11 5 1 20 9 1818 6 19 5 12 11 16 1 2 20 15 3 17 4 13 7 8 10 14 96 16 12 2 19 3 5 11 17 1 15 20 18 4 13 7 8 10 14 920 1 11 5 6 12 3 15 16 7 17 4 19 13 2 18 8 14 10 9

• Correct Rank 1 Classification: 14/20 (13/20)• Correct Rank <= 2 Classification: 16/20 (14/20)

Algorithmic improvements

• Metal Artifact Reduction: using morphology (opening) only in slices ~ artifacts

• Non-rigid registration– fine tuning of regularization parameters to improve skull-

skull matching and extrapolation quality– Alternative deformation models

• Statistical Deformation Models (based on sDT or original CT of database)

– Combination with local models (e.g. nose (De Greef))– Combination with point/surface model (Claes)

• Weighted averaging of sDT– Weights ~ skull overlap– Weights ~ class similarity (gender, age, BMI)– Spatially varying weights

• Statistical Modes of Variation

Introduction

Data

Method

Results

Discussion

Conclusion

Metal ArtifactsIntroduction

Data

Method

Results

Discussion

Conclusion

Metal Artifact RemovalIntroduction

Data

Method

•MAR

•sDT

•Warping

•Reconstruction

Results

Discussion

Conclusion

Algorithmic improvements

• Metal Artifact Reduction: using morphology (opening) only in slices ~ artifacts

• Non-rigid registration– fine tuning of regularization parameters to improve skull-skull

matching and extrapolation quality, again using leave-one-out cross-validation

– Alternative deformation models• Statistical Deformation Models (based on sDT or original CT of

database)

– Combination with local models (e.g. nose (De Greef))– Combination with point/surface model (Claes)

• Weighted averaging of sDT– Weights ~ skull overlap– Weights ~ class similarity (gender, age, BMI)– Spatially varying weights

• Statistical Modes of Variation

Introduction

Data

Method

Results

Discussion

Conclusion

Algorithmic improvements

• Metal Artifact Reduction: using morphology (opening) only in slices ~ artifacts

• Non-rigid registration– fine tuning of regularization parameters to improve skull-

skull matching and extrapolation quality– Alternative deformation models

• Statistical Deformation Models (based on sDT or original CT of database)

– Combination with local models (e.g. nose (De Greef))– Combination with point/surface model (Claes)

• Weighted averaging of sDT– Weights ~ skull overlap– Weights ~ class similarity (gender, age, BMI)– Spatially varying weights

• Statistical Modes of Variation

Introduction

Data

Method

Results

Discussion

Conclusion

Attribute-modulated reconstruction

• All reconstructions so far made with all data in the database, irrespective of gender, age and BMI!

sDT = i wi sDTi , wi = 1/N

Attribute-weighted interpolation• How to bias reconstruction to specific attribute values? (k-)Nearest Neighbour?• Problem: small database, hence weak statistical model (PCA, PLS, …)• Solution(?): “Shape by Example”

– Given attribute values (gender, age, BMI) pi and q of template subjects i and target subject, resp.– Find weight wi(q) to apply to sDTi in weighted average

sDT = I wi(q) sDTi , wi(pj) ij

– Determined using RBF smoothest approximation

Example

Example

All Females only Males only

AWI Females+BMI Ground truth

Algorithmic improvements

• Metal Artifact Reduction: using morphology (opening) only in slices ~ artifacts

• Non-rigid registration– fine tuning of regularization parameters to improve skull-

skull matching and extrapolation quality– Alternative deformation models

• Statistical Deformation Models (based on sDT or original CT of database)

• Weighted averaging of sDT– Weights ~ skull overlap– Weights ~ class similarity (gender, age, BMI)– Spatially varying weights

• Statistical Modes of Variation

Introduction

Data

Method

Results

Discussion

Conclusion

Conclusion

• “Proof of concept” of volumetric cranio-facial reconstruction

• Validation procedure required on a representative database

• Metal Artifact Reduction is required • Missing Data problem using deformation model• Comments?

Introduction

Data

Method

Results

Discussion

Conclusion