Landmark localization and registration of 3D facial scans

for the evaluation of orthodontic treatments in maxillofacial and oral surgery

School of EECS: Prathap Nair, Dr Andrea Cavallaro

School of Medicine and Dentistry: Dr Lifong Zou

Mid-project update

What is the problem?

• To quantify 3D facial asymmetry

• Clinical diagnosis• Treatment planning• Post-treatment monitoring• Statistical studies on a large population

• What is rigid registration?• Alignment of 2 or more faces

• Classical approach: Iterative Closest Point (ICP) algorithm• Advantage

• no prior info needed• Disadvantage

• random points used for matching can lead to erroneous results

Approach: rigid registration

Example

Our approach

• Rigid registration based on landmarks • Landmark detection via Statistical Shape Analysis

BtG project: Achievement 1

• Improved accuracy

Red – before BtG

Green – after BtG

Approach: overview

Test Scan

Reference scan

Detection of

Landmark

Points

Detection of

Landmark

Points

Coarse registration using

Key landmark

s

Detection of Stable regions

Fine registrat

ion using the

Semantic

Regions

Distance

estimation

Test scan Reference scan

Approach: overview

Test Scan

Reference scan

Detection of

Landmark

Points

Detection of

Landmark

Points

Coarse registration using

Key landmark

s

Detection of Stable regions

Fine registrat

ion using the

Semantic

Regions

Distance

estimation

Test scan Reference scan

Approach: overview

Test Scan

Reference scan

Detection of

Landmark

Points

Detection of

Landmark

Points

Coarse registration using

Key landmark

s

Detection of Stable regions

Fine registrat

ion using the

Semantic

Regions

Distance

estimation

Test scan Reference scan

Approach: overview

Test Scan

Reference scan

Detection of

Landmark

Points

Detection of

Landmark

Points

Coarse registration using

Key landmark

s

Detection of Stable regions

Fine registrat

ion using the

Semantic

Regions

Distance

estimation

Key Landmar

ks

Coarse registration

Test scan Reference scan

Approach: overview

Test Scan

Reference scan

Detection of

Landmark

Points

Detection of

Landmark

Points

Coarse registration using

Key landmark

s

Detection of Stable regions

Fine registrat

ion using the

Semantic

Regions

Distance

estimation

Test scan Reference scan

Approach: overview

Test Scan

Reference scan

Detection of

Landmark

Points

Detection of

Landmark

Points

Coarse registration using

Key landmark

s

Detection of Stable regions

Fine registrat

ion using the

Semantic

Regions

Distance

estimation

Fine registration

Approach: overview

Test Scan

Reference scan

Detection of

Landmark

Points

Detection of

Landmark

Points

Coarse registration using

Key landmark

s

Detection of Stable regions

Fine registrat

ion using the

Semantic

Regions

Distance

estimation

ICP Proposed approach

Example

BtG project: Achievement 2

• User friendly GUI• To ease burden on clinicians• User-feedback mechanisms

Conclusions

• Achievements• Improved landmark localisation accuracy • More user-friendly GUI with the user feedback

• Current work• Clinical evaluation of the landmark detection accuracy • Validation of 3D facial scan registration accuracy• Further improving the GUI based on clinician feedback

Contact:

prathap.nair@elec.qmul.ac.uk

andrea.cavallaro@elec.qmul.ac.uk