CATRA: Interactive Measuring and Modelling of Cataracts

CATRA: Interactive Measuring and Modeling of Cataracts

Vitor F. Pamplona Erick B. Passos Jan Zizka Manuel M. Oliveira Everett Lawson Esteban Clua Ramesh Raskar

MIT Media Lab – Camera Culture

CATRA: Quantitative Maps for Self-assessment of Early Cataracts

Vitor Pamplona , Ankit Mohan, Manuel Oliveira, Ramesh Raskar, ACM SIGGRAPH 2010

NETRA: Near Eye Tool for Refractive Assessment

Main Cause of Preventable Blindness

Slit-Lamp Microscope

Computational Ophthalmology

Four Stages of Interaction

Opacity Map(position, size)

Attenuation Map(brightness)

Contrast Map(contrast) PSF Map

CCCCCC

CCC

CC

CCC

CCC

CC

CC

Occlusion Scattering

Four Stages of Interaction

Opacity Map(position, size)

Attenuation Map(brightness)

Contrast Map(contrast) PSF Map

CCCCCC

CCC

CC

CCC

CCC

CC

CC

3mm0.6mm

Occlusion Scattering

Forward Scattering Sensed on Fovea

LCD1 LensLCD2

Projection on the Fovea

Light Box

Testing Sections

Forward Scattering Sensed on Fovea

LCD1 LensLCD2

Projection on the Fovea

Light Box

Testing Sections

Testing Sections of the Eye Lens

LCD1 LensLCD2

Trading Resolution for Brightness

LCD1 LensLCD2

Collimated

Beams

Testing Section

Healthy Eye Point Spread Function = Peak

Eye PSF Visual Representation

Mild Cataracts Point Spread Function

Eye PSF Visual Representation

Advanced Cataracts Point Spread Function

Eye PSF Visual Representation

CATRA: Measuring, Modeling and Simulation

to estimate PSF and simulate visionWe want to map the aperture

Peak

σ

Cataract ScreeningPresence of

Cataracts(Binary Answer)

Detecting the Presence of Cataracts

Moving patterns on LCD1

LCD1 LCD2

Detecting the Presence of Cataracts

LCD1 LCD2

Detecting the Presence of Cataracts

LCD1 LCD2 Eye lens

Detecting the Presence of Cataracts

Perceived ImageLCD1 LCD2 Eye lens

Cataract ScreeningPresence of

Cataracts(Binary Answer)

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Presence of Cataracts

(Binary Answer)

Estimating an Opacity MapMoving patterns

on LCD1

LCD1 LCD2

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Presence of Cataracts

(Binary Answer)

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Brightness Test(Attenuation Map)

Presence of Cataracts

(Binary Answer)

LCD1 LCD2

Estimating Attenuation MapsDecreasing Brightness

on the clear path

Estimating Attenuation Maps

LCD1 LCD2

Same PerceivedBrightness

Decreasing Brightness

on the clear path

Estimating Attenuation Maps

Perceived ImageLCD1 LCD2

Decreasing Intensity of the Clear Path

To Match Brightness

Eye

Estimating Attenuation Maps

Perceived ImageLCD1 LCD2

Decreasing Intensity of the Clear Path

To Match Brightness

Eye

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Brightness Test(Attenuation Map)

Presence of Cataracts

(Binary Answer)

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Brightness Test(Attenuation Map)

Peak

Presence of Cataracts

(Binary Answer)

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Brightness Test(Attenuation Map)

Sub-apertureContrast Test

(Contrast Map)

Presence of Cataracts

(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

Contrast Test

Perceived ImageLCD1 LCD2

Rotated Low Contrast Letter C

Increasing Contrast

Eye

Contrast Test

Perceived ImageLCD1 LCD2

Rotated Low Contrast Letter C

Increasing Contrast

Eye

Contrast Test

Perceived ImageLCD1 LCD2

Rotated Low Contrast Letter C

Increasing Contrast

Eye

Contrast Test

Perceived ImageLCD1 LCD2

Rotated Low Contrast Letter C

Increasing Contrast

Press the right key

Eye

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Brightness Test(Attenuation Map)

Sub-apertureContrast Test

(Contrast Map)

Presence of Cataracts

(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Brightness Test(Attenuation Map)

Sub-apertureContrast Test

(Contrast Map) σ

Presence of Cataracts

(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Brightness Test(Attenuation Map)

Sub-apertureContrast Test

(Contrast Map)

Sub-aperture PSF Match(PSF Map)

Presence of Cataracts

(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

Point Spread Function Matching

Perceived ImageLCD1 LCD2 Eye

Perceived ImageLCD1 LCD2 Eye

Point Spread Function Matching

Perceived ImageLCD1 LCD2 Eye

Point Spread Function Matching

Perceived ImageLCD1 LCD2

Sub-aperture Point Spread Function

Eye

Point Spread Function Matching

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Brightness Test(Attenuation Map)

Sub-apertureContrast Test

(Contrast Map)

Sub-aperture PSF Match(PSF Map)

Presence of Cataracts

(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)

Brightness Test(Attenuation Map)

Sub-apertureContrast Test

(Contrast Map)

Sub-aperture PSF Match(PSF Map)

σ

Presence of Cataracts

(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

Contrast Tests when High-Attenuated

Sub-apertureContrast Test

(Contrast Map)

Sub-aperture PSF Match(PSF Map)

Brightness Test(Attenuation Map)

Peak

Peak

CCCCCC

CCC

CC

CCC

CCC

CC

CC

High Attenuation

Low Attenuation

Reducing Search Space for PSF

Position, Size and Shape

(Opacity Map)

Brightness Test(Attenuation Map)

Sub-apertureContrast Test

(Contrast Map)

Sub-aperture PSF Match(PSF Map)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

Low Attenuation

High Attenuation

Presence of Cataracts

(Binary Answer)

Dual Monitor Stack

Dual Monitor Stack

Dual Monitor StackLens

Medical Monochrome

Monitors LCD Stack

Cell Phone Prototype

Lens

Pinhole Mask

Validation using CamerasThin Diffuser to

Simulate Cataracts

Estimated Attenuation Map Estimated PSF Map

Validation using Cameras

Simulated Cataract

EstimatedOpacity Map

Measured Attenuation

(Pictures)

Estimated Attenuation

(User Interaction)

Scratching Contact Lenses

Advanced Cataracts

MildCataracts

EarlyCataracts

Measured: 0.5mm2

Estimated: 0.45mm2

Elderly Evaluation

Rendering Cataract Effects

Eye’s PSF = Summation of Sub-aperture PSF’s

For the point of focus only

Depth-dependent PSF

Focal PointCloser Far

Scene for Simulation

Depth Map

Picture with Cataract Lenses

Simulation with Estimated PSF

Bokeh Effect

Limitations

• Active user participation

• One clear light path

• Retinal diseasesPatience and concentration are required to follow steps

Future Clinical TestsClinical Testing Partners

for CATRALast year’s NETRA Prototypes

(29 partners in 14 countries)

Acknowledgments

Bruce SherleenShrikant CaioDon Rob

TiagoTyler Fabiano

Computational Ophthalmology• A Co-design of Optics and CG for Measuring Cataracts

– Forward Scattering– Foveal Image

• Four-stage Interactive Maps – A Radar for the Cloud Cover– Better than Subjective Rating

• Simulation of an individual’s cataract-affected eye

CCCCCC

CCC

CC

CCC

CCC

CC

CC

CATRA: Quantitative Maps for Self-assessment of Early Cataracts

More at: eyecatra.com

Mass-use Devices Scientific Instruments

EyeNETRA Initiative

EyeCATRA.com

CCCCCC

CCC

CC

CCC

CCC

CC

CC

LCD2LCD1

Vitor Pamplona, Ankit Mohan, Manuel Oliveira, Ramesh Raskar SIGGRAPH 2010

NETRA Near Eye Tool for Refractive Assessment

Preventable Blindness

Cataract Types

Nuclear Sub-capsular Cortical

Cataract Types

Nuclear Sub-capsular Cortical

Cataract Types

Nuclear Sub-capsular Cortical

Cataract Types

Nuclear Sub-capsular Cortical

Easier to detect