1

Vessels delineation in retinal images using COSFIRE filters

George Azzopardi1,3, Nicola Strisciuglio1,2, Mario Vento2, Nicolai Petkov1

1University of Groningen (The Netherlands) - 2University of Salerno (Italy) – 3University of Malta

university of salerno

BICV  Summer  School,  Shenyang,  China,  2015   1  

Delineation

Automated delineation of elongated structures gained great interest in the image processing community Applications: detection of crack in walls, segmentation of rivers in aerial images, extraction of blood vessels from biomedical images

BICV  Summer  School,  Shenyang,  China,  2015   2  

Motivation

BICV  Summer  School,  Shenyang,  China,  2015  

› The structure of the retinal vascular tree can reveal signs of cardiovascular diseases

for vessel delineation can speed-up the › An automatic process diagnosis process

3  

Related works

BICV  Summer  School,  Shenyang,  China,  2015  

•  Mainly based on convolution and matched filters [3-4], mathematical morphology [5]

•  Suffer from high sensitivity to noise

› Supervised Methods

•  Based on pixel-wise feature vectors computation and classification with machine learning tools [6-9]

•  High computational time

•  Complex learning procedures

› Unsupervised Methods

4  

Filter Configuration

BICV  Summer  School,  Shenyang,  China,  2015  

› The CORF/COSFIRE filter is trainable

Prototype pattern DoG Response Local intensity maxima

4 2 1

3 5

5  

Filter Configuration

BICV  Summer  School,  Shenyang,  China,  2015  

› The CORF/COSFIRE filter is trainable

Local intensity maxima

4 2 1

3 5

Filter Model

6  

Pipeline

BICV  Summer  School,  Shenyang,  China,  2015   7  

Rotation Invariance

45° 60° 75°

BICV  Summer  School,  Shenyang,  China,  2015  

90° 105°

0° 15° 30°

120° 135° 150° 165°

8  

Rotation Invariance

45° 60° 75°

BICV  Summer  School,  Shenyang,  China,  2015   9  

Data sets

BICV  Summer  School,  Shenyang,  China,  2015  

› DRIVE: 40 JPEG images at 768x584 pixels (20 training, 20 testing) › STARE: 20 JPEG images at 700x605 pixels › CHASE_DB1: 28 JPEG images at 1280x960 pixels

DRIVE STARE CHASE_DB1

10  

Configured Filters

BICV  Summer  School,  Shenyang,  China,  2015  

› A bar-selective (symmetric) COSFIRE filter is configured to detect vessels

Original image Bar selective

(12 orientations)

Filter output

11  

Configured Filters

BICV  Summer  School,  Shenyang,  China,  2015  

› A bar-selective (symmetric) COSFIRE filter is configured to detect vessels

Original image Ground truth Filter output

12  

Configured Filters

BICV  Summer  School,  Shenyang,  China,  2015  

› A bar-ending-selective (asymmetric) COSFIRE filter is configured to be responsive on vessel-endings

Ground truth Asymmetric filter output

13  

Configured Filters

BICV  Summer  School,  Shenyang,  China,  2015  

› A bar-ending-selective (asymmetric) COSFIRE filter is configured to be responsive on vessel-endings

Ground truth Symmetric filter output Asymmetric filter output

14  

Performance Evaluation

BICV  Summer  School,  Shenyang,  China,  2015  

› We measured the performance in terms of:

•  Matthews Correlation Coefficient (MCC)

•  Sensitivity (Se)

•  Specificity (Sp)

•  Accuracy (Acc)

•  Area under ROC curve (AUC)

15  

ROC curves

BICV  Summer  School,  Shenyang,  China,  2015  

› A r e a u n d e r R O C curve

› DRIVE = 0.9614 › STARE = 0.9563 › CHASE_DB1= 0.9487

Close to Human observer performance (no statistical difference)

16  

Results Comparison (1/3)

BICV  Summer  School,  Shenyang,  China,  2015   17  

Results Comparison (2/3)

BICV  Summer  School,  Shenyang,  China,  2015   18  

Results Comparison (3/3)

BICV  Summer  School,  Shenyang,  China,  2015   19  

Time Efficiency

BICV  Summer  School,  Shenyang,  China,  2015  

› Most efficient method for vessel delineation in retinal images ever published

*Processing time is reported for DRIVE and STARE data sets

20  

Robustness to noise

BICV  Summer  School,  Shenyang,  China,  2015   21  

Supervised method

•  Delineation of vessels of different thickness •  Selection of the most relevant filters for the application at hand by

Generalized Matrix Learning Vector Quantization (GMLVQ)

…

COSFIRE filter-bank Pixel-wise features Filters selection Classification

Relevances (GMLVQ) SVM

BICV  Summer  School,  Shenyang,  China,  2015   22  

A bank of B-COSFIRE filters

•  We configure a bank of 21 vessels detector and 21 vessel-endings detector (deal with vessels of different thickness)

•  We describe each pixel with a vector of the responses of the configured filters

Vessel selective (12 orientations)

Vessel-ending selective (24 orientations)

BICV  Summer  School,  Shenyang,  China,  2015   23  

A bank of B-COSFIRE filters

BICV  Summer  School,  Shenyang,  China,  2015   24  

Filter selection

•  GMLVQ evaluates the relevance of the single and all of pairs of filters (matrix of relevance)

•  We select the filters with local relevance maxima

BICV  Summer  School,  Shenyang,  China,  2015   25  

Classification

•  We use the responses of the selected filters to form pixel-wise feature vectors to describe vessel and non-vessel pixels

•  In order to account for skewness in the data we perform the Inverse

Hyperbolic Sine Transformation: •  We train a SVM classifier to distinguish between vessel and non-vessels

pixels

BICV  Summer  School,  Shenyang,  China,  2015   26  

Results

B-COSFIRE B-COSFIRE

›  We  compute  the  final  results  for  a  given  threshold,  the  one  that  provides  the  maximum  average  MCC  on  a  given  data  set  

BICV  Summer  School,  Shenyang,  China,  2015   27  

Results Comparison

BICV  Summer  School,  Shenyang,  China,  2015   28  

1

Automa'c  differen'a'on  of  u-­‐  and  n-­‐serrated  pa4erns  in  DIF  images

Chenyu  Shi,  Jiapan  Guo,  George  Azzopardi,  Nicolai  Petkov  

BICV  Summer  School,  Shenyang,  China,  2015   29  

Background

•  A type of skin disease: pemphigoid •  A special pemphigoid: Epidermolysis bullosa acquisita (EBA) •  EBA : an autoimmune blistering disease and shares similar clinical

features with other types

Reference:        Buijsrogge,  J.J.A.,  Diercks,  G.F.H.,  Pas,  H.H.,  Jonkman,  M.F.:  The  many  faces  of  epidermolysis  bullosa  acquisita  aFer  serraGon  paHern  analysis  by  direct  immunofluorescence  microscopy.  BriGsh  Journal  of  Dermatology  165(1),  92–98  (JUL2011)    

BICV  Summer  School,  Shenyang,  China,  2015   30  

Serration pattern analysis

EBA à u-serrated pattern à finger-like shapes Others à n-serrated pattern à undulating n-shapes

BICV  Summer  School,  Shenyang,  China,  2015   31  

So far there are no automatic techniques to distinguish between these two types of serration patterns.

u-­‐serrated  pa4erns   n-­‐serrated  pa4erns  

BICV  Summer  School,  Shenyang,  China,  2015   32  

Method

Step 1: Segmentation of the region of interest Step 2: Detect ridges and determine the orientations of each location Step 3: Use normalized histogram of orientations as the feature vector

BICV  Summer  School,  Shenyang,  China,  2015   33  

Segmentation of the region of interest

BICV  Summer  School,  Shenyang,  China,  2015   34  

Examples of DIF images

BICV  Summer  School,  Shenyang,  China,  2015   35  

Detect  ridges  with  CORF  model  

Rota'on-­‐tolerant  result  

BICV  Summer  School,  Shenyang,  China,  2015   36  

Determine  orienta'on  of  the  boundary  

BICV  Summer  School,  Shenyang,  China,  2015   37  

Use normalized histogram of orientations as the feature vector

BICV  Summer  School,  Shenyang,  China,  2015   38  

Experiments

Data set (Medical Hospital in Groningen) •  416  DIF  images  

•  240  n-­‐serrated  •  176  u-­‐serrated  

Results

•  RecogniTon  rate  of  84.6%  on  the  UMCG  public  data  set  of  26  images,  which  is  comparable  to  the  performance  of  medical  experts  82.1%.  

BICV  Summer  School,  Shenyang,  China,  2015   39  

Conclusions

•  Highly effective and robust approach for vessel segmentation in retinal images and ridge detection

•  We proposed a general framework for the delineation of elongated patterns: the features are domain-independent

•  The B-COSFIRE filter is versatile as it can be configured to detect any pattern of interest

•  Very efficient

BICV  Summer  School,  Shenyang,  China,  2015   40  

Outlook

BICV  Summer  School,  Shenyang,  China,  2015  

•  Vessel delineation with push-pull inhibition?

•  Use bifurcation- and crossover-selective filters

•  Delineation of 3D vessels in angiography images of the brain by adding depth information to the model

•  Parallelization of the algorithm, which can run on GPUs

Anyone interested for an internship in Malta?

41  

References

BICV  Summer  School,  Shenyang,  China,  2015  

› [1] A CORF computational model of a simple cell that relies on LGN input

outperforms the Gabor function model. Biological Cybernetics 106, 177-189. › [2] Trainable COSFIRE filters for keypoint detection and pattern recognition. IEEE

Transactions on Pattern Analysis and Machine Intelligence 35, 490-503. › [3] Al-Rawi, M., Qutaishat, M., Arrar, M., 2007. An improved matched filter for

blood vessel detection of digital retinal images. Computer in biology and medicine 37, 262-267.

› [4] Hoover, A., Kouznetsova, V., Goldbaum, M., 2000. Locating blood vessels in

retinal images by piecewise threshold probing of a matched filter response. IEEE Transactions on medical imaging 19, 203-210.

› [5] Mendonca, A.M., Campilho, A., 2006. Segmentation of retinal blood vessels by

combining the detection of centerlines and morphological reconstruction. IEEE Transactions on Medical Imaging 25, 1200-1213.

42  

References

BICV  Summer  School,  Shenyang,  China,  2015  

› [6] Ricci, E., Perfetti, R., 2007. Retinal blood vessel segmentation using line operators and support vector classification. IEEE Transactions on medical imaging 26, 1357-1365.

› [7] Staal, J., Abramo, M., Niemeijer, M., Viergever, M., van Ginneken, B., 2004.

Ridge-based vessel segmentation in color images of the retina. IEEE Transactions on medical imaging 23, 501-509.

› [8] Marin, D., Aquino, A., Emilio Gegundez-Arias, M., Manuel Bravo, J., 2011. A

New Supervised Method for Blood Vessel Segmentation in Retinal Images by Using Gray-Level and Moment Invariants-Based Features. IEEE Transactions on medical imaging 30, 146-158.

› [9] Soares, J.V.B., Leandro, J.J.G., Cesar, Jr., R.M., Jelinek, H.F., Cree, M.J., 2006.

Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification. IEEE Transactions on medical imaging 25, 1214-1222.

43