AUTOMATED LESION DETECTORS · tor, Retinal Fundus Image, Computer-aided Diagnosis, Wavelets, Multiscale Anal-ysis, Variational Segmentation, Cartoon+Texture Decomposition. 1.Introduction

Pre-Publicacoes do Departamento de MatematicaUniversidade de CoimbraPreprint Number 14–40

AUTOMATED LESION DETECTORSIN RETINAL FUNDUS IMAGES

ISABEL NARRA FIGUEIREDO, SUNIL KUMAR, CARLOS MANTA OLIVEIRA, JOAODIOGO RAMOS AND BJORN ENGQUIST

Abstract: Diabetic retinopathy (DR) is a sight-threatening condition occurringin persons with diabetes, which causes progressive damage to the retina. The earlydetection and diagnosis of DR is vital for saving the vision of diabetic persons. Theearly signs of DR which appear on the surface of the retina are the dark lesions suchas microaneurysms (MAs) and hemorrhages (HEMs), and bright lesions (BLs) suchas exudates. In this paper, we propose a novel automated system for the detectionand diagnosis of these retinal lesions through the processing of retinal fundus images.We devise appropriate binary classifier detectors for these three different types oflesions. Some novel contextual/numerical features are derived, for each lesion type,depending on its inherent properties. This is performed by analysing several waveletbands (resulting from the isotropic undecimated wavelet transform decomposition ofthe retinal image green channel) and by using an appropriate combination of Hessianmultiscale analysis, variational segmentation and cartoon+texture decomposition.The proposed methodology has been validated on several medical data sets usingstandard performance measures such as sensitivity and specificity. The individuallyperformance, per frame, of the MAs detector is 93% sensitivity and 89% specificity,of the HEMs detector is 86% sensitivity and 90% specificity, and of the BLs detectoris 90% sensitivity and 97% specificity. Regarding the collective performance of thesebinary detectors, as an automated screening system for DR (meaning that a patientis considered to have DR if it is a positive patient for at least one of the detectors) itachieves in average 95%-100% of sensitivity and 70% of specificity at a per patientbasis.

Keywords: Microaneurysm Detector, Hemorrhage Detector, Bright Lesion Detec-tor, Retinal Fundus Image, Computer-aided Diagnosis, Wavelets, Multiscale Anal-ysis, Variational Segmentation, Cartoon+Texture Decomposition.

1. IntroductionDiabetic retinopathy (DR) is a sight-threatening consequence of Diabetes

Mellitus which affects the small blood vessels in the retina, the nerve layerthat lines the back of your eye. Often there are no visual symptoms in theearly stages of diabetic retinopathy. Early diagnosis and treatment are es-sential to prevent significant vision loss from DR. So, it is vital, for a diabetic

Received November 24, 2014.

1

2 I.N. FIGUEIREDO, S. KUMAR, C.M. OLIVEIRA, J.D. RAMOS AND B. ENGQUIST

patient to have regular eye examinations. One of the easiest way to diag-nose DR is by analyzing fundus images. They are the visual records whichdocument the current ophthalmoscopic appearance of a patient’s retina, theretinal vasculature, and the optic nerve head (optic disc) from which theretinal vessels enter the eye. The acquisition of fundus images is inexpensive,non-invasive and easy to perform. Therefore they are adapted for the largescale screening purposes.

DR can be mainly classified into two classes: Non-proliferative diabeticretinopathy (NPDR) and Proliferative diabetic retinopathy (PDR). NPDRis the earliest stage of DR and can be described as mild, moderate or se-vere. With this condition, the walls of the blood vessels in the retina becomeweak. Tiny bulges protrude from the vessel walls, sometimes leaking or ooz-ing fluid and blood into the retina. Also, sometimes, deposits of cholesterolor other fats from the blood may leak into the retina. NPDR can causeseveral changes in the eye, including microaneurysms (MAs), hemorrhages(HEMs) and bright lesions (BLs) (as for instance hard and soft exudates).PDR is the more advanced form of the disease, where fragile new blood ves-sels form on the surface of the retina over time. These abnormal vessels canbleed or develop scar tissue causing severe loss of vision. PDR may causemore severe vision loss than NPDR because it can affect both central andperipheral vision. Due to this reason it is very significant to diagnose andtreat DR in the non-proliferative stage.

Computer-aided detection and diagnosis of DR with retinal fundus imagessignificantly lessens the burden of the implementation of a large scale screen-ing of the diabetic patients. Recent years have seen the development of themethods for the accurate detection of MAs, HEMs and BLs by consideringthem individually and in a collective way. Without being exhaustive we men-tion here some of these related works. A survey of various methods relatedto DR can be found in [28]. The segmentation of exudates using fuzzy c-means clustering algorithm is done in [22]. After extracting color, size, edgestrength, and texture features from the regions and a multilayer neural net-work classifier is used to classify the images. In [3] a new hybrid classifier asan ensemble of Gaussian mixture model and support vector machine is pro-posed for exudate detection. In [29] a new candidate segmentation method,based on mathematical morphology, is proposed. Extracting some features,a random forest algorithm is used to detect the exudates among the candi-dates. An automatic MAs detector based on template matching in wavelet

AUTOMATED LESION DETECTORS IN RETINAL FUNDUS IMAGES 3

space using a Gaussian template is proposed in [24]. The detection of MAsand HEMs based on recursive region growing segmentation algorithm anda moat operator is done in [25]. After extracting the potential candidatesand formulating a set of features of MAs, HEMs and exudates, [2] used ahybrid classifier which is a weighted combination of multivariate m-Mediodsand a Gaussian mixture model. An ensemble-based framework for the de-tection of MAs is proposed in [4]. The algorithm in [23] assessed the needfor referral in DR detection with a decision based on the fusion of resultsby meta-classification. In [12] the detection of HEMs is done based on thehue, saturation and value (HSV) model and the Mahalanobis distance. Anovel splat feature classification method is proposed in [27] to detect retinalhemorrhages.

The present paper describes an intelligent automated system for the binaryclassification of different types of DR lesions through the processing of eyefundus images.

We derive some novel contextual detectors for the lesions (MAs, HEMs andBLs) depending on their intrinsic properties. The numerical features, whichdefine each detector, rely on the analysis of several wavelet bands (result-ing from the isotropic undecimated wavelet transform [26] decomposition ofthe retinal image), and includes an appropriate combination of Hessian mul-tiscale analysis (more precisely, using the Hessian eigenvalues), variationalsegmentation [6, 9] and cartoon+texture decomposition [7]. These featuresare then used to classify the retinal image into separate categories of DR.We perform a thorough testing of the proposed method on several rich datasets to ensure its good performance in realistic environment.

The rest of the paper is structured as follows. Section 2 presents in detailthe methodology, by explaining the definition of the MAs, HEMs and BLsdetectors. This includes the extraction of the candidates for the differentlesions, description of all the features and the binary classifiers for the le-sions. Section 3 presents the experimental results on several medical datasets prepared by the experts. Then, the paper ends with a discussion of theresults and some conclusions in Section 4.

2.MAs, HEMs and BLs DetectorsIn this section we present the detailed description of the method for the

detection of the different lesions. The procedure relies on four major math-ematical techniques (i- wavelets, ii- multiscale analysis of the image Hessian


matrix, iii- variational image segmentation, iv- cartoon and texture imagedecomposition) which are appropriately and sequentially applied and com-bined. Firstly we outline these 4 techniques in an abstract framework fora scalar image. Then we proceed with the description of the first step ofthe method: the pre-processing step and the optic disk detection in retinalfundus images. Finally we give a detailed explanation of the three lesiondetectors: microaneurysms (MAs), hemorrhages (HEMs) and bright lesions(BLs). Once these feature detectors are extracted, the decision criteria, onwhether a frame has or does not have one of these type of lesions, are basedon simple thresholding approaches.

2.1. The Isotropic Undecimated Wavelet Transform. The IsotropicUndecimated Wavelet Transform (IUWT) [26] is characterized by a simpleimplementation of its direct and inverse transform. Denoting by C0 = I ascalar input image, at each subsequent iteration j, the scaling coefficients Cjare computed by lowpass filtering (where filtering is applied by convolutionand separably along each dimension), and the wavelet coefficients Wj (here-after called wavelet levels) are defined just by scaling coefficient subtraction,i.e, Wj+1 := Cj+1−Cj. Starting with the filter h0 = [1/16, 1/4, 3/8, 1/4, 1/16],which is derived from the cubic B-spline, at each scale j ≥ 2 the filter hjis augmented by inserting 2j − 1 zeros between each pair of adjacent co-efficients of h0. The scaling coefficient Cj+1 is defined from the previouscoefficient Cj using separable convolution with hj in each dimension. Thereproduction of the original image I is achieved just by adding all the waveletcoefficients and the final computed scaling coefficient in the following way:I := Cm +

∑mj=1Wj. In addition we denote by In a partial reconstruction of

I defined by

In =n∑j=2

Wj, n = 2, 3, 4, 5, (1)

with Wj the wavelet level at iteration j.

2.2. Multiscale Analysis of the Image Hessian Matrix. For lookingfor blob structures in a scalar image, we use herein a strategy based on theeigenvalues of the image Hessian matrix and multiscale image analysis. Fora scalar image I : Ω ⊆ R2 → R, where Ω is the pixel domain, we first define


the Hessian matrix of one point (x, y) ∈ Ω, and at a scale s, by

Hs(x, y) :=

(Isxx(x, y) Isxy(x, y)

Isxy(x, y) Isyy(x, y)

).

Here Isxx, Isxy and Isyy are the second-order partial derivatives of I and the

scale s is involved in the calculation of these derivatives (for instance usingGaussian convolution). Suppose λs,1 and λs,2 are the two eigenvalues of theHessian matrix Hs. Note that in a blob region, these two eigenvalues havethe same sign and similar magnitudes (see [17]). Without loss of generalitywe assume that |λs,1| ≤ |λs,2|. Then, setting Fs := λ2

s,1 + λ2s,2

f1 := exp(−βF 2

s

)and f2 := 1− exp

(−α(λs,1λs,2

)2),

and motivated by [17, 14], we define the blob (Bs) detector (at each pointof the domain) by

Bs :=

0, if λs,1λs,2 < 0 or |λs,2 − λs,1| > δ(1− f1)f2, otherwise.

Here α and β are parameters which control the sensitivity of the functionBs and δ a user chosen threshold. The values of α, β and δ are consideredto be 1, 1 and 10−4, respectively. In order to automatically detect blobsof different sizes, a multiscale approach is necessary. The response of thedetector function Bs will be maximum at a scale that approximately matchesthe size of the blob to be detected. Hence, at each point of the domain, wedefine the function

B := maxsmin≤s≤smax

(Bs.I) , (2)

where smin and smax are the minimum and maximum scales at which the blobsare expected to be found. Finally, in the subsequent Sections we considerthe maximum of the function B as an input to our binary (blob) classifier

Bmax := maxi,j

Bi,j, 1 ≤ i ≤ Nx, 1 ≤ j ≤ Ny, (3)

where Nx and Ny are the width and height of the image in pixels.

2.3. Variational Image Segmentation. The segmentation scheme, wehave applied, relies on a reformulation of the Chan and Vese variationalmodel [6, 9], and is described as follows: We first compute the constantsc1 and c2 (representing the averages of a scalar image I : Ω ⊆ R2 → R,


in a two-region image partition). We then solve the following minimizationproblem

minu,v

TVg(u) +

1

2θ‖u− v‖2

L2(Ω) +

∫Ω

(λ r(I, c1, c2) v + α ν(v)

)dx dy

(4)

where TVg(u) :=∫

Ω g(x, y)|∇u| dx dy is the total variation norm of the func-

tion u, weighted by a positive function g; r(I, c1, c2)(x, y) :=(c1−I(x, y)

)2−(c2− I(x, y)

)2is the fitting term, θ > 0 is a fixed small parameter, λ > 0 is a

constant parameter weighting the fitting term, and α ν(v) is a term resultingfrom a reformulation of the model as a convex unconstrained minimizationproblem (see [6, Theorem 3]). Here, u represents the two-phase segmentationand v is an auxiliary unknown. The segmentation curve, which divides theimage into two disjoint parts, is a level set of u, (x, y) ∈ Ω : u(x, y) = µ,where in general µ = 0.5 (but µ can be any number between 0 and 1, with-out changing the segmentation result, because u is very close to a binaryfunction).

The above minimization problem is solved by minimizing u and v sepa-rately, and iterated until convergence. In short we consider the following twosteps:1. v being fixed, we look for u that solves

minu

TVg(u) +

1

2θ‖u− v‖2

L2(Ω)

. (5)

2. u being fixed, we look for v that solves

minv

1

2θ‖u− v‖2

L2(Ω) +

∫Ω

(λ r(I, c1, c2) v + α ν(v)

)dx dy

. (6)

We refer to [6, Propositions 3, 4] for the solution of (5)-(6). For our exper-iments g is considered to be the edge indicator function g(∇u) := 1

1+β‖∇u‖2

with β = 10−3 (‖.‖ denotes the Euclidean norm). The values of θ and λ areconsidered to be 1 and 0.01, respectively.

2.4. Cartoon and Texture Image Decomposition. Given a scalar imageI : Ω ⊆ R2 → R, we decompose it into the cartoon (geometric) part C andthe texture (oscillatory) part T using an algorithm of Buades et al. [7]. Thisalgorithm is briefly described as follows. We define the local total variation


(LTV) for each pixel x in Ω, by the convolution (denoted by ’∗’)

LTVσ(I)(x) := Gσ ∗ |∇I|(x)

where Gσ is a Gaussian kernel of variance σ. The main characteristic of atextural region is its high total variation (due to its oscillations) and theproperty of decreasing very fast under low-pass filtering (as opposed to car-toon, or equivalently, non-oscillatory regions). We then compute the relativereduction rate of LTV by

λ(x) :=LTVσ(I)(x)− LTVσ(Rσ ∗ I)(x)

LTVσ(I)(x),

where Rσ is a low pass filter. λ is a local indicator of the oscillations in I:if λ(x) ≈ 0 there is little reduction of LTV with the low-pass filter, but ifλ(x) ≈ 1, this means the reduction is strong, and consequently the pixel xbelongs to a textured region.

Depending on the relative reduction of LTV, taking the weighted averageof I and Rσ ∗ I, we get the cartoon part C, and hence the texture part T ,

C(x) := w(λ(x)

)(Rσ ∗ I

)(x) +

(1− w

(λ(x)

))I(x),

T (x) := I(x)− C(x).(7)

The weight w : [0, 1] → [0, 1] is a nondecreasing piecewise affine functionthat is constant and equal to zero, near zero, and constant and equal to 1,near 1. If λ(x) is very small, then I is non oscillatory around x and thereforew ≈ 0, so C ≈ I, that is the image is cartoon. If instead λ(x) is big, thenthe image I is locally oscillatory in the neighborhood of x.

2.5. Pre-processing and Optic Disk Detection. Retinal fundus imagesnormally contain a main region at the center of the image, which is called thefield of view (FOV), surrounded by dark background pixels. For automateddetection of lesions we only need to process the main region, so we separate itfrom the background, in a preprocessing step. The separation is performed byeroding the green channel of the image, using a square structuring element.

It is also needed to extract the optic disc (OD) from each image to eliminateany spurious and false regions caused by their similarities with bright lesions.The segmentation of the OD is done in two steps. We first localize the ODusing its geometric features and then use the circular Hough transform (CHT)in the localized region to detect the OD boundary. The herein proposed


approach for the OD detection was presented by the authors in a conferencepaper in [11], and it is summarized here for convenience of the reader.

It is well known that the OD appears most contrasted in the green channelcompared to the red and blue channels of the RGB image. More importantly,it appears as a concave region (protrusion) in the green channel. Motivatedfrom [13, 15], we define

ODloc := −Kmin(H, 0),

where K and H are, respectively, the Gaussian and mean curvatures of thegreen channel image. This geometrical function was introduced in [13, 15] asa protrusion measure for detecting colonic polyps (which are round-shapedprotusion objects) in wireless capsule endoscopy images. Mathematically, thevalue of the ODloc function is closely related to the size of the protrusions inthe images. Therefore, the optic disc location in the frame can be inferred byidentifying the locations where ODloc is higher. Hence, applying a thresholdwe can separate the candidate region(s) for the location of the optic disc.We extract the highest 15% of pixels of ODloc. The entire thresholded imageis scanned to count the number of connected components. The connectedcomponents are found using an algorithm by Haralick and Shapiro [18]. Foreach connected component, in the thresholded image, we compute the areaand the ODloc mean pixel intensity. The component corresponding to themaximum of the product between the area and theODloc mean pixel intensityis considered to be the location of the OD. We then compute the centroid ofthe same component, which is identified with the estimated OD center andconsequently the OD location.

Afterwards, the detection of the OD boundary is done based on the CHT[20, 19], which relies on equations for circles. The parametric representationof a circle is x := a+ r ∗ cos(θ), y := b+ r ∗ sin(θ), where a and b representthe coordinates for the center, θ ∈ [0, 2π] and r is the circle radius. Weconsider r between rmin and rmax which are, respectively, one-tenth and one-fifth of the image (as these measurements are estimated to be the average ODdiameter) divided by two. The minimum value makes sure that the OD cup(an internal circular object in the OD) is not considered, while the maximumvalue is such that too wide areas are eliminated. The CHT transforms a setof feature points in the image space into a set of accumulated votes in aparameter space. Then, for each feature point, votes are accumulated in anaccumulator array for all parameter combinations. The array elements that


Figure 1. Two pairs of ODloc function in the green channel andthe corresponding optic disk boundary determined by CHT.

contains the highest number of votes indicate the presence of the desiredshape, i.e. the OD in our case. Based on our previous optic disc localization(the above mentioned computed centroid), we define a region of interest(ROI) of size m×n, where m and n are one-ninth of the respective dimensionsof the image multiplied by two. Finally to segment the OD boundary fromthe retinal image we apply the CHT to the edges of the ROI, which arecomputed using Canny edge detector [8].

The OD detection for a disease and a non-disease image (exhibiting BLs)is shown in Figure 1.

2.6. Microaneurysm Detection. A microaneurysm is a round dark-redstructure (a very small spot whose size is approximately less than 125 mi-crons in its longest dimension) caused by thickening of capillary basementmembranes within the retina. We have observed that MAs are visible andenhanced at wavelet level W2, or at the sum of wavelets levels W2 and W3,of the green channel of the input fundus image. These facts have motivatedus to devise a MAs detector, based on circular blob detection in these twowavelet images, as described in the sequel of the current Section.

For each image we take I := IG (in Section 2.1), with IG the green channelof the input fundus image, and we compute (1), i.e. In =

∑nj=2Wj, n = 2, 3.

Then, we extract the 20% darkest pixels within the FOV of In to identifythe MA candidates. We then take the product of this thresholded imagewith In, and remove the pixels corresponding to the vessels and OD. Theresulting image is denoted by Tn. The segmentation of the vessels is done byusing an approach similar to [5] [briefly it includes the following proceduredescribed in points i) to iii) : i) We compute the sum of the wavelet levelsW2 and W3 of the green channel of the image and extract the darkest 10%of the pixels within the FOV of the same. ii) The vasculature can be seen


(a) (b)

−10

−5

0

5

10

−25

−20

−15

−10

−5

0

5

10

15

20

(c)

−20

−15

−10

−5

0

5

10

15

20

−40

−30

−20

−10

0

10

20

30

40

(d)

0

1

2

3

4

5

0

1

2

3

4

5

6

7

8

(e)

0

1

2

3

4

5

6

7

8

9

2

4

6

8

10

12

14

(f)

Figure 2. Visualization of the main steps of MAs Detector. (a)and (b) Retinal fundus images having MAs. (c) and (d) Pair ofwavelet level W2 and sum of wavelet levels W2 and W3, for (a)and (b), respectively. (e) and (f) Pair of blob functions MA2 andMA3, for (a) and (b), respectively.

as a large connected structure in the binary image, along with some isolatedsmall objects, which are removed from the binary image. In a similar way we


fill small holes present within the thresholded regions. iii) We remove objectsof size less than 150 pixels and fill holes of size greater than 20 pixels]. Afterremoving the vessels from Tn we also apply to Tn the adaptive Wiener filter,using neighborhoods of size [15, 15], for smoothing and enhancing. We haveobserved that there are still some elongated structures left in Tn. To removethem we apply a technique similar to the vessel segmentation. We take thesum of the wavelet levels W1, W2 and W3 of Tn and extract the darkest 10%of the pixels within the FOV of the same. From the same binary image weremove objects of size less than 75 pixels and fill holes of size greater than20 pixels. The pixels corresponding to the resulting binary image are thenremoved from Tn and the corresponding image is denoted by Fn. Finally, weperform the inversion of the image in the following way

Mn :=(

maxi,j

Fn;i,j

)− Fn;i,j, 1 ≤ i ≤ Nx, 1 ≤ j ≤ Ny, (8)

where Nx and Ny are the width and height of the image Fn in pixels. We thenlook for blob structures in each image Mn (n = 2, 3), using the eigenvaluesof the Hessian matrix of Mn and multiscale image analysis (as described inSection 2.2, with I := Mn). Hence, at each point of the domain we definethe functions, see (2),

MAn := maxsmin≤s≤smax

(Bs.Mn) ,

where smin and smax are the minimum and maximum scales at which theMAs are expected to be found. For our experiments we consider scale s tovary between 2 and 8.

Finally, we consider the maximum of each function MAn as an input to ourMA binary classifier (or equivalently the Mas detector), see (3)

MAmaxn := max

i,jMAn;i,j, 1 ≤ i ≤ Nx, 1 ≤ j ≤ Ny.

Consequently, if for a frame

(MAmax2 ≥ MAL

2 ) ∧ (MAmax3 ≥ MAL

3 ) (9)

then it is classified as a frame containing a MA, otherwise the frame is consid-ered to be a normal frame. Here the pair of lower bounds (MAL

2 ,MAL3 ) filters

out frames that are unlikely to have MAs (in the experiments we considerthe values (3.5, 5.5), see Table 2 in Section 3).


The previous functions, I2, I3, MA2, MA3, involved in the definition ofthe MA detector are illustrated in Figure 2 for two retinal fundus imagesexhibiting MAs.

2.7. Hemorrhage Detection. Due to the difficulty of identifying HEMs,resulting of their diversity and variability in appearance (shape, size, texture)we have devised 3 criteria, where the third is based directly on 3 differentHEMs sizes. Therefore there is a total of five criteria for HEMs. Then aframe is considered as containing an HEM if it satisfies at least one of thesefive criteria. The motivation for the definition of each criterion (that reliesessentially on the hemorrhage visible information) is justified for each case.The input for each HEM criterion is always the green channel of the retinalfundus image, because it exhibits the best contrast for visual HEM detectionin wavelets bands.

2.7.1.HEM Criterion 1. This first criterion is quite similar to the detectionof MAs. The main difference is that because hemorrhages have a bigger sizethan MAs, and as we know that larger features are visible with improvedcontrast on higher wavelet levels, we use I4 =

∑4j=2Wj (see (1), where now

the input scalar image is the green channel of the fundus image), instead ofI2 or I3 as for MAs. To identify the hemorrhage regions we extract the 20%darkest pixels, which we call Rh. Hemorrhages are know be medium size darkred spots, so we create a binary mask by extracting objects of size between40 and 500 pixels from Rh and call this mask Mh. Finally, we consider theproduct of I4, Rh, Mh, and remove the pixels corresponding to the OD. Tothe resulting image we apply the adaptive Wiener filter using neighborhoodsof size [15, 15] and do the inversion similar to (8) to get the image Hh. Asa result of these thresholding and smoothing operations, the HEM lesionsappear as isolated bright patches or blobs outstanding to their surroundingsin Hh. We then look for blob structures in Hh, using the technique describedpreviously in Section 2.2 (with I := Hh), and where now smin and smax arethe minimum and maximum scales at which the HEMs are expected to befound. From that we get the measure HEMmax for our binary classifier (see(3)). If the frame passes the criterion

HEMmax ≥ HEML1 (10)

then the frame is labelled as a hemorrhage frame. The threshold HEML1 ischosen very high (12 in the experiments, see Table 2 in Section 3) so that


(a)

−20

−15

−10

−5

0

5

10

15

20

(b)

0

2

4

6

8

10

12

(c)

1

2

3

4

5

6

(d) (e)

0

2

4

6

8

10

12

(f)

Figure 3. Illustration of HEM Criterion 2 (and also HEM Cri-terion 1 in subfigure (d)). (a) Original image with HEMs. (b)Sum of wavelet levels W2, W3 and W4. (c) Image Hh. (d) Com-puted blob function in image Hh. (e) Binary segmentation imageS (via thresholding). (f) Image Hh in the regions of S.

we can detect HEM with marginal false positives. In Figure 3, the subfigure(c) depicts the blob function, and there we can see that HEMmax attains ahigher value than 12.

2.7.2. HEM Criterion 2. One of the most frequent features of some hemor-rhages is that it appears as a red spot, with uneven density and surroundinga smaller punctuate lesion considered to be a MA. Thus this second criterionis a procedure that intends to quantify this property.

In this case we compute essentially other measures from Hh and also useHEMmax (both defined in HEM Criterion 1). We use a binary segmentation


via thresholding, such that for a pixel x

S(x) := H(Hh(x)− θ)where H is the Heaviside function taken pixel-wise

H(x) :=

0, if x < 0,1, if x ≥ 0,

and the scalar threshold θ is defined by

θ :=1

2maxi,j

Hh;i,j 1 ≤ i ≤ Nx, 1 ≤ j ≤ Ny.

We find the connected components of S using an algorithm by Haralick andShapiro [18]. For each connected component we compute the mean pixelintensity in Hh. We consider the maximum of the mean intensities for ourbinary classification and denote it by Hmean. This maximum value is consid-ered since we expect the mean of the pixels to be high corresponding to theselected HEM region. We also compute the eccentricity of the componentcorresponding to the maximum mean and denote it by Hecc. This informationis useful since we expect the selected HEM region to be round in shape afterthresholding and smoothing operations. Thus criterion 2 is

(HEMmax ≥ HEML2) ∧ (Hmean ≥ HLmean) ∧ (Hecc ≤ HU

ecc). (11)

The frame is checked for this criterion. If it passes, then it is classified asa HEM frame, otherwise we continue to the next criterion of our HEMsclassification. The lower bounds HEML2, HL

mean and the upper bound HUecc

discard frames without HEMs (the chosen values in the experiments are 6, 9and 0.7, respectively, see Table 2 in Section 3).

The main steps of this HEM Criterion 2 are illustrated in Figure 3.

2.7.3. HEM Criterion 3. This final criterion includes 3 subcriteria denotedby 3-a, 3-b and 3-c. It is based on different sizes of HEMs. Essentially welook for three sizes of hemorrhages regardless of their shapes : between 100and 500 pixels, or between 500 and 1100 pixels, or big objects but less than1500 pixels.

To this end, and for each size class, we essentially perform, firstly, a suit-able segmentation of the HEM regions in the frame, then select (with anappropriate procedure that involves the cartoon part of the image) the bestcandidate hemorrhage region and finally quantify same features in this se-lected candidate for the binary HEM classification. We explain with all the


details criterion 3-a, and shorten the descriptions of criteria 3-b and 3-c,which are similar.HEM Criterion 3-a. To achieve a better segmentation, we first enhance andsmooth the hemorrhages, and consequently they will have a more roundishshape. To this end we compute the Laplacian of Hh (defined in HEM Cri-terion 1, but using the adaptive Wiener filter with neighborhoods of size[25, 25]). We set

Ls := −s2∆Gs ∗Hh, (12)

where Gs is a Gaussian kernel of variance s, ∆ is the Laplacian operator and∗ represents the convolution. Taking the maximum with respect to the scaleswe get

L := maxsmin≤s≤smax

Ls. (13)

We then normalize the pixel intensity values of L and Hh in the range 0to 255 and to emphasize the hemorrhage regions we consider Dh := L +Hh. Secondly, we apply the blob detection algorithm described previouslyin Section 2.2 to Dh and denote the resulting image by Ph (that is in (2)I := Dh, B is denoted by Ph and the scale vector s = [10 12 14 16]). We nowperform the two-phase variational segmentation, described in Section 2.3, onthe scalar image Ph, aiming at separating the hemorrhage regions, from theother parts: the segmentation mask is denoted by u, and contains the HEMcandidates.

Decomposing I4 = Ch + Th, into the cartoon (geometric) part Ch and thetexture (oscillatory) part Th, using the algorithm of Section 2.4, we set thenSh = Ch.Rh.u, for discarding some superfluous segments from the segmen-tation mask u. In fact, because HEM regions appear as medium/large sizegeometric objects in I4 they go to the cartoon part Ch (rather than to Th),which is relevant for the procedure described in the next sentence. After-wards, for each connected component in Sh we compute the area and themean intensity of the pixels. We consider at most five components corre-sponding to the higher values of the product between the area and the meanintensity for the location of the HEMs. The resulting group of candidates isdenoted by C.

Then, within the FOV of the sum of the wavelet levels W2 and W3, we com-pute the darkest 10% of the pixels. From the resulting image a binary imageis created and the objects of size less than 150 pixels are removed and theholes of size greater than 20 pixels are filled. We denote the resulting binary


image by Qh. We shall compute some measures for hemorrhage classificationfrom Qh.

In this first criterion we quantify some properties related to HEMs of sizebetween 100 and 500 pixels from a binary image Mh (created in a similarway as the image also called Mh in HEM Criterion 1, for objects between40 and 500 pixels). Then to the product I4.Rh.Mh we proceed exactly as inHEM Criterion 1 to generate an image Hh for which we compute HEMmax

a

(following the technique of Section 2.2, with I := Hh, and the subscript lettera symbolizes criterion 3-a).

If the location of the previous value HEMmaxa matches with the location

of a candidate region in C, then based on the location of the centroid of thelatter we define a region of interest (ROI) of size 60 × 60 pixels in Qh. Ifthere is no match found then we continue to the next step, HEM criterion3-b. In the ROI we look for the object which is at the minimum Euclidiandistance from the centroid of the matched component. For the same objectwe compute the solidity (a scalar value specifying the proportion of the pixelsin the convex hull that are also in the object) and denote it by S. If for theframe the following condition is satisfied

(HEMmaxa ≥ HEML3) ∧ (S ≥ SL), (14)

(where HEML3 and SL are pre-defined values) then the frame is a HEMframe, else we continue to look for the objects of other sizes.HEM Criterion 3-b. This criterion is similar to the previous HEM criterion3-a (including formulas (12), (13) and the group of candidates C, obtainedvia I4). The only difference now is that we extract the 20% darkest pixelsfrom the sum of wavelet levels up to level 5, i.e. I5 =

∑5j=2Wj, and then from

there we select the objects of size between 500 and 1100 pixels, to create thebinary image Mh. Again we get an image Hh (as in the previous criterion)and applying the blob detection technique of Section 2.2 to Hh we obtainthe scalar value HEMmax

b (the subscript letter b symbolizes criterion 3-b). Ifthe location of HEMmax

b matches with the location of a candidate region inC (as defined exactly in HEM Criterion 3-a) then based on the location ofthe centroid of the latter we apply the previously described procedure forchecking the matching in Qh (as defined in HEM Criterion 3-a) and obtainthe solidity measure S. If there is no match found then we continue to thenext step, HEM Criterion 3-c. If for the frame

(HEMmaxb ≥ HEML4) ∧ (S ≥ SL), (15)


(a)

−20

−10

0

10

20

30

40

50

60

70

80

(b) (c)

(d) (e) (f)

Figure 4. Visualization of the main steps of HEM Criterion 3-a.(a) Original image with HEMs. (b) sum of wavelet levels W2, W3

and W4. (c) Image Ph for segmentation. (d) Final segmentationsuperimposed on the original image. (e) Blob function related toobjects of size between 100 and 500 pixels. (f) Image Qh in theROI for computing the solidity.

(where HEML4 and SL are pre-defined values) then the frame is a HEMframe, else we move to our final HEM step.HEM Criterion 3-c. This criterion is again analogous to the previous HEMcriteria (including formulas (12), (13) and the group of candidates C, obtainedvia I4). The main alteration is that the binary mask Mh is now created fromthe extraction of the 20% darkest pixels of I5 and by removing objects of sizeless than 1500 pixels (in this process we also fill the holes of size greater than20 pixels). Again we consider the product of I5, Mh and the 20% darkestpixels of I5, remove from it the OD, and the resulting image is smoothed


(a) (b) (c)

(d) (e) (f)

Figure 5. Illustration of the main steps for HEM Criterion 3-b.(a) Original image with HEMs. (b) Sum of wavelet levels W2, W3

and W4. (c) Image Ph for segmentation. (d) Final segmentationC superimposed on the original image. (e) Blob function relatedto objects of size between 500 and 1100 pixels. (f) Image Qh inthe ROI for computing the solidity.

using the adaptive Wiener filter of neighborhoods of size [15, 15]. After thatthe inversion is performed similar to (8) to obtain Hh. We again apply theblob detection technique of Section 2.2 to Hh, to get a scalar value HEMmax

c

(the subscript letter c symbolizes criterion 3-c), whose location is used toobtain the solidity measure S (in a similar way to the previous two criteria).If no match is found in the procedure then the frame is considered to be anormal frame. Our final criterion is

(HEMmaxc ≥ HEML5) ∧ (S ≥ SL). (16)


(a)

−25

−20

−15

−10

−5

0

5

10

15

20

(b) (c)

(d) (e) (f)

Figure 6. Visual representation of the relevant parts of HEMCriterion 3-c. (a) Original image with HEMs. (b) Sum of waveletlevels W2, W3 and W4. (c) Image Ph for segmentation. (d) Finalsegmentation C superimposed on the original image. (e) Blobfunction related to objects of size up to 1500 pixels. (f) ImageQh in the ROI for computing the solidity.

If the frame satisfies this criterion then it is considered to be a HEM frame,otherwise a normal frame.

Again in these criteria 3-a, 3-b, 3-c, the pairs of lower bounds (HEML3,SL),(HEML4,SL), (HEML5,SL) filter out frames that are unlikely to have HEMs.In the experiments we consider the values (6, 0.7), (7, 0.7) and (8, 0.7), re-spectively (see Table 2 in Section 3).

The segmentation of the HEM candidates as well the main steps of HEMCriteria 3-a, 3-b and 3-c, for retinal images with HEMs, are displayed inFigures 4, 5 and 6, respectively.


2.8. Bright Lesion Detection. For the BLs detection a preprocessing stepis essential to algorithm success, aimed at reducing the intra- and inter-image variability. We employ the technique proposed in [10, 16] to normalizeluminosity and contrast in fundus images based on a statistical model ofthe image. In this section we always work with the corresponding enhancedimages.

The herein proposed technique for the BLs detection is an improved andextended version of the approach presented by the authors in a conferencepaper in [12]. To detect BLs our strategy is based on identifying separatelysmall and medium/large BL lesion regions. We propose three criteria. Thefirst is similar to the MAs detection, and aims at identifying small dottedBLs. After that, for the other two criteria we utilize the high wavelet levelsand also the texture and cartoon decomposition of the image. In fact, doingthe texture+cartoon decomposition of the wavelet levels of the green channelof the enhanced image, we have observed that clusters of small dotted BLs,that are spread across the image, are more prominent in the texture part.On the other hand, large/medium BL regions, that are more homogeneous,go to the cartoon part and, in general, create void regions in the texturepart. These facts are also in good agreement with the comments reported inSection 2.4

2.8.1. BL Criterion 1. We start by computing W2, the wavelet level atiteration 2 of the green channel of the enhanced image. Extracting the 20%highest pixels in W2 we consider the product of the thresholded image withW2 and remove the pixels corresponding to the OD. The resulting imagedenoted by Fb contains the small dotted BLs. To smooth and enhance theselesions, we apply the adaptive Wiener filter to Fb using neighborhoods of size[15, 15]. Consequently, the small dotted BLs appear as isolated bright patchesor blobs outstanding to their surroundings. We then apply the previouslydescribed blob detection technique (of Section 2.2) to the scalar image Fband obtain the measure BLmax (see (2)) for the binary classification. If

BLmax ≥ BLL1 (17)

then the frame is classified as containing a BL (the lower bound BLL1 discardsframes without BL and its value is 11, see Table 2 in Section 3).

The approach for BLs detection in [12] utilized only a criterion similar to(17). Our next two BL criteria are different from [12]. Now we look for smalldotted BLs that are more spread in the images and large/medium BLs that


are more homogeneous. To describe them we consider Ib =∑4

j=2Wj, whereWj is the wavelet level at iteration j of the green channel of the enhancedimage, see (1). Let I tb denote the image corresponding to the 20% highestpixels in Ib. We decompose Ib into Ib = Cb + Tb, where Cb and Tb representthe cartoon and texture decomposition of Ib according to the algorithm ofSection 2.4.

2.8.2.BL Criterion 2. To extract the candidate regions corresponding to theclusters of dotted BLs spread regions, we perform the nonlinear convolution-type transform of I tb.Tb in the following way

χt = Ls(|I tb.Tb|τ),

where Ls is a linear operator convolving the image with a Gaussian kernelwith variance s, and the positive scalar τ < 1 de-emphasizes very small re-gions with strong texture (see [21], where we have used a similar argument).Here, the convolution is performed separably along each dimension. Aftercomputing χt we apply a two-phase segmentation on χt as described in Sec-tion 2.3. As a result of this we obtain the segmentation mask containing thecandidate regions. After removing the pixels corresponding to the OD, welook for the region in the segmented mask that matches the location of thevalue BLmax in (17). The selected candidate is denoted by Mt. To sharpenthe lesions we apply the unsharp masking. The sharpening process works byutilizing a slightly blurred version of the image. We compute

Ub = I tb.Tb + (I tb.Tb − χt).

From Ub we get the feature Nt = 1|Mt|||∇Ub||L2(Mt), where |Mt| is the sum of

the pixels in Mt (and L2(Mt) is the space of square-integrable functions inMt). A high value for Nt indicates that Mt is a high textured (oscillatory)region.

Our next feature, in this criterion, is based on the observation that clus-ters of small dotted BLs, that are more spread, create holes in the vesselssegmentation. We obtain the segmentation of the vessels from the sum ofthe wavelet levels W2 and W3 of the green channel image using the methoddescribed in Section 2.6. Then in the region resulting from the intersectionof Mt with the detected vessels we compute the feature E , which is the dif-ference between the number of components to the number of holes in theregion.


(a)

−10

−5

0

5

10

15

20

25

30

35

40

(b)

0

1

2

3

4

5

6

(c)

−15

−10

−5

0

5

10

15

20

(d)

0

0.1

0.2

0.3

0.4

0.5

0.6

(e) (f)

(g)

−20

−10

0

10

20

30

40

(h) (i)

Figure 7. Depiction of the several main procedures for BL Cri-terion 2 (and also for BL Criterion 1 in subfigure (c)). (a) Origi-nal image with a cluster of dotted BLs. (b) Sum of wavelet levelsW2, W3 and W4. (c) Computed blob function in wavelet levelW2. (d) Texture Tb (e) Convolution type transform χt. (f) Seg-mentation of χt superimposed on the original image. (g) Selectedsegmented candidate maskMt superimposed on the original im-age. (h) Sharped image Ub in the final region (for computingfeature Nt). (i) Intersection of the maskMt with the vessels (forcomputing feature E).


Finally the criterion for the detection of clusters of small dotted BLs is

(BLmax ≥ BLL2) ∧ (Nt ≥ N Lt ) ∧ (E ≤ EU). (18)

If the frame satisfies this criterion, then it is a BL frame, otherwise we con-tinue to our final criterion of the BLs classification. In the experiments (seeTable 2 in Section 3) the lower bounds BLL2 and N L

t are 6 and 1.3 and theupper bound EU is 0.

Figure 7 illustrates the main computations, described in this BL Criterion2, for one frame, with very bad quality, but clearly exhibiting a cluster ofdotted BLs.

2.8.3. BL Criterion 3. In our final criterion we look for the large/mediumBLs that are more homogeneous. Such regions are missed by the previousBL criteria. Firstly, we consider Ib.I

tb and remove the pixels corresponding

to the OD. We also apply the adaptive Wiener filter using neighborhoodsof size [15, 15]. Similarly to the procedure described in BL Criterion 1, forthe wavelet level W2, we apply the blob detection technique, starting fromIb.I

tb (and not W2), and obtain the feature BLmax2:4 (as in Section 2.2, (3),

and the subscript 2:4 means that in Ib the sum of wavelet levels from 2 upto 4 is used). Secondly, we extract the candidate regions correspondingto the large/medium BLs. For that (and similarly to the clusters of smalldotted BL), we compute the nonlinear convolution-type transform of I tb.CbAfterwards we perform a two-phase segmentation on χc (as described inSection 2.3) to get the segmentation mask containing the candidates. Wethen remove the pixels corresponding to the OD and search for the segmentedregion that matches the location of the BLmax2:4 , determined above, to get theselected candidate region Mc. Then we compute the maximum value in theimage region Mc ∩

(Cb.I

tb

)and denote it by Cmax. We also compute the

feature Nc = 1|Mc|||∇(I tb.Tb)||L2(Mc), where |Mc| is the sum of the pixels in

Mc. As opposed to Nt in BL Criterion 2, a low value of Nc indicates thatMc is a cartoon region (and not a textured region). Our final criterion is

(BL ≤ BLmax2:4 ≤ BU) ∧ (N Lc ≤ Nc ≤ N U

c ) ∧ (CL ≤ Cmax ≤ CU). (19)

If the frame satisfies this criterion, then it considered to be a BL frame,otherwise it is considered to be a normal frame. The values for the lowerand upper bounds are indicated in Table 2 in Section 3. The upper boundsBU , CU are imposed for deleting fames with artifacts and [N L

c ,N Uc ] is an

estimated low valued interval for the feature Nc.


Figure 8 displays the computations of this BL Criterion 3, for one framewhich has medium/large BL lesions.

3. ExperimentsWe now assess the performance of the proposed methodology for automated

detection and diagnosis of DR. We have done two different evaluations: firstby applying each detector individually and then by considering the detectorsin a collective way (that is, a patient is considered a DR patient if he is apositive patient for at least one detector).

3.1. Data sets. For evaluating the performance of the MAs, HEMs andBLs detectors, individually, three data sets, named DMA, DHEM and DBL

were prepared by the experts: DMA contains 484 frames out of which 316are normal and 168 frames contain MAs, DHEM contains 492 frames out ofwhich 309 are normal and 183 frames contain HEMs and DBL contains 607frames out of which 396 are normal and 211 frames contain the BLs.

Secondly for using the proposed detectors as a system for automated di-agnosis of diabetic retinopathy (DR), and test the system performance, weconsider four data sets prepared by the experts, named DR1, DR2, DR3 andDR4 (see Table 1).

All these images are provided by the company Retmarker(http://www.retmarker.com/), and were obtained from patients screened ac-cording to the Diabetic Retinopathy Screening Program of Portugal. Thescreening includes the acquisition of 2 images per eye and per patient (ex-cept in DR3 and DR4, where a few of the normal patients have less than 4images) without pupil dilation. The images corresponding to the data setsDR1 and DR2 are taken by the Topcon TRC NW-100 non-mydrictic retinalcamera, and for the data sets DR3 and DR4 by a nonmydriatic 45-degreefixed Canon CR6-45NM fundus camera attached to a Sony Power HD 3CCDdigital color camera. The images were stored in jpeg files (no compression)with resolutions of 1024 × 682, 768 × 584, and 768 × 768 pixels. The trueclassification of the disease patients, with DR, and normal patients is shownin Table 1. The classification is done on a per patient basis. This meansthat if at least one image of a patient contains any type of lesion related tothe definition of DR, it is considered as a disease (DR) patient. This groundtruth classification is also provided by the company Retmarker and is thehuman grading at the Diabetic Retinopathy Screening Program.


(a)

−40

−20

0

20

40

60

(b)

0

5

10

15

20

25

30

(c)

−40

−30

−20

−10

0

10

20

30

40

50

60

(d)

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

(e) (f)

(g)

0

10

20

30

40

50

60

(h)

−2

0

2

4

6

8

(i)

Figure 8. Pictures of the principal stages of BL Criterion 3. (a)Original image with medium/large BLs. (b) Sum Ib of waveletlevels W2,W3 and W4. (c) Computed blob function in the sumIb. (d) Cartoon Cb. (e) Convolution type transform χc. (f)Segmentation of χc superimposed on the original image. (g) Finalsegmentation regionMc superimposed on the original image. (h)Cartoon Cb in restricted to the selected segmented candidateMc.(i) Texture Tb in the final region (for computing Nc).


Table 1. Expert Classification of DR Data sets

Data sets DR Patients (Images) Normal Patients (Images) Total Patients (Images)

DR1 11 (44) 345 (1351) 356 (1395)

DR2 18 (72) 341 (1364) 359 (1436)

DR3 466 (1854) 4961 (19651) 5427 (21505)

DR4 225 (900) 5144 (20534) 5369 (21434)

3.2. Results. We use sensitivity and specificity to assess the performance ofour methodology, which are widely used measures in the medical community.Their definitions are illustrated in the following formulas

sensitivity =number of TP

number of TP + number of FN, (20)

specificity =number of TN

number of TN + number of FP, (21)

where TP, FN, FP and TN represent the number of true positives, falsenegatives, false positives and true negatives frames, respectively. If the framebelongs to the class of disease (DR) frames and it is classified by the algorithmas negative, it is counted as a false negative; if it is classified as positive, itis counted as a true positive. If the frame belongs to the class of non-disease(i.e. normal) frames and it is classified as positive, it is counted as a falsepositive; if it is classified as negative, it is counted as a true negative.

Sensitivity represents the ability of the method to correctly classify theframe containing a lesion (in our case MAs, HEMs and BLs) as lesion frame,while specificity represents the ability of the method to correctly classify anon lesion frame as normal frame. The above definitions are on a per framebasis, and are used to assess the performance of each individual detector(MAs or HEMs or BLs). The performance of the MAs detector, HEMsdetector and BLs detector in terms of sensitivity and specificity is shown inTable 3 (with respect to the previous mentioned data sets DMA, DHEM andDBL, respectively).

Our binary classifier, for each detector, depends on certain parameterswhich are listed in Table 2 along with their values. These parameters arethresholds for the binary classifiers. Their values were chosen based on theintrinsic definitions of the associated criteria: see the qualitative justificationof these values for MAs in (9), for HEMs in (10)-(11) and after in (16), and


Table 2. Values of the parameters used in the classifier

Detector Parameter Value Equation Section

MAL2 3.5 9 2.6

MAs MAL3 5.5 9 2.6

HEML1 12 10 2.7

HEML2 6 11 2.7

HLmean 9 11 2.7

HUecc 0.7 11 2.7

HEMs HEML3 6 14 2.7

SL 0.7 14 2.7

HEML4 7 15 2.7

SL 0.7 15 2.7

HEML5 8 16 2.7

SL 0.7 16 2.7

BLL1 11 17 2.8

BLL2 6 18 2.8

NLt 1.3 18 2.8

EU 0 18 2.8

BLs BL 18 19 2.8

BU 40 19 2.8

NLc 0.01 19 2.8

NUc 0.2 19 2.8

CL 35 19 2.8

CU 60 19 2.8

finally for BLs in (17)-(18)-(19). The values were selected from the commonsense considerations, not from the goal of improving the performance for anyparticular data set that we used. In fact we utilized several big datasets andimages acquired with different devices, and it can be seen from the tablesdisplayed in this section that the detectors have a consistent performance.Moreover, we would like to stress that the generality of these thresholds wereset by analysing the ROC (receiver operating characteristic) curves for eachindividual criterion, in large data sets.


Table 3. Performance evaluation of each detector (per frame)

Detector Sensitivity Specificity

MAs 93.45 88.92

HEMs 86.34 90.19

BLs 89.81 97.47

Next we evaluate the performance of the methods in a collective way asa system for the detection and diagnosis of DR. Since here we have theclassification that is on a per patient basis, the assessment of the performanceof our system was done on a per patient basis. For this purpose we use amodified definition of sensitivity and specificity, in (20) and (21), where thereTP, FN, FP and TN are considered on a per patient basis instead of a perframe basis. A patient is labelled to be a positive patient (i.e. belonging tothe class of DR) if at least one frame of the patient is classified as a positiveframe by our methodology, i.e. if it is a positive for at least one of the threedetectors (MAs, HEMs or BLs). Accordingly we compute the number of TP,FN, FP and TN, and hence the sensitivity and specificity.

Table 4. Performance evaluation of the proposed system for DRdetection (per patient)

Data sets Sensitivity Specificity

DR1 100 70.7

DR2 100 73

DR3 89.3 58.9

DR4 92 57.6

The evaluation of the proposed system in terms of sensitivity and specificityis given in Table 4. This Table shows that the sensitivity and specificityvalues in data sets DR3 and DR4 are worse than in DR1 and DR2. Thisis because the two latter datasets contain images with very bad quality and


artifacts (in fact these images are nonmydriatic photographs without pupildilation, which originates lower quality images). This leads to an increase offalse positives and some false negatives (some examples are shown in Figure9).

Figure 9. Some examples of bad quality DR retinal fundus im-ages, not detected by the proposed system, from the data setsDR3 and DR4.

4. Discussion and ConclusionsWe developed a system for automated detection and diagnosis of DR,

through the processing of retinal fundus images, by devising appropriateMAs, HEMs and BLs binary classifier detectors. The preprocessing phaseinvolves the extraction of the background pixels that enables further compu-tations on the foreground pixels only, as well as the removal of main image


structures, such as the optic disk and blood vessels, so that they cannot inter-fere in the detection of the lesions. The extraction of the candidates for differ-ent types of lesions (MAs, HEM and BLs) is done by analysing different sub-band wavelet images (resulting from the isotropic undecimated wavelet trans-form decomposition of the green channel of the retinal image), and applyingHessian multiscale analysis, variational segmentation, cartoon+texture de-composition. These techniques are combined in a sequential and suitableway and this combination depends on the visual information transmitted bythe different lesions (that by nature are very diverse in shape, size, textureand color). After that some novel contextual features are derived and quan-tified, for each lesion depending on their properties. Then the correspondentnumerical feature values are used to obtain a binary classification method.Firstly, the testing of the proposed methods for the MAs, HEMs and BLsdetection is done individually. Secondly, we consider the proposed methodsas a system for DR detection and performed a thorough testing of the sys-tem on rich data sets to ensure its good performance in realistic situations.Standard performance measures such as sensitivity and specificity are usedfor its evaluation.

For general use of an automated system, high sensitivity is a safety issueand is therefore more important than high specificity, which is an efficacy is-sue [1]. The main function of any automated system is to decrease the amountof frames that require manual inspection. The proposed system achieves avery high sensitivity (between 90% and 100%), and equally importantly, avery good specificity (approximately 70%, in datasets with an average goodquality images).

AcknowledgmentThis work was partially supported by the project PTDC/MATNAN/0593/2012,

and also by CMUC (Center for Mathematics, University of Coimbra) andFCT (Portugal), through European program COMPETE/ FEDER and projectPEst-C/MAT/UI0324/2013. The authors would also like to thank Dr. GoncaloQuadros for having suggested the study of the interesting and challengingretina topic.


References[1] M. D. Abramoff, J. M. Reinhardt, S. R. Russell, J. C. Folk, V. B. Mahajan, M. Niemeijer,

and G. Quellec. Automated early detection of diabetic retinopathy. Ophthalmology, 117:1147– 1154, 2010.

[2] M. U. Akram, S. Khalid, A. Tariq, S. A. Khan, and F. Azam. Detection and classification ofretinal lesions for grading of diabetic retinopathy. Computers in Biology and Medicine, 45:161– 171, 2014.

[3] M. U. Akram, A. Tariq, S. A. Khan, and M. Y. Javed. Automated detection of exudatesand macula for grading of diabetic macular edema. Computer Methods and Programs inBiomedicine, 114:141 – 152, 2014.

[4] B. Antal and A. Hajdu. An ensemble-based system for microaneurysm detection and diabeticretinopathy grading. IEEE Transactions on Biomedical Engineering, 59:1720–1726, 2012.

[5] P. Bankhead, C. N. Scholfield, J.G. McGeown, and T. M. Curtis. Fast retinal vessel detectionand measurement using wavelets and edge location refinement. PloS one, 7(3):e32435, 2012.

[6] X. Bresson, S. Esedoglu, P. Vandergheynst, J.-P. Thiran, and S. Osher. Fast global minimiza-tion of the active contour/snake model. J Math. Imaging Vis., 28:151–167, 2007.

[7] A. Buades, T. M. Le, J. M. Morel, and L. A Vese. Fast cartoon + texture image filters. IEEETransactions on Image Processing, 19:1978–1986, 2010.

[8] J. Canny. A computational approach to edge detection. IEEE Transactions on Pattern Anal-ysis and Machine Intelligence, 8:679–698, 1986.

[9] T. F. Chan and L. A. Vese. Active contours without edges. IEEE Transactions on ImageProcessing, 10:266–277, 2001.

[10] J. Ferreira, R. Bernardes, P. Baptista, and J. Cunha-Vaz. Earmarking retinal changes in asequence of digital color fundus photographs. In IFMBE Proc, volume 11, pages 1727 – 1983,2005.

[11] I. N. Figueiredo and S. Kumar. Automatic optic disc detection in retinal fundus images basedon geometric features. In Image Analysis and Recognition, International Conference ICIAR2014. Aurelio Campilho and Mohamed Kamel, 2014.

[12] I. N. Figueiredo and S. Kumar. Wavelet-based computer-aided detection of bright lesions inretinal fundus images. In CompIMAGE 2014, LNCS 8641, pages 234–240, 2014.

[13] I. N. Figueiredo, S. Kumar, and P. N. Figueiredo. An intelligent system for polyp detection inwireless capsule endoscopy images. In Computational Vision and Medical Image Processing,VIPIMAGE 2013, pages 229–235. Joao Tavares & Natal Jorge (eds), 2014 Taylor & FrancisGroup, London, ISBN 978-1-138-00081-0 (ECCOMAS Thematic Conference on Computa-tional Vision and Medical Image Processing).

[14] I. N. Figueiredo, S. Kumar, C. Leal, and Pedro N. Figueiredo. Computer-assisted bleed-ing detection in wireless capsule endoscopy images. Computer Methods in Biomechanics andBiomedical Engineering: Imaging & Visualization, 1:198–210, 2013.

[15] P. N. Figueiredo, I. N. Figueiredo, S. Prasath, and R. Tsai. Automatic polyp detection inpillcam colon 2 capsule images and videos: preliminary feasibility report. Diagnostic andTherapeutic Endoscopy, pages 1–7, 2011.

[16] M. Foracchia, E. Grisan, and A. Ruggeri. Luminosity and contrast normalization in retinalimages. Medical Image Analysis, 9(3):179 –190, 2005.

[17] A. F. Frangi, W. J. Niessen, K. L. Vincken, and M. A. Viergever. Multiscale vessel enhance-ment filtering. In Medical Image Computing and Computer-Assisted Intervention, pages 130–137, Springer, Berlin, 1998.

[18] R. M. Haralock and L. G. Shapiro. Computer and Robot Vision. Addison-Wesley LongmanPublishing Co., Inc., Boston, MA, USA, 1991.


[19] P. V. C. Hough. Methods and means for recognizing complex patterns. U.S. Patent 3 069 654,Dec. 1962.

[20] C. Kimme, D. Ballard, and J. Sklansky. Finding circles by an array of accumulators. Commun.ACM, 18:120–122, 1975.

[21] A. Mamonov, I. N. Figueiredo, P. N. Figueiredo, and R. Tsai. Automated polyp detection incolon capsule endoscopy. IEEE Transactions on Medical Imaging, 33(7):1488–1502, 2014.

[22] A Osareh, B. Shadgar, and R. Markham. A computational-intelligence-based approach for de-tection of exudates in diabetic retinopathy images. IEEE Transactions on Information Tech-nology in Biomedicine, 13:535–545, 2009.

[23] R. Pires, H. F. Jelinek, J. Wainer, S. Goldenstein, E. Valle, and A. Rocha. Assessing the needfor referral in automatic diabetic retinopathy detection. IEEE Transactions on BiomedicalEngineering, 60:3391–3398, 2013.

[24] G. Quellec, M. Lamard, P.M. Josselin, G. Cazuguel, B. Cochener, and C. Roux. Optimalwavelet transform for the detection of microaneurysms in retina photographs. IEEE Transac-tions on Medical Imaging, 27:1230–1241, 2008.

[25] C. Sinthanayothin, J. F. Boyce, T. H. Williamson, H. L. Cook, E. Mensah, S. Lal, andD. Usher. Automated detection of diabetic retinopathy on digital fundus images. DiabeticMedicine, 19:105–112, 2002.

[26] J.-L. Starck, J. Fadili, and F. Murtagh. The undecimated wavelet decomposition and itsreconstruction. IEEE Transactions on Image Processing, 16:297–309, 2007.

[27] L. Tang, M. Niemeijer, J. M. Reinhardt, M. K. Garvin, and M.D. Abramoff. Splat featureclassification with application to retinal hemorrhage detection in fundus images. IEEE Trans-actions on Medical Imaging, 32:364–375, 2013.

[28] R. J. Winder, P. J. Morrow, I. N. McRitchie, J. R. Bailie, and P.M. Hart. Algorithms for digitalimage processing in diabetic retinopathy. Computerized Medical Imaging and Graphics, 33:608– 622, 2009.

[29] X. Zhang, G. Thibault, E. Decenciere, B. Marcotegui, B. Lay, R. Danno, G. Cazuguel, G. Quel-lec, M. Lamard, Pascale M., A. Chabouis, Z. Victor, and A. Erginay. Exudate detection in colorretinal images for mass screening of diabetic retinopathy. Medical Image Analysis, 18:1026 –1043, 2014.

Isabel Narra FigueiredoCMUC, Department of Mathematics, University of Coimbra, Portugal.

E-mail address: [email protected]: http://www.mat.uc.pt/∼isabelf

Sunil KumarCMUC, Department of Mathematics, University of Coimbra, Portugal.

E-mail address: [email protected]

Carlos Manta OliveiraUniversidade Nova de Lisboa, Portugal and Retmarker

E-mail address: [email protected]: http://www.retmarker.com/

Joao Diogo RamosRetmarker


E-mail address: [email protected]: http://www.retmarker.com/

Bjorn EngquistDepartment of Mathematics and the Institute for Computational Engineering and Sci-ences, The University of Texas at Austin, Austin, USA.

E-mail address: [email protected]

Documents

AUTOMATED LESION DETECTORS · tor, Retinal Fundus Image, Computer-aided Diagnosis, Wavelets, Multiscale Anal-ysis, Variational Segmentation, Cartoon+Texture Decomposition. 1.Introduction