Transliten dan rapikan - Copy.doc

8/10/2019 Transliten dan rapikan - Copy.doc

1/22

Article history:The pomegranate is a fruit with excellent organoleptic andnutritional properties, but the fact that it is

Received 11 February 2!

dif"cult to peel a#ects its commercialisation and decreases its

potential consumption$ %ne solution isReceived in revised form 21 &ay 2!

to mar'et the arils of pomegranate in a ready(to(eat form$)owever, after the peeling process, unwanted

Accepted * &ay 2!

Available online 12 +une 2!material, such as internal membranes anddefective arils, is extracted together with good arils and must

product or deteriorates its appearance$ For di#erent reasons,the commercial sorting machines that are

eywords:

-mage analysiscurrently available for similar commodities .cherries, nuts, rice,etc$/ are not capable of handling and sort(

Real(timeing pomegranate arils, thus ma'ing it necessary to build speci"ce0uipment$ This wor' describes the

Fruit sortingdevelopment of a computer vision(based machine to inspect the rawmaterial coming from the extraction

&achineryprocess and classify it in four categories$ The machine is capable ofdetecting and removing unwanted

ualitymaterial and sorting the arils by colour$ The prototype is composed of threeunits, which are designed

-nspectionto singulate the obects to allow them be inspected individually andsorted$ The inspection unit relies

the R34 ratio and the other is a more complex approach

based on 5ayesian 6inear 7iscriminant Analysis

.67A/ in the R45 space$ 5oth methods o#ered an averagesuccess rate of 89 on a validation set, the

former being more intuitive for the operators, as well as fasterand easier to implement, and for these


2/22

reasons it was included in the prototype$ ubse0uently, thecomplete machine was tested in industry

by wor'ing in real conditions throughout a wholepomegranate season, in which it automatically sorted

more than nine tons of arils$ 2! ;lsevier 6td$ All rights reserved$

1$ -ntroductionother agricultural inputs .fertilisers, phytosanitary products and

so forth/ in large 0uantities and this ma'es growingthem easier

pain produces about 2, tons of pomegranate .uence its internal 0uality and properties, but do degrade its'et, but theirdescriptions lie beyond the scope of this wor'$ %ne of

external appearance and, thus preventing the a#ected fruits fromtheir mainproblems, however, is that fragments of internal mem(

being mar'eted$ )owever, manual peeling and extraction of the ar(brane or s'inand other unwanted material are released during the

ils is dif"cult, and this gives rise to a certain degree of reection byextractionprocess$ &oreover, some defective arils .bro'en, abnor(

the consumer in favour of other fruits that are easier to eat$ %n themally shapedor with di#erent physiological disorders/ may ap(

other hand, the nutritional and anticarcinogenic properties ofpear, together witharils of di#erent colours ranging from white

pomegranates have been widely demonstrated .chubert et al$,to red$ 7efectivearils may shorten the shelf life of the product,

and arils with di#erent colours in the samepac'age may degrade


3/22

2?/$ Furthermore, pomegranate trees do not re0uire water orthe appearance ofthe product and hence reduce its price$ For all

these reasons, it is crucial for manufacturers ofready(to eat pome(

granate arils to "nd automatic solutions to inspectand sort the

@orresponding author$ Fax: B*= 8CD 1 D=11$

product$;(mail address:blascoEosivagva$es.+$5lasco/$

2D(!??=3G ( see front matter2! ;lsevier 6td$ All rights reserved$

doi:1$11D3$foodeng$2!$C$*C

2!+$ 5lasco et al$ 3 +ournal of Food ;ngineering 8 .28/ 2?H*=

-n current production lines, this raw material inspection has7esigningmechanisms for separating and transporting the arils$

three maor obectives: .1/ to remove the pieces of internal7eveloping real(timecomputer vision algorithms for inspecting

membranes and s'inI .2/ to remove unwanted arils, such as thoseand classifying

the arils$ that are too small, bro'en or have brownish coloursI and .*/ to sort7evelopingalgorithms and practical devices for synchronising

the 0uality arils by colour in order to render the appearance of thethe inspectionunit with the sorting system$

product uniform in the pac'age$ ince manual inspection is too5uilding a systemfor classifying the arils into categories$

expensive and subective, computer vision opens up the possibility7evelopingcommunication and control procedures to supervise

of constructing an automatic sorting machine$the whole machine, which included afriendly interface with the

&achine vision has been largely employed for fresh fruit inspec(user$

tion, and attempts to imitate human operators .5lasco et al$, 2*I

5rosnan and un, 2=/$ This technology is nowadays available forThe proposedsolution had to be developed as a prototype for


4/22

the automatic sorting of most popular, relatively big fruits, such assorting all theobects that left the extracting machine into up to

oranges .Aleixos et al$, 22I5lasco et al$, 2?a, b/, apples .6eemansfourcategories, while ensuring a minimum production of ?C 'g

and 7estain, 2=/, peaches .&iller and 7elwiche, 1881IJwiggelaarof arils per hour$ et al$, 188D/ and tomatoes .;dan et al$, 188?/$omputer vision algorithms had toassess the colour of each ob(

omputer vision systems mounted in sorting machines re0uireect individually,and had to be capable of discriminating between

auxiliary devices to ma'e the obects enter and leave the "eld ofarils and otherundesired material, mainly composed of frag(

view of the cameras$ They usually wor' in combination with othermented s'inand internal membranes$ The control algorithms

devices that split the production into di#erent categories$ -t is of(had tosynchronise image ac0uisition from two di#erent camerasI

ten advisable to separate the obects individually during these pro(analyse theimages and the displacement of the obects on the con(

cesses in order to facilitate both the analysis of the obects in theveyor beltsI andactivate the sorting unit$

image and the action of the sorting mechanisms$ 6arge fruits areReal(time imageprocessing was essential to achieve the re(

relatively easy to separate from each other by mechanical means0uiredthroughput$ -ntuitive and fast classi"cation techni0ues

and this allows the computer vision system to inspect each indi(had to becompared in order to decide which one should be imple(

vidual obect more easily$mented in the prototype$ The optimal solution had to bethe one

)owever, with the exception of the following, very little informa(that consumedless computing time and was easier to operate by

tion is available on the development of machine vision systems ina non(

experienced user, without compromising 0uality standards$ the case of processed and small fruits: olives .7KaL et al$, 2=/, dates&oreover,the solution had to be economically feasible and easy to

.Mulfsohn et al$, 188*/, mandarin segments .5lasco et al$, 2?c/, rai(maintain$

sins .)uxoll et al$, 188C/ or canned peaches .NiLmanos et al$, 188?/$

-nspection of pomegranate arils has not yet been automated to date


5/22

because they are particularly dif"cult to manage due to their small*$ &aterials andmethods

siLe, and because it is a wet and stic'y product that is very dif"cult

to separate$ &oreover, arils are more fragile than whole fruits whichThe prototype

.Fig$ 1/ basically consisted of three maor ele( complicates the way they are handled considerably$ All these rea(ments thatcorresponded to the feeding, inspection and sorting

sons ma'e it necessary to develop a speci"c sorting machine$units$ These aredescribed below$

Additional dif"culties such as the range of possible colours of

the arils .from white to dar' red or brown/, the high speed of the*$1$ Feeding unit:eparation and transport of the obects

conveyor belts, and the bright spots produced by water on the s'in

of arils, further complicate colour estimation$ upervised segmen(Raw materialcoming from the extracting machine travelled on

tation techni0ues based on statistics, such as 5ayesian methods,a single 2Cmm(wide conveyor belt, and then dropped down onto

can facilitate image analysis .&archant and %nyango, 2*/,a vibrating, tiltedplate$ The inclination of the plate and its vibrat(

although they re0uire the participation of an expert to train theing movementmade the material move forward and disaggregate

system properly$ impler techni0ues, such as thresholding, provideas it reachedthe end of the plate$ -n the following step, it was split

faster image segmentation, but they can only be used if the coloursinto six *mm wide conveyor belts that moved at a relatively

of the obects belonging to the di#erent classes are distinctive andhigher speed.$?CH1$2C m3s/$ This action produced an individual

well de"ned .4unase'aran, 18!?I Jion et al$, 188C/$separation of the di#erentobects that made up the raw input$ This

The Agricultural ;ngineering entre of the Nalencian -nstitute ofstep was crucialfor subse0uent processes, since it allowed the

Agricultural Research .-N-A/ and the pomegranate producer Frutassystem to usesimple image analysis techni0ues and to expel each

&ira )ermanos, have collaborated over two years in order to de(obect into itscorresponding outlet$ The narrow siLe of the belts


6/22

sign and develop a computer vision based prototype .patent pend(preventedlarge accumulations of material and facilitated the

ing/ to automatically inspect and sort pomegranate arils dependingseparation of theobects$

on their commercial 0uality, and to remove unwanted materialsThe colour of theconveyor belts was chosen to enhance the

produced during the extraction of the arils$ The results of prelimin(contrastbetween the belts and arils in order to ma'e segmentation

ary trials underta'en during the "rst season, wor'ing on(site theof the imageseasier$ onveyor belts of di#erent colours were

producer facilities, have been presented elsewhere .5lasco et al$,tested beforebeing installed on the prototype$ These preliminary

2!/$ The present paper discusses the technological challenges in(tests

consisted in ac0uiring and segmenting images of arils and

volved and the engineering solutions implemented in the develop(internalmembranes placed over pieces of white, blue, dar' green,

ment of the new prototype$light green and grey belts and then measuring the ratioof success(

ful classi"cation .5lasco et al$, 2!/$

2$ %bective*$2$ -nspecting unit and computer vision system

The obective of this wor' was to develop an engineering solu(The prototype usedtwo progressive scan cameras to ac0uire

tion for the automatic sorting of pomegranate arils$ This tas'C12*!= pixel R45.Red, 4reen and 5lue/ images with a

included:resolution of $? mm3pixel$ 5oth cameras were connected to a

computer, the so(called OOvision computerP .


7/22

stored them in the computerQs memory$ The illumination systemac0uisition of theimage independent of the speed of the belts,

consisted of two = w daylight compact >uorescent tubes locatedand thusensures that there are never any overlaps or gaps be(

on both sides of each conveyor belt, thus ma'ing a total of 1=tween consecutiveimages$

tubes$ Tubes were used together with high fre0uency electronicegmentationconsists in determining which regions of the im(

ballast to avoid the >ic'er e#ect$ This illumination was powerfulage correspondto bac'ground and which represent the obects of

enough and did not produce shadows$ The in>uence of bright spotsinterest$ Meopted for a pixel(oriented segmentation algorithm,

in the scene caused by direct lighting on wet surfaces, which couldbecause these

algorithms are normally faster than other ap(

alter the perception of the colour by the inspection system, wasproaches.region(oriented algorithms, textural analysis, etc$/$ The

minimised by using cross(polarised "lters placed above the lampsconveyor beltswere blue and conse0uently had high 5 values

and on the camera lenses$ The scene captured by each camera hadand low Rvalues of the R45 coordinates$ The colour of pomegran(

a length of approximately *D mm along the direction of theate arils variesbetween white and red, which correspond to high R

movement of the obects and a width that allowed the system tovalues$ -nternalmembranes are mostly white, and hence have high

inspect three conveyor belts at the same time$ The entire systemR, 4 and 5values$ onse0uently, the segmentation algorithm used

was housed in a stainless steel chamber$ A picture of the wholea pre(de"nedthreshold in the R band$


8/22

All source codes were speci"cally written for this application with(the pea' onthe right represents pixels from the aril$ The value of

out the use of any commercial library in order to ensure the controlthisthreshold was selected manually and at random between the

of the operations and real(time responses$two pea's by an expert$ %ne of the maor achievements of the software that was devel(%nce thebac'ground had been removed, each connected region

oped is the possibility of wor'ing with two cameras at the samewas labelled as apossible obect of interest .under normal circum(

time, since the ac0uisition of the images is a very time(consumingstances, itshould be an aril or some other material/$ -n the same

process .= ms per image/$ The software was designed to processoperation, theprogram estimated the siLe and centroid of each of

one image obtained with one camera in parallel with the ac0uisi(these obectsand the average R45 coordinates of their pixels$ ;xtre(

tion of another image with the other camera$ The result is thatmely small or largeobects were classi"ed as unwanted material$

the processing of one image and the ac0uisition of the next overlapFinally, theaverage colour coordinates were used to classify the

in time, thus saving time and optimising the operation$obect into one of four pre(de"ned categories$ The procedure to

The ac0uisition of the images is triggered by pulses receiveddetermine theclass is described below$ After processing each im(

from an optical encoder attached to the shaft of the carrier rollerage, themachine vision computer sent the category and position

*+$ 5lasco et al$ 3 +ournal of Food ;ngineering 8 .28/ 2?H*=

of the obect to a second computer .called thecontrol computer/,and they need afast way to adapt the inspection software to the

which trac'ed the obect until it was sorted$ This communicationevolution of thecolour of the product$ olour thresholds are intu(


9/22

was implemented via T


10/22


11/22

there were changes in the performance of theclassi"cation

algorithms$

*$*$ The sorting unit

The sorting area followed the inspection chamber.Fig$ C/$ Three

outlets were placed on one side of each of theconveyor belts$ -n

front of each outlet air eectors were suitablyplaced to expel the

product$

The separation of the arils was monitored by thecontrol com(

puter, in which a board with *2 digital outputs wasmounted$ This

board was used to manage the air eectors$ Thecomputer trac'ed

the movement of the obects on the conveyor beltsby reading

the signals produced by the optical encoderattached to the shaft

of the carrier roller$

Table 1

5asic statistics of the R34 ratio for each colourcategory

&eantandard


12/22

Mhite 12C$1 !$2 11C$ 1**$* 1!H1**

+$ 5lasco et al$ 3 +ournal of Food ;ngineering 8 .28/ 2?H*=*1

*$=$ Testing the prototype under commercialconditions$ 4lobal

evaluation of its performance

%nce the prototype was ready, it was installed incommercial

facilities for industrial testing and con"gured toseparate arils

and unwanted material as described above$ Theprototype was

tested in commercial conditions over a period ofsix months, be(

tween eptember and February 2C32D,which was the pome(

granate season in pain$

7uring the tests, the prototype inspected morethan nine tons of

obects$ -t was unviable to evaluate the results ofthe individual

classi"cation of each obect, since wor'ing undercommercial con(

ditions made it impossible to stop the line in orderto compare the

classi"cation of single obects made by themachine and human

experts$ For this reason, the evaluation wasperformed on the

batches produced by the prototype$ A panel ofthree experts ana(


13/22

lysed random samples obtained from the di#erentoutlets of the

machine, each expert giving an overall subectiveopinion of the

performance of the inspection system for eachcategory$ ;ach ex(

pert decided whether the automaticclassi"cation was good, regu(

lar or poor and the "nal decision of the panel wasconsidered to be

the one on which two or three of them had agreed.Fig$ D/$

=$ Results and discussion

The main outcome of this wor' was theimplementation of an

engineering solution for sorting pomegranate arilsunder commer(

cial conditions$ The development and successful

commissioning of

the automatic inspection machine constituted animportant engi(

neering achievement in itself$ The following sections describe

some of the 'ey results which contributedto this development$Fig$ C$7etails of the grading area showing the air eectors and

the outlets$

The control computer received the category and the position of=$1$ Feeding unit

each obect in image coordinates via T


14/22

for each obect$ The positions of both the obect in the image andunder industryconditions$ -t re0uired precise levelling in order

the belt were employed to calculate the moment at which the eec(to distributeraw material uniformly on the six conveyor belts$

tor had to be activated$ The control computer stored the number of;xcessiveamounts of raw material on one or several belts proved

encoder pulse counts necessary for the obect to reach the eector,to have animportant in>uence on the performance of the machine

and then decreased this value every time it received a pulse fromas a whole$

the encoder$ Mhen the count reached Lero, the obect was sup(%ther solutionshad been considered, but the arils were always

posed to pass ust in front of the eector and the computer openedprevented fromrolling freely when they reached the belts$ The

the appropriate electro(valve in order to remove it from the belt$movement of theobects on the belt produced errors in the syn(

As has been said earlier, since synchronism was based on countingchronismbetween the inspection and the sorting units and, as a re(

encoder pulses, it was independent of the speed of the belt$ Thissult, some piecesdid not reach the proper outlet$

design made it possible to achieve a position trac'ing accuracy of5iconic rollerswith a rotational movement have been employed

$* mm$in other machines for small fruit inspection, li'e olives .7KaL et al$,

*2+$ 5lasco et al$ 3 +ournal of Food ;ngineering 8 .28/ 2?H*=

2/$ These devices allow the system to inspect practically theTable =

whole surface of the obects by ac0uiring images while the obectonfusionmatrix of the classi"cation performed at the end of the season using

rotates$ &oreover, they provide a way to isolate individual obects$thresholds on theR34 ratio

)owever, arils are sorted only by colour, but the colour is distrib(Actual lassi"ed as


15/22

uted uniformlyI for this reason a single image provided alltheMhite


16/22

values within the categories prevented them from being separatedwellclassi"ed, and unwanted material detected with a success rate

automatically by this simple method$ )owever, the use of the R34of 19,although 19 of white arils were wrongly classi"ed in this

ratio improved the results considerably .89 success/$class$ There was a smallamount of confusion between white, pin'

Results of this classi"cation method based on the R34 ratio wereand brown arils,but it has to be ta'en into consideration that pin'

acceptable both at the beginning and the end of the season$Tables *and whiteare secondary 0ualities$ Around *9 confusion was found

and =show the confusion matrix on the validation set using thebetween the redand brown classes$ The average rate of success of

thresholds de"ned inTable 2at the beginning and the end of thethis method was

close to 829 in all the categories at the beginning

season, respectively$ These tables also show that unwanted materialof theseason$ )owever, tests performed at the end of the season

was the class in which best correct classi"cation was obtainedshowed thatsuccess decreased in all categories, but particularly

.8!9/$ This can be easily explained by the fact that the colour andin the redcategory and mostly in the brown category, which fell

the siLe of foreign material were normally very di#erent from thoseto !*9 .TableD/$ The confusion between both classes increased

of the arils .the sorting algorithm considered both parameters in or(to more than19$

der to produce the "nal decision of the class/$ The most sensitiveThe analysis ofthe coef"cients of the classi"cation functions

classes from the commercial point of view were red and brown arilsIallowed usto deduce the parameters that had a higher power to

successful classi"cation reached a rate of 89 at the beginning ofthediscriminate between the classes .Table ?/$ ince the span of the

season and !C9 at the end$ onfusion between these two classes Table C

onfusion matrix of the classi"cation performedat the beginning of the season using

Table 267A on the average R45 values of the obects

Average success rate of classi"cation on the validation set using thresholds on the


17/22

di#erent colour descriptorsActual lassi"ed as

Mhite


18/22

Wnwanted material 1$? $ $ $ 8!$*Wnwanted material $? $* $ $ 88$

Results refer to the validation set$Results refer to the validation set$

+$ 5lasco et al$ 3 +ournal of Food ;ngineering 8 .28/ 2?H*=**

the application of machine vision in order toautomate this tas',Table ?

and have made the mar'eting of ready(to(eatpomegranate arilsoef"cients of the classi"cation functions obtained using 67A

a feasible proposition$lass variables Mhite


19/22

ince the average success rate of both methods .multiple

could handle a maximum throughput of ?C 'g3h$The feeding unit,

thresholds on the R34 ratio and 67A on average values of obects/

based on a tilted plate and narrow conveyorbelts, was an e#ective

were similar .they di#ered by about 19 in each category/, we

device for separating fragments of raw materialcoming from the

decided to use the simplest one, which allowed us to increase

extracting machine$ The inspection unit, whichhad two cameras

processing time$ &oreover, thresholds are easier to tune by the

connected to a computer vision system, hadenough capacity to

operators when they want to change the sorting results in real

achieve real(time speci"cations and enoughaccuracy to ful"l the

time$

commercial re0uirements$ The sorting unit was

able to classify the product into four categories$ ynchronisationbetween the last

=$2$2$


20/22

ing speed of more than 2C images3s, but it was limited to 12

images3s due to mechanical limitations$ This opens up the possibil(

Ac'nowledgements

ity of including more sophisticated solutions in the prototype, for

instance, learning algorithms for automatically adapting the classi(

This wor' was partially funded by the ;uropeanommission

"cation parameters to the changes in the colour of the arils, similar

.;/ through


21/22

iments demonstrated that two consecutive obects re0uired apomegranate.


22/22

four months$ At the end of the season, the classi"cation perfor(algorithm for theclassi"cation of olives by machine vision$ Food Research

mance decreased, because the confusion between red and brown-nternational**, *CH*8$

arils increased$ The performance of the prototype was ?C 'g of arils;dan, Z$,

Documents

Transliten dan rapikan - Copy.doc