Transliten dan rapikan - Copy.doc

Embed Size (px)

Citation preview

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    Article history:The pomegranate is a fruit with excellent organoleptic andnutritional properties, but the fact that it is

    Received 11 February 2!

    dif"cult to peel a#ects its commercialisation and decreases its

    potential consumption$ %ne solution isReceived in revised form 21 &ay 2!

    to mar'et the arils of pomegranate in a ready(to(eat form$)owever, after the peeling process, unwanted

    Accepted * &ay 2!

    Available online 12 +une 2!material, such as internal membranes anddefective arils, is extracted together with good arils and must

    product or deteriorates its appearance$ For di#erent reasons,the commercial sorting machines that are


    -mage analysiscurrently available for similar commodities .cherries, nuts, rice,etc$/ are not capable of handling and sort(

    Real(timeing pomegranate arils, thus ma'ing it necessary to build speci"ce0uipment$ This wor' describes the

    Fruit sortingdevelopment of a computer vision(based machine to inspect the rawmaterial coming from the extraction

    &achineryprocess and classify it in four categories$ The machine is capable ofdetecting and removing unwanted

    ualitymaterial and sorting the arils by colour$ The prototype is composed of threeunits, which are designed

    -nspectionto singulate the obects to allow them be inspected individually andsorted$ The inspection unit relies

    the R34 ratio and the other is a more complex approach

    based on 5ayesian 6inear 7iscriminant Analysis

    .67A/ in the R45 space$ 5oth methods o#ered an averagesuccess rate of 89 on a validation set, the

    former being more intuitive for the operators, as well as fasterand easier to implement, and for these

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    reasons it was included in the prototype$ ubse0uently, thecomplete machine was tested in industry

    by wor'ing in real conditions throughout a wholepomegranate season, in which it automatically sorted

    more than nine tons of arils$ 2! ;lsevier 6td$ All rights reserved$

    1$ -ntroductionother agricultural inputs .fertilisers, phytosanitary products and

    so forth/ in large 0uantities and this ma'es growingthem easier

    pain produces about 2, tons of pomegranate .uence its internal 0uality and properties, but do degrade its'et, but theirdescriptions lie beyond the scope of this wor'$ %ne of

    external appearance and, thus preventing the a#ected fruits fromtheir mainproblems, however, is that fragments of internal mem(

    being mar'eted$ )owever, manual peeling and extraction of the ar(brane or s'inand other unwanted material are released during the

    ils is dif"cult, and this gives rise to a certain degree of reection byextractionprocess$ &oreover, some defective arils .bro'en, abnor(

    the consumer in favour of other fruits that are easier to eat$ %n themally shapedor with di#erent physiological disorders/ may ap(

    other hand, the nutritional and anticarcinogenic properties ofpear, together witharils of di#erent colours ranging from white

    pomegranates have been widely demonstrated .chubert et al$,to red$ 7efectivearils may shorten the shelf life of the product,

    and arils with di#erent colours in the samepac'age may degrade

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    2?/$ Furthermore, pomegranate trees do not re0uire water orthe appearance ofthe product and hence reduce its price$ For all

    these reasons, it is crucial for manufacturers ofready(to eat pome(

    granate arils to "nd automatic solutions to inspectand sort the

    @orresponding author$ Fax: B*= 8CD 1 D=11$

    product$;(mail address:blascoEosivagva$es.+$5lasco/$

    2D(!??=3G ( see front matter2! ;lsevier 6td$ All rights reserved$


    2!+$ 5lasco et al$ 3 +ournal of Food ;ngineering 8 .28/ 2?H*=

    -n current production lines, this raw material inspection has7esigningmechanisms for separating and transporting the arils$

    three maor obectives: .1/ to remove the pieces of internal7eveloping real(timecomputer vision algorithms for inspecting

    membranes and s'inI .2/ to remove unwanted arils, such as thoseand classifying

    the arils$ that are too small, bro'en or have brownish coloursI and .*/ to sort7evelopingalgorithms and practical devices for synchronising

    the 0uality arils by colour in order to render the appearance of thethe inspectionunit with the sorting system$

    product uniform in the pac'age$ ince manual inspection is too5uilding a systemfor classifying the arils into categories$

    expensive and subective, computer vision opens up the possibility7evelopingcommunication and control procedures to supervise

    of constructing an automatic sorting machine$the whole machine, which included afriendly interface with the

    &achine vision has been largely employed for fresh fruit inspec(user$

    tion, and attempts to imitate human operators .5lasco et al$, 2*I

    5rosnan and un, 2=/$ This technology is nowadays available forThe proposedsolution had to be developed as a prototype for

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    the automatic sorting of most popular, relatively big fruits, such assorting all theobects that left the extracting machine into up to

    oranges .Aleixos et al$, 22I5lasco et al$, 2?a, b/, apples .6eemansfourcategories, while ensuring a minimum production of ?C 'g

    and 7estain, 2=/, peaches .&iller and 7elwiche, 1881IJwiggelaarof arils per hour$ et al$, 188D/ and tomatoes .;dan et al$, 188?/$omputer vision algorithms had toassess the colour of each ob(

    omputer vision systems mounted in sorting machines re0uireect individually,and had to be capable of discriminating between

    auxiliary devices to ma'e the obects enter and leave the "eld ofarils and otherundesired material, mainly composed of frag(

    view of the cameras$ They usually wor' in combination with othermented s'inand internal membranes$ The control algorithms

    devices that split the production into di#erent categories$ -t is of(had tosynchronise image ac0uisition from two di#erent camerasI

    ten advisable to separate the obects individually during these pro(analyse theimages and the displacement of the obects on the con(

    cesses in order to facilitate both the analysis of the obects in theveyor beltsI andactivate the sorting unit$

    image and the action of the sorting mechanisms$ 6arge fruits areReal(time imageprocessing was essential to achieve the re(

    relatively easy to separate from each other by mechanical means0uiredthroughput$ -ntuitive and fast classi"cation techni0ues

    and this allows the computer vision system to inspect each indi(had to becompared in order to decide which one should be imple(

    vidual obect more easily$mented in the prototype$ The optimal solution had to bethe one

    )owever, with the exception of the following, very little informa(that consumedless computing time and was easier to operate by

    tion is available on the development of machine vision systems ina non(

    experienced user, without compromising 0uality standards$ the case of processed and small fruits: olives .7KaL et al$, 2=/, dates&oreover,the solution had to be economically feasible and easy to

    .Mulfsohn et al$, 188*/, mandarin segments .5lasco et al$, 2?c/, rai(maintain$

    sins .)uxoll et al$, 188C/ or canned peaches .NiLmanos et al$, 188?/$

    -nspection of pomegranate arils has not yet been automated to date

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    because they are particularly dif"cult to manage due to their small*$ &aterials andmethods

    siLe, and because it is a wet and stic'y product that is very dif"cult

    to separate$ &oreover, arils are more fragile than whole fruits whichThe prototype

    .Fig$ 1/ basically consisted of three maor ele( complicates the way they are handled considerably$ All these rea(ments thatcorresponded to the feeding, inspection and sorting

    sons ma'e it necessary to develop a speci"c sorting machine$units$ These aredescribed below$

    Additional dif"culties such as the range of possible colours of

    the arils .from white to dar' red or brown/, the high speed of the*$1$ Feeding unit:eparation and transport of the obects

    conveyor belts, and the bright spots produced by water on the s'in

    of arils, further complicate colour estimation$ upervised segmen(Raw materialcoming from the extracting machine travelled on

    tation techni0ues based on statistics, such as 5ayesian methods,a single 2Cmm(wide conveyor belt, and then dropped down onto

    can facilitate image analysis .&archant and %nyango, 2*/,a vibrating, tiltedplate$ The inclination of the plate and its vibrat(

    although they re0uire the participation of an expert to train theing movementmade the material move forward and disaggregate

    system properly$ impler techni0ues, such as thresholding, provideas it reachedthe end of the plate$ -n the following step, it was split

    faster image segmentation, but they can only be used if the coloursinto six *mm wide conveyor belts that moved at a relatively

    of the obects belonging to the di#erent classes are distinctive andhigher speed.$?CH1$2C m3s/$ This action produced an individual

    well de"ned .4unase'aran, 18!?I Jion et al$, 188C/$separation of the di#erentobects that made up the raw input$ This

    The Agricultural ;ngineering entre of the Nalencian -nstitute ofstep was crucialfor subse0uent processes, since it allowed the

    Agricultural Research .-N-A/ and the pomegranate producer Frutassystem to usesimple image analysis techni0ues and to expel each

    &ira )ermanos, have collaborated over two years in order to de(obect into itscorresponding outlet$ The narrow siLe of the belts

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    sign and develop a computer vision based prototype .patent pend(preventedlarge accumulations of material and facilitated the

    ing/ to automatically inspect and sort pomegranate arils dependingseparation of theobects$

    on their commercial 0uality, and to remove unwanted materialsThe colour of theconveyor belts was chosen to enhance the

    produced during the extraction of the arils$ The results of prelimin(contrastbetween the belts and arils in order to ma'e segmentation

    ary trials underta'en during the "rst season, wor'ing on(site theof the imageseasier$ onveyor belts of di#erent colours were

    producer facilities, have been presented elsewhere .5lasco et al$,tested beforebeing installed on the prototype$ These preliminary

    2!/$ The present paper discusses the technological challenges in(tests

    consisted in ac0uiring and segmenting images of arils and

    volved and the engineering solutions implemented in the develop(internalmembranes placed over pieces of white, blue, dar' green,

    ment of the new prototype$light green and grey belts and then measuring the ratioof success(

    ful classi"cation .5lasco et al$, 2!/$

    2$ %bective*$2$ -nspecting unit and computer vision system

    The obective of this wor' was to develop an engineering solu(The prototype usedtwo progressive scan cameras to ac0uire

    tion for the automatic sorting of pomegranate arils$ This tas'C12*!= pixel R45.Red, 4reen and 5lue/ images with a

    included:resolution of $? mm3pixel$ 5oth cameras were connected to a

    computer, the so(called OOvision computerP .

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    stored them in the computerQs memory$ The illumination systemac0uisition of theimage independent of the speed of the belts,

    consisted of two = w daylight compact >uorescent tubes locatedand thusensures that there are never any overlaps or gaps be(

    on both sides of each conveyor belt, thus ma'ing a total of 1=tween consecutiveimages$

    tubes$ Tubes were used together with high fre0uency electronicegmentationconsists in determining which regions of the im(

    ballast to avoid the >ic'er e#ect$ This illumination was powerfulage correspondto bac'ground and which represent the obects of

    enough and did not produce shadows$ The in>uence of bright spotsinterest$ Meopted for a pixel(oriented segmentation algorithm,

    in the scene caused by direct lighting on wet surfaces, which couldbecause these

    algorithms are normally faster than other ap(

    alter the perception of the colour by the inspection system, wasproaches.region(oriented algorithms, textural analysis, etc$/$ The

    minimised by using cross(polarised "lters placed above the lampsconveyor beltswere blue and conse0uently had high 5 values

    and on the camera lenses$ The scene captured by each camera hadand low Rvalues of the R45 coordinates$ The colour of pomegran(

    a length of approximately *D mm along the direction of theate arils variesbetween white and red, which correspond to high R

    movement of the obects and a width that allowed the system tovalues$ -nternalmembranes are mostly white, and hence have high

    inspect three conveyor belts at the same time$ The entire systemR, 4 and 5values$ onse0uently, the segmentation algorithm used

    was housed in a stainless steel chamber$ A picture of the wholea pre(de"nedthreshold in the R band$

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    All source codes were speci"cally written for this application with(the pea' onthe right represents pixels from the aril$ The value of

    out the use of any commercial library in order to ensure the controlthisthreshold was selected manually and at random between the

    of the operations and real(time responses$two pea's by an expert$ %ne of the maor achievements of the software that was devel(%nce thebac'ground had been removed, each connected region

    oped is the possibility of wor'ing with two cameras at the samewas labelled as apossible obect of interest .under normal circum(

    time, since the ac0uisition of the images is a very time(consumingstances, itshould be an aril or some other material/$ -n the same

    process .= ms per image/$ The software was designed to processoperation, theprogram estimated the siLe and centroid of each of

    one image obtained with one camera in parallel with the ac0uisi(these obectsand the average R45 coordinates of their pixels$ ;xtre(

    tion of another image with the other camera$ The result is thatmely small or largeobects were classi"ed as unwanted material$

    the processing of one image and the ac0uisition of the next overlapFinally, theaverage colour coordinates were used to classify the

    in time, thus saving time and optimising the operation$obect into one of four pre(de"ned categories$ The procedure to

    The ac0uisition of the images is triggered by pulses receiveddetermine theclass is described below$ After processing each im(

    from an optical encoder attached to the shaft of the carrier rollerage, themachine vision computer sent the category and position

    *+$ 5lasco et al$ 3 +ournal of Food ;ngineering 8 .28/ 2?H*=

    of the obect to a second computer .called thecontrol computer/,and they need afast way to adapt the inspection software to the

    which trac'ed the obect until it was sorted$ This communicationevolution of thecolour of the product$ olour thresholds are intu(

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    was implemented via T

  • 8/10/2019 Transliten dan rapikan - Copy.doc


  • 8/10/2019 Transliten dan rapikan - Copy.doc


    there were changes in the performance of theclassi"cation


    *$*$ The sorting unit

    The sorting area followed the inspection chamber.Fig$ C/$ Three

    outlets were placed on one side of each of theconveyor belts$ -n

    front of each outlet air eectors were suitablyplaced to expel the


    The separation of the arils was monitored by thecontrol com(

    puter, in which a board with *2 digital outputs wasmounted$ This

    board was used to manage the air eectors$ Thecomputer trac'ed

    the movement of the obects on the conveyor beltsby reading

    the signals produced by the optical encoderattached to the shaft

    of the carrier roller$

    Table 1

    5asic statistics of the R34 ratio for each colourcategory


  • 8/10/2019 Transliten dan rapikan - Copy.doc


    Mhite 12C$1 !$2 11C$ 1**$* 1!H1**

    +$ 5lasco et al$ 3 +ournal of Food ;ngineering 8 .28/ 2?H*=*1

    *$=$ Testing the prototype under commercialconditions$ 4lobal

    evaluation of its performance

    %nce the prototype was ready, it was installed incommercial

    facilities for industrial testing and con"gured toseparate arils

    and unwanted material as described above$ Theprototype was

    tested in commercial conditions over a period ofsix months, be(

    tween eptember and February 2C32D,which was the pome(

    granate season in pain$

    7uring the tests, the prototype inspected morethan nine tons of

    obects$ -t was unviable to evaluate the results ofthe individual

    classi"cation of each obect, since wor'ing undercommercial con(

    ditions made it impossible to stop the line in orderto compare the

    classi"cation of single obects made by themachine and human

    experts$ For this reason, the evaluation wasperformed on the

    batches produced by the prototype$ A panel ofthree experts ana(

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    lysed random samples obtained from the di#erentoutlets of the

    machine, each expert giving an overall subectiveopinion of the

    performance of the inspection system for eachcategory$ ;ach ex(

    pert decided whether the automaticclassi"cation was good, regu(

    lar or poor and the "nal decision of the panel wasconsidered to be

    the one on which two or three of them had agreed.Fig$ D/$

    =$ Results and discussion

    The main outcome of this wor' was theimplementation of an

    engineering solution for sorting pomegranate arilsunder commer(

    cial conditions$ The development and successful

    commissioning of

    the automatic inspection machine constituted animportant engi(

    neering achievement in itself$ The following sections describe

    some of the 'ey results which contributedto this development$Fig$ C$7etails of the grading area showing the air eectors and

    the outlets$

    The control computer received the category and the position of=$1$ Feeding unit

    each obect in image coordinates via T

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    for each obect$ The positions of both the obect in the image andunder industryconditions$ -t re0uired precise levelling in order

    the belt were employed to calculate the moment at which the eec(to distributeraw material uniformly on the six conveyor belts$

    tor had to be activated$ The control computer stored the number of;xcessiveamounts of raw material on one or several belts proved

    encoder pulse counts necessary for the obect to reach the eector,to have animportant in>uence on the performance of the machine

    and then decreased this value every time it received a pulse fromas a whole$

    the encoder$ Mhen the count reached Lero, the obect was sup(%ther solutionshad been considered, but the arils were always

    posed to pass ust in front of the eector and the computer openedprevented fromrolling freely when they reached the belts$ The

    the appropriate electro(valve in order to remove it from the belt$movement of theobects on the belt produced errors in the syn(

    As has been said earlier, since synchronism was based on countingchronismbetween the inspection and the sorting units and, as a re(

    encoder pulses, it was independent of the speed of the belt$ Thissult, some piecesdid not reach the proper outlet$

    design made it possible to achieve a position trac'ing accuracy of5iconic rollerswith a rotational movement have been employed

    $* mm$in other machines for small fruit inspection, li'e olives .7KaL et al$,

    *2+$ 5lasco et al$ 3 +ournal of Food ;ngineering 8 .28/ 2?H*=

    2/$ These devices allow the system to inspect practically theTable =

    whole surface of the obects by ac0uiring images while the obectonfusionmatrix of the classi"cation performed at the end of the season using

    rotates$ &oreover, they provide a way to isolate individual obects$thresholds on theR34 ratio

    )owever, arils are sorted only by colour, but the colour is distrib(Actual lassi"ed as

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    uted uniformlyI for this reason a single image provided alltheMhite

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    values within the categories prevented them from being separatedwellclassi"ed, and unwanted material detected with a success rate

    automatically by this simple method$ )owever, the use of the R34of 19,although 19 of white arils were wrongly classi"ed in this

    ratio improved the results considerably .89 success/$class$ There was a smallamount of confusion between white, pin'

    Results of this classi"cation method based on the R34 ratio wereand brown arils,but it has to be ta'en into consideration that pin'

    acceptable both at the beginning and the end of the season$Tables *and whiteare secondary 0ualities$ Around *9 confusion was found

    and =show the confusion matrix on the validation set using thebetween the redand brown classes$ The average rate of success of

    thresholds de"ned inTable 2at the beginning and the end of thethis method was

    close to 829 in all the categories at the beginning

    season, respectively$ These tables also show that unwanted materialof theseason$ )owever, tests performed at the end of the season

    was the class in which best correct classi"cation was obtainedshowed thatsuccess decreased in all categories, but particularly

    .8!9/$ This can be easily explained by the fact that the colour andin the redcategory and mostly in the brown category, which fell

    the siLe of foreign material were normally very di#erent from thoseto !*9 .TableD/$ The confusion between both classes increased

    of the arils .the sorting algorithm considered both parameters in or(to more than19$

    der to produce the "nal decision of the class/$ The most sensitiveThe analysis ofthe coef"cients of the classi"cation functions

    classes from the commercial point of view were red and brown arilsIallowed usto deduce the parameters that had a higher power to

    successful classi"cation reached a rate of 89 at the beginning ofthediscriminate between the classes .Table ?/$ ince the span of the

    season and !C9 at the end$ onfusion between these two classes Table C

    onfusion matrix of the classi"cation performedat the beginning of the season using

    Table 267A on the average R45 values of the obects

    Average success rate of classi"cation on the validation set using thresholds on the

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    di#erent colour descriptorsActual lassi"ed as


  • 8/10/2019 Transliten dan rapikan - Copy.doc


    Wnwanted material 1$? $ $ $ 8!$*Wnwanted material $? $* $ $ 88$

    Results refer to the validation set$Results refer to the validation set$

    +$ 5lasco et al$ 3 +ournal of Food ;ngineering 8 .28/ 2?H*=**

    the application of machine vision in order toautomate this tas',Table ?

    and have made the mar'eting of ready(to(eatpomegranate arilsoef"cients of the classi"cation functions obtained using 67A

    a feasible proposition$lass variables Mhite

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    ince the average success rate of both methods .multiple

    could handle a maximum throughput of ?C 'g3h$The feeding unit,

    thresholds on the R34 ratio and 67A on average values of obects/

    based on a tilted plate and narrow conveyorbelts, was an e#ective

    were similar .they di#ered by about 19 in each category/, we

    device for separating fragments of raw materialcoming from the

    decided to use the simplest one, which allowed us to increase

    extracting machine$ The inspection unit, whichhad two cameras

    processing time$ &oreover, thresholds are easier to tune by the

    connected to a computer vision system, hadenough capacity to

    operators when they want to change the sorting results in real

    achieve real(time speci"cations and enoughaccuracy to ful"l the


    commercial re0uirements$ The sorting unit was

    able to classify the product into four categories$ ynchronisationbetween the last


  • 8/10/2019 Transliten dan rapikan - Copy.doc


    ing speed of more than 2C images3s, but it was limited to 12

    images3s due to mechanical limitations$ This opens up the possibil(


    ity of including more sophisticated solutions in the prototype, for

    instance, learning algorithms for automatically adapting the classi(

    This wor' was partially funded by the ;uropeanommission

    "cation parameters to the changes in the colour of the arils, similar

    .;/ through

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    iments demonstrated that two consecutive obects re0uired apomegranate.

  • 8/10/2019 Transliten dan rapikan - Copy.doc


    four months$ At the end of the season, the classi"cation perfor(algorithm for theclassi"cation of olives by machine vision$ Food Research

    mance decreased, because the confusion between red and brown-nternational**, *CH*8$

    arils increased$ The performance of the prototype was ?C 'g of arils;dan, Z$,