Tao

Developing Language Processing Components with GATE Version 7 (a User Guide)For GATE version 7.2-snapshot (development builds) (built December 5, 2012)

rmish gunninghm hin wynrd ulin fonthev lentin ln xirj eswni sn oerts qenevieve qorrell edm punk engus oerts hni hmljnovi homs reitz wrk eF qreenwood rorio ggion tohnn etrk oyong vi im eterset al

he niversity of he0eldD heprtment of gomputer iene PHHIEPHIP

httpXGGgteFFukG

his user mnul is freeD ut plese onsider mking dontionF rwv versionX httpXGGgteFFukGuserguideWork on GATE has been partly supported by EPSRC grants GR/K25267 (Large-Scale Information Extraction), GR/M31699 (GATE 2), RA007940 (EMILLE), GR/N15764/01 (AKT) and GR/R85150/01 (MIAKT), AHRB grant APN16396 (ETCSL/GATE), Matrixware, the Information Retrieval Facility and several EU-funded projects: (SEKT, TAO, NeOn, MediaCampaign, Musing, KnowledgeWeb, PrestoSpace, h-TechSight, and enIRaF).

Developing Language Processing Components with GATE Version 7The University of Sheeld, Department of Computer Science Regent Court 211 Portobello Sheeld S1 4DP United Kingdom http://gate.ac.uk

2012 The University of Sheeld, Department of Computer Science

This work is licenced under the Creative Commons Attribution-No Derivative Licence. You are free to copy, distribute, display, and perform the work under the following conditions:

Attribution You must give the original author credit. No Derivative Works You may not alter, transform, or build upon this work.

With the understanding that:

Waiver

Any of the above conditions can be waived if you get permission from the copyright holder.

Other Rights

In no way are any of the following rights aected by the license: your fair dealing or fair use rights; the author's moral rights; rights other persons may have either in the work itself or in how the work is used, such as publicity or privacy rights.

Notice For any reuse or distribution, you must make clear to others the licence termsof this work.

For more information about the Creative Commons Attribution-No Derivative License, please visit this web address: http://creativecommons.org/licenses/by-nd/2.0/uk/

Brief Contents

I GATE BasicsI sntrodution P snstlling nd unning qei Q sing qei heveloper R giyviX the qei gomponent wodel S vnguge esouresX gorporD houments nd ennottions T exxsiX xerlyExew snformtion ixtrtion ystem

3S PU QU UI WQ IIU

II GATE for Advanced UsersU qei imedded V teiX egulr ixpressions over ennottions W exxsgX exxottionsEsnEgontext IH erformne ivlution of vnguge enlysers II ro(ling roessing esoures IP heveloping qei

135IQU IVW PPW PQW PTW PUU

III CREOLE PluginsIQ qzetteers IR orking with yntologies IS xonEinglish vnguge upport IT homin pei( esoures IU rsers IV whine verningiii

289PWI QII QSI QSW QTU QVI

iv

Contents

IW ools for elignment sks PH gomining qei nd swe PI wore @giyviA lugins

RQI RRU RSW

IV The GATE Family: Cloud, MIMIR, TeamwarePP qei gloud PR qei wmir

525SPU SSI

PQ qei emwreX e eEsed gollortive gorpus ennottion ool SQU

Appendicese ghnge vog f ersion SFI lugins xme wp g ysolete giyvi lugins h hesign xotes i ent sks for qei p xmedEintity tte whine tterns q rtEofEpeeh gs used in the repple gger eferenes

553SSQ SVU SVW SWU THS TIQ TPI TPQ

Contents

I GATE BasicsI sntrodutionIFI IFP IFQ row to se this ext F F F F F F F F F F F F F F F F F F F F F F F F gontext F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F yverview F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IFQFI heveloping nd heploying vnguge roessing pilities IFQFP fuiltEsn gomponents F F F F F F F F F F F F F F F F F F F F IFQFQ edditionl pilities in qei heveloperGimedded F F F IFQFR en ixmple F F F F F F F F F F F F F F F F F F F F F F F F F ome ivlutions F F F F F F F F F F F F F F F F F F F F F F F F F F eent ghnges F F F F F F F F F F F F F F F F F F F F F F F F F F F IFSFI ersion UFI @xovemer PHIPA F F F F F F F F F F F F F F F F purther eding F F F F F F F F F F F F F F F F F F F F F F F F F F F hownloding qei F F F F F F F F F F F F F F F F snstlling nd unning qei F F F F F F F F F F F PFPFI he isy y F F F F F F F F F F F F F F F PFPFP he rrd y @IA F F F F F F F F F F F F F PFPFQ he rrd y @PAX uversion F F F F F F PFPFR unning qei heveloper on nixGvinux sing ystem roperties with qei F F F F F F F gon(guring qei F F F F F F F F F F F F F F F F F fuilding qei F F F F F F F F F F F F F F F F F F F PFSFI sing qei with wvenGsvy F F F F F F F ninstlling qei F F F F F F F F F F F F F F F F F rouleshooting F F F F F F F F F F F F F F F F F F F he qei heveloper win indow voding nd iewing houments F F greting nd iewing gorpor F F F F orking with ennottions F F F F F F QFRFI he ennottion ets iew F F v F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

3V V W W II IP IP IR IS IS IU

S

IFR IFS IFT PFI PFP

P snstlling nd unning qei

PU

PFQ PFR PFS PFT PFU QFI QFP QFQ QFR

PU PU PU PV PW PW QH QP QQ QR QS QS QV RH RQ RS RS

Q sing qei heveloper

QU

vi

ContentsQFRFP he ennottions vist iew F F F F F F F F F F F F F F F F F F F F QFRFQ he ennottions tk iew F F F F F F F F F F F F F F F F F F F F QFRFR he goEreferene iditor F F F F F F F F F F F F F F F F F F F F F F QFRFS greting nd iditing ennottions F F F F F F F F F F F F F F F F F QFRFT hemEhriven iditing F F F F F F F F F F F F F F F F F F F F F F F QFRFU rinting ext with ennottions F F F F F F F F F F F F F F F F F F QFS sing giyvi lugins F F F F F F F F F F F F F F F F F F F F F F F F F F QFT snstlling nd updting giyvi lugins F F F F F F F F F F F F F F F F QFU voding nd sing roessing esoures F F F F F F F F F F F F F F F F F QFV greting nd unning n epplition F F F F F F F F F F F F F F F F F F F QFVFI unning n epplition on htstore F F F F F F F F F F F F F F QFVFP unning s gonditionlly on houment petures F F F F F F F QFVFQ hoing snformtion ixtrtion with exxsi F F F F F F F F F F F F QFVFR wodifying exxsi F F F F F F F F F F F F F F F F F F F F F F F F F QFW ving epplitions nd vnguge esoures F F F F F F F F F F F F F F F QFWFI ving houments to pile F F F F F F F F F F F F F F F F F F F F F QFWFP ving nd estoring vs in htstores F F F F F F F F F F F F F QFWFQ ving epplition ttes to pile F F F F F F F F F F F F F F F F QFWFR ving n epplition with its esoures @eFgF qeigloudFnetA QFIH ueyord hortuts F F F F F F F F F F F F F F F F F F F F F F F F F F F F F QFII wisellneous F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F QFIIFI topping qei from estoring heveloper essionsGyptions F F QFIIFP orking with niode F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F RT RT RU RV SI SP SQ SS ST SV SV SW TH TH TI TI TP TQ TR TS TU TU TV

R giyviX the qei gomponent wodelRFI RFP RFQ RFR RFS RFT RFU

RFV

he e nd giyvi F F F F F F F F F F F F F F F F F he qei prmework F F F F F F F F F F F F F F F F F F he vifeyle of giyvi esoure F F F F F F F F F F roessing esoures nd epplitions F F F F F F F F F vnguge esoures nd htstores F F F F F F F F F F F fuiltEin giyvi esoures F F F F F F F F F F F F F F F giyvi esoure gon(gurtion F F F F F F F F F F F F RFUFI gon(gurtion with wv F F F F F F F F F F F F F RFUFP gon(guring esoures using ennottions F F F F RFUFQ wixing the gon(gurtion tyles F F F F F F F F F RFUFR voding hirdErty virries using ephe svy oolsX row to edd tilities to qei heveloper F F F F RFVFI utting your tools in suEmenu F F F F F F F F peturesX imple ettriuteGlue ht F F F F F F F F gorporX ets of houments plus petures F F F F F F houmentsX gontent plus ennottions plus petures ennottionsX hireted eyli qrphs F F F F F F F F F F F F

F F F F F F F F F F F F F F F F F









UI

UP UQ UQ UR US US UT UU VP VU VW WH WI

S vnguge esouresX gorporD houments nd ennottionsSFI SFP SFQ SFR

WQ

WQ WR WR WR

ContentsSFRFI ennottion hems F F F F F F F F F F F F F F F F F F F F SFRFP ixmples of ennotted houments F F F F F F F F F F F F SFRFQ gretingD iewing nd iditing hiverse ennottion ypes houment pormts F F F F F F F F F F F F F F F F F F F F F F F F F SFSFI heteting the ight eder F F F F F F F F F F F F F F F F SFSFP wv F F F F F F F F F F F F F F F F F F F F F F F F F F F F F SFSFQ rwv F F F F F F F F F F F F F F F F F F F F F F F F F F F F SFSFR qwv F F F F F F F F F F F F F F F F F F F F F F F F F F F F SFSFS lin text F F F F F F F F F F F F F F F F F F F F F F F F F F SFSFT p F F F F F F F F F F F F F F F F F F F F F F F F F F F F F SFSFU imil F F F F F F F F F F F F F F F F F F F F F F F F F F F F SFSFV hp piles nd y0e houments F F F F F F F F F F F F F SFSFW swe ge houments F F F F F F F F F F F F F F F F F F SFSFIH goxvvGsyf houments F F F F F F F F F F F F F F F F F F wv snputGyutput F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

vii

SFS

SFT TFI TFP

WR WT WW WW IHI IHP IIH III III IIP IIQ IIR IIR IIS IIT

T exxsiX xerlyExew snformtion ixtrtion ystemhoument eset F F F F F F F F F F F F F F F F F okeniser F F F F F F F F F F F F F F F F F F F F TFPFI okeniser ules F F F F F F F F F F F F F TFPFP oken ypes F F F F F F F F F F F F F F TFPFQ inglish okeniser F F F F F F F F F F F F TFQ qzetteer F F F F F F F F F F F F F F F F F F F F TFR entene plitter F F F F F F F F F F F F F F F F TFS egix entene plitter F F F F F F F F F F F F TFT rt of peeh gger F F F F F F F F F F F F F TFU emnti gger F F F F F F F F F F F F F F F F TFV yrthogrphi goreferene @yrthowtherA F F TFVFI qei snterfe F F F F F F F F F F F F F TFVFP esoures F F F F F F F F F F F F F F F F TFVFQ roessing F F F F F F F F F F F F F F F F TFW ronominl goreferene F F F F F F F F F F F F TFWFI uoted peeh umodule F F F F F F F TFWFP leonsti st umodule F F F F F F F F TFWFQ ronominl esolution umodule F F TFWFR hetiled hesription of the elgorithm F TFIH e lkEhrough ixmple F F F F F F F F F F F TFIHFI tep I E okenistion F F F F F F F F F F TFIHFP tep P E vist vookup F F F F F F F F F F TFIHFQ tep Q E qrmmr ules F F F F F F F F

IIU

IIV IIW IIW IPH IPI IPI IPQ IPR IPS IPT IPT IPU IPU IPU IPU IPV IPV IPW IPW IQQ IQR IQR IQR

II GATE for Advanced UsersU qei imedded

135IQU

viii

ContentsUFI UFP UFQ UFR uik trt with qei imedded F F F F F F F F F F F F F esoure wngement in qei imedded F F F F F F F F sing giyvi lugins F F F F F F F F F F F F F F F F F F F vnguge esoures F F F F F F F F F F F F F F F F F F F F F F UFRFI qei houments F F F F F F F F F F F F F F F F F F UFRFP peture wps F F F F F F F F F F F F F F F F F F F F F UFRFQ ennottion ets F F F F F F F F F F F F F F F F F F F F UFRFR ennottions F F F F F F F F F F F F F F F F F F F F F F UFRFS qei gorpor F F F F F F F F F F F F F F F F F F F F roessing esoures F F F F F F F F F F F F F F F F F F F F F gontrollers F F F F F F F F F F F F F F F F F F F F F F F F F F F wodelling eltions etween ennottions F F F F F F F F F hupliting esoure F F F F F F F F F F F F F F F F F F F F UFVFI hrle properties F F F F F F F F F F F F F F F F F F ersistent epplitions F F F F F F F F F F F F F F F F F F F F yntologies F F F F F F F F F F F F F F F F F F F F F F F F F F F greting xew ennottion hem F F F F F F F F F F F F F greting xew giyvi esoure F F F F F F F F F F F F F edding upport for xew houment pormt F F F F F F F sing qei imedded in wultithreded invironment F sing qei imedded within pring epplition F F F UFISFI huplition in pring F F F F F F F F F F F F F F F F F UFISFP pring pooling F F F F F F F F F F F F F F F F F F F F F UFISFQ purther reding F F F F F F F F F F F F F F F F F F F F sing qei imedded within omt e epplition UFITFI eommended hiretory truture F F F F F F F F F UFITFP gon(gurtion piles F F F F F F F F F F F F F F F F F F UFITFQ snitiliztion gode F F F F F F F F F F F F F F F F F F qroovy for qei F F F F F F F F F F F F F F F F F F F F F F F UFIUFI qroovy ripting gonsole for qei F F F F F F F F UFIUFP qroovy sripting F F F F F F F F F F F F F F F F F UFIUFQ he riptle gontroller F F F F F F F F F F F F F F UFIUFR tility methods F F F F F F F F F F F F F F F F F F F F ving gon(g ht to gteFxml F F F F F F F F F F F F F F F ennottion merging through the es F F F F F F F F F F F F he veftErnd ide F F F F F F F F F F F F F F F F F F F F VFIFI wthing intire ennottion ypes F F F F F F F VFIFP sing petures nd lues F F F F F F F F F F F F VFIFQ sing wetEroperties F F F F F F F F F F F F F F VFIFR fuilding omplex ptterns from simple ptterns VFIFS wthing imple ext tring F F F F F F F F F VFIFT sing empltes F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IQU IQV IRI IRQ IRQ IRQ IRS IRT IRV ISH ISH ISQ ISS IST ISU ISV ISW ITH ITQ ITR ITT ITW IUH IUI IUP IUP IUP IUQ IUR IUR IUS IUW IVR IVT IVU

UFS UFT UFU UFV UFW UFIH UFII UFIP UFIQ UFIR UFIS

UFIT

UFIU

UFIV UFIW VFI

V teiX egulr ixpressions over ennottions

IVW

IWI IWI IWP IWP IWQ IWS IWS

ContentsVFIFU wultiple tternGetion irs F F F F F F F F F F F VFIFV vr wros F F F F F F F F F F F F F F F F F F F F F VFIFW wultiEgonstrint ttements F F F F F F F F F F F F VFIFIH sing gontext F F F F F F F F F F F F F F F F F F F F VFIFII xegtion F F F F F F F F F F F F F F F F F F F F F F F VFIFIP isping peil ghrters F F F F F F F F F F F F VFP vr ypertors in hetil F F F F F F F F F F F F F F F F F F VFPFI iqulity ypertors F F F F F F F F F F F F F F F F F VFPFP gomprison ypertors F F F F F F F F F F F F F F F VFPFQ egulr ixpression ypertors F F F F F F F F F F F VFPFR gontextul ypertors F F F F F F F F F F F F F F F F VFPFS gustom ypertors F F F F F F F F F F F F F F F F F VFQ he ightErnd ide F F F F F F F F F F F F F F F F F F F F VFQFI e imple ixmple F F F F F F F F F F F F F F F F F VFQFP gopying peture lues from the vr to the r VFQFQ yptionl or impty vels F F F F F F F F F F F F F VFQFR r wros F F F F F F F F F F F F F F F F F F F F F VFR se of riority F F F F F F F F F F F F F F F F F F F F F F F VFS sing hses equentilly F F F F F F F F F F F F F F F F F VFT sing tv gode on the r F F F F F F F F F F F F F F F VFTFI e wore gomplex ixmple F F F F F F F F F F F F F VFTFP edding peture to the houment F F F F F F F F VFTFQ pinding the okens of wthed ennottion F F VFTFR sing xmed floks F F F F F F F F F F F F F F F F VFTFS tv r yverview F F F F F F F F F F F F F F F F F VFU yptimising for peed F F F F F F F F F F F F F F F F F F F F VFV yntology ewre qrmmr rnsdution F F F F F F F F F VFW erilizing tei rnsduer F F F F F F F F F F F F F F F F VFWFI row to erilizec F F F F F F F F F F F F F F F F F F VFWFP row to se the erilized qrmmr pilec F F F F VFIH xotes for wontrel rnsduer sers F F F F F F F F F F F VFII tei lus F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

ix

IWU IWV PHH PHI PHP PHR PHR PHS PHS PHT PHT PHU PHU PHU PHV PHW PHW PIH PIQ PIR PIS PIU PIV PPH PPH PPQ PPR PPR PPS PPS PPS PPT

W exxsgX exxottionsEsnEgontextWFI WFP

WFQ

snstntiting h F F F F F F F F F F F F F F F F F erh qs F F F F F F F F F F F F F F F F F F F F WFPFI yverview F F F F F F F F F F F F F F F F F WFPFP yntx of ueries F F F F F F F F F F F F F WFPFQ op etion F F F F F F F F F F F F F F F F WFPFR gentrl etion F F F F F F F F F F F F F F WFPFS fottom etion F F F F F F F F F F F F F F sing h from qei imedded F F F F F F F WFQFI row to instntite serhledtstore WFQFP row to serh in this dtstore F F F F F

F F F F F F F F F F

F F F F F F F F F F

F F F F F F F F F F

F F F F F F F F F F

F F F F F F F F F F

PPW

PQH PQI PQI PQP PQQ PQR PQS PQS PQS PQT

x

Contents

IH erformne ivlution of vnguge enlysers

IHFI wetris for ivlution in snformtion ixtrtion F F F F F F F F IHFIFI ennottion eltions F F F F F F F F F F F F F F F F F F F F IHFIFP gohen9s upp F F F F F F F F F F F F F F F F F F F F F F F IHFIFQ reisionD ellD pEwesure F F F F F F F F F F F F F F F F IHFIFR wro nd wiro everging F F F F F F F F F F F F F F F F IHFP he ennottion hi' ool F F F F F F F F F F F F F F F F F F F F F IHFPFI erforming ivlution with the ennottion hi' ool F F IHFPFP greting qold tndrd with the ennottion hi' ool IHFQ gorpus ulity essurne F F F F F F F F F F F F F F F F F F F F F IHFQFI hesription of the interfe F F F F F F F F F F F F F F F F F IHFQFP tep y step usge F F F F F F F F F F F F F F F F F F F F F IHFQFQ hetils of the gorpus sttistis tle F F F F F F F F F F F IHFQFR hetils of the houment sttistis tle F F F F F F F F F F IHFQFS qei imedded es for the mesures F F F F F F F F F IHFQFT seXevlXqpr F F F F F F F F F F F F F F F F F F F F F F F F F IHFR gorpus fenhmrk ool F F F F F F F F F F F F F F F F F F F F F F IHFRFI repring the gorpor for se F F F F F F F F F F F F F F F IHFRFP he(ning roperties F F F F F F F F F F F F F F F F F F F F F IHFRFQ unning the ool F F F F F F F F F F F F F F F F F F F F F F IHFRFR he esults F F F F F F F F F F F F F F F F F F F F F F F F F IHFS e lugin gomputing snterEennottor egreement @seeA F F F F F IHFSFI see for glssi(tion F F F F F F F F F F F F F F F F F F F F IHFSFP see por xmed intity ennottion F F F F F F F F F F F F IHFSFQ he fhwEfsed see ores F F F F F F F F F F F F F F F F IHFT e lugin gomputing the fhw ores for n yntology F F F F F IHFU ulity essurne ummriser for emwre F F F F F F F F F F F IIFI yverview F F F F F F F F F F F F F F F IIFIFI petures F F F F F F F F F F F IIFIFP vimittions F F F F F F F F F IIFP qrphil ser snterfe F F F F F F IIFQ gommnd vine snterfe F F F F F F IIFR epplition rogrmming snterfe IIFRFI vogRjFproperties F F F F F F F IIFRFP fenhmrk log formt F F F IIFRFQ inling pro(ling F F F F F F IIFRFR eporting tool F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F







PQW

PRH PRH PRI PRR PRS PRT PRT PRV PSH PSH PSH PSI PSP PSP PSS PST PST PSU PSV PSW PTH PTP PTQ PTR PTS PTT PTW PUH PUH PUH PUI PUP PUP PUQ PUQ PUR

II ro(ling roessing esoures

PTW

IP heveloping qei

IPFI eporting fugs nd equesting petures F F F F F F F F F F F F F F F F F F F F PUU IPFP gontriuting thes F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F PUU IPFQ greting xew lugins F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F PUV

PUU

ContentsIPFQFI ht to gll your lugin F F F F F F IPFQFP riting xew F F F F F F F F F F IPFQFQ riting xew F F F F F F F F F F IPFQFR riting edy wde9 epplition IPFQFS histriuting our xew lugins F F F IPFR pdting this ser quide F F F F F F F F F F IPFRFI fuilding the ser quide F F F F F F F IPFRFP wking ghnges to the ser quide F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

xi

PUV PUV PVP PVS PVS PVT PVU PVV

III CREOLE PluginsIQ qzetteersIQFI sntrodution to qzetteers F F F F F F F F F F F F F F F IQFP exxsi qzetteer F F F F F F F F F F F F F F F F F F F F IQFPFI greting nd wodifying qzetteer vists F F F IQFPFP exxsi qzetteer iditor F F F F F F F F F F F F IQFQ yntoqzetteer F F F F F F F F F F F F F F F F F F F F F F IQFR qze yntology qzetteer iditor F F F F F F F F F F F F IQFRFI he qze qzetteer vist nd wpping iditor IQFRFP he qze yntology iditor F F F F F F F F F F F IQFS rsh qzetteer F F F F F F F F F F F F F F F F F F F F F IQFSFI rerequisites F F F F F F F F F F F F F F F F F F F IQFSFP rmeters F F F F F F F F F F F F F F F F F F F IQFT plexile qzetteer F F F F F F F F F F F F F F F F F F F F IQFU qzetteer vist golletor F F F F F F F F F F F F F F F F IQFV yntooot qzetteer F F F F F F F F F F F F F F F F F F IQFVFI row hoes it orkc F F F F F F F F F F F F F F F IQFVFP snitilistion of yntooot qzetteer F F F F F IQFVFQ imple steps to run yntooot qzetteer F F F IQFW vrge uf qzetteer F F F F F F F F F F F F F F F F F F IQFWFI uik usge overview F F F F F F F F F F F F F F IQFWFP hitionry setup F F F F F F F F F F F F F F F F IQFWFQ edditionl ditionry on(gurtion F F F F F F IQFWFR roessing esoure gon(gurtion F F F F F F IQFWFS untime on(gurtion F F F F F F F F F F F F F IQFWFT emnti inrihment F F F F F F F F F F F F IQFIHhe hred qzetteer for multithreded proessing F IRFI ht wodel for yntologies F F F F F F F F F F F IRFIFI rierrhies of glsses nd estritions IRFIFP snstnes F F F F F F F F F F F F F F F F F IRFIFQ rierrhies of roperties F F F F F F F F IRFIFR ss F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

289PWIPWI PWP PWQ PWQ PWS PWS PWS PWT PWT PWU PWU PWV PWW QHH QHH QHP QHQ QHT QHT QHU QHV QHV QHW QHW QIH

IR orking with yntologies

QII

QIP QIP QIQ QIR QIT

xii

ContentsIRFP yntology ivent wodel F F F F F F F F F F F F F F F F F F F F F F F F F F F IRFPFI ht rppens when esoure is heletedc F F F F F F F F F F F IRFQ he yntology luginX gurrent smplementtion F F F F F F F F F F F F F F IRFQFI he yvswyntology vnguge esoure F F F F F F F F F F F F IRFQFP he gonnetesmeyntology vnguge esoure F F F F F F F F IRFQFQ he greteesmeyntology vnguge esoure F F F F F F F F F IRFQFR he yvswP fkwrdsEgomptile vnguge esoure F F F IRFQFS sing yntology smport wppings F F F F F F F F F F F F F F F F F IRFQFT sing figyvsw F F F F F F F F F F F F F F F F F F F F F F F F F F IRFQFU he sesmegvs ommnd line interfe F F F F F F F F F F F F F F IRFR he yntologyyvswP pluginX kwrdsEomptile implementtion IRFRFI he yvswyntologyv vnguge esoure F F F F F F F F F F IRFS qei yntology iditor F F F F F F F F F F F F F F F F F F F F F F F F F F F IRFT yntology ennottion ool F F F F F F F F F F F F F F F F F F F F F F F F F IRFTFI iewing ennotted ext F F F F F F F F F F F F F F F F F F F F F F IRFTFP iditing ixisting ennottions F F F F F F F F F F F F F F F F F F F IRFTFQ edding xew ennottions F F F F F F F F F F F F F F F F F F F F F F IRFTFR yptions F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IRFU eltion ennottion ool F F F F F F F F F F F F F F F F F F F F F F F F F IRFUFI hesription of the two views F F F F F F F F F F F F F F F F F F F F IRFUFP grete new nnottion nd instne from text seletion F F F F F IRFUFQ grete new nnottion nd dd lel to existing instne from seletion F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IRFUFR grete nd set properties for nnottion reltion F F F F F F F F F IRFUFS helete instneD lel or property F F F F F F F F F F F F F F F F F IRFUFT hi'erenes with ye nd yntology iditor F F F F F F F F F F F F IRFV sing the ontology es F F F F F F F F F F F F F F F F F F F F F F F F F F F IRFW sing the ontology es @old versionA F F F F F F F F F F F F F F F F F F F IRFIHyntologyEewre tei rnsduer F F F F F F F F F F F F F F F F F F F F IRFIIennotting ext with yntologil snformtion F F F F F F F F F F F F F F IRFIPopulting yntologies F F F F F F F F F F F F F F F F F F F F F F F F F F F IRFIQyntology es nd smplementtion ghnges F F F F F F F F F F F F F F F IRFIQFI hi'erenes etween the implementtion plugins F F F F F F F F F IRFIQFP ghnges in the yntology es F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F text F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F QIT QIV QIW QPH QPQ QPR QPR QPR QPS QPT QPU QPU QPW QQR QQR QQR QQU QQU QQV QQW QRH QRH QRH QRI QRI QRI QRQ QRR QRS QRT QRV QRV QRW

IS xonEinglish vnguge upport

ISFI vnguge sdenti(tion F F F F F F F ISFIFI pingerprint qenertion F F F ISFP prenh lugin F F F F F F F F F F F F ISFQ qermn lugin F F F F F F F F F F F ISFR omnin lugin F F F F F F F F F F ISFS eri lugin F F F F F F F F F F F F ISFT ghinese lugin F F F F F F F F F F F ISFTFI ghinese ord egmenttion

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

F F F F F F F F

QSI

QSP QSP QSQ QSQ QSR QSR QSR QSS

Contents

xiii

ISFU rindi lugin F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F QSU

IT homin pei( esoures

ITFI fiomedil upport F F F F F F F F F F F F F F ITFIFI efxi F F F F F F F F F F F F F F F F ITFIFP wetwp F F F F F F F F F F F F F F F ITFIFQ qpell iomedil spelling suggestion ITFIFR fehi F F F F F F F F F F F F F F F ITFIFS winighemGhrug gger F F F F F F F ITFIFT eqene F F F F F F F F F F F F F F F F ITFIFU qixse F F F F F F F F F F F F F F F F ITFIFV enn fiogger F F F F F F F F F F F F ITFIFW wuttionpinder F F F F F F F F F F F F ITFIFIH xormqene F F F F F F F F F F F F F F

F F F F F F F F F F F F F F F F F F F F F F F F F F F nd orretion F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F








QSW

QTH QTH QTI QTQ QTR QTR QTR QTR QTS QTT QTT

IU rsers

IUFI winir rser F F F F F F F F F F F F F F F F F IUFIFI ltform upported F F F F F F F F F F F IUFIFP esoures F F F F F F F F F F F F F F F F IUFIFQ rmeters F F F F F F F F F F F F F F F IUFIFR rerequisites F F F F F F F F F F F F F F F IUFIFS qrmmtil eltionships F F F F F F F IUFP e rser F F F F F F F F F F F F F F F F F F IUFQ vi rser F F F F F F F F F F F F F F F F F IUFQFI equirements F F F F F F F F F F F F F F IUFQFP fuilding vi F F F F F F F F F F F IUFQFQ unning the rser in qei F F F F F IUFQFR iewing the rse ree F F F F F F F F F IUFQFS ystem roperties F F F F F F F F F F F F IUFQFT gon(gurtion piles F F F F F F F F F F F IUFQFU rser nd qrmmr F F F F F F F F F F IUFQFV wpping xmed intities F F F F F F F F IUFQFW pgrding from fughrt to vi F IUFR tnford rser F F F F F F F F F F F F F F F F F IUFRFI snput equirements F F F F F F F F F F F IUFRFP snitiliztion rmeters F F F F F F F F IUFRFQ untime rmeters F F F F F F F F F F

QTW

QTW QUH QUI QUI QUI QUP QUP QUR QUS QUS QUS QUT QUT QUU QUV QUW QUW QVH QVH QVI QVI QVR QVS QVS QVT QVU RHH

IV whine verning

IVFI wv qenerlities F F F F F F F F F F F F F F F F F F F F F F F F F F IVFIFI ome he(nitions F F F F F F F F F F F F F F F F F F F F F IVFIFP qeiEpei( snterprettion of the eove he(nitions IVFP fth verning F F F F F F F F F F F F F F F F F F F F F F F F IVFPFI fth verning gon(gurtion pile ettings F F F F IVFPFP gse tudies for the hree verning ypes F F F F F F F

QVQ

xiv

ContentsIVFPFQ row to se the fth verning in qei heveloper IVFPFR yutput of the fth verning F F F F F F F F F F F F F IVFPFS sing the fth verning from the es F F F F F F F IVFQ whine verning F F F F F F F F F F F F F F F F F F F F F F F F IVFQFI he heei ilement F F F F F F F F F F F F F F F F F F IVFQFP he ixqsxi ilement F F F F F F F F F F F F F F F F F F F IVFQFQ he iue rpper F F F F F F F F F F F F F F F F F F F F IVFQFR he weix rpper F F F F F F F F F F F F F F F F F F IVFQFS he w vight rpper F F F F F F F F F F F F F F F F F F IVFQFT ixmple gon(gurtion pile F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F RHV RHW RIT RIU RIU RIW RIW RPH RPI RPR

IW ools for elignment sks

IWFI sntrodution F F F F F F F F F F F F F F IWFP he ools F F F F F F F F F F F F F F F IWFPFI gompound houment F F F F IWFPFP gompoundhoumentpromml IWFPFQ gompound houment iditor IWFPFR gomposite houment F F F F F IWFPFS heletewemers F F F F F F IWFPFT withwemers F F F F F F IWFPFU ving s wv F F F F F F F F IWFPFV elignment iditor F F F F F F F IWFPFW ving piles nd elignments F IWFPFIH etionEyEetion roessing

F F F F F F F F F F F F









F F F F F F F F F F F F F F F F F F F F F F F F F F F F F






RQQ

RQQ RQQ RQR RQT RQT RQU RQV RQW RQW RQW RRT RRU

PH gomining qei nd swe

PHFI imedding swe ei in qei F F F F F F F F F F F PHFIFI wpping pile pormt F F F F F F F F F F F F F F PHFIFP he swe gomponent hesriptor F F F F F F PHFIFQ sing the enlysisingine F F F F F F F F F PHFP imedding qei gorpusgontroller in swe F F PHFPFI wpping pile pormt F F F F F F F F F F F F F F PHFPFP he qei epplition he(nition F F F F F F F PHFPFQ gon(guring the qeiepplitionennottor F PIFI er qroup ghunker F F F F F F F F F F F F F F PIFP xoun hrse ghunker F F F F F F F F F F F F F PIFPFI hi'erenes from the yriginl F F F F F PIFPFP sing the ghunker F F F F F F F F F F F PIFQ ggerprmework F F F F F F F F F F F F F F F F PIFQFI reegger"wultilingul y gger PIFQFP qixse nd houle uotes F F F F F F PIFR ghemistry gger F F F F F F F F F F F F F F F F PIFRFI sing the gger F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

RRW

RSH RSH RSR RSS RST RST RSU RSV RTP RTP RTP RTP RTQ RTT RTV RTW RTW

PI wore @giyviA lugins

RTI

ContentsPIFS emnt emnti ennottion ervie F F F F F F PIFT vupedi emnti ennottion ervie F F F F F F F PIFU ennotting xumers F F F F F F F F F F F F F F F F PIFUFI xumers in ords nd xumers F F F F F PIFUFP omn xumerls F F F F F F F F F F F F F F PIFV ennotting wesurements F F F F F F F F F F F F F PIFW ennotting nd xormlizing htes F F F F F F F F PIFIHnowll fsed temmers F F F F F F F F F F F F F PIFIHFI elgorithms F F F F F F F F F F F F F F F F F PIFIIqei worphologil enlyzer F F F F F F F F F F PIFIIFI ule pile F F F F F F F F F F F F F F F F F F F PIFIPplexile ixporter F F F F F F F F F F F F F F F F F F PIFIQgon(gurle ixporter F F F F F F F F F F F F F F F PIFIRennottion et rnsfer F F F F F F F F F F F F F F PIFIShem inforer F F F F F F F F F F F F F F F F F F PIFITsnformtion etrievl in qei F F F F F F F F F F PIFITFI sing the s puntionlity in qei F F F PIFITFP sing the s es F F F F F F F F F F F F F F PIFIUesphinx e grwler F F F F F F F F F F F F F F PIFIUFI sing the grwler F F F F F F F F F F F PIFIUFP roxy on(gurtion F F F F F F F F F F F F F PIFIVordxet in qei F F F F F F F F F F F F F F F F F PIFIVFI he ordxet es F F F F F F F F F F F F F PIFIWue E eutomti ueyphrse hetetion F F F F F F PIFIWFI sing the uie ueyphrse ixtrtor9 PIFIWFP sing ue gorpor F F F F F F F F F F F F F PIFPHennottion werging lugin F F F F F F F F F F F F PIFPIgopying ennottions etween houments F F F F PIFPPypenglis lugin F F F F F F F F F F F F F F F F F PIFPQvingipe lugin F F F F F F F F F F F F F F F F F F F PIFPQFI vingipe okenizer F F F F F F F F F F F PIFPQFP vingipe entene plitter F F F F F F F PIFPQFQ vingipe y gger F F F F F F F F F PIFPQFR vingipe xi F F F F F F F F F F F F F PIFPQFS vingipe vnguge sdenti(er F F F F F PIFPRypenxv lugin F F F F F F F F F F F F F F F F F F PIFPRFI snit prmeters nd models F F F F F F F F PIFPRFP ypenxv s F F F F F F F F F F F F F F F PIFPRFQ ytining nd generting models F F F F F PIFPSgontent hetetion sing foilerpipe F F F F F F F F PIFPTsnter ennottor egreement F F F F F F F F F F F F PIFPUhem ennottion iditor F F F F F F F F F F F F F PIFPVgoref ools lugin F F F F F F F F F F F F F F F F F PIFPWumed pormt F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

xv

RTW RUH RUI RUP RUS RUT RUW RVI RVI RVP RVQ RVS RVT RVU RVW RWH RWP RWR RWS RWT RWV RWV SHP SHR SHR SHT SHU SHV SHW SIH SII SII SII SIP SIP SIQ SIR SIR SIT SIT SIU SIV SIV SPP

xvi

ContentsPIFQHwediiki pormt F F F F F F F F F F PIFQIermider term extrtion tools F F PIFQIFI ermnk lnguge resoures PIFQIFP ermnk ore gopier F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F SPP SPQ SPQ SPS

IV The GATE Family: Cloud, MIMIR, TeamwarePP qei gloudPPFI PPFP PPFQ PPFR PPFS qei gloud serviesX n overview F F F F F F F F F F F gomprison with other systems F F F F F F F F F F F F F row to uy servies F F F F F F F F F F F F F F F F F F F F riing nd disounts F F F F F F F F F F F F F F F F F F F ennottion tos on qeigloudFnet F F F F F F F F F F PPFSFI he ennottion ervie ghrges ixplined F F F PPFSFP ennottion to ixeution in hetil F F F F F F F PPFT unning gustom ennottion tos on qeigloudFnet PPFTFI repring our epplitionX he fsis F F F F PPFTFP he qeigloudFnet environment F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

527SPWSQH SQH SQI SQP SQQ SQQ SQR SQR SQS SQS SQW SRI SRI SRQ SRQ SRR SRR SRS SRS SRT SRV SSH

PQ qei emwreX e eEsed gollortive gorpus ennottion ool SQWPQFI sntrodution F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F PQFP equirements for wultiEole gollortive ennottion invironments PQFPFI ypil hivision of vour F F F F F F F F F F F F F F F F F F F F PQFPFP emoteD lle ht torge F F F F F F F F F F F F F F F F F PQFPFQ eutomti nnottion servies F F F F F F F F F F F F F F F F F F PQFPFR ork)ow upport F F F F F F F F F F F F F F F F F F F F F F F F F PQFQ emwreX erhitetureD smplementtionD nd ixmples F F F F F F F PQFQFI ht torge ervie F F F F F F F F F F F F F F F F F F F F F F F PQFQFP ennottion ervies F F F F F F F F F F F F F F F F F F F F F F F PQFQFQ he ixeutive vyer F F F F F F F F F F F F F F F F F F F F F F F PQFQFR he ser snterfes F F F F F F F F F F F F F F F F F F F F F F F F PQFR rtil epplitions F F F F F F F F F F F F F F F F F F F F F F F F F F

PR qei wmir

SSQ

Appendicese ghnge vogeFI ersion UFI @xovemer PHIPA F F F F F F eFIFI xew plugins F F F F F F F F F F F eFIFP virry updtes F F F F F F F F F eFIFQ qei imedded es hnges eFP ersion UFH @perury PHIPA F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

555SSSSSS SSS SST SST SSU

ContentseFPFI wjor new fetures F F F F F F F F F F F eFPFP emovl of depreted funtionlity F F eFPFQ yther enhnements nd ug (xes F F eFQ ersion TFI @epril PHIIA F F F F F F F F F F F F eFQFI xew giyvi lugins F F F F F F F F F eFQFP yther new fetures nd improvements eFR ersion TFH @xovemer PHIHA F F F F F F F F F F eFRFI wjor new fetures F F F F F F F F F F F eFRFP freking hnges F F F F F F F F F F F F eFRFQ yther new fetures nd ug(xes F F F F eFS ersion SFPFI @wy PHIHA F F F F F F F F F F F F eFT ersion SFP @epril PHIHA F F F F F F F F F F F F eFTFI tei nd teiErelted F F F F F F F F eFTFP yther ghnges F F F F F F F F F F F F F eFU ersion SFI @heemer PHHWA F F F F F F F F F F eFUFI xew petures F F F F F F F F F F F F F F eFUFP tei improvements F F F F F F F F F F eFUFQ yther improvements nd ug (xes F F eFV ersion SFH @wy PHHWA F F F F F F F F F F F F F eFVFI wjor xew petures F F F F F F F F F F eFVFP yther xew petures nd smprovements eFVFQ pei( fug pixes F F F F F F F F F F F eFW ersion RFH @tuly PHHUA F F F F F F F F F F F F F eFWFI wjor xew petures F F F F F F F F F F eFWFP yther xew petures nd smprovements eFWFQ fug pixes nd yptimiztions F F F F F eFIH ersion QFI @epril PHHTA F F F F F F F F F F F F eFIHFI wjor xew petures F F F F F F F F F F eFIHFP yther xew petures nd smprovements eFIHFQ fug pixes F F F F F F F F F F F F F F F F eFII tnury PHHS F F F F F F F F F F F F F F F F F F eFIP heemer PHHR F F F F F F F F F F F F F F F F F eFIQ eptemer PHHR F F F F F F F F F F F F F F F F F eFIR ersion Q fet I @eugust PHHRA F F F F F F F F eFIS tuly PHHR F F F F F F F F F F F F F F F F F F F F eFIT tune PHHR F F F F F F F F F F F F F F F F F F F F eFIU epril PHHR F F F F F F F F F F F F F F F F F F F F eFIV wrh PHHR F F F F F F F F F F F F F F F F F F F eFIW ersion PFP ! eugust PHHQ F F F F F F F F F F F eFPH ersion PFI ! perury PHHQ F F F F F F F F F F eFPI tune PHHP F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

xvii

SSU SSV SSV STH STH STH STP STP STP STQ STR STS STS STT STT STU STW SUH SUH SUI SUQ SUR SUR SUR SUT SUV SUW SUW SUW SVI SVP SVP SVQ SVQ SVR SVR SVS SVS SVS SVT SVT

f ersion SFI lugins xme wp

SVW

xviii

Contents

g ysolete giyvi lugins

gFI yntotext tpeg gompiler F F F F F F F F F F F gFP qoogle lugin F F F F F F F F F F F F F F F F F F gFQ hoo lugin F F F F F F F F F F F F F F F F F F gFQFI sing the hoo F F F F F F F F F F F gFR qzetteer isul esoure E qei F F F F F F gFRFI hisply wodes F F F F F F F F F F F F F gFRFP viner he(nition ne F F F F F F F F F gFRFQ viner he(nition oolr F F F F F F F gFRFR ypertions on viner he(nition xodes gFRFS qzetteer vist ne F F F F F F F F F F F gFRFT wpping he(nition ne F F F F F F F F gFS qoogle rnsltor F F F F F F F F F F F F F hFI tterns F F F F F F F F F F F F hFIFI gomponents F F F F F F hFIFP wodelD viewD ontroller hFIFQ snterfes F F F F F F F hFP ixeption rndling F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F


















SWI

SWI SWP SWP SWQ SWQ SWR SWR SWS SWS SWS SWT SWT SWW THH THP THQ THQ

h hesign xotes

SWW

i ent sks for qei

iFI helring the sks F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F iFP he pkgegpp tsk E undling n pplition with its dependenies F F F iFPFI sntrodution F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F iFPFP fsi sge F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F iFPFQ rndling xonElugin esoures F F F F F F F F F F F F F F F F F F F F F iFPFR tremlining your lugins F F F F F F F F F F F F F F F F F F F F F F F F iFPFS fundling ixtr esoures F F F F F F F F F F F F F F F F F F F F F F F F iFQ he expndreoles sk E werging ennottionEhriven gon(g into reoleFxml winFjpe F F F F F F F F F F F (rstFjpe F F F F F F F F F F F F (rstnmeFjpe F F F F F F F F F nmeFjpe F F F F F F F F F F F pFRFI erson F F F F F F F F F pFRFP votion F F F F F F F F pFRFQ yrgniztion F F F F F pFRFR emiguities F F F F F F pFRFS gontextul informtion nmepostFjpe F F F F F F F F dtepreFjpe F F F F F F F F F dteFjpe F F F F F F F F F F F F reldteFjpe F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

THU

THU THU THU THV THW TIP TIP TIR TIS TIT TIU TIU TIU TIU TIV TIV TIV TIV TIW TIW TIW

p xmedEintity tte whine tternspFI pFP pFQ pFR

TIS

pFS pFT pFU pFV

ContentspFW pFIH pFII pFIP pFIQ pFIR pFIS pFIT pFIU pFIV pFIW numerFjpe F F F F ddressFjpe F F F F urlFjpe F F F F F F identi(erFjpe F F F jotitleFjpe F F F F (nlFjpe F F F F F F unknownFjpe F F F nmeontextFjpe orgontextFjpe F loontextFjpe F lenFjpe F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F

I

TIW TPH TPH TPH TPH TPH TPI TPI TPI TPP TPP

q rtEofEpeeh gs used in the repple gger eferenes

TPQ TPS

P

Contents

Part I GATE Basics

Q

Chapter 1 Introductionoftwre doumenttion is like sexX when it is goodD it is veryD very goodY nd when it is dD it is etter thn nothingF @enonymousFA here re two wys of onstruting softwre designX one wy is to mke it so simple tht there re oviously no de(ieniesY the other wy is to mke it so omplited tht there re no ovious de(ieniesF @gFeFF roreA e omputer lnguge is not just wy of getting omputer to perform operE tions ut rther tht it is novel forml medium for expressing ides out methodologyF husD progrms must e written for people to redD nd only iniE dentlly for mhines to exeuteF @he truture nd snterprettion of gomputer rogrmsD rF eelsonD qF ussmn nd tF ussmnD IWVSFA sf you try to mke something eutifulD it is often uglyF sf you try to mke something usefulD it is often eutifulF @ysr ildeA1 qei2 is n infrstruture for developing nd deploying softwre omponents tht proess humn lngugeF st is nerly IS yers old nd is in tive use for ll types of omputtionl tsk involving humn lngugeF qei exels t text nlysis of ll shpes nd sizesF prom lrge orportions to smll strtupsD from multiEmillion reserh onsorti to undergrdute projetsD our user ommunity is the lrgest nd most diverse of ny system of this typeD nd is spred ross ll ut one of the ontinents3 F qei is open soure free softwreY users n otin free support from the user nd developer ommunity vi qeiFFuk or on ommeril sis from our industril prtnersF e re the iggest open soure lnguge proessing projet with development tem more thn doule the size of the lrgest omprle projets @mny of whih re integrted with1 These were, at least, our ideals; of course we didn't completely live up to them. . . 2 If you've read the overview at http://gate.ac.uk/overview.html, you may prefer to skip to Section 1.1. 3 Rumours that we're planning to send several of the development team to Antarctica on one-way ticketsare false, libellous and wishful thinking.

S

T

Introduction

qei4 AF wore thn S million hs een invested in qei development5 Y our ojetive is to mke sure tht this ontinues to e money well spent for ll qei9s usersF he qei fmily of tools hs grown over the yers to inlude desktop lient for developersD work)owEsed we pplitionD tv lirryD n rhiteture nd proessF qei isX an IDED

proessing omponents undled with very widely used snformtion ixtrtion system nd omprehensive set of other plugins for hosted lrgeEsle text proessingD qei gloud @httpXGGgteloudFnetGAF ee lso ghpter PPFa cloud computing solution

qei heveloperX n integrted development environment6 for lnguge

a web appD qei emwreX ollortive nnottion environment for ftoryE style semnti nnottion projets uilt round work)ow engine nd hevilyE optimised kend servie infrstrutureF ee lso ghpter PQF a multi-paradigm search repositoryD

qei wmirD whih n e used to index nd serh over textD nnottionsD semnti shems @ontologiesAD nd semnti metEdt @instne dtAF st llows queries tht ritrrily mix fullEtextD struturlD linguisti nd semnti queries nd tht n sle to terytes of textF ee lso ghpter PRF qei imeddedX n ojet lirry optimised for inlusion in diverse pplitions giving ess to ll the servies used y qei heveloper nd moreFa frameworkD an architecture X

ompositionFa process

highElevel orgnistionl piture of how lnguge proessing softwre

for the retion of roust nd mintinle serviesF

e lso developX wikiGgwD qei iki @httpXGGgtewikiFsfFnetGAD minly to host our own wesites nd s tested for some of our experiments por more informtion on the qei fmily see httpXGGgteFFukGfmilyG nd lso rt s of this ookF yne of our originl motivtions ws to remove the neessity for solving ommon engineering prolems efore doing useful reserhD or reEengineering efore deploying reserh results into pplitionsF gore funtions of qei tke re of the lion9s shre of the engineeringX4 Our philosophy is reuse not reinvention, so we integrate and interoperate with other systems e.g.:LingPipe, OpenNLP, UIMA, and many more specic tools.

5 This is the gure for direct Sheeld-based investment only and therefore an underestimate. 6 GATE Developer and GATE Embedded are bundled, and in older distributions were referred to just as

`GATE'.

Introduction modelling nd persistene of speilised dt strutures

U

mesurementD evlutionD enhmrking @never elieve omputing reserher who hsn9t mesured their results in repetle nd open setting3A visulistion nd editing of nnottionsD ontologiesD prse treesD etF (nite stte trnsdution lnguge for rpid prototyping nd e0ient implementtion of shllow nlysis methods @teiA extrtion of trining instnes for mhine lerning pluggle mhine lerning implementtions @ekD w vightD FFFA yn top of the ore funtions qei inludes omponents for diverse lnguge proessing tsksD eFgF prsersD morphologyD tggingD snformtion etrievl toolsD snformtion ixtrtion omponents for vrious lngugesD nd mny othersF qei heveloper nd imedded re supplied with n snformtion ixtrtion system @exxsiA whih hs een dpted nd evluted very widely @numerous industril systemsD reserh systems evluted in wgD igD egiD hgD slD xgsD etFAF exxsi is often used to rete hp or yv @metdtA for unstrutured ontent @semantic annotationAF qei version I ws written in the midEIWWHsY t the turn of the new millennium we omE pletely rewrote the system in tvY version S ws relesed in tune PHHWY nd version T " in xovemer PHIHF e elieve tht qei is the leding system of its typeD ut s sientists we hve to dvise you not to tke our word for itY tht9s why we9ve mesured our softwre in mny of the ompetitive evlutions over the lst dedeEndEEhlf @wgD igD egiD hg nd moreY see etion IFR for detilsAF e invite you to give it tryD to get involved with the qei ommunityD nd to ontriute to humn lnguge sieneD engineering nd developmentF his ook desries how to use qei to develop lnguge proessing omponentsD test their performne nd deploy them s prts of other pplitionsF sn the rest of this hpterX etion IFI desries the est wy to use this ookY etion IFP rie)y notes tht the ontext of qei is pplied lnguge proessingD or Language EngineeringY etion IFQ gives n overview of developing using qeiY etion IFR lists pulitions desriing qei performne in evlutionsY etion IFS outlines wht is new in the urrent version of qeiY etion IFT lists other pulitions out qeiF

V

Introduction

xoteX if you don9t see the omponent you need in this doumentD or if we mention omE ponent tht you n9t see in the softwreD ontt gteEusersdlistsFsoureforgeFnet7 ! vrious omponents re developed y our ollortorsD who we will e hppy to put you in ontt withF @yften the proess of getting new omponent is s simple s typing the v into qei heveloperY the system will do the restFA

1.1

How to Use this Text

he mteril presented in this ook rnges from the oneptul @eFgF wht is softwre rhiteturec9A to prtil instrutions for progrmmers @eFgF how to del with qei exeptionsA nd linguists @eFgF how to write pttern grmmrAF purthermoreD qei9s highly extensile nture mens tht new funtionlity is onstntly eing dded in the form of new pluginsF smportnt funtionlity is s likely to e loted in plugin s it is to e integrted into the qei oreF his presents something of n orgnistionl hllengeF yur @no dout imperfetA solution is to divide this ook into three prtsF rt s overs instlltionD using the qei heveloper qs nd using exxsiD s well s providing some kground nd theoryF e reommend the new user to egin with rt sF rt ss overs the more dvned of the ore qei funtionlityY the qei imedded es nd tei pttern lnguge mong other thingsF rt sss provides referene for the numerous plugins tht hve een reted for qeiF elthough exxsi provides good strting pointD the user will soon wish to explore other resouresD nd so will need to onsult this prt of the textF e reommend tht rt sss e used s refereneD to e dipped into s neessryF sn rt sssD plugins re grouped into rod res of funtionlityF

1.2

Context

qei n e thought of s oftwre erhiteture for vnguge ingineering gunninghm HHF oftwre erhiteture9 is used rther loosely here to men omputer infrstruture for softE wre developmentD inluding development environments nd frmeworksD s well s the more usul use of the term to denote mroElevel orgnistionl struture for softwre systems hw 8 qrln WTF vnguge ingineering @viA my e de(ned sX F F F the disipline or t of engineering softwre systems tht perform tsks involvE ing proessing humn lngugeF foth the onstrution proess nd its outputs7 Follow the `support' link from http://gate.ac.uk/ to subscribe to the mailing list.

Introductionre mesurle nd preditleF he literture of the (eld reltes to oth ppliE tion of relevnt sienti( results nd ody of prtieF gunninghm WW

W

he relevnt sienti( results in this se re the outputs of gomputtionl vinguistisD xtE url vnguge roessing nd erti(il sntelligene in generlF nlike these other disiplinesD viD s n engineering disiplineD entils predictabilityD oth of the proess of onstruting viE sed softwre nd of the performne of tht softwre fter its ompletion nd deployment in pplitionsF ome working de(nitionsX IF gomputtionl vinguistis @gvAX siene of lnguge tht uses omputtion s n investigtive toolF PF xturl vnguge roessing @xvAX siene of omputtion whose sujet mtE ter is dt strutures nd lgorithms for omputer proessing of humn lngugeF QF vnguge ingineering @viAX uilding xv systems whose ost nd outputs re mesurle nd preditleF RF oftwre erhitetureX mroElevel orgnistionl priniples for fmilies of systemsF sn this ontext is lso used s infrstrutureF SF oftwre erhiteture for vnguge ingineering @eviAX softwre infrstruE tureD rhiteture nd development tools for pplied gvD xv nd viF @yf ourse the prtie of these (elds is roder nd more omplex thn these de(nitionsFA sn the sienti( endevours of xv nd gvD qei9s role is to support experimenttionF sn this ontext qei9s signi(nt fetures inlude support for utomted mesurement @see ghpter IHAD providing level plying (eld9 where results n esily e repeted ross di'erent sites nd environmentsD nd reduing reserh overheds in vrious wysF

1.3

Overview

1.3.1 Developing and Deploying Language Processing Facilitiesqei s n rhiteture suggests tht the elements of softwre systems tht proess nturl lnguge n usefully e roken down into vrious types of omponentD known s resoures8 F8 The terms `resource' and `component' are synonymous in this context. `Resource' is used instead of just`component' because it is a common term in the literature of the eld: cf. Evaluation conference series [LREC-1 98, LREC-2 00]. the Language Resources and

IH

Introduction

gomponents re reusle softwre hunks with wellEde(ned interfesD nd re populr rhiteturl formD used in un9s tv fens nd wirosoft9s FxetD for exmpleF qei omponents re speilised types of tv fenD nd ome in three )voursX

vngugeesoures @vsA represent entities suh s lexionsD orpor or ontologiesY roessingesoures @sA represent entities tht re primrily lgorithmiD suh s prsersD genertors or ngrm modellersY isulesoures @sA represent visulistion nd editing omponents tht prtiipte in qssF

hese de(nitions n e lurred in prtie s neessryF golletivelyD the set of resoures integrted with qei is known s giyviX golletion of iusle yjets for vnguge ingineeringF ell the resoures re pkged s tv erhive @or te9A (lesD plus some wv on(gurtion dtF he te nd wv (les re mde ville to qei y putting them on we serverD or simply pling them in the lol (le speF etion IFQFP introdues qei9s uiltEin resoure setF hen using qei to develop lnguge proessing funtionlity for n pplitionD the developer uses qei heveloper nd qei imedded to onstrut resoures of the three typesF his my involve progrmmingD or the development of vnguge esoures suh s grmmrs tht re used y existing roessing esouresD or mixture of othF qei heveloper is used for visulistion of the dt strutures produed nd onsumed during proessingD nd for deuggingD performne mesurement nd so onF por exmpleD (gure IFI is sreenshot of one of the visulistion toolsF qei heveloper is nlogous to systems like wthemti for wthemtiinsD or tfuilder for tv progrmmersX it provides onvenient grphil environment for reserh nd development of lnguge proessing softwreF hen n pproprite set of resoures hve een developedD they n then e emedded in the trget lient pplition using qei imeddedF qei imedded is supplied s series of te (lesF9 o emed qeiEsed lnguge proessing filities in n pplitionD these te (les re ll tht is neededD long with te (les nd wv on(gurtion (les for the vrious resoures tht mke up the new filitiesF9 The main JAR le (gate.jar) supplies the framework. Built-in resources and various 3rd-party librariesare supplied as separate JARs; for example (guk.jar, the GATE Unicode Kit.) contains Unicode support (e.g. additional input methods for languages not currently supported by the JDK). They are separate because the latter has to be a Java extension with a privileged security prole.

Introduction

II

pigure IFIX yne of qei9s visul resoures

1.3.2 Built-In Componentsqei inludes resoures for ommon vi dt strutures nd lgorithmsD inluding doE umentsD orpor nd vrious nnottion typesD set of lnguge nlysis omponents for snformtion ixtrtion nd rnge of dt visulistion nd editing omponentsF qei supports douments in vriety of formts inluding wvD pD emilD rwvD qwv nd plin textF sn ll ses the formt is nlysed nd onverted into sinE gle uni(ed model of annotationF he nnottion formt is modi(ed form of the sE i formt qrishmn WU whih hs een mde lrgely omptile with the etls formt fird 8 viermn WWD nd uses the now stndrd mehnism of stndEo' mrkup9F qei doumentsD orpor nd nnottions re stored in dtses of vrious sortsD visulised vi the development environmentD nd essed t ode level vi the frmeworkF ee ghpter S for more detils of orpor etF e fmily of roessing esoures for lnguge nlysis is inluded in the shpe of exxsiD e xerlyExew snformtion ixtrtion systemF hese omponents use (nite stte tehniques to implement vrious tsks from tokenistion to semnti tgging or ver phrse hunkingF ell exxsi omponents ommunite exlusively vi qei9s doument nd nnottion resouresF ee ghpter T for more detilsF yther giyvi resoures re desried in rt sssF

IP

Introduction

1.3.3 Additional Facilities in GATE Developer/Embeddedhree other filities in qei deserve speil mentionX teiD tv ennottion tterns ingineD provides regulrEexpression sed ptE ternGtion rules over nnottions ! see ghpter VF he nnottion di'9 tool in the development environment implements performne metris suh s preision nd rell for ompring nnottionsF ypilly lnguge nlysis omponent developer will mrk up some douments y hnd nd then use these long with the di' tool to utomtilly mesure the performne of the omponentsF ee ghpter IHF quD the qei niode uitD (lls in some of the gps in the thu9s10 support for niodeD eFgF y dding input methods for vrious lnguges from rdu to ghineseF ee etion QFIIFP for more detilsF

1.3.4 An Examplehis setion gives very rief exmple of typil use of qei to develop nd deploy lnguge proessing pilities in n pplitionD nd to generte quntittive results for sienti( pulitionF vet9s imgine tht developer lled ptim is uilding n emil lient11 for gyerdyne ystems9 lrge orporte sntrnetF sn this pplition she would like to hve lnguge proessing system tht utomtilly spots the nmes of people in the orportion nd trnsforms them into milto hyperlinksF e little investigtion shows tht qei9s existing omponents n e tilored to this purposeF ptim strts up qei heveloperD nd retes new doument ontining some exmple emilsF he then lods some proessing resoures tht will do nmedEentity reognition @ tokeniserD gzetteer nd semnti tggerAD nd retes n pplition to run these omponents on the doument in sequeneF rving proessed the emilsD she n see the results in one of severl viewers for nnottionsF he qei omponents re deent strtD ut they need to e ltered to del speilly with people from gyerdyne9s personnel dtseF herefore ptim retes new yerE9 versions of the gzetteer nd semnti tgger resouresD using the ootstrp9 toolF his tool retes diretory struture on disk tht hs some tv stu odeD wke(le nd n wv10 JDK: Java Development Kit, Sun Microsystem's Java implementation. Unicode support is being activelyimproved by Sun, but at the time of writing many languages are still unsupported. In fact, Unicode itself doesn't support all languages, e.g. Sylheti; hopefully this will change in time. specic viruses and hadn't heard of Gmail or Thunderbird.

11 Perhaps because Outlook Express trashed her mail folder again, or because she got tired of Microsoft-

Introduction

IQ

on(gurtion (leF efter severl hours struggling with dly written doumenttionD ptim mnges to ompile the stus nd rete te (le ontining the new resouresF he tells qei heveloper the v of these (les12 D nd the system then llows her to lod them in the sme wy tht she loded the uiltEin resoures erlier onF ptim then retes seond opy of the emil doumentD nd uses the nnottion editing filities to mrk up the results tht she would like to see her system produingF he sves this nd the version tht she rn qei on into her seril dtstoreF prom now on she n follow this routineX IF un her pplition on the emil test orpusF PF ghek the performne of the system y running the nnottion di'9 tool to ompre her mnul results with the system9s resultsF his gives her oth perentge ury (gures nd grphil disply of the di'erenes etween the mhine nd humn outputsF QF wke edits to the odeD pttern grmmrs or gzetteer lists in her resouresD nd reompile where neessryF RF ell qei heveloper to reEinitilise the resouresF SF qo to IF o mke the ltertions tht she requiresD ptim reEimplements the exxsi gzetteer so tht it regenertes itself from the lol personnel dtF he then lters the pttern grmmr in the semnti tgger to prioritise reognition of nmes from tht soureF his ltter jo involves lerning the tei lnguge @see ghpter VAD ut s this is sed on regulr expressions it isn9t too di0ultF iventully the system is running nielyD nd her ury is WQ7 @there re still some proE lem sesD eFgF when people use niknmesD ut the performne is good enough for proE dution useAF xow ptim stops using qei heveloper nd works insted on emedding the new omponents in her emil pplition using qei imeddedF his pplition is written in tvD so emedding is very esy13 X the qei te (les re dded to the projet gveerD the new omponents re pled on we serverD nd with little ode to do initilistionD loding of omponents nd so onD the jo is (nished in hlf dy ! the ode to tlk to qei tkes up only round ISH lines of the eventul pplitionD most of whih is just opied from the exmple in the sheffield.examples.StandAloneAnnie lssF feuse ptim is worried out gyerdyne9s unethil poliy of developing kynet to help the lrge orportes of the est strengthen their strngleEhold over the orldD she wnts to get jo s n demi insted @so tht her onsiene will only hve to ope with the12 While developing, she uses a file:/... URL; for deployment she can put them on a web server. 13 Languages other than Java require an additional interface layer, such as JNI, the Java Native Interface,which is in C.

IR

Introduction

torture of studentsD s opposed to humnityAF he tkes the ury mesures tht she hs ttined for her system nd writes pper for the tournl of xsturtium vogrithm snitement desriing the pproh used nd the results otinedF feuse she used qei for developmentD she n ite the repetility of her experiments nd o'er ess to exmple inry versions of her softwre y putting them on n externl we serverF end everyody lived hppily ever fterF

1.4

Some Evaluations

his setion ontins n inomplete list of pulitions desriing systems tht used qei in ompetitive quntittive evlution progrmmesF hese progrmmes hve hd signi(nt impt on the lnguge proessing (eld nd the widespred presene of qei is some mesure of the mturity of the system nd of our understnding of its likely performne on diverse text proessing tsksF

vi

et al.

HUd desries the performne of n wEsed lerning system in the xgsET tent etrievl skF he system hieved the est result on two of three mesures used in the tsk evlutionD nmely the Ereision nd pEmesureF he system oE tined lose to the est result on the remining mesure @eEreisionAFteringF st uses qei for informtion extrtion nd the wwe system to rete sumE mries nd semnti representtions of doumentsF yne system on(gurtion rnked Rth in the e eople erh PHHU evlutionF nents nd the eri plugin ville in qei to produe summries in inglish from mixture of inglish nd eri doumentsF

ggion HU desries rossEsoure oreferene resolution system sed on semnti lusE

ggion HT desries rossElingul summriztion system whih uses wwe ompoE

ypenEhomin uestion ensweringX he niversity of he0eld hs long history

of reserh into openEdomin question nsweringF qei hs formed the E sis of muh of this reserh resulting in systems whih hve rnked highly durE ing independent evlutions sine IWWWF he (rst suessful question nswering system developed t the niversity of he0eld ws evluted s prt of ig V nd used the vsi informtion extrtion system @the forerunner of exxsiA whih ws distriuted with qei rumphreys et al. WWF purther reserh ws reported in ott 8 qizusksF HHD qreenwood et al. HPD qizusks et al. HQD qizusks et al. HR nd qizusks et al. HSF sn PHHR the system ws rnked Wth out of PV prtiipting groupsF inition ptterns mnully implemented in qei s well s lerned tei ptterns

ggion HR desries tehniques for nswering de(nition questionsF he system uses defE

Introduction

IS

indued from orpusF sn PHHRD the system ws rnked Rth in the igGe evluE tionsF

ggion 8 qizusks HR desries multidoument summriztion system impleE

mented using summriztion omponents omptile with qei @the wwe sysE temAF he system ws rnked Pnd in the houment nderstnding ivlution proE grmmesFet al.

wynrd

surprise lnguge progrmF exxsi ws dpted to geuno with four person dys of e'ortD nd hieved n pEmesure of UUFS7F nfortuntelyD ours ws the only system prtiipting3et al.

HQe nd wynrd

et al.

HQd desrie prtiiption in the shi

wynrd

designed for the egi tsk @eutomti gontent ixtrtionAF elthough ompriE son to other prtiipting systems nnot e reveled due to the stipultions of egiD results show VP7EVT7 preision nd rellFet al. et al.

HP nd wynrd

et al.

HQ desrie results otined on systems

rumphreys qizusks

WV desries the vsiEss system used in wgEUF WS desries the vsiEss system used in wgETF

1.5

Recent Changes

his setion detils reent hnges mde to qeiF eppendix e provides omplete hnge logF

1.5.1 Version 7.1 (November 2012)xew pluginshe TermRaider plugin @see etion PIFQIA provides toolkit nd smple pplition for term extrtionF wo new pluginsD Tagger_Zemanta @see etion PIFSA nd Tagger_Lupedia @see etion PIFTA provide s tht wrp online nnottion servies provided y emnt nd yntotextF e new plugin nmed Coref_Tools inludes frmework for fst oEreferene proessingD nd one tht performs orthogrphil oEreferene in the style of the exxsi yrthomtherF ee etion PIFPV for full detilsF e new Congurable Exporter in the ools pluginD llowing nnottions nd fetures to e exported in formts spei(ed y the user @eFgF for use with externl mhine lerning toolsAF ee etion PIFIQ for detilsF

IT

Introduction

upport for reding numer of new doument formts hs lso een ddedX PubMed and the Cochrane Library CoNLL IOB MediaWiki

formts @see etion PIFPWAF

formt @see etion SFSFIHAF

mrkupD oth plin text nd wv dump (les suh s those from ikipedi @see etion PIFQHAF

sn dditionD redyEmde pplitions hve een dded to mny existing plugins @notly the Lang_* nonEinglish lnguge pluginsA to mke it esier to experiment with their sF

virry updtespdted the tnford rser plugin @see etion IUFRA to version PFHFR of the prser itselfD nd dded runEtime prmeters to the to ontrol the prser9s dependeny optionsF he wesurement nd xumer tggers hve een upgrded to use teiC insted of teiF his should result in fster proessingD nd lso llows for more memory e0ient duplition of instnesD iFeF when pool of pplitions is retedF he ypenxv plugin hs een ompletely revised to use ephe ypenxv IFSFP nd the orresponding set of modelsF ee etion PIFPR for detilsF he ntive lunher for qei on w y now works with yrle tv U s well s epple tv TF

qei imedded es hngesome of the most signi(nt hnges in this version re under the onnet in qei imE eddedX he lss loding rhiteture underlying the loding of plugins nd the genertion of ode from tei grmmrs hs een reEworkedF he new version llows for the omplete unloding of plugins nd for etter memory hndling of generted lssesF hi'erent plugins n now lso use di'erent versions of the sme Qrd prty lirriesF here hve lso een numer of hnges to the wy plugins re @unAloded whih should provide for more onsistent ehviourF he qei wv formt hs een updted to hndle more vlue types @essentilly every dt type supported y trem @httpXGGxstremFodehusForgGfqFhtmlA should e usle s feture nme or vlueF piles in the new formt n e opened without error y older qei versionsD ut the dt for the previouslyEunsupported types will e interpreted s tringD ontining n wv frgmentF

Introduction

IU

he s de(ned in the exxsi plugin re now desried y nnottions on the tv lsses rther thn expliitly inside reoleFxmlF he min reson for this hnge is to enle the de(nitions to e inherited to ny sulsses of these sF greting n empty sulss is ommon wy of providing with di'erent set of defult prmeters @this is used extensively in the lnguge plugins to provide ustom gzetteers nd nmed entity trnsduersAF his hs the dded ene(t of ensuring tht new fetures lso utomtilly perolte down to these sulssesF sf you hve developed your own tht extends one of the exxsi ones you my (nd it hs quired new prmeters tht were not there previouslyD you my need to use the driddengreolermeter nnottion to suppress themF he orpus prmeter of vngugeenlyser @n interfe mostD if not llD s impleE mentA is now nnotted s dyptionl s most implementtions do not tully require the prmeter to e setF hen sving n pplition the plugins re now sved in the sme order in whih they were originlly loded into qeiF his ensures tht dependenies etween plugins re orretly mintined when pplitions re restoredF es support for working with reltions etween nnottions ws ddedF ee etion UFU for more detilsF he method of populting orpus from single (le hs een updted to llow ny mime type to e used when reting the new doumentsF end numerous smller ug (xes nd performne improvementsF F F

1.6

Further Reading

vots of doumenttion lives on the qei we siteD inludingX qei online tutorilsY the min system doumenttion treeY tvho es doumenttionY rwv of the soure odeY omprehensive list of qei pluginsF por more detils out he0eld niversity9s work in humn lnguge proessing see the xv group pges or A Denition and Short History of Language Engineering @gunninghm WWAF por more detils out snformtion ixtrtion see IE, a User Guide or the qei si pgesF

IV

Introduction

e list of pulitions on qei nd projets tht use it @some of whih re ville onEline from httpXGGgteFFukGgteGdoGppersFhtmlAX

PHIH fonthevet al.

ronmentD emphsising the di'erent roles tht users ply in the orpus nnottion proE essF guge interfesF here is other relted work y hmljnoviD egtonoviD nd gunE ninghm on using qei to uild nturl lnguge interfes for quering ontologiesF @rindi nd qujrtiAF

IH desries the emwre weEsed ollortive nnottion enviE

hmljnovi IH presents the use of qei in the development of ontrolled nturl lnE

eswni 8 qizusks IH disusses the use of qei to proess outh esin lnguges PHHW ggion 8 punk HW fouses in detil on the use of qei for mining opinions nd ftsfor usiness intelligene gthering from we ontentF qeiF

eswni 8 qizusks HW presents in more detil the text lignment omponent of fonthevet al. HW is the rumn vnguge ehnologies9 hpter of emnti unowlE edge wngement9 @tohn hviesD wrko qroelnik nd hunj wldeni edsFA et al.

hmljnovi

s prt of the ey reserh projetF

HW disusses the use of semnti nnottion for softwre engineeringD

vlvik 8 wynrd HW reviews the urrent stte of the rt in emil proessing nd

ommunition reserhD fousing on the roles plyed y emil in informtion mngeE mentD nd ommeril nd reserh e'orts to integrte semntiEsed pproh to emilF ing tsksF pirstlyD n w with uneven mrgins @wwA is proposed to del with the prolem of imlned trining dtF eondlyD w tive lerning is employed in order to llevite the di0ulty in otining lelled trining dtF he lgorithms re presented nd evluted on severl snformtion ixtrtion @siA tsksF

vi

et al.

HW investigtes two tehniques for mking ws more suitle for lnguge lernE

PHHV egtonoviet al. HV presents our pproh to utomti ptent enrihmentD tested in lrgeEsleD prllel experiments on y nd iy doumentsF

Introduction

IW

hmljnovi

et al.

for querying ontologies using unonstrined lngugeEsed queriesF

HV presents uestionEsed snterfe to yntologies @uestsyA E tool

hmljnovi 8 fonthev HV presents semntiEsed prototype tht is mde for

n openEsoure softwre engineering projet with the gol of exploring methods for ssisting openEsoure developers nd softwre users to lern nd mintin the system without mjor e'ortFet al.

hell lle

HV presents erviepinderF

vi 8 gunninghm HV desries our wEsed system nd severl tehniques we deE

veloped suessfully to dpt w for the spei( fetures of the pEterm ptent lsE si(tion tskF mehnis methods for informtion retrievl nd nturl lnguge proessingF exmines the extent to whih they re redy for use in the rel worldF

vi 8 fonthev HV reviews the reent developments in pplying geometri nd quntum wynrd HV investigtes the stte of the rt in utomti textul nnottion toolsD nd wynrdet al. HV disusses methods of mesuring the performne of ontologyEsed informtion extrtion systemsD fousing prtiulrly on the flned histne wetri @fhwAD new metri we hve proposed whih ims to tke into ount the more )exile nture of ontologillyEsed pplitionsF et al. HV investigtes xv tehniques for ontology popultionD using omE intion of ruleEsed pprohes nd mhine lerningF et al.

wynrd ln

essing strutured informtionD tht is domin independent nd esy to use without triningF

HV presents the uestsy system nturl lnguge interfe for E

PHHU punk punket al.

informtion extrtionFet al.

HU desries n ontologilly sed pproh to multiEsoureD multilingul HU presents ontrolled lnguge for ontology editing nd softwre imE

plementtionD sed prtly on stndrd xv toolsD for proessing tht lnguge nd mnipulting n ontologyFet al.

wynrd

indued y hnges to the ontologiesD nd @PA the evolution of the ontology indued y hnges to the underlying metdtF

HU proposes methodology to pture @IA the evolution of metdt

PH

Introductionet al.

wynrd

min ontologiesD whih enles the extrtion of relevnt informtion to e fed into models for nlysis of (nnil nd opertionl risk nd other usiness intelligene pplitions suh s ompny intelligeneD y mens of the fv stndrdF PHHUF yur rossEdoument oreferene system uses n inEhouse gglomertive lustering implementtion to group douments referring to the sme entityFet al.

HU desries the development of system for ontent mining using doE

ggion HU desries experiments for the rossEdoument oreferene tsk in emivl

ggion

the ontext of prtil eEusiness pplition for the i wsxq rojet where the gol is to gther interntionl ompny intelligene nd ountryGregion informtionF ontology s n essentil prt of the extrtion proessD y tking into ount the reltions etween oneptsF

HU desries the pplition of ontologyEsed extrtion nd merging in

vi

et al.

HU introdues hierrhil lerning pproh for siD whih uses the trget

vi

et al.

tion lelsD whih n e seen s the lel reltion sensitive version of importnt mesures suh s verged preision nd pEmesureD nd presents the results of pplyE ing the new evlution mesures to ll sumitted runs for the xgsET pEterm ptent lssi(tion tskFet al.

HU proposes some new evlution mesures sed on reltions mong lssi(E

vi vi

HU desries the lgorithms nd linguisti fetures used in our prtiipting system for the opinion nlysis pilot tsk t xgsETFproh for the spei(s of the pEterm ptent lssi(tion sutsk t xgsET tent etrievl skF

et al.

HUd desries our wEsed system nd the tehniques we used to dpt the pE

vi 8 hweEylor HU studies tpneseEinglish rossElnguge ptent retrievl using

uernel gnonil gorreltion enlysis @uggeAD method of orrelting liner relE tionships etween two vriles in kernel de(ned feture spesF

PHHT eswniet al. HT @roeedings of the Sth snterntionl emnti e gonferene @sgPHHTAA sn this pper the prolem of dismiguting uthor instnes in onE tology is ddressedF e desrie weEsed pproh tht uses vrious fetures suh s pulition titlesD strtD initils nd oEuthorship informtionF et al. HT emnti ennottion nd rumn vnguge ehnology9D ontriE ution to emnti e ehnologyX rends nd eserh9 @hviesD tuder nd rE renD edsFA et al. HT emnti snformtion eess9D ontriution to emnti e ehnologyX rends nd eserh9 @hviesD tuder nd rrenD edsFA

fonthev

fonthev

Introduction

PI

fonthev 8 ou HT presents n ontology lerning pproh tht IA exploits rnge

of informtion soures ssoited with softwre projets nd PA relies on tehniques tht re portle ross pplition dominsF

hvis

et al. HT desries work in progress onerning the pplition of gontrolled vnE guge snformtion ixtrtion E gvsi to ersonl emnti iki E emperE ikiD the gol eing to permit users who hve no speilist knowledge in ontology tools or lnguges to semiEutomtilly nnotte their respetive personl iki pgesF

vi 8 hweEylor HT studies mhine lerning lgorithm sed on ugge for rossE

lnguge informtion retrievlF he lgorithm is pplied to tpneseEinglish rossE lnguge informtion retrievlF

wynrd

et al. HT disusses existing evlution metrisD nd proposes new method for evluting the ontology popultion tskD whih is generl enough to e used in vriety of situtionD yet more preise thn mny urrent metrisF et al.

ln

simply y using restrited version of the inglish lngugeF he ontrolled lnguge desried is sed on n open voulry nd restrited set of grmmtil onE strutsF

HT desries n pproh tht llows users to rete nd edit ontologies

ln ng

et al. HT desries the retion of linguisti nlysis nd orpus serh tools for umerinD s prt of the development of the igvF et al. HT proposes n w sed pproh to hierrhil reltion extrtionD using fetures derived utomtilly from numer of qeiEsed openEsoure lnguge proessing toolsF

PHHS eswniet al. HS @roeedings of pifth snterntionl gonferene on eent edvnes in xturl vnguge roessing @exvPHHSAA st is fullEfetured nnottion indexing nd serh engineD developed s prt of the qeiF st is powered with ephe vuene tehnology nd indexes vriety of douments supported y the qeiF

fonthev HS presents the yxyw system whih uses xturl vnguge qenertion@xvqA tehniques to produe textul summries from emnti e ontologiesF of the inylopedi of vnguge nd vinguistisF

gunninghm HS is n overview of the (eld of snformtion ixtrtion for the Pnd idition gunninghm 8 fonthev HS is n overview of the (eld of oftwre erhiteture forvnguge ingineering for the Pnd idition of the inylopedi of vnguge nd vinE guistisF

PP

Introductionet al.

howmn howmn howmn

use mteril from the snternet to ugment television news rodstsFet al.

HS @iuro sntertive elevision gonferene perA e system whih n HS @orld ide e gonferene perA he e is used to ssist the HS @eond iuropen emnti e gonferene perA e system tht

nnottion nd indexing of rodst newsFet al.

semntilly nnottes television news rodsts using news wesites s resoure to id in the nnottion proessF sed si system whih uses the w with uneven mrgins s lerning omponent nd the qei s xv proessing moduleF

vi

et al.

HS @roeedings of he0eld whine verning orkshopA desrie n w

vi

et al.

verning @goxvvEPHHSAA uses the uneven mrgins versions of two populr lerning lgorithms w nd ereptron for si to del with the imlned lssi(tion proE lems derived from siFet al.

HS @roeedings of xinth gonferene on gomputtionl xturl vnguge

vi

@ighnEHSAA system for ghinese word segmenttion sed on ereptron lerningD simpleD fst nd e'etive lerning lgorithmFet al.

HS @roeedings of pourth sqrex orkshop on ghinese vnguge proessing

oljnr

priendly yntology euthoring sing gontrolled vngugeF ogrphil summries from multiple doumentsFet al.

HS @niversity of he0eldEeserh wemorndum gEHSEIHA serE

ggion 8 qizusks HS desries experiments on ontent seletion for produing iE rsu HS @roeedings of the Pnd iuropen orkshop on the sntegrtion of unowlE

edgeD emnti nd higitl wedi ehnologies @isw PHHSAAhigitl wedi reserE vtion nd eess through emntilly inhned eEennottionF

ng

et al. HS @roeedings of the PHHS siiiGsgGegw snterntionl gonferene on e sntelligene @s PHHSAA ixtrting homin yntology from vinguisti esoure fsed on eltedness wesurementsF

PHHR fonthev HR @vig PHHRA desries lexil nd ontologil resoures in qei used forxturl vnguge qenertionFet al.

fonthev

HR @txviA disusses developments in qei in the erly nughtiesF

gunninghm 8 ott HR @txviA is the introdution to the ove olletionF gunninghm 8 ott HR @txviA is olletion of ppers overing mny importntres of oftwre erhiteture for vnguge ingineeringF

Introduction

PQ

himitrov viet al.

et al.

oreferene resolutionF

HR @enphor roessingA gives lightweight method for nmed entity

HR @whine verning orkshop PHHRA desries n w sed lerning lgoE rithm for si using qeiFet al.

wynrd

gzetteer lists from multiElnguge dtFet al.

HR @vig PHHRA presents lgorithms for the utomti indution of HR @i PHHRA disusses ontologyEsed si in the hehight projetF

wynrd wynrd

et al. HR @eswe PHHRA presents utomti retion nd monitoring of seE mnti metdt in dynmi knowledge portlF

ggion 8 qizus

Documents

Tao