Download pdf - CEOS Open Search Developer Guideceos.org/.../02/CEOS-Open-Search-Developer-Guide.pdf · title="This parameter follows the elastic search free text search implementations" />

1

CEOSOpenSearchDeveloperGuide

AbstractCEOShasdevelopedasetofconventionsaroundtheATOMsyndicationspecificationandOpenSearch‘query-response’specificationtoenablelightweightmechanismsofdatadiscoveryandaccess(CEOSOpenSearchBestPractices).Theresultisadatadiscoveryframeworkthatis,

• Lightweightandsimple• Standards-based• REST-like• Lowentrycost

ThepurposeofthisdocumentistoprovidedevelopmentguidelinestobothclientandserverimplementersthatachievethefollowingwithrespecttoanyEarthDataOpenSearchimplementation,

• PromotetheuseoftheOpenSearchstandardasameansofdatadiscoveryforEarthDataproviders

• DefinetheexpectationsandrequirementsofcandidateOpenSearchimplementations

• Removeambiguityinimplementationwherepossible• FacilitatetheaggregationofresultsbetweendisparateEarthDataproviders

viaOpenSearchcommonstandards• AllowforclientstoaccesssearchenginesviaanOpenSearchDescriptor

Document(OSDD)withnoaprioriknowledgeoftheinterface• FacilitatesmoothintegrationbetweenrelatedOpenSearchimplementations,

suchasacollectionresourcethatreferstogranuleresourcecollectionsfromanotherprovider(InternationalDirectoryNetwork(IDN)andtheCEOSWGISSIntegratedCatalog(CWIC)isagoodexample)

TableofcontentsCEOSOpenSearchDeveloperGuide...................................................................................................1Abstract.....................................................................................................................................................1Tableofcontents...................................................................................................................................1Introduction............................................................................................................................................2Specificationandextensionadherence.......................................................................................3TheOpenSearchDescriptorDocument.......................................................................................3AnatomyofanOSDD–theserverstory.................................................................................3AnatomyofanOSDD–theclientstory...................................................................................7

TheSearchRequest..............................................................................................................................7AnatomyofaSearchRequest–theserverstory................................................................7QueryParameters............................................................................................................................8

2

Resultsetnavigationparameters..........................................................................................10AnatomyofaSearchRequest–theclientstory..............................................................11Metricsparameters......................................................................................................................11FullExampleofa‘developerguide’compliantOpenSearchRequest....................11

TheSearchResponse.......................................................................................................................12Anatomyofaresponse–theserverstory..........................................................................12Anatomyofaresponse–theclientstory...........................................................................15Resultordering–theserverstory.........................................................................................16Resultordering–theclientstory..........................................................................................16Handlingerrors–theserverstory........................................................................................17

Two-stepSearching..........................................................................................................................17Theserverstory............................................................................................................................17Theclientstory..............................................................................................................................18

AppendixA–SummaryTable......................................................................................................20AppendixB–Examples...................................................................................................................21OSDDwithparameterextension............................................................................................21OSDDwithoutparameterextension.....................................................................................23ATOMresult....................................................................................................................................24

IntroductionTheA9searchtechnologycompany,asubsidiaryofAmazon,originallydevelopedtheOpenSearchspecification,whichwasreleasedtothegeneralcommunityin2005,andadoptedinavarietyofsettings.TheOpenSearchconventioniscurrentlymaintainedonwww.opensearch.org,asiteprovidedbyA9tothecommunity.AnOpenSearchDescriptorDocument(OSDD)servedbyanOpenSearchimplementationdefinesadiscoveryserviceintermsof,

• HTTPGETqueryURL,definingsimplekeyword/valuepairqueryparameters• Resultformatsitsupports(ATOM,RSS,HTML)

Giventhatdocument,aclientcanexecuteaqueryviaasimpleHTTPGETinvocation.Theserverwillrespondwithanavigableresultsetcontainingreferencestopertinentinventoryintermsof

• Spatialextent• Temporalextent• Metadata,dataanddocumentationURLs• Linkstofurthersearches(possiblyrepresentedbyadditionalOSDDs)

Clientsthatisassimpleasawebbrowserorascomplexasanaggregatedportalimplementationcanusetheseresults.

3

Figure1:SearchOverview

SpecificationandextensionadherenceThisguideregardsthefollowingspecificationasmandatorywhenimplementingOpenSearch,

• CEOSOpenSearchBestPractices

TheOpenSearchDescriptorDocumentTheprimarypurposeofanOpenSearchDescriptorDocument(OSDD)istodescribetoaclient(machineorhuman)howtosearchaninventoryandwhatresultstoexpect.Searchexpectationsaredefinedintermsofprotocol,endpointandqueryparameters.Resultexpectationsaredefinedintermsofformatandcontent.AugmentinganOSDDwithparameterelementsasperDraft2oftheparameterextensiongivesustheabilityto

1. Increasethespecificityofsearchparameters.2. Allowaclient,intheory,toprogrammaticallyconstructauserinterfacefor

anarbitraryOpenSearchimplementationon-the-flysolelyfromanOSDD.Bothoftheseabilitiesarecrucialtooneofourgoals,theabilitytoaggregatediscoveryacrossmultiple,disparateproviders.

AnatomyofanOSDD–theserverstoryTheykeyelementofanOSDDistheURLelement.Takenasawhole,includinganychildparameterelements,theentiretyofanOpenSearchimplementationforacollectionofresourcescanbedescribedtoaclient.Forexample,<Url type="application/atom+xml" type=”collection” template="http://foo.gov/opensearch/datasets.atom?q={searchTerms?}”/>

4

Tellsaclientthataninventoryof‘dataset’resourcescanbesearchedfromtheendpoint‘foo.gov/opensearch’usingthehttpprotocolandthattheresultscanbefilteredbytheoptionalqueryparameter‘q’.Bindingtheparameter‘q’tothe‘searchTerms’descriptor,wherethenamespace‘os’referstotheOpenSearchspecificationnamespace,conveysmeaning.Inthiscase,theOpenSearchspecificationdefinesthismeaningas‘keywordorkeywordsdesiredbythesearchclient’PlaceholdersforoptionalsearchparametersinURLtemplatesmustincludeaquestionmarkcharacter‘?’attheendofthesearchterm.Toincreaseinteroperabilityitisrecommendedthatyoumakeasmanyofyoursearchparametersoptionalaspossible.Thetypeattributedefinestheformatofresultsreturnedbyanyqueryagainstthisresourcecollection.Inthiscase,theresultsformatisATOM.Giventheexampleabovethefollowingrequestsarevalid,http://foo.gov/opensearch/datasets.atom http://foo.gov/opensearch/datasets.atom?q=foo Giventhisexample, <Url type="application/atom+xml" template="http://foo.gov/opensearch/datasets.atom?q={searchTerms}”/> ThefollowingrequestisNOTvalid, http://foo.gov/opensearch/datasets.atom Therequiredparameter“q”andattendantkeywordsarenotincluded. OpenSearchserverdevelopersshouldprovideanOpenSearchimplementationthatsupportstheATOMresultsformatthatmayberequestedviaHTTPcontentnegotiationorresourceextension.Notethatintheaboveexamplesa‘dataset’resourceisinferredbytheresourceURL.WecouldformalizethisbindingbyusingarelationattributeintheURLelementoftheOSDD.Forexample,<Url type="application/atom+xml" rel=”collection” template="http://foo.gov/opensearch/datasets.atom?q={searchTerms}”/> OpenSearchserverdevelopersshoulddescribetheresourcecollectionendpointbyarelationattribute.Thesuggestedvaluesoftherelationattributecanbe‘collection’or‘results’(granules).

5

Wecanincreasethespecificityofaqueryparameterbyleveragingtheparameterextension.Forexample,anOpenSearchAPIthatcanonlyreturnamaximumof2000resultscanbedescribedasfollows,<Parameter name="count" value="{count}" minimum="0" maxInclusive="2000"/> OpenSearchserverdevelopersshoulduse‘parameter’elementsaspertheOpenSearchparameterextensiontodescribetheirsearch-engine-specificsearchparameters.AclientmaywishtogenerateauserinterfaceatruntimebyparsingtheOSDD.Leveragingtheparameterextensioncomesclose1toallowingustoachievethis.Forexample,thefollowingparameterdefinitionmayinstructaUItoprovideabrowseradiowidgetbyvirtueofthevalue‘{browse}’whichdescribesaBooleanproperty.Thatwidgetwouldhaveadisplaynameof‘Browse?’andwillbedeemedoptionalbyvirtueoftheminimumvalueofone.<Parameter name="browse" uiDisplay= “Browse required?”

value="{foo:browse}" title="inventory which has a browse image" minimum=”1” maximum=”1”>

<Option value="true" label="Yes"/> <Option value="false" label="No"/> </Parameter>

OpenSearchserverdevelopersshouldusename,value,titleanduiDisplay1attributesfortheirnon-standard2parameterdefinitions.SomecommonsearchparametershavecharacteristicsthataredifficulttocaptureusingtheparameterextensionasitisinDraft2.Freetextsearching,forexample,maysupportquerysyntax.Giventhatthereisnowidelyadoptedfreetextquerysyntaxstandardandthatmostimplementationswillprovidethissupportthrough3rdpartyproductslikeLuceneorElasticSearch,weareproposinganaugmentationtotheparameterextensionthatwillallowaparametertoexhibitaprofileanalogousto‘myfreetextsearchquerylanguagebehaveslikeLucene’

Forexample, <Parameter name="keyword"

1WewillpetitionOpenSearchtoaddthe‘uiDisplay’attributetotheparameterelement2Bynon-standardwemeanparametersotherthanthosedefinedintheOpenSearchspecificationsandextensions

6

value="{searchTerms}" title="inventory containing all the specified keywords separated

by space, case-insensitive, wildcards are supported" minimum="0"> <link rel="profile" href="http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html" title="This parameter follows the elastic search free text search implementations" />

</Parameter> OpenSearchserverdevelopersshouldsupplyanimplementationwithfreetextsearchcapabilitiesanddefinethepropertiesofthatcapabilityviaanATOMlinkreferringtoaprofile. ThegeometryparameterasdefinedinthegeoextensionallowsfortherepresentationofaspatialconstraintviaWellKnownText(WKT).ThisWKTstringmayrepresentanumberofdifferentgeometriessuchaspoint,lineorpolygon.ItislikelythatanOpenSearchimplementationonlysupportsasubsetofthesegeometries.Weareproposinganaugmentationtotheparameterextensionthatwillallowaparametertoexhibitanumberofprofilesanalogousto‘mygeometrysearchparameterssupportspoint,lineandpolygon’Forexample,<params:Parameter name="geometry"

value="{geo:geometry}" title="inventory which has a spatial extent overlapping this geometry" minimum="0"> <atom:link rel="profile" href="http://www.opengis.net/wkt/LINESTRING" title="This service accepts WKT LineStrings"/>

<atom:link rel="profile" href="http://www.opengis.net/wkt/POINT" title="This service accepts WKT Points"/> <atom:link rel="profile" href="http://www.opengis.net/wkt/POLYGON" title="This service accepts WKT Polygons"/>

</params:Parameter> OpenSearchserverdevelopersshouldsupplyanimplementationwithgeometrysearchcapabilitiesdefineanenumerationofsupportedgeometrytypesviaATOMlinksreferringtogeometrytypeprofiles.

YoumaywishtotrackclientusagethroughOpenSearchimplementationsformetricspurposes.YoucandothisbypassingaclientIDfromimplementationtoimplementation.

7

OpenSearchserverdevelopersshouldexposeanOSDDviatheprovisionofaclient-suppliedidentifierInthesearchsectionofthisdocumentwetalkaboutmetricsrelatedtoanOpenSearchimplementation.AkeyelementofmetricssupportistheabilitytoidentifyaclientandtheirusageofyourAPIinsomemanner.Tofacilitatethiswehaveintroducedtheconceptofanon-mandatoryclientidparameter.Inordertominimizetheobtrusivenessontheclientofaclientidwesuggestthefollowing,

1. ExposeyourOSDDviaanOpenSearchhtmllandingpage.2. YourOSDDisdynamicallygeneratedbasedonthesubmissionofanhtml

formonthatpagethathasasingle,mandatory‘clientId’parameter.3. SubmittingtheformwillreturnanOSDDwiththatclientIDembeddedinthe

URLtemplateattribute.Forexampleinvoking,HTTP GET foo.gov/datasets/openSearchDescriptorDocument.xml?clientId=foo

WouldreturnanOSDDcontaining,<Url type="application/atom+xml" template="http://foo.gov/opensearch/datasets.html?clientId=foo”/> Consequently,ifaclientabidesbythecontractoftheURLtemplate,allqueriestoyourAPIwillcontaintheclientIdparameter.

AnatomyofanOSDD–theclientstoryOpenSearchclientdevelopersshouldaccessanOSDDusinganidentifieriftheserverprovidessuchaninterface OpenSearchclientdevelopersshouldleverageasmuchinformationprovidedintheOSDDaspossiblewheninteractingwithanOpenSearchimplementation

TheSearchRequestAnOpenSearchrequestisaHTTPrequest(normally‘GET’butthiscanbedefinedwithinanOSDD)directedtoaspecificcollectionofresources.Thoseresourcesnormallydescribedatasetsorgranulesinearthdatadiscovery.

AnatomyofaSearchRequest–theserverstory

8

Figure2:OpenSearchqueryURL

Thesetofresourcesreturnedcanbeconstrainedandtraversedbyattachingparameterstoyourrequest.TherequestedformatofyourresultscanbedefinedintherequestbycontentnegotiationviatheHTTP‘accept’headerorusingaresourceextension.Exampleofformatrequestbyresourceextension,https://foo.gov/opensearch/datasets.atomExampleofformatrequestbyHTTP‘accept’header,Accept: application/xml+atom: https://foo.gov/opensearch/datasets

QueryParametersWerecommendtheuseofkeyword,spatialandtemporalconstraintsforyoursearchimplementation.Youmaydefineothersearchparametersbuttheabove3havebeenfoundtoaccountforover90%ofuserqueriesinarecentstudyconductedbyNASA’sECHOsystem.ItisexpectedthatmultiplequeryparameterswillbeAND’dtogethertoformaquery.

SearchTerms–OpenSearchSpecificationThesearchTermsparameterisexpectedtobeasimplesetofkeywordstobeusedinafreetextsearch.Thiscontrastssomewhatwiththetraditionalmethodofsearching(e.g.,withinEOSDISVersion0),whichemphasizedstructuredsearchesbasedonspecificattributeslikesatelliteandinstrument.However,freetextsearchissimplertoimplementattheclientendandmoreuniversal,notrelyingonacommonschema,andwithjudicioususeofacronymsbytheclientoruser(e.g.,“MODIS”)canbenearlyasprecise.NotethatsearchTermsareanimportantdiscriminatorfordatacollections;however,theyareoftennotusefulforfileswithinadatacollection,wherethediscriminatorismorelikelytobespaceortimecoordinates(see‘two-stepsearching)ofthefiles.ThussearchTermsisnotarequiredparameter;ratheritistheserverthatspecifiesinatemplatewhethertheyarerequiredforthatparticularsearchtype.Keywordconstraintsarekeytodiscoveringtheappropriatedatacollectionsforauser.

9

Notethatifyourkeywordsearchalgorithmdoesnotconformtoanyprofile(LuceneorElasticSearchforexample)thenthefollowingisexpected:WhenmultiplesearchTermsareincluded,theyshouldbeseparatedby‘+’,e.g.,“MODIS+fires”.ImplementationsarerequiredtotreatthiscombinationasaBoolean“AND”operationbydefault.Youshouldspecifymulti-wordphrasesbysettingdoublequotes(“)aroundthephrase.Inthiscase,animplementationmusttreatthisasaphrasesearch.OpenSearchserverdevelopersshould,ifappropriate,filterresultsbyafreetextkeywordconstraintasspecifiedintheOpenSearchspecificationSpatialandtemporalconstraintsnarrowdownaresultsettoauser’sdesiredareaandtimeofinterest.

Spatial–OpenSearchGeoExtensionBoundingboxisthesimplestandmostusedtypeofspatialconstraint.Asperthegeoextensionwerecommendsearchprovidersdefineaboundingboxconstraintas4longitude/latitudecoordinatesexpressedasEPSG:4326decimaldegrees.Theorderofthosecoordinatesshouldbewest,south,eastandnorth.Forexample,datasets.atom?box=-180.0,-90.0,180.0,90.0 Theimpliedrelationbetweenaspatialconstraintandanyinventoryreturnedisthatthespatialextentofaresourceoverlapsthespatialconstraintinthequery.SearchprovidersmayimplementotherspatialconstraintsspecifiedinOpenSearchgeoextensionversion1.0draft2butsupportforGeoBoundingBoxsupportisrequired.ItshouldbenotedthatwhileaboundingboxspatialsearchconstraintisexpressedintheGEOform,boundingboxspatialextentsshouldbeexpressedasGEORSSboundingboxes(seeresultsection)OpenSearchserverdevelopersshouldfilterresultsbyageoboundingboxconstraintasspecifiedintheOpenSearchGeoextension

Temporal–OpenSearchTimeExtensionTemporalconstraintscanbeexpressedasintervalsintimethatmaybeopenended.AupperorlowertemporalboundcanbeexpressedasadateordatetimeconstraintspecifiedintheRFC-3339formataspertheOpenSearchTimeextension.

10

Exampleofafullyspecifiedrange,datasets.atom?startTime=2001-01-01T22:00:00Z&endTime=2001-01-01T22:00:00Z Exampleofanopen-endedrange,datasets.atom?startTime=2001-01-01T22:00:00Z Exampleofanopen-endedrangewithdateonly,datasets.atom?startTime=2001-01-01 Theimpliedrelationbetweenatemporalconstraintandanyinventoryreturnedisthatthetemporalextentofaresourceoverlapsthetemporalconstraintinthequery.OpenSearchserverdevelopersshouldfilterresultsbytimestartandtimeendasspecifiedintheOpenSearchTimeextension

ResultsetnavigationparametersWerecommendusingtheindexqueryparametertonavigatethroughresultssets.Anydefaultsusedbyyourimplementationwhennotsuppliedbytheclientshouldberenderedinyourresult.WerecommendthatastartIndexvalueof‘1’refertothefirstresultofaresultset.Example,datasets.atom?startIndex=1&count=10 Willreturntheresults1through10ofaresultset.datasets.atom?startIndex=2&count=10 Willreturntheresults2through11ofaresultset.Certainimplementationsmayprefertousethe‘startPage’parameterratherthan‘startIndex’.Thisisacceptablebutwerecommendtheuseof‘startIndex’.Note:thebaseOpenSearchspecificationallowsstartPageasaparameterbutdoesnotdefinestartPageintheresultbody,onlystartIndex.ItisbecauseofthisdiscrepancythatwerecommendtheuseofstartIndex.However,weunderstandthatimplementationsofyoursearchAPImayprecludetheuseifindexnavigationdirectly.

11

OpenSearchserverdevelopersshouldallowaclienttonavigatearesultsetusingthe‘startIndex’and‘count’parametersasspecifiedintheOpenSearchspecification

AnatomyofaSearchRequest–theclientstoryOpenSearchclientdevelopersshouldpreserveanynon-variablesearchparameters(suchasclientid)giventothembytheOSDDforanysearchesrelatedtothatOSDD

MetricsparametersIntheOSDDsectionwetalkabouttheneedfortheusertosupplyaclientIDwhenobtaininganOSDD.TheOSDDreturnedwillembedthatclientIDintheURLtemplate.Intheory,allrequestssentbythatclientwillhavethatparameterpresent.ThepurposeofthisIDistoallowthetrackingofmetricsbasedonparticularclientsandusers.TofacilitatethisneedwerecommendthatsearchprovidersimplementingOpenSearchusethisclientIDreferencewhenloggingorgeneratingtheirownmetrics.Furthermore,werecommendthattheclientIDbepropagatedtoothersearches(see‘Twostepsearching).WestronglyrecommendthattheclientIDshouldbeamandatoryparameterwhenobtaininganOSDDthroughyourAPIbutoptionalinanysearchrequest.OpenSearchserverdevelopersshouldallowaclienttospecifya‘clientId’andthatIDmustbepropagatedtoanysecondaryOpenSearchimplementationsWhilethisisnotanidealmeansofprovidinguserinformation,itappearstobethebestwaytoprovide‘optin’,non-obtrusivesupport.ThesuppliedclientIDmaybe(a)non-uniqueand(b)notmeaningful.Thisriskisbalancedbythepossiblelackofinterestinaprovider’sOpenSearchAPIifamorethoroughhandshake(designatedIDsforexample)isrequired.Example,datasets.atom?clientId=foo

FullExampleofa‘developerguide’compliantOpenSearchRequesthttps://foo.gov/opensearch/datasets.atom?clientId=foo&searchTerms=air+temperature&box=10,10,10,10&startTime=startTime=2001-01-01T22:00:00Z&endTime=2001-01-01T22:00:00Z&pageNumber=1&count=10

12

TheSearchResponseAnOpenSearchresponseisanATOMfeedwithzeroormoreATOMentries.Eachentryrepresentsasingleresourcepertainingtothequerysubmitted.

Anatomyofaresponse–theserverstoryAnATOMresponseisoneATOMfeedelementcontainingthefollowing,1. Informationaboutthesearchimplementationintermsoftitle,authorandID.2. Informationaboutthenatureoftheresultsetintermsoftotalnumberofresults,

numberofresultsreturnedandhowmanytheclientaskedfor.3. Navigationinformationfortraversingthatresultsetincludinglinkstothe

previous,next,firstandlastresultsintheset.4. Zeroormoreentriespertainingtoresourcesmatchingtheclientquery

Figure3:OpenSearchResponseoverview

ResultsetnavigationbycontentAsdescribedintherequestsection,OpenSearchserverdevelopersshouldallowaclienttonavigatearesultsetusingthe‘startIndex’/’startPage’and‘count’parametersasspecifiedintheOpenSearchspecificationThisrecommendationhasanimpactonhowaproviderdescribestheirresultset.Forexample,<feed> …

<os:totalResults>3415</os:totalResults> <os:itemsPerPage>10</os:itemsPerPage> <os:startIndex>1</os:startIndex> …

</feed> Indicatesthattheclientaskedforthefirst10resultswherethereare10resultsperpageandthereareatotalof3415results.Thefirst10entries(1-10)wouldbe

13

representedinthefeed.Iftheclientaskedforthesecondindexonwards(startIndex=2)thentheentries2to11wouldberepresentedinthefeed.Inthecasewherezeroresultsarereturnedthefollowingwouldbecontainedinthefeed,<feed> …

<os:totalResults>0</os:totalResults …

<subtitle type="text">Your search yielded zero matches</subtitle>

</feed> OpenSearchserverdevelopersshouldnotinclude‘itemsPerPage’and‘startindex’inafeedresultwhenzeroentriesarereturnedforaqueryandthatthefeedsubtitleshouldcontainhuman-readabletextdescribingthatstate.

ResultsetnavigationbyHATEOASInordertofacilitatetraversalofaresultsetwecanleveragetheRESTconceptofHypermediaastheEngineofApplicationState(HATEOAS)andprovidelinkstothecurrent,previous,next,firstandlastpagesintheresultsetasfollows,<feed> …

<os:totalResults>55</os:totalResults> <os:itemsPerPage>10</os:itemsPerPage> <os:startIndex>30</os:startIndex>

<link href="foo.gov/opensearch/datasets.atom?startIndex=1&numberOfResults=10" rel="first" type="application/atom+xml"/> <link href="foo.gov/opensearch/datasets.atom? startIndex=20&numberOfResults=10" rel="prev" type="application/atom+xml"/> <link href="foo.gov/opensearch/datasets.atom? startIndex=30&numberOfResults=10" rel="self" type="application/atom+xml"/> <link href="foo.gov/opensearch/datasets.atom? startIndex=40&numberOfResults=10" rel="next" type="application/atom+xml"/> <link href="foo.gov/opensearch/datasets.atom? startIndex=50&numberOfResults=10" rel="last" type="application/atom+xml"/> … </feed>

14

OpenSearchserverdevelopersshouldprovidenavigationlinksforthefirst,previous,current,nextandlastpagesofaresultset.

ResultmetadataEachresourceinaresultwillberepresentedbyanATOM‘entry’element.ThatelementwillcontainanID,alinktothemetadatafromwhichthisentrywasderived,and,ifapplicable,aspatialextentandatemporalextent.OpenSearchserverdevelopersshouldprovidealinkofrelation‘alternate’foreachentryinaresultsetfeed.Thatlinkshouldpointtotheoriginalmetadatafromwhichthisentrywasderived. Whenrenderingaspatialextentforanentrywerecommendthatanimplementationprovideaminimum-boundingrectangle(MBR)torepresentmorecomplexgeometriessuchaspolygoninadditiontotheaccuratespatialextent.Therationalebehindthisisthatboundingrectangleisthelowestcommondenominatorofspatialconstraint/extentsupportedbyprovidersandclients.YoumayalsorenderyouroriginalextentinWKTasperthegeoextension.Forexample,anMBRrepresentingapointanditsoriginalrepresentation, <georss:box>39.1 -96.6 39.1 -96.6</georss:box> <georss:point>39.1 -96.6</georss:point> OpenSearchserverdevelopersshouldrenderspatialextentsusingaminimum-boundingrectangleinthe‘georss’formataswellasthenativerepresentationofthatextent.Whenrenderingatemporalextent,animplementationshoulduseaDublinCoredateelement,whichwillsupportdateanddate-timeranges,open-endeddateanddate-timerangesandsingledatedanddate-times.Forexampleadate-timerange,<dc:date xmlns:dc="http://purl.org/dc/elements/1.1/">1984-12-25T00:00:00.000Z/1988-03-04T00:00:00.000Z</dc:date> anopen-endeddate-timerange,<dc:date xmlns:dc="http://purl.org/dc/elements/1.1/">1984-12-25T00:00:00.000Z/</dc:date> asingledate-time,<dc:date xmlns:dc="http://purl.org/dc/elements/1.1/">1984-12-25T00:00:00.000Z</dc:date>

15

asingledate,<dc:date xmlns:dc="http://purl.org/dc/elements/1.1/">1984-12-25</dc:date> OpenSearchserverdevelopersshouldrendertemporalextentsusingaDublinCoredateelement.Entriesshouldalsorenderlinkelementspointingtovariousartifactsassociatedwiththeresource.Thosemayinclude

• Data• Additionalmetadata• Browseimagery• Documentation.

OpenSearchserverdevelopersshouldusethefollowingrelationvalueswhendescribingartifactsassociatedwitharesource,Artifact Definition Relation ExampleMetadata filewith(usually)

structuredinformationaboutcorrespondingdatafiles

via <linkhref="foo.xml"rel="via"/>

Browse imageofthedatatypicallyusedformakingdatarequestdecisions

icon <linkhref="sample.gif"rel="icon"/>

Documentation Filewithhuman-readableinformationabouttheresource

describedBy <linkhref="foo.pdf"rel="describedBy"/>

Data linkrepresentingadatafileorothersciencedataresource;maybelargeinsize

enclosure <linkhref="foo.hdf"rel="enclosure"/>

OSDD linktoanOpenSearchDescriptionDocument;usefulforrecursivesearching

search <linkhref=”osdd.xml”rel=”search”/>

Table1:LinkrelationsforOpenSearchresponseartifacts

Anatomyofaresponse–theclientstory

ResultsetnavigationWerecommendthatserverimplementationsuseHATEOStoprovideresultsetnavigation.Clientimplementationsareatthemercyoftheimplementationstheyinteractwith.WhensuppliedwithbothmechanismsweadvocateusingHATEOSasitisamoreRESTfulsolutionandprobablymoresimpletoimplementontheclientsidesincetherequestisalreadyformulatedintheresponse.

16

ResultmetadataThepurposeofresultmetadatawithrespecttoclientimplementationistoprovidetheuserwithinformationtohelpdeterminedataofinterest.Consequently,aclientshouldprovidethefollowing:

• AmeansofrenderingcanonicalmetadataassociatedwitharesultentrygivenaURL

• AmeansofrenderingbrowseimageryassociatedwitharesultentrygivenaURL

• AmeansofrenderingdocumentationassociatedwitharesultentrygivenaURL

• AmeansofaccessingdataassociatedwitharesultentrygivenaURL• Ameansof‘drillingdown’onadatasetsearchforgranules(orfurther

dependingonserverimplementation)(see‘Twostepsearching’)ThiscanbeachievedbyleveragingthereltypeofeachlinkpresentedinaresultentryasperTable1.

Resultordering–theserverstoryResultorderingisoptionalasperOGCrecommendations.WerecommendthatifyousupportsearchbyfreetextthatyouemployrelevancyrankingaspertheOpenSearchRelevancyExtensionandorderyourresultsbythatrankingindescendingorder.Forexample,<feed xmlns:relevance="http://a9.com/-/opensearch/extensions/relevance/1.0/"> <entry> … <relevance:score>0.98</relevance:score> .. </entry> <entry> … <relevance:score>0.75</relevance:score> .. </entry> … </feed>

Resultordering–theclientstoryWhenportrayingresultsfromaserviceimplementationthatsupportrelevancyandorderingtheclientmaywanttoleveragetherelevancyscores:

• Displayingtherelevancyscore• Aggregatingmorethanoneresultsourceandordering,assumingthetwo

sourcesabidebythesamerelevancyrules.

17

Handlingerrors–theserverstoryIntheeventofanerrortheserviceimplementationshouldrespondwiththeappropriateHTTPerrorcodeandaresponsebodythatverboselydescribestheerrorinATOMformat.Theseerrorswillgenerallyfallintotwocategories,

• UserinputerrorswillberespondedtowithaBAD_REQUEST400responseoranothererrorinthe4xxrange,iftheerrorcorrespondstooneoftheexistingHTTPerrorcodes.

• ProcessingerrorswillberespondedtowithanINTERNAL_SERVER_ERROR500responseoranothererrorinthe5xxrange,iftheerrorcorrespondstooneoftheexistingHTTPerrorcodes.

Notethataquerythatyieldszeroresultsisnottreatedasanerrorstateand,therefore,shouldreturnasuccessfulHTTPcode.

Two-stepSearchingOneserioushurdletoovercomeinsearchingfordataistheenormousnumberofdataitemstoaccountforinresponses,aswellastheexpectednumberofsuccessful“hits”foraquery.Inordinarywebsearches,theenduserisusuallylookingforasmallnumberofwebpagesordocuments.Relevancerankingtypicallydoesagoodjobofpresentingthesesuccessfulhitsnearthetopofthereturnedlist,followedbysinglepoint-and-clickretrievals.However,whensearchingforEarthsciencedatacoveringlargetimeperiodsorspatialareas,auserwilloftenspecifyasetofconstraintstofindanappropriatedatacollectiontogetherwithspace-timecriteriaforgranuleswithinthatdatacollection.Often,theprecisionofthedatacollectionsreturnedforthesearchislow,withmanyspurioushits.However,thespace-timeprecisionofthefilesisoftenquitehigh:thatis,theusertrulywantstouseallthedatagranulesofadesirabledatacollectionsetthatfallwithinthespace-timeregionofinterest.Thus,searchingforalldatasatisfyingbothdatasetcontentandspace-timeregionatthesametimecanproduceagreatmanyspurioushits,i.e.,allthegranulesfordatacollectionsthatarenotdesired.Thetwo-stepsolutionbreaksasearchintotwodistinctparts.Aclientwillfirstidentifydatasetsofinterest(step1)andthensearchforgranulesassociatedwiththosedatasets(step2).

TheserverstoryTofacilitatethis,whenanOpenSearchimplementationrepresentingdataset/collectionresourcesreturnsresults,eachentryinthatresultsetshouldcontainanATOMlinkofrelationtype‘search’ifapplicable.ThatATOMlinkshouldpointtoaURLofanOSDDpertainingtogranulesassociatedwiththatentry(i.e.dataset/collection).

18

Forexample,

<feed … > … <entry … > <id>http://foo.gov/opensearch/datasets/foo</id> <link rel=‘search’ type=‘application/opensearchdescription+xml’>https://datacenter.nasa.gov/opensearch/granules/descriptorDocument?datasetId=foo&clientId=d</link> … </entry> … </feed>

Thisrelationshipisformalizedbytwoattributesinthelinkelement:

• rel=’search’• type=’application/opensearchdescription+xml’

Althoughtwosteps,datasetandgranule,areillustratedhere,thenotionofmorestepscouldbeused.Thatis,onecouldhaveanOpenSearchserverthatsearchestop-level(searchengine)documents,returningtheURLstoOSDDsforeachsearchengine.EachstepintheprocessreturnslinkstoOpenSearchDescriptiondocuments,whichinturninformtheclientonhowtoexecutethenextlevelsearch,untilthelowestlevel,wherethereturnconsistsoflinkstoactualdataofinterest.Thetwo-stepsearchconceptisanoptionalaspectoftheconvention.Thereareusecaseswhereonlyadatasetqueryisneeded,aswellassomewhereonlythegranule-levelqueryisneeded.

TheclientstoryAclientcanparseanOSDDandformulateasearchforgranulesbelongingtothatdataset.

19

Figure4:Two-stepoverview

Forexample,

1. GivenandatasetOSDD,searchfordatasetsofinterest

GET http://foo.gov/opensearch/datasets?searchTerms=MODIS&clientId=d

2. Parsesearchresults

<feed … > … <entry … > <id>http://foo.gov/opensearch/datasets/MODIS_dataset</id> <link rel=‘search’ type=‘application/opensearchdescription+xml’>bar.gov/opensearch/granules/descriptorDocument?datasetId=MODIS_dataset&clientId=d</link> … </entry> … </feed>

3. RetrievegranuleOSDD4. ParsegranuleOSDD

< OpenSearchDescription> … <Url type="application/atom+xml"

template="bar.gov/opensearch/granules.atom?datasetId= MODIS_dataset&clientId=d&box={geo:box?}">

… </Url> …

20

</OpenSearchDescription>

5. Searchforgranulesofinterest

GET http://bar.gov/opensearch/granules.atom?datasetId= MODIS_dataset&clientId=d&box=1,2,3,4

AppendixA–SummaryTableElementorAttribute

Description Namespace Example

titleelement Human-readabletextdescribingaboutanentry,usuallyrelativelyshort

ATOM

idelement UniqueidentifierforareturnedresourceinATOMresponse

ATOM

linkelement URLforaresourcereferencedinATOMresponse

ATOM

relattribute Relationshiprepresentedbyagiven<link>element.

ATOM rel=”enclosure”(usedtodescribe<link>elementspointingtodataresources)

typeattribute Typeofresource,typicallygivenasamime-type

ATOM type=”application/x-netcdf”(describeslinkelementsthatarenetCDFfiles).

Url Externalresourcelocation

OpenSearch <Urltype="text/html"template="http://example.com/search?q={searchTerms}&pw={startPage?}"/>

templateattribute templateattributeof<Url>elementRequiredinOSDD

OpenSearch

date TemporalcoverageofdatainanATOMresponse.

DublinCore

geo:box Minimumboundingrectangle,requiredinATOMresponseifspatialinformationis

OpenSearchGeo

21

included

geo:geometry OptionalWell-Known-Textbasedgeometry

OpenSearchGeo

Table2:summaryofcommonelementsandattributes

AppendixB–Examples

OSDDwithparameterextension<?xml version="1.0"?> <OpenSearchDescription

xmlns="http://a9.com/-/spec/opensearch/1.1/" xmlns:echo="http://www.echo.nasa.gov/esip" xmlns:geo="http://a9.com/-/opensearch/extensions/geo/1.0/" xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/" xmlns:esipdiscovery="http://commons.esipfed.org/ns/discovery/1.2/" esipdiscovery:version="1.2"

xmlns:params="http://a9.com/-/spec/opensearch/extensions/parameters/1.0/" xmlns:atom="http://www.w3.org/2005/Atom" >

<ShortName>ECHO Dataset Search</ShortName> <Description>NASA ECHO Dataset search using geo, time and

parameter extensions</Description> <Tags>ECHO NASA CWIC CEOS ESIP OGC dataset</Tags>

<Contact>[email protected]</Contact> <Url

type="application/atom+xml" params:method="GET"

template="https://api.echo.nasa.gov:443/opensearch/datasets .atom?keyword={searchTerms?}&instrument={echo:instrument?}&satellite={echo:satellite?}&boundingBox={geo:box?}&geometry={geo:geometry?}&placeName={geo:name?}&startTime={time:start?}&endTime={time:end?}&cursor={startPage?}&numberOfResults={count?}&uid={geo:uid?}&clientId=foo"> <params:Parameter

name="keyword" value="{searchTerms}" minimum="0"> <atom:link

rel="profile" href="http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html" title="This parameter follows the elastic search free text search implementations" />

</params:Parameter> <params:Parameter

name="instrument" value="{echo:instrument}" title="Inventory associated with a satellite instrument expressed by this short name" minimum="0"/>

22

<params:Parameter name="satellite" value="{echo:satellite}" title="Inventory associated with a Satellite/platform expressed by this short name" minimum="0"/>

<params:Parameter name="boundingBox" value="{geo:box}" title="Inventory with a spatial extent overlapping this bounding box" minimum="0"/>

<params:Parameter name="geometry" value="{geo:geometry}" title="Inventory with a spatial extent overlapping this geometry" minimum="0"> <atom:link

rel="profile" href="http://www.opengis.net/wkt/LINESTRING" title="This service accepts WKT LineStrings"/>

<atom:link rel="profile" href="http://www.opengis.net/wkt/POINT" title="This service accepts WKT Points"/>

<atom:link rel="profile" href="http://www.opengis.net/wkt/POLYGON" title="This service accepts WKT Polygons"/>

</params:Parameter> <params:Parameter

name="placeName" value="{geo:name}" title="Inventory with a spatial location described by this name" minimum="0"/>

<params:Parameter name="startTime" value="{time:start}" title="Inventory with a temporal extent containing this start time" minimum="0"/>

<params:Parameter name="endTime" value="{time:end}" title="Inventory with a temporal extent containing this end time" minimum="0"/>

<params:Parameter name="cursor" value="{startPage}" minimum="0"/>

<params:Parameter name="numberOfResults" value="{count}" minimum="0" maxInclusive="2000"/>

<params:Parameter name="uid" value="{geo:uid}"

23

title="Inventory associated with this unique ID" minimum="0"/>

</Url> <Query

role="example" echo:instrument="AMSR-E" echo:satellite="Aqua" title="Sample search" geo:box="-180.0,-90.0,180.0,90.0" time:start="2002-05-04T00:00:00-0400"

time:stop="2009-05-04T00:00:00-0400"/> <Attribution>NASA ECHO</Attribution> <SyndicationRight>open</SyndicationRight>


OSDDwithoutparameterextension<?xml version="1.0"?> <OpenSearchDescription

xmlns="http://a9.com/-/spec/opensearch/1.1/" xmlns:echo="http://www.echo.nasa.gov/esip" xmlns:geo="http://a9.com/-/opensearch/extensions/geo/1.0/" xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/" xmlns:esipdiscovery="http://commons.esipfed.org/ns/discovery/1.2/" esipdiscovery:version="1.2"

xmlns:params="http://a9.com/-/spec/opensearch/extensions/parameters/1.0/" xmlns:atom="http://www.w3.org/2005/Atom" >

<ShortName>ECHO Dataset Search</ShortName> <Description>NASA ECHO Dataset search using geo, time and

parameter extensions</Description> <Tags>ECHO NASA CWIC CEOS ESIP OGC dataset</Tags>

<Contact>[email protected]</Contact> <Url

type="application/atom+xml" params:method="GET"

template="https://api.echo.nasa.gov:443/opensearch/datasets .atom?keyword={searchTerms?}&instrument={echo:instrument?}&satellite={echo:satellite?}&boundingBox={geo:box?}&geometry={geo:geometry?}&placeName={geo:name?}&startTime={time:start?}&endTime={time:end?}&cursor={startPage?}&numberOfResults={count?}&uid={geo:uid?}&clientId=foo">

</Url> <Query

role="example" echo:instrument="AMSR-E" echo:satellite="Aqua" title="Sample search" geo:box="-180.0,-90.0,180.0,90.0" time:start="2002-05-04T00:00:00-0400"

time:stop="2009-05-04T00:00:00-0400"/> <Attribution>NASA ECHO</Attribution> <SyndicationRight>open</SyndicationRight>


24

ATOMresult<?xml version="1.0" encoding="UTF-8"?> <feed esipdiscovery:version="1.2"

xmlns=”http://www.w3.org/2005/Atom” xmlns:dc="http://purl.org/dc/terms/" xmlns:echo="http://www.echo.nasa.gov/esip" xmlns:esipdiscovery="http://commons.esipfed.org/ns/discovery/1.2/" xmlns:georss="http://www.georss.org/georss/10" xmlns:gml="http://www.opengis.net/gml" xmlns:os="http://a9.com/-/spec/opensearch/1.1/" xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/"> <updated>2014-08-01T17:22:04.288Z</updated> <id>https://api.echo.nasa.gov:443/opensearch/datasets.atom</id> <author>

<name>ECHO</name> <email>[email protected]</email>

</author> <title type="text">ECHO dataset metadata</title> <os:totalResults>3434</os:totalResults> <os:itemsPerPage>1</os:itemsPerPage> <os:startIndex>1</os:startIndex> <Query searchTerms="FIFE" role="request"/> <subtitle type="text">Search for ‘FIFE’</subtitle> <link

href="https://api.echo.nasa.gov:443/opensearch/granules/descriptor_document.xml" hreflang="en-US" rel="search" type="application/opensearchdescription+xml"/>

<link href="https://api.echo.nasa.gov/opensearch/datasets.atom? keyword=FIFE&numberOfResults=1&cursor=1" hreflang="en-US" rel="self" type="application/atom+xml"/>

<link href="https://api.echo.nasa.gov/opensearch/datasets.atom? keyword=FIFE&numberOfResults=1&cursor=3434" hreflang="en-US" rel="last" type="application/atom+xml"/>

<link href="https://api.echo.nasa.gov/opensearch/datasets.atom? keyword=FIFE&numberOfResults=1&cursor=2" hreflang="en-US" rel="next" type="application/atom+xml"/>

<link href="https://api.echo.nasa.gov/opensearch/datasets.atom? keyword=FIFE&numberOfResults=1&cursor=2" hreflang="en-US" rel="first" type="application/atom+xml"/>

<link

25

href="https://wiki.earthdata.nasa.gov/display/echo/Open+Search+API+release+information" hreflang="en-US" rel="describedBy" title="Release Notes" type="text/html"/>

<entry> <id>https://api.echo.nasa.gov:443/opensearch/datasets.atom?uid=C179003030-ORNL_DAAC</id> <author>

<name>ECHO</name> <email>[email protected]</email>

</author> <title type="text">15 Minute Stream Flow Data: USGS (FIFE)</title> <summary type="text">ABSTRACT: USGS 15 minute stream flow data for Kings Creek on the Konza Prairie</summary> <updated>2008-12-02T00:00:00.000Z</updated> <echo:datasetId>15 Minute Stream Flow Data: USGS (FIFE)</echo:datasetId> <echo:shortName>doi:10.3334/ORNLDAAC/1</echo:shortName> <echo:versionId>1</echo:versionId> <echo:dataCenter>ORNL_DAAC</echo:dataCenter> <echo:processingLevelId>3</echo:processingLevelId> <link

href="http://daac.ornl.gov/cgi-bin/dsviewer.pl?ds_id=1" hreflang="en-US" rel="enclosure"/>

<link href="http://daac.ornl.gov/FIFE/guides/15_min_strm_fl

ow.html" hreflang="en-US" rel="describedBy" title="USGS 15 minute stream flow data for Kings Creek on the Konza Prairie (VIEW RELATED INFORMATION)"/>

<georss:point>39.1 -96.6</georss:point> <georss:box>39.1 -96.6 39.1 -96.6</georss:box> <link

href=”http://gcmd.nasa.gov/getdif.htm?FIFE_STRM_15M” hreflang="en-US" rel="enclosure" title="doi:10.3334/ORNLDAAC/1" type="text/html"/>

<link href="https://api.echo.nasa.gov:443/opensearch/granules.atom?clientId=our_html_ui&shortName=doi:10.3334/ORNLDAAC/1&versionId=1&dataCenter=ORNL_DAAC" hreflang="en-US" rel="search" title="Search for granules" type="application/atom+xml"/>

<link href="https://api.echo.nasa.gov:443/opensearch/granules/descriptor_document.xml?clientId=our_html_ui&shortName=doi:10.3334/ORNLDAAC/1&versionId=1&dataCenter=ORNL_DAAC"

26

hreflang="en-US" rel="search" title="Custom ECHO Granule OpenSearch Descriptor Document" type="application/opensearchdescription+xml"/>

<link href="https://api.echo.nasa.gov:443/catalog-rest/echo_catalog/datasets/C179003030-ORNL_DAAC.xml" hreflang="en-US" rel="alternate" title="Product metadata" type="application/xml"/> <dc:date>1984-12-25T00:00:00.000Z/1988-03-04T00:00:00.000Z</dc:date>

</entry> </feed>