213

IX. Data Server Configuration

A. Data services ThebackboneofthePacIOOSdatasystemisthedataservice.Ontheback-endofthissystemisanOPeNDAP-basedarchitecture.Inthissection,thedatatransporttoolsaredescribed.

1. TDS

a) configuration Therearetwodifferenttypesofconfigurationfilesthatareimportant.Thefirst,threddsConfig.xml,isfortheTDSoperations.TheotherconsistsofsetsofXMLfilesthatcomprisethedatacatalog.Themaincatalogiscatalog.xml.Thisfilecontainsdescriptionsofallthedatatobeserved,eitherbydirectlyspecifyingthedatasetname(andpath)orbyspecifyingsomeotherserver,orbyspecifyingasub-catalog.Forexample,theentry

<datasetRoot path="my/test" location="/export/lawelawe1/" /> <dataset name="Levitus" ID="levitus" urlPath="my/test/levitus.nc" > <serviceName>thisDODS</serviceName> </dataset>

specifiestheNetCDFfilelevitus.nc(specifiedwiththeurlPathtag),foundinthedirectory/export/lawelawe1(specifiedwiththelocationtag).ThedatasetwillshowupontheTHREDDSpageunderthelink"Levitus"(specifiedbythenametag).TheIDandpathtagsareusedtouniquelyidentifythedataset.Asub-catalogisspecifiedinthecatalog.xmlfilewithacatalogReftag,e.g.,

<catalogRef xlink:href="idd/roms\_scalar.xml" xlink:title="ROMS scalar vars" name="" />

Thexlink:hreftagspecifiesthelocationandnameofthenewcatalog(notethedirectory/usr/local/tomcat/content/thredds/iddcontainsallthesesub-catalogs).Again,thexlink:titletagiswhatshowsupontheTDSwebpage(asalinktotheDAPpage).

214

Theinitialsub-catalogsthathavebeendevelopedareforthemodeloutput,andrequiredsomespecialcodeforthenon-standardgridandvariables.Forexample,theroms_scalar.xmlfilestartswiththefollowingheaderinformation:

<?xml version="1.0" encoding="UTF-8"?> <catalog name="HI-ROMS Catalog" xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0" xmlns:xlink="http://www.w3.org/1999/xlink"> <service name="allServices" base="" serviceType="compound"> <service name="dapService" serviceType="OpenDAP" base="/thredds/dodsC/"/> <service name="wcs" serviceType="WCS" base="/thredds/wcs/"/> <service name="wms" serviceType="WMS" base="/thredds/wms/"/> <service name="ncss" serviceType="NetcdfSubset" base="/thredds/ncss/grid/"/> </service>

Therestofthexmlfiledefinesadatasettag,whereinthemetadata,linknamesandvariablesaredefinedorre-defined).Forexample:

<dataset name="HI-ROMS Nowcast/Forecast Test"> definesthenameattribute(thisshowsupastextontheTDSpageforthisdatasetasmoreorlessatitle).Thisisfollowedbysomemetadataattributes.ThecriticalnextpartisthedatasetFMRCtag:

<datasetFmrc name="HI-ROMS Forecast Model Test" path="hioos/roms-scalar">

<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">

<aggregation dimName="runtime" type="forecastModelRunCollection"

recheckEvery="10min"> <scan location="/export/lawelawe1/model/ocn/roms/2009/" suffix=".nc" dateFormatMark="hiig_#yyyyMMdd" subdirs="false" olderThan="5 min"/> </aggregation>

ThefirstlinecontainsthetextthatwillshowupasthelinktotheDAPpage(i.e.,"HI-ROMSForecastModelTest"willbedisplayed).Thescanlocationtaggivesthedirectorywherethedataare,withattributessuffixthewildcardand

215

dateFormatMarkdescribingthewaytheindividualdatasetnamescontaindateinformation(e.g.,thefileforMay21,2009wouldhavethenamehiig_20090521.nc).ThenextsectionofthexmlfilecontainsamapfromtheNetCDFvariablenamestomore"standard"names.ThisisdoneforalltheROMSoceanvariables.Inthiscase,theoceanmodelusesdifferent(x,y,z)gridsforallthevariablesasfollows:

eta (xi_rho,eta_rho)T,s (xi_rho,eta_rho,s_rho)u (xi_u,eta_u,s_rho)v (xi_v,eta_v,s_rho)w (xi_rho,eta_rho,s_w)

andtheassociatedxml(onevariableexample)is:

<variable name="xi_rho" shape="xi_rho" type="double"> <attribute name="units" value="degrees_east"/> <values start="-163.83069727891120" increment="0.03860544217687"/> </variable>

Finally,thexmlfilecan"hide"acertainnumberofvariablesfromtheserver(variablesthatpeopletypicallydon'tneed):

<remove type="variable" name="ubar"/> <remove type="variable" name="DU_avg1"/> <remove type="variable" name="DU_avg2"/>

willhidethevariablesubar,DU_avg1andDU_avg2fromtheTDSlist.

b) example usage OncetheTDSisconfigured,itrestartswithtomcat(oritcanbedeployedwiththetomcatadminpage).ThetopcatalogforTDS,catalog.html,listsallthedatagroups(Figure25).Thedatagroupsaredefinedin/usr/local/tomcat/content/thredds/catalog.html.Selectingoneofthecategoriesbringsupamorespecificpageforthatdatacategory.Thesedata-specificpagesaredefinedin/usr/local/tomcat/content/thredds/idd/*.xml.

216

Figure25.ThetopcatalogpageforthePacIOOSTDS.Thedatacategoriesandlinksaredefinedinthefile/usr/local/tomcat/contents/thredds/catalog.xml.Thepagecanbeaccessedathttp://oos.soest.hawaii.edu/thredds.

Forexample,clickingon“AtmosphericModelOutput”bringsupthedataaccesspagesfortheatmosphericmodeloutput(Figure26).Thisoutputisdescribedintheprevioussectionandincludesthreelocally-runWRFmodelconfigurationsandtheglobalNCEPoutput.Thedatasetattributesaredefinedinatm_mod.xml,alongwithsimilarfilesfortheotherdatasets,in/usr/local/tomcat/contents/thredds/idd.

217

Figure26.ExampleTDSpageforPacIOOSatmosphericmodeloutput.

Selectingoneofthesedatasetsgivesthespecificpageforthatdataset.Figure27showsthepagefortheNCEPmodel.Similartootherforecastmodels,theNCEPpagehasthreepossiblechoices.Thefirst,“BestTimeSeries”willconstructatimeseriesbasedondailyruns.If,forsomereason,thereisnooutputforaparticularday,theserverwillusetheforecastforthepreviousday’srun.Thus,asingleURLwillprovideacontinuoustimeseries.

Figure27.TDSpageforNCEPglobalatmosphericmodeloutput.

TheotherthreelinksprovideOPeNDAPURL’sfordifferenttime-stepsampling.Thefirst,“ForecastModelRun”,provideslinkstoeachday’smodeloutput.Thesearenotaggregatedinanyway,soapage(URL)iscreatedforeachoutputfile.Thesecond,“ConstantForecastDate”givesapage(URL)foreachtime-step(forecasttime)ofeachfile.Thefinallinkisto“ConstantForecastOffset”.Thisisawaytoaccessaspecificforecasttimefromeachdataset,forexample,onecouldaccessthe12-hourforecastcreatedeachday.SelectinganygivestheTDSpageforaccessingthedata.Thesepagesareallsimilarinthesensethattheycontainthreemainitems:metadataabouttheparticulardataset,linkstoaccessingthedataset(e.g.,OPeNDAP,WMS,etc.),anddirectlinkstothe

218

dataviewers(e.g.,LAS,DChart,etc.).Figure28showsthepageforthe“BestTimeSeries”oftheNCEPGlobalAtmosphericModel.

Figure28."BestTimeSeries"pageforNCEPglobalatmosphericmodel.

219

Themetadataforeachdatasetincludessummarydocumentation,contributors,keywords,projects,creators,publishers,variables,coverage(spatialandtemporal),andproperties.Theremaininginformationonthepageprovidesalistoftheaccesspagesandviewers.Inthiscase,thedataareavailableviaOPeNDAP,NetcdfSubset,WMS,WCS,ISO(metadatapages)andNCML.TheviewersincludeIDV,NetCDF-Java,Godiva2,LAS,ERDDAP,DChartandthePacIOOSviewer.TheseareallexamplesofOPeNDAP-enableclientsoftwareandservers.Figure29showstheOPeNDAPpage.

Figure29.OPeNDAPinfopageforNCEPglobalatmosphericmodel,"BestTimeSeries".

TheOPeNDAPpageinFigure29providestheURLtoaccessthedataviaclientsoftware(inDataURLbox),butalsogivesthevariables,dimensionsandattributes.Thesecanbeaccessbyselecting“GetASCII”or“GetBinary”.Incomparison,Figure30showstheresultoftheNetCDFsubsetpage(Accesslinknumber2inFigure29).Thistoolallowsonetosubsetthedatasetviaamoreuse-friendlypage.

220

Figure30.NetCDFsubsetpageforNCEPglobalatmosphericmodel"BestTimeSeries".

2. ERDDAP

a) configuration ERDDAPrunswithintomcatsimilartoTDS.PacIOOShashadproblemswithtomcatrestarts,soERDDAPisisolatedonasingleinstanceoftomcat,installedin/usr/local/tomcat-erddap/content/erddap.Therearetwoconfigurationfiles,setup.xmlforthesystem-wideERDDAPconfigurationanddatasets.xmlfortheentiredatacatalog(unlikeTDS,ERDDAPhasasinglefilewithalldatasetslisted).Thesetupfilescontainsinformationabouttheorganization,pointsofcontact,entryURLs,etc.ThedatasetcataloghasalltheinformationaboutspecificdatasetsthatareservedbyERDDAP.Thefilecanspecifylocalfilesandcatalogs,and/orremotecatalogs.

221

TheinitialERDDAPpagehasasearchtoolthatjustaccessestheERDDAPdatasets,andPacIOOSbypassesthistolinkstraighttothedatalisting(Figure11).Eachdatasetislistedbytitleandtherearemanylinkstovariousfeatures.Thedatasetitselfislistedunder“Title”.Totherightarevariousmeta-datalinks,includingapopupboxwithashortdescription(“Summary”),meta-datainformationfromthefileitself(“Meta-dataInformation”link),“BackgroundInformation”providesalinktotheoriginaldatasource(configurable).FurthertotherightarelinkstogetRSSore-mailalertswhenthedataareupdated.Totheleftofthedatalistarethedataaccesslinks.ThefirstonesprovidetheDataAccessForm(“GridDAPData”forgriddeddataand“TableDAPData”fornon-griddeddata).ThereisanadditionalcolumnforgeneratingamapviaaWebMappingService(WMS).

b) example usage ERDDAPrunsintomcat,andduetoconflictsitisconfiguredinasingleinstanceoftomcat.Theservicerunsonport8080andisproxied,similartoTDS,ashttp://oos.soest.hawaii.edu/erddap(fromhttp://lawelawe:8080/erddap).

222

Figure31.PacIOOSERDDAPtop-pageshowingthedatacataloglist.Eachdatasetisidentifiedbyatitle,andERDDAPprovideslinkstoDAP,WMSandvariousmetadatapages.

223

Figure32.DataAccessFormsfromERDDAP.TheupperpanelshowsanexampleDAFforgriddeddata,andthelowerpanelisforpointdata.Inbothcases,theDAFallowstheusertosubsetinspaceand/ortime,pickthevariableofinterest,andtheneitherdownloadthedatainanumberofformats(Filetypepull-down)orprovidetheURLforadataservice.

224

Figure33.ERDDAPWMSexample.TheserviceprovidesaURLsothattheimagemaybeusedinaWMS-enabledclient.

225

Figure34.ERDDAPGraphfunction.ThefinaloptioninERDDAPisto"makeagraph".Theaboveexampleshowsthispageforaparticulardatasetanddemonstrateshowuserscanselectgraphtype,axes,colors,times,etc.Theresultcanthenbesavedtoafileorincludedintoawebpage.

3. GeoServer

a) configuration ThePacIOOSGeoServeraccessesaPostGISdatabaseandprovideslayerstotheGeoExplorer.Inaddition,itprovidesaccessviaWMS,WFS,WCS,TMSandGWC.Theconfigurationiscomplex[FILLINHERE]

Server AdminAcctGeoNetworkserver(45) AdministratorGeoNetwork adminDatabaseServer(46) AdministratorPostGRESQL postgresMapServer(48) AdministratorGeoServer pacioosTestServer(50) AdministratorArcGISServer pacioos

226

b) example usage [FILLINHERE]

4. Sensor Observation Service TheOpenGeospatialConsortium(OGC)webservices(OWS)includeservicesformaps(WMS),features(WFS)andvectordata(WCS).SensorObservationService(SOS)issomethinglikeaWFS,althoughSOSisspecificallyforsensordata,whileWFSismoregeneric.Theseserviceshavethefollowingoperations:

• SOS:GetCapabilities,DescribeSensor,GetObservation• WMS:GetCapabilities,GetMap,GetFeatureInfo• WCS:GetCoverage• WFS:GetCapabilities,GetFeature,DescribeFeatureType

FollowingtheIOOSguidelines,SOSisimplementedfollowingtheOOSTethysconventionandcode-base.Atthetimewhenitwasinstalled,therewereseveralpackages,butmostreliedonreadingdatafromMySQLdatabase.SincePacIOOSisusingaflat-filesystem,andsincemostdataareinNetCDF,thedecisionwasmadetousetheSOSversionwritteninJavathatreadsNetCDFfilesinaflatfilesystem.ThisistheOOSTethysversionwrittenbyLuisBermudez.SensorObservationServiceisamethodforprovidingsensordatafromalocaldatabasetoremoteusers.TodatetherehasbeennorequestfordataviaSOS,andtherehaveyettobeidentifiedSOS-enabledclienttools,sothedevelopmentandmaintenanceofthisparticularservicehashadlowpriority.TheNationalDataBuoyCenter(NDBC)runsafairlyrobustSOS,andsoitwillbeusedasanexampleforthistypeofdataservice.First,alinkismadetotheSOSserver,e.g.,http://sdf.ndbc.noaa.gov/sos/server.php,andargumentsareappendedasdescribedbelow:GetCapabilities:Thisfunctionreturnsmetadataaboutthisservice.(Notfullyimplemented.)Parametersinclude:

A. request=GetCapabilitiesB. service=SOS

DescribeSensor:Thisfunctionreturnsdetailedsensorcharactertistics.(Notfullyimplemented.Onlyreturnspositionatthistime.)Parametersinclude:

C. request=DescribeSensorD. OutputFormat=text/xml;subtype="sensorML/1.0.0"E. procedure=urnofsensortodescribe(seeprocedurein

GetCapabilitiesresponse)F. service=SOSG. version=1.0.0

GeoNodeServer AdministratorGeoNode pacioosServer Administrator

227

GetObservation:Thisfunctionreturnsobservationdataofthespecifiedtypeforthestationswithinthespecifiedstartandstopdates.(Notfullyimplemented-Currentlyreturnssinglepointorverticalprofileforaspecifiedstationanddate/timeordate/timerange.Boundingboxtobeadded.)Parametersinclude:

H. request=GetObservationI. service=SOSJ. version=1.0.0K. offering=urn:x-noaa:def:station:noaa.nws.ndbc::<stationID>L. observedproperty=oneofthefollowing:

1. currents2. salinity3. waterlevel4. watertemperature5. waves6. winds

M. responseformat=oneofthefollowing:1. text/xml;schema="ioos/0.6.1"2. application/ioos+xml;version=0.6.1

N. eventtime=<DateTimeorDateTime1/DateTime2>(optionalparameter)

1. DateTimeisformattedas2008-06-04-T00:00:00Zor2008-06-04-T00:00Z.(Atthistime,onlythesecondsareoptional)

2. Whenthisparameterisomitted,themostrecentobservationisreturned.

O. featureofinterest=BBOX:<min_lon>,<min_lat>,<max_lon>,<max_lat>(Notyetimplemented)

ThereturnisanXMLfilethatcanbeingestedintoclientsoftwareortranslatedviaXSLT.AnexampleSOSserverisavailablefromOOSTethysindifferentformats.Theimplementationsarespecifiedbyoperatingsystem(e.g.,Linux,Mac,etc.),databasetype(e.g.,MySQL,SQLite,xenia,etc.),andwebscripting(e.g.,php,python,java,etc.).TheinitialSOSforHiOOSisbasedonthejavaimplementationforOPeNDAP/NetCDFsources.

a) configuration Asajavaprogram,thefilescomeasa.warfileandareinthe/usr/local/apache-tomcat-6.0.18/webapps/directory.Theoostethys/WEB_INF/classes/configsubdirectorycontainstheconfigurationfileoostethys.xml.Thisfileiswherethedatasetsgetspecified.

b) example usage TheSOSisminimallyused,soitisonlybrieflydescribedhere.Theentrypageisathttp://oos.soest.hawaii.edu/oostethys/andisshowninFigure35.Thetoplinkcheckstheserver,andthebottomlinkgoestothecode.Themiddlelinkisforthecapabilitiespage(xmlresponse).

228

Figure35.PacIOOSSOSinterface.

5. DAPPER

DAPPERisanOPeNDAPserverusedprimarilyfornon-gridded,stationdata(e.g.,profiledata,insitumoorings,etc.).Itcanreaddatafromeitheralocaldatabase(andaggregate)orfromaremoteOPeNDAPserver.TheoutputfromDAPPERismainlytoDChart(seesectiononDChart)buttheoreticallycanservedatatoanyOPeNDAP-enabledclientorotherOPeNDAPservers.DAPPERusesMySQLtoorganizeandaccessdatasets.

a) configuration ThemainconfigurationforDAPPERis/usr/local/dapper-2.2.0/etc/dapper.properties.Inthisfilethedatabases,cache,filedirectory,etc.aredefined.FilesgetloadedinDAPPEReitherviarawNetCDF(directorydefinedindapper.properties}orbythescriptdapperload.sh.Thefirststepistocreatea"database"andthena"dataset".NotetheseareequivalenttotheMySQLterms"database"and"table".Asanexample:

dapperload.sh -u dapsql -p sirot create nearshore ns01

willcreateadatabasecalled"nearshore"withadatasetcalled"ns01".TheusernameandpasswordfordapperloadarecreatedinMySQL.Notethatthedapperloaddatabasesanddatasetscanbequeriedand/ormodifiedusingrawMySQLcommands.Forexample,

mysql -u root –p; show databases; use nearshore; show tables;

willlistallthedatabases,selectthenear-shoreone,andthenlistallthetables(datasets)inthatdatabase.

DAPPERwasdesignedtoloadandserveprofiledata,i.e.,datathatisafunctionofdepthbutnottime.Inordertoloadtime-seriesintoDAPPER,thedatasetname

229

needstoeitherinclude"_time_"orendwith"_time".The"dataset"nameiswhatisdisplayedintheDAPurl.Thenextstepisloadafile(orsetoffiles)intothe"dataset".Thisisdoneagainusingdapperload.sh:

dapperload.sh -u dapsql -p sirot load nearshore ns01 ns01_2008_04_17.nc

Thiswillloadthedatafromns01_2008_04_17.ncintotheDAPPERdatasetns01.Asetorlistoffilescanbeloadedusingthe"-l"option:

dapperload.sh -u dapsql -p sirot -l file_list.01 load nearshore ns01

willloadallthefileslistedinfile_list.01.Table58summarizestheDAPPERdatabases.

database filesais hourlyAISbathy miscbathymetryfilesbeach_cam beachphotosdhw degreeheatingweeksdataglider gliderdatahfr HFRdatamodel atm,nocm,ocn,ore,scud,

tidemodeloutputnearshore alldailynsXXNetCDFfiles singlefileswithalltimesnss individualNSfilessatellite ghrsstsealevel UHSLC“fast”hourlydata UHSLC“fast”dailydata UHSLC“fast”monthlydatauhslc individualfilesfromSCLwaterqualbuoy timeseriesfromWQBwqb individualfilesfromWQB

Table58.DAPPERdatabaseentries.

DAPPERreferencesdatasetsviathedatasetnamebyappendinga.cdp.Thedatasetnameisalsoaddedinthedapper.propertiesfile.

b) example usage Theserverhasascript/usr/local/dapper-2.2.0/dapper.shtostopandstarttheDAPPERserver.Oncestarted,DAPPERrunsviaport9090andcanbeaccessedathttp://lawelawe:9090/dods(proxiedto

230

http://oos.soest.hawaii.edu/dapper)SimilartoSOS,theuserinterfacetoDAPPERisunimpressivesincethemainpurposeofDAPPERisasadataservertoDChart.Error!Referencesourcenotfound.showstheDAPPERentrypage.

Figure36.PacIOOSDAPPERpage.

6. DataTurbine DataTurbine(DT)isareal-timestreamingdataenginethatallowsforquickdata-streamingfromavarietyofsensors.Itactsasa"blackbox"towhichapplicationsanddevicessendandreceivedata.ItismainlyusedinPacIOOStogetdatafromWQB’sandNSS’sintoadatabase.

a) configuration TherearetwoinstancesofPacIOOSDT,oneattheKiloNalusite,andtheotheronthemachineinChee’slab(bbl.acnl.hawaii.edu).ThefirstrecordsthedatacomingoffsensorsatKN,andthenanrsyncisdonewiththelocalmachine.Further,therearetwowebinterfacestotheDT.Oneisa“realtimebuffer”,athttp://bbl.ancl.hawaii.edu:8080/RBNB/,andtheotheristhearchivesideofDT,athttp://bbl.ancl.hawaii.edu/kilonalu-data/.Itisnotclearhowdatagetwrittentooneortheother.Theringbuffersitelistsalltheinstrumentscurrentlybeingarchived,andindividualpageshavethe“latest”datavalue.Thearchivesidehasallthepriordata,andisarrangedbylocation,instrumentanddate.

231

Figure37.PacIOOSDataTurbunepages.Theleftsideisthereal-time(ringbuffer)andtherightisthearchive.

b) example usage MostoftheaccesstoDTisdoneviaautomaticscripting,eitherthroughpython(e.g.,thewaterqualitypage)orbywget(e.g.,theNSSarchive).NotethatthewaterqualitypagereadsfromtheRBNBsideofDT,andtheNSS/WQBdataconversionsaredoneusingthearchiveside.

B. Data Browsing Tools Thedataservicesdescribedintheprevioussectionprovideaccesstothedatastreams.Inthissection,thetoolsthatallowuserstoview,subsetandoptionallydownloaddataaredescribed.Again,aswiththedatatransporttools,thesoftwareusediseitherdevelopedlocallyorisfree,open-sourcecode.Alltoolsprovideweb-basedaccesstothedataandthereforetheonlyrequirementisawebbrowserandnetworkaccess.Itshouldbenotedthatsomeofthedataservertoolsprovidebothtransportandbrowsecapability.Themainuse,atleastwithinPacIOOS,iswherethesetoolsaredescribed.Forexample,ERDDAPprovidesOPeNDAPcapabilities,butalsoallowsuserstodisplayandsubsetdataviaon-linemapsandfigures.Finally,themainGISserviceinPacIOOS,GeoServer,isdescribedinthissection,butitismorethanadata

232

browsingtoolasitprovidesGISanalysiscapabilities.Atthispoint,however,itismainlyusedtosearchanddisplaymapsandfeatures.

1. LAS Themainwebbrowsingtoolthatallowsforon-the-flyfigurepreparationandoptionallydatadownloadingistheLiveAccessServer(LAS).LASwasdevelopedattheNOAAPacificMarineEnvironmentalLab(PMEL).ItisaJava-basedsetofroutinesthatallowuserstobrowsedatainagraphicalway.Thecodeisrunfrom/usr/local/las.v7.2.1(aliasedto/usr/local/las).Therearescriptsforstartingtheservice(startserver.sh)andstoppingit(stopserver.sh).

a) configuration LASrunsinatomcatcontainer,liketheserversdescribedabove.Thedatasetsareconfiguredusingxmlfilesin/usr/local/tomcat/content/las/conf/server.MostfilesarereadfromtheTDS,andtheconfigurefilegivestheURL.Forexample,theatmosphericmodeloutput,WRF,fortheHawaiianIslandshasaconfigurationfilewrf_hi.xml,andtheTDSURLisgiveninthedatasetstag:

<datasets> <wrf_hi name="WRF Hawaii Regional Atmospheric Model" url="http://lawelawe.soest.hawaii.edu:8080/thredds/dodsC/hioos/model/atm/wrf_hi/WRF_Hawaii_Regional_Atmospheric_Model_best.ncd">

Thexmlfilealsocontainstagsforvariablenamesandattributes(ifnecessary).ItisalsopossibletoputdataintoLASwithoutreadingfromaTDS.

b) example usage TheinterfaceforLASishttp://oos.soest.hawaii.ed/las.Error!Referencesourcenotfound.showsthetoppage.Thefirststepisto“choosedataset”usingtheboxintheupperleftcorner.Whenclicked,thisbringsuptheentirelistofdata.Forthisexample,thedatasetchosenistheROMSoutputfortheHawaiianIslands(Figure39).Onceselected,thedatasetvariablesaredisplayed;forthisexample,salinitywaschosen.Theresultingpage(Figure40)showsthevariableselectedasacolorshadedplot.Inthetopleftcornerausercanchangethedomaineitherthroughclick-drag(selectthesquareiconabovethemap)orbyenteringthelatitudeandlongituderangesintheboxesbelowthemap.Thedefaultfordataofthistypeisalat/lonmapview.Ifadifferentplotisselected,however,theoptionsandmapchange.Forexample,ifatime-seriesisdesired(andselected),theusercanonlyenterasinglepointinthemap(againeitherviaclickorenteringlat/lonvaluesintheboxes),andthe“Date”pull-downchangesfromasingletimetoastartandendtime.LASwillrecognizethedatadimensionswhen

233

loaded,sodatawithoutlat/lonrangeswillonlypermittime-seriesplotting,orifdatadonothavedepthattributes,depthprofilesaren’tlistedasoptions.Onceallthedesiredrangesareselected,aplotisgeneratedbyselectingthe“UpdatePlot”boxalongthetop.Therearemoreselectionsatthetopmenutoanimatetheplot,exporttoGoogleEarth,save,print,etc.

Figure38.ThePacIOOSLASmainpage.

234

Figure39.DataselectionmenuintheLAS.

Figure40.ExampleLASplot.

235

2. DChart DChartisaweb-browsingtoolusuallyusedtosubsetanddisplayinsituprofiledata,althoughitcanbeusedtodisplayandsavedatafromanyOPeNDAPsource.ItwasdevelopedbyJoeSirotatNOAA'sPacificMarineEnvironmentalLab(PMEL).DChartcomeswithDAPPERandthusthedatasetsservedareusuallyassociatedwiththeDAPPERdatabase.Itcanreaddatafromthelocalarchive(viaDAPPER)orfromaremoteOPeNDAPserver.Itthenprovidestheuserwithoptionstoselectcertainprofiles(geographically,ortemporally),variablesandtimeranges.TheoutputformatsincludeaNetCDFdownload,adatadisplayinthebrowserorviaGoogleEarthand/orGoogleMap.Figure41isanexampleviewofthePacIOOSDChart.

Figure41.DChartpageshowinganexampledataset(temperaturevs.salinityforanearshoresensor).

a) configuration DChartcomesbundledwithDAPPERandisasubdirectoryundertheDAPPERdirectorystructure.DChartgetsstartedwhenDAPPERisstarted,andthefilesthatcontrolDChartareinasubdirectoryofDAPPER(/usr/local/dapper/dchart).

b) example usage Theuserfirstselectsadataset(thelistisgeneratedfromdatainDAPPER).Onceselected,theoptionschangeaccordingly.Forexample,thevariablelistisgeneratedbasedonthedata.Alocationmapisgivenintheupperlefttoshowwherethedataarecollected.Thecenterpanelgivesoptionsforsettingtheplottype.Userscanchooseatime-series,propertyvs.property,ortosavethedatatoafile.Panelson

236

therightarethenupdatedgiventhetypeofplotselected.Therearepanelstoselectthevariableofchoiceandsliderstoselecttimeandspatialranges.Finally,theplotisgeneratedbyclickingonthe“plotselected”tab.Oncethefigureisshown,thereareoptionstozoomand/orpantheimagebyclickingonthearrows.Ifinsteadofafiguretheoptionto“exportdata”isselected,apopupwillgiveuserstheoptionstosavethedatainaspecificformat.

3. ERDDAP TheERDDAPserver,describedearlier,actsasbothadataserviceandasadatabrowsingtool.SectionC2givesthedetailsofERDDAP.

4. Voyager (Data Viewer) Thecoredata-browsingtoolistheHawaiiDataViewer.Thisisamap-basedtoolfordisplayingdataasoverlays.Theviewer,developedbyJohnMaurer,usestheGoogleMapAPIasfoundationfortheoverlays.TheviewwasinitiallydevelopedtoshowthePacIOOSassetsintheHawaiianIslands,andthushasbeencalledtheHawaiiDataViewer,buttechnicallyitcanbeusedtoshowanygeographiclocation.TheviewerislinkedtothewebpageswhentheHawaiianIslandsgrouppagesareactive,ordirectlyathttp://www.soest.hawaii.edu/hioos/map/.Figure12showstheviewerwiththedefaultbackground.Thelistontheleftcontainsalltheavailablelayers.Somearestaticdatasetsorpremadeimages,othersaregeneratedandupdatedbyfeedsfromremoteservers,andstillothersaregeneratedfromthePacIOOSservers.Thelistisever-growing,butincludes:

• dopplerradar:livefeedsfromtheNationalWeatherService,themapsgetupdatedevery2minutes;optionsincludeshort-rangereflectivity,velocity,precipitation,etc.

• hazardso airqualityo droughto earthquakeso emergencyshelterso floodhazardzoneso tsunamievaczoneso tsunamiheightso unexplodedordnanceo volcanoes

• nauticalcharts• bathymetry(depth)• waves

o PacIOOSwavebuoyso NDBCbuoyso CDIPwavebuoys

237

• tideandwaterlevelo DARTstationso IOCstationso NOSshorestations

• surfacecurrents(HFR)• satellitedata

o ASCATo AVHRRo GOESo Jasono SeaWifso MODISo QuikSCAT

• weatherforecast• surfforecast• tideforecast• oceanforecast• maritimeboundaries

o threenauticalmileso territorialseao contiguouszoneo EEZ

• geographicboundarieso antemeridiano equatoro internationaldatelineo primemeridiano tropicofcancero tropicofcapricorn

• navigationo aidstonavigation(beacons,buoys,lighthouses,lights)o boatingfacilities(anchorageareas,commercialharbors,launch

ramps,mooringbuoys,pieronly,smallboatharbors)o FADso Obstructionso Shiplocationso Shiproutes

• seafloorinfrastructureo dumpingsites(dumping,discontinued,explosives)o sewerlineso underwatercables

• fishandwildlifeo monkseals

• protectedareaso bottomfishareas

238

o CoconutIslando Conservationdistrictso Fishreplenishmento Fisheriesmanagemento Kaho’olaweo Nationalmonumento Naturalareareserveo Whalesanctuaryo Wildliferefugeo Wildlifesanctuary

• benthichabitatso biologyo geomorphologyo geography

• waterqualityo nearshoresensorso waterqualitybuoyso waterqualitysites

• waterresourceso streamgaugeso raingauges

• oceanobservatorieso HOTo KiloNalu

Figure42.HawaiiDataViewertoppage.

239

Theviewerhasmanyfeaturesformodifyingorinteractingwithlayers.Mostoftheseareavailableviaoptionsonthepage,andsomeareinternalfeatures(forexample,[shift-click-drag]willzoomtoaspecifiedregion).Ofthepageoptions,therearethreetypes:layoutoptionsatthetop,drawingatthelowerleft,anddata-specificoptions.Atthetopofthepagearequick-zoomoptionstospecificislandsandbeaches(notethatzoom-to-beachisnotyetimplemented).Alsoalongthetopoftheviewerisanoptiontocyclethroughdomains(arrows),anda“link”box.Thisboxallowsuserstosave,viaasingleURL,themapwithallparametersandoverlays(forexampleausercanzoomtoacertainarea,overlayspecificdata,andthensavetheresultasaURLtoreproducetheimage).Thislinkcaneitherbeemailedtoanotheruserorsavedasalinkinawebpage.Finally,consistentwiththeGoogleMapAPI,thebasemapcanbechangedtoasatellite,terrainormapview.Inthelowerleftcornerareanothersetofoptions.Theseallowtheusertodrawlatitudeandlongitudegridlines,clickonapointtogetrangelines,drawalinetoshowdistance,ordrawapolygontogetarea.Therearealsolinkstocleartheseorchangeunits.Finally,eachdatasethasit’sownsetofoptionsoncedisplayed.Theseinclude,forexample,changingthetransparencyofanoverlay,changingcolorranges,animation,etc.

5. Explorer (GeoServer) Geospatialdataservices,includingtheOpenGeospatialConsortium(OGC)standardsWebMapService(WMS),WebFeatureService(WFS)andWebCoverageService(WCS)areavailableviathePacIOOSGeoServer.GeoServerisanopen-sourceserverwritteninJavathatallowsuserstoshareandeditgeospatialdata.Designedforinteroperability,itpublishesdatafromanymajorspatialdatasourceusingopenstandards.GeoServerhasevolvedtobecomeaneasymethodofconnectingexistinginformationtoVirtualGlobessuchasGoogleEarthandNASAWorldWindaswellastoweb-basedmapssuchasOpenLayers,GoogleMapsandBingMaps.GeoServerfunctionsasthereferenceimplementationoftheOGCWFSstandard,andalsoimplementstheWMSandWCSspecifications.GeoServerreadsavarietyofformats,includingPostGIS43,Shapefiles,GeoTIFF,etc.,andthroughstandardprotocolsitcanproduceKML,GML,Shapefile44,GeoRSS,PDF,

43PostGISisanopen-sourceprogramthataddssupportforgeographicobjectstothePostgreSQLobject-relationaldatabase.PostGISfollowstheSimpleFeaturesforSQLspecificationfromOGC.44OpenLayersisanopensourceJavaScriptlibraryfordisplayingmapdatainwebbrowsers.ItprovidesanAPIforbuildingrichweb-basedgeographicapplicationssimilartoGoogleMaps.

240

GeoJSON,GIF,andmore.ItalsoincludesanintegratedOpenLayersclientforpreviewingdatalayers.Figure13isaschematicoftheservice.InthePacIOOSimplementation,thedatabaseisPostgres,theinterfaceisPostGIS;thewebbrowserisOpenLayers.

Figure43.SchematicrepresentationofGeoServer.

ThePacIOOSGeoServerishostedonseparateservermachine,andtheinterfaceisfoundat:http://pacioos-mapserver2.ancl.hawaii.edu:8080/geoexplorer/index_pacioos.htmlFigure14showsthedefaultentrypage.NotethatthePacIOOSwebsitelinkstoindividualdomainsviathe“dashboard”.Forexample,iftheregion“AmericanSamoa”isselected,then“PacificDataExplorer”linkstothedomainofAmericanSamoa(andnottheentirePacific).

241

Figure44.PacIOOSGeoServerpage.

Differentdata/layersareobtainedbyselectingthe“Overlays”folder.ThisbringsupawindowwhereinausercanselectdatafromthePacIOOSdatabaseorotherdistributeddata.Atthetopareiconsthatwhenclickedprovidesomemappingtools(print,rulers,metadata,pan,zoom)andaGoogleEarthoption.

C. Discovery Tools Theabilitytosearchfordataanddata-servicesviathewebishandledwithaWebCatalogService(WCS).Thistypeofserviceworksasalinkbetweenmetadata(bothaboutdataanddataservices)anduserrequeststoandaboutdataholdings.TheOGCCatalogServicedefinescommoninterfacestodiscover,browseandquerymetadataaboutdata,servicesandotherresources.InadditiontotheWCS,auserinterfaceisusedtoassistinqueries.Thesearetypicallycomposedofmaps,pull-downsorfill-inboxesthatusersentertheirsearchcriteria.Thebackengine,then,istheCWSthatsearchesbothlocal,andoptionallyremote,datacatalogs.Datacatalogsfromremoteserversthatareaccessinacatalogservicearereferredtoas“brokeredresources”.Apremiumisthereforeplacedonmetadata;havingmetadatainarecognizedstandardallowsthedatasettobe“harvested”bythecatalogservice.Beyondthe

242

metadata,thelayout,orsearchprocessisalsodoneinastandardorspecificway.These“publishedinterfaces”includethingslike

• CSW/ebRIM-CIM e-businessRegistryInformationModel• CSW/ebRIM-EO• CSW/ISO• CSW/ISO-GEO• ESRI-GEOPORTAL• GENESI-DR• GI-CAT• OPENSEARCH• OPENSEARCH-ESIP• OPENSEARCH-GEO• OAI-PMH

Andmetadatastandardsare1. ISO19115/19115part2/19139-descriptionsofgeodata,datasets,

imagery2. FGDCContentStandard-descriptionsofgeodatasets3. FGDCBiologicalDataProfile-descriptionsofbiologicaldata4. DarwinCore-descriptionsofbiologicaldata5. DDMS-descriptionsoffeaturetypesandschemas6. ABCD-descriptionsofspeciesdatasets7. EO-HMA-descriptionsofearthobservationproducts8. DublinCore-“universal”descriptionsofresources

Anddata-servicesthathandlemetadataare:• ATOM• CDI• CORE/SIMPLE• CSW/CORE,CCMD,ISO,OPENSEARCH,ebRIM,ebRIM-CIM,ebRIM-EO• DEGREE• DIF• DOWNLOAD• ESRI-GEOPORTAL• FILE• GBIF• GDACS• GEONETWORK• GEORSS• GI-AXE• GI-CAT• ISO19115-2• NCML-CF• NCML-OD• NETCDF-CF• OPENSEARCH

243

• SOS• STATIC• THREDDS,THREDDS-NCISO,THREDDS-NCISO-PLUS• WAF• WCS/WFS/WMS/WPS• Web20

PacIOOSiscurrentlyrunningtwoCWS:RAMADDAandGI-CAT.Thesearedescribedinmoredetailnext.Eventuallyonemightbedroppeddependingonexperiences.

1. RAMADDA TheRepositoryforArchiving,ManagingandAccessingDiverseDAta(RAMADDA)isacatalogservicedevelopedatUnidata(likeTHREDDS).Moreprecisely,itisapublishingplatformandcontentmanagementsystem.Itcanruneitherin“standalone”mode(asacommand-lineprogram)orviatomcat.PacIOOSinitiallyranramadda-1.1withintomcat,anditseemedtocauseproblemswiththeotherservices(tomcatwasrestartingeveryday).Withtheupgradetoramadda-1.3,thechoicewasmadetousethestandaloneversion.

a) configuration TheRAMADDAfilesarein/usr/local/ramadda-1.3,anditisstartedbyrunning:sh ramadda.sh & ThisstartsRAMADDArunning,andintheprogramport8082isspecified(RCFproxiesthis).Theinterfaceisthenavailableathttp://oos.soest.hawaii.edu/repository.Thereisanadministratorlogin(usernameramadda_admin,passwordisthesameasourtomcatmanager).TheadministrationpageisshowninError!Referencesourcenotfound..Onceloggedinasadministrator,newcatalogitemscanbeenteredandsitecontentandlayoutcanbechanged.

244

Figure45.RAMADDAAdministrationpage.

Theoptionsarefolder,file,link,homepage,server-sidefiles,remoteftpandopendaplink.

b) example usage RAMADDAisorganizedaroundahierarchicalcollectionofFoldersandEntries.Youcanopenafolderbyclickingonitsfoldericon.Mousingoveranentryoffolderwillshowasmallpopupiconlistingentriesinthatitem;clickingonitwillshowmoredetailedinformation.

245

Figure46.ExampleRAMADDAdatapage.

Filesmaybeaddedindividually,orviatheadministrativepagethroughtheTREDDScatalog.ForPacIOOSwejustloadtheTDScatalog.ThemainuseofRAMADDAistosearchfordata.Thisisinitiatedviathe“search”optionatthetop.

2. GI-CAT GI-CATisabrokercatalogservicemadebyESSI-Lab.Itallowsclientstodiscoverandevaluategeo-informationresourcesoverafederationofdatasources.Italsopublishesdifferentcataloginterfaces,allowingdifferentclientstousetheservice.ThisisarelativelynewutilityemployedbyPacIOOS,soitisnotyetfullyoperational.Inadditiontothecatalogservice,GI-CATcomeswithaweb-baseddatasearchtoolcallGI-portal.

a) configuration TheconfigurationofGI-CATisdoneon-lineusingGI-conf.TheconfiguratorislinkedoffthemainGI-CATpage,ordirectlyathttp://oos.soest.hawaii.edu/gi-cat/gi-conf/index.html.Onceloggedin,theconfigurationtoolallowsfor“globalsettings”andconfiguration.TheglobalsettingsincludepasswordandendpointforGI-CAT;homepagetitle,subtitleandlogo;proxyandharvestingsettings.Initially,onlytheendpointhasbeenchangedfromdefaulttohttp://oos.soest.hawaii.edu/gi-cat.

246

Theconfigurepagestartswith“configurations”.The“PacIOOS”configurationwasaddedusing“new”.Oncecreated,theconfigurationwas“opened”andmodifiedtohavespecific“publishedinterfaces”and“brokeredresources”.TheformerincludeCSW/ISO,GI-CAT,ESRI-GEOPORTALandOPENSEARCH;thelatterincludesthespecificcatalogsthatarebrokered(atpresentjustthePacIOOSTDSandGeoNetwork).Figure47showsanexampleconfigurationpage.

Figure47.GI-CATconfigurationpageforPacIOOS.

Eachindividualbrokeredresource,asadded,requiresatitle,endpoint,typeandharvesterschedule.Figure48showsanexample.Thekeyisthe“endpointfield”,whichistypicallytheTDScatalogpage,orinthecaseofGeoNetwork,thecswpage.SinceittakesalongtimetoharvesttheentirePacIOOScatalog,individualitemswereadded.Thislistwillcertainlybemodifiedandexpandedasthecatalogservicebecomesbetterunderstood.

247

Figure48.Examplesettingsforabrokeredresource(CoastWatchTDScatalog).

b) example usage TheusageofGI-CATismainlyviatheapplicationGI-portal.GI-portalaccessestheharvestedcatalogsinGI-CATandprovidesaninterfaceforuserstosearchfordatabyvariablename,location(lat/lon),ortime.Atpresentthistooljustrespondswithmetadatainformation,unlikeRAMADDAthatprovidesthelinkstotheactualdataservices.Itisunclearatthispointhowusefulthiswillbe.

3. GeoNetwork

D. Web pages ThePacIOOSwebpagesareobviouslythekeycomponentoftheentireendeavor,andcreationandmaintenanceofthesefallsupontheDMG.Dataservicesarealsointegrallytiedtothewebpages,sodataservicesandwebservicesworktogether.

248

Thewebpagesessentiallywentthroughthreeiterations.Thefirstwasputtogetherbyanad-hocgroupoflocaldevelopers,includingSharonDeCarlo,PaulJohnson,JimPotemra,PeterHacker,MarcieGrabowskiandothers.Theseinitialwebpageswereorganizedaroundthefourthemes,anddataserviceswerelimited.

Figure49.InitialPacIOOS(thenHiOOS)webpage.

Later,mainlythroughinputanddesignideasfromDougLuther,thepageswereredonetoaddressspecificusergroups(Figure17).Thetoppagehadlinksforsurfers,swimmers,etc.ThepagesweredevelopedbyChipFletcher’sprogrammer(ChynLim).Userswouldself-identifyandfindinformationaboutparticulartopics.Thesepageswereconfusingtosome,muchappreciatedbyothers.Inmid-2010,JenniferPatterson(formerlywiththeKiloNaluproject)washiredtocompletelyrevamptheHIOOSpages,intheprocessconverteverythingtothePacIOOSbrand,anddevelopthereal-timedataqualitypages.Thecurrentpageshaveaconsistentformatthatincludesa“dashboard”foreachregion.Thepagesareallservedfromlawelaweandareproxiedashttp://oos.soest.hawaii.edu,orequivalently,http://pacioos.org(thelatterpurchasedfromGoDaddy.com).Therearethreemainfeaturesonthewebsite:regional“dashboards”thatappearwhenspecificregionsareselected,IOOSgeneral

249

contentatthebottomofthepage(linkstootherregionsandpartners),andmenusalongthetop.

Figure50.SecondversionofthePacIOOSwebpage(stillreferredtoasHiOOS).

Thedashboardsincludereal-timeandforecastinformationaboutlocaltides,weather,waves,etc.

Figure51.PacIOOStoppage(http://pacioos.org).

250

Themenusatthetopprovidelinkstoallthesubsequentpages.Oneofthese,“FocusAreas”,containsinformationaboutthemaingroups,i.e.,waterquality,oceanconditions,modeling,etc.Anotherimportantlinkis“DataAccess”,whichprovideslinkstothedataservers,browsersandproducts.

E. Statistics AnimportantmetricforthePacIOOSDMSisaccessstatistics.Theseareseparatedbetweenhitstothewebpage,whicharetrackedviaGoogleAnalytics,andaccessviadataservices.ThelatteraretrackedwithAWStats.AdvancedWebStatistics,orAWStats,isafreetoolthatgeneratesadvancedweb,streaming,ftpormailserverstatistics,graphically.ThisloganalyzerworksasaCGIorfromcommandlineandshowsallpossibleinformationintheapacheloginagraphicalway.Itusesapartialinformationfiletobeabletoprocesslargelogfiles,oftenandquickly.ItcananalyzelogfilesfromallmajorservertoolslikeApachelogfiles(NCSAcombined/XLF/ELFlogformatorcommon/CLFlogformat),WebStar,IIS(W3Clogformat)andalotofotherweb,proxy,wap,streamingservers,mailserversandsomeftpservers.

a) configuration Atpresent,PacIOOSisnotrunninganftpserver,butAWStatsisconfiguredtoanalyzetheaccesstoTDS,ERDDAP,SOS,LAS,andDChart.BeforemovingthePacIOOSserverfromSOEST(POST-31)toITS,wewouldgetweblogsfromRCF.Nowweuseourown.Thesearefoundin/usr/local/apache/logsandarenamedaccess_log(forthecurrentlog).Theseget“rotated”everymonth,andthepastfourmonthsaresaved(e.g.,access_log-20170901.gzwillhavethelogsforthemonthofAugust).Awstatshasafewscriptsthatwillscanandparsetheapachelogfileandgetthestatisticsandthenautomaticallygenerateanhtmlfile.Thefirststepistopulloutalltheactivityforthepreviousdate;thisisdoneusinggreponthedatefield.Theoutputiswrittentoafile(/usr/local/awstats/logstats/oos.log).Next,grepisusedto“ignore”certainIPaddressesknowntoeitherbelocalorfromrobots.Finally,twoawstatsscriptsarerun,logresolvemerge.plandfixhostname.pl.Theresultingasciifileisthenfurtherparsedforeachserver,e.g.,TDS-specificcallsarewrittentotds.log,LAS-specificcallstolas.logandsoon.Oncewehaveserver-specificfilesforeachtimeperiod,awstats.plisruntoproducethewebpageswithupdatedstatistics.Itshouldbenotedthatoncethisscriptisrun,theinformationissavedin/usr/local/awstats/datainafilethathastheserver

251

anddateinthename,e.g.,awstats122016.pacioos-TDS.txt.Ifforsomereasonthestatsneedtobere-done,thisfilewillneedtoberemoved.Theentireprocessisdoneviathecronjobupdate_http_stats.sthatrunsdailyat03:10(thiswasdonesinceRCFrotatedtheirlogsat03:09daily).ItupdatesAWStats,e.g.,/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -config=pacioos-TDS update Thisisrunforeachconfigurationfile,andthereisoneforTDS,LAS,Dchart,ERDDAPandSOS.Theconfigurationfilesarein/usr/local/awstats/wwwroot/cgi-binandhavefilenamesawstats.pacioos-LAS.conf,fortheLASexample.Thekeylinesinthisfilearespecifications(pathandfilename)forthelogfile,domain,hostnameanddatadirectory.Thisfilealsohasthenameforthelogodisplayedinthewebstatspage.

b) example usage Figure52showsanexamplestatspagefortheTDSinMarch.Thepagecontainsstatisticsforwebhits,pages,downloadamounts,etc.

Figure52.AWStatspage.

252

F. Hardware Forhistoricalreasons,thePacIOOSdataserverhardwareisspreadacrossthreedifferentlocations,includingITS,HIGandKeller.Originally,itallstartedwithasingleserver(lawelawe)inPOST-31.Thismachinewasco-locatedwiththemodelingcluster.InMarchof2015bothweremovedtothenewITSbuilding.1.ITSTheoriginalhardwareconsistedofasingleDellserver(PowerEdge2950)and40TBXyratexRAIDarray.ThesewerepurchasedinApril2008andarereferredtoaslawelawe.soest.hawaii.edu,themainPacIOOSdataservermachine.SincethemovetoITS,thedomainnamewaschangedtopacioos.hawaii.edu(IPisnow128.171.123.50).ItisrunningCentOS-5andisthemainserverhostingthePacIOOSwebservices,THREDDS,ERDDAP,etc.;basicallyeverythingexceptthegeospatialstuff.Attachedviafibreisat48TBZyratex(48bayswith1TBdrives)RAIDarray.2.Keller-103AbackupserverwithattachedRAIDwaspurchasedinmid-2011.TheintentofthesemachinesweretobethemainPacIOOSservers,withlawelaweactingasthedevelopmentalmachine.Thetwoservers(DellPowerEdgeR610)werepurchasedinJuly2011,thetwoRAIDarrays(Infotrend48-TBenclosures)inMay2011.InJune2014asecondpurchasewasmade,thisforasingleDellserver(PowerEdgeR620)and96TBRAID.Thissystemisrunningavirtualserver,central.pacioos.hawaii.edu(128.171.128.2).Togetin,needtoconnecttotheVPN128.171.128.2(userjimp),thencansshusing192.168.100.#orthealternateIPlistedbelowinTable59:

IP AltIP Name192.168.100.2 JoeCentOS(MissionControl)192.168.199.3 Dellswitch192.168.100.9 VMAServer192.168.100.10 JoeNAS192.168.100.11 ESX1192.168.100.12 ESX2192.168.100.13 ESX3192.168.100.31 Infotrend-1192.168.100.32 Infotrend-2192.168.100.33 Infotrend-3192.168.100.37 CloneofSOW192.168.100.39 Brocadeswitch192.168.100.40 128.171.128.179 SonOfLawelawe192.168.100.41 Lawelawe_Sr192.168.100.42 128.171.128.6 Matlab_NFS192.168.100.43 128.171.128.181 KoheiGeoServer192.168.100.47 PacIOOS_BeachSafety_W2k8192.168.100.48 PACIOOS_MAP_SERVER2_W2k8192.168.100.49 PACIOOS_PGSQL_DB2_W2k8

253

192.168.100.50 128.171.128.3 Abe_GeoNode_Ubuntu192.168.100.58 PacIOOS1-345192.168.100.61 BBL072512_61192.168.100.62 BBLVM_62kilonalu-sharedPW4kn192.168.100.63 LawelaweSR192.168.100.64 JimPNOAAFisheries192.168.100.65 AT&TXP32bit192.168.100.101 128.171.128.2-6 pfSense2(All)192.168.100.102 128.171.128.178-191 pfSense176192.168.100.250 vSphereCenter5_w2008R2192.168.100.252 Windows2012R2HyperV192.168.100.253 CabinetAPCbar192.168.100.254 128.171.128.2-p3389 Vcenter.pacioos.local

Table59.PacIOOSVMclusterIPandmachinenames.

3.HIG-319ThereareanumberofmachinesinHIG-319thatwereacquiredoverthepastseveralyears.TheoldestmachineisaDellPowerEdge2950andrunstheBBLservices.ThismachinewasinheritedfromGenoPawlak.IthandlestheDataTurbineinstance.ThefirstpurchasedmachinewasinitiallycalledpacioosandwasintendedtohandleallthegeospatialservicesforPacIOOS.ThismachineisaDellPowerEdgeR510andwaspurchasedinFebruary2010.Itrunsthegeospatialservices(GeoServer,GeoExplorer,GeoNetwork)aswellastheBeachSafetySiteandtheDBEDTCableSurveySite.Inmid-2014asetofmachineswasacquiredfromtheoldEPSCoRgrant(machinestransferredfromUHHilo).

• DellPowerEdge2950(bbl)• DellPowerEdgeR510(geospatialstuff)• 3DellPowerEdgeR710• 2DellPowerEdgeR210• 1DellPowerEdge2950

Tag No. Asset Description MnfctrModel No. Serial No.

In Service Cost Location

146393 11453Infortrend 48TB RAID array Storehouse 24FG1840 ES450000A104900041 5/13/11 14,447.00 Keller

146394 11483Infortrend 48TB RAID array Storehouse 24FG1840 ES450000A104900042 5/13/11 14,447.00 Keller

146723 11637Server Dell PowerEdge Dell R610 26RZXQ1 7/19/11 5,480.64 Keller (pacioos2)

146724 11642Server Dell PowerEdge Dell R610 26R4YQ1 7/19/11 5,480.64

Keller (lawelawe2)

138148 9549Server Dell PowerEdge Dell 2950 III 3SLG2G1 4/22/08 5,233.03 ITS (lawelawe)

143198 10080Xyratex 48TB RAID array Hitachi

UltraStar STH77088 SHU808110002812 11/5/08 35,462.00 ITS (old RAID)

145245 10677Server Dell PowerEdge Dell R510 2168M1 2/25/10 5,216.64 HIG (pacioos)

144614 928Server Dell PowerEdge Dell 2950 9FF0W31 5/14/09 6,192.77 HIG (epscor)

147208 538Storage array with switch Dell MD3600F 9C4L6S1, DN9ZJN1 8/29/11 29,563.75 HIG (epscor)

147331 452 Server Dell Dell R710 FLZ84S1 9/25/11 12,704.09 HIG (epscor)

145783 Server Dell PowerEdge Dell R710 GRGFJN1 HIG (epscor)

254

145782 Server Dell PowerEdge Dell R710 6562NM1 HIG (epscor)

Server Dell PowerEdge Dell R210 6LZJJN1 HIG (epscor)

Server Dell PowerEdge Dell R210 7LZJJN1 HIG (epscor)

Server Dell PowerEdge Dell R620 11/1/14 5,441 Keller

Infortrend 96TB RAID array Storehouse 11/1/14 16,872 Keller

Table60.PacIOOSdataservingmachines.

G. Software Therearemanydifferentprogramsrunningonthevariousmachines.Error!Referencesourcenotfound.showsasummaryofthemainprogramsthatareneededforthedataservices.Server/Page Version Dependency ProtocolSupported Machine

apache-http 2.2.8 lawelaweapache-tomcat 8.0.24 lawelawejava 1.8.0_60 lawelawepython 2.7 DonotinstallPython

3+. lawelawe

THREDDS 4.6.3 Tomcat7+Java7+

OPeNDAPWMS(ncWMS)WCSSOS(ncSOS1.2.1)ISO19115(ncISO)NCSS

lawelawe

LAS 8.4rc2 ferret6.9.5Tomcat7+Java7+needsJDKforinstall(notjustJRE)TDSant1.8+

OPeNDAP(F-TDS) lawelawe

ferret 6.9.5 lawelawencSOS 1.2.1 TDS SOS lawelaweERDDAP 1.64 Java8+

Tomcat(preferablyitsowncontainer)

OPeNDAPISO19115FGDCRSSERDDAPRESTWMS(EPSG:4326)

lawelawe

Dapper/DChart TomcatJava

OPeNDAP lawelawe

GeoServer 2.4.8 TomcatJava7+PostgreSQL/PostGISother?

WMSWMS-CWMTSTMSWCSWFSGWC

geo

GeoWebCache tbd Tomcat WMS geo

255

Java WMS-CWMTSTMS

GeoExplorer custom GeoServerTomcat

geo

pycsw 1.8.2 Python2.6or2.7Apache(cgi-binOK,WSGIpreferred)pythonmodules:lxml,SQLAlchemy,pyproj,Shapely,OWSLib.

CSW lawelawe

PostgreSQL 9.1.3(lawelawe)

SQL lawelawegeo

PostGIS 2.0.0(lawelawe)

PostgreSQL9.1.3geos-3.3.3json-c-0.9

lawelawegeo

DataTurbine PHP 5.2.6

lawelawe

WordPress 4.3.1 PHPMySQL

SOL

MySQL 5.0.67 lawelaweImageMagick 6.7.9-6 LZMAcompression

-disable-openmp? lawelawe

MapServer 6.0.3 ApachePostgreSQL/PostGISlibpng,freetype2.x,GD(libgd)2.0.29+,zlib,libproj,libcurl,OGR,GDAL,AGG,libtiff,libgeotiff,libjpeg,GEOS4.10+,libxml,libpq

WMSWCSWFS

lawelawe

XSLT Pythonlibxml2Pythonlibxslt

lawelawe

MetadataPages XSLT(Pythonlibxml2andlibxslt)TDS(ncISO)ISO19115

lawelawe

StatusMonitor sendmailApachePython:scipy.io.netcdf,netCDF4.

lawelawe

Voyager TDSERDDAPMapServerGeoServerPythonImageMagickMySQLPostGIS/PostgreSQLCasperJS

lawelawe

256

wget,curl.ingestcrons

VoyagerMobile VoyagerGit(forPhoneGap>iOSapp)

lawelawe

Kiosks Troy’sPHPTDSJohn’skiosk.pyandget_weather.py(Apachecgi-bin)

lawelawe

ShorelineChange GeoServer SOLBeachSafety CableSurvey SentinelSite lawelaweSharkTracks VoyagerMobile

Voyager lawelawe

OceanAcidification John’sget_co2_data.pycron

lawelawe

SeaLevelRise GeoServer lawelaweCoralCover GeoServer lawelaweHabitatBlueprint GeoServer

TDSMapServerImageMagick

lawelawe

ICAC GeoServer lawelaweWaianaeOrdnance lawelaweFukushima lawelaweMamalaBay lawelaweWeatherStations ERDDAP lawelaweWaveGlidersPage lawelaweTable61.Dataserverversionsandrequirements.