Batch processing with - Massachusetts Institute of Technologygeoweb.mit.edu/~floyd/courses/gg/201707_EOS/pdf/14-sh_gamit.pdf · Batch processing with sh_gamit M. A. Floyd Massachusetts

Batchprocessingwithsh_gamit

M.A.FloydMassachusettsInstituteofTechnology,Cambridge,MA,USA

GPSDataProcessingandAnalysiswithGAMIT/GLOBKEarthObservatoryofSingapore

17–21July2017

http://geoweb.mit.edu/~floyd/courses/gg/201707_EOS/MaterialfromR.W.King,T.A.Herring,M.A.Floyd(MIT)andS.C.McClusky (nowatANU)

Outline

• Setup,operationandoptionsforGAMITprocessingwithsh_gamit• Directorystructures• Mainfunctionsingamit

• ProgramscalledthatruntheGAMITprocessing• Filesthatareimportantinprocessing• Summaryfiles• Residualplots• Problemsthatcanhappenandsuggestions

2017/07/17 Batchprocessingwithsh_gamit 1

Overviewofsh_gamit:Gettingstarted

• Tostartsh_setup willcreate/tables,/rinex,/gsoln directoriesandthenlocalspecificscanbeset.• Intables/,process.defaults andsites.default arethetwomainfilesthatneedtobeedited;sittbl.mayalsoneededitingtoensuresomeconstrainedstationsinthenetworktobeprocessed;sestbl.iseditedifnon-standardprocessing.• Intables/,apriori coordinatefilecreated(nameinprocess.defaults).Additionalcoordinatesareputinto./tables/lfile.• Inrinex/,localRINEXfilesneedtobecopiedin;rinex datainarchiveswillautomaticallybedownloaded

• sh_gamit -expt [expt-name] -s [yr] [start-doy] [stop-doy]• Commonoptionsare:-dopt –copt –rx_doy_minus -netext


Directorystructure

• Toplevel:globaltablesandsurveydirectories• Withineachtop-leveldirectory:

brdc/gfiles/glbf/gsoln/igs/rinex/tables/day1/day2/(thesedirectoriesarecreatedasneeded)

• Generally50-60sitesisthelargestnetworkprocessedinGAMIT;largernumbersofstationsrequiresub-nettingofsites(seenetsel,global_sel andsh_network_sel).• Tablesarelinkedfromdaydirectoriestoexperimenttables/andthento~/gg/tables• GAMITprocessingoccursinthedaydirectories• GLOBKprocessingoccursingsoln/


Filesprovidedorcreatedautomatically

• Satelliteorbits• IGSsp3-files(tabular)and/org-files(ICsforGAMIT)• ARCintegratestogett-files(tabular)

• Earthorientationparameters(EOPs:ut1.;wob.)— downloadedifneededforcurrentday• Leap-secondfile— linkedtogg/tables(update~yearlyorwhenleapsecond)• Satelliteclock(j-)files— fromRINEXnavigation(brdc)file• Rcvr/antcharacteristics(rcvant.dat,hi.dat)— linkedtogg/tables• Differentialcodebiases(dcb.dat)— update~monthly• Antennaphasecentermodels(antmod.dat)— linkedtogg/tables(alsoneedstobeupdatedwhennewantennasadded).• Luni-solarephemeridesandnutation(soltab.,luntab.,nutabl.)linkedtogg/tables(needtoupdateyearly)• Oceantidegrid(optional)– linkedtogg/tables• Atmosphericloadinggrid(optional)– needtoupdateyearly• Mappingfunctiongrid(optional)– needtoupdateyearly


Filesyouneedtoworryabout

• RINEXfiles– localpluslistinsites.defaults• Controlfiles

• process.defaults :minoreditsforeachsurvey• sestbl.:experimentandmodelssetup(unchangedformostprocessing)• sites.defaults :sitestoincludeoromitandsourceofmetadata• sittbl.:sitesconstrainedforambiguityresolution• globk_comb.cmd :use_site,apr_neu,apr_svs,apr_wob,apr_ut1,sig_neu andmar_neu commands

• glorg_comb.cmd :apr_file,pos_org,stab_site commands

• a prioricoordinates(apr-file,l-file)• Metadata(station.info)• Differentialcodebiases(dcb.dat)– downloadcurrentvalues1/month• Satellitecharacteristics(svnav.dat)– downloadcurrentw/eachnewlaunch


process.defaults

• Controls:• Dataandprocessingdirectorystructure• Somesessionparameters(e.g.starttime,lengthanddatainterval,andapr-filename)• Peripheralbook-keeping(e.g.filestocompress,archiveordelete,andemailaddressforsummary)


sites.defaults

• Controlssitestobeinincludedinexperimentofgivenname• WhetherornotthesesitesshouldbedownloadedfromapublicFTPserver(use“ftprnx”flag)• WhetherornotthesesitesshouldhavetheirmetadataupdatedfromtheRINEXfileheader(notrecommended,particularlyforcontinuoussites;use“xstinfo”flag)

• Mayuseonesites.defaults filewithmultipleexperimentnamesanduse“-expt”optioninsh_gamit toprocessonlycertainsites• Mayuseadifferentexperimentsetupfordifferentdays• sh_gamit willlookfirstfor“sites.defaults.YYYYDDD”,thenageneric“sites.defaults”file


autcln.cmd

• Controlsallpartsofthephasecleaningalgorithm• Defaultsgenerallyworkwellforallexperiments• Mayoccasionallywishtochange:

• elevationmask• criteriatokeepmoredatafromsiteswithbadapriorico-ordinates


.apr-file

• Controlsapriori(input)coordinatesofsites• Convergenceof(non-linear)processingisabout1:1000,i.e.10maccuracyforapriorico-ordinatewillresultinfinalcoordinateaccuratetoabout10mm• Importanttohavegoodaprioricoordinates

• Utilitiesinclude:sh_rx2apr• .TheexperimentL-fileisinitializedeachdaywiththecoordiantes intheapr-filespecifiedinprocess.defaults (whileretaininganyentriesaddedduringpriorprocessingforsitesnotintheapr_file


station.info

• Controlssiteoccupationmetadata• Sitename• Startandstoptimesofoccupation• Receiverandantennainformation(types,serialnumbers,firmware,heights)

• Utilitiesincludesh_upd_stnfo whichinvokesprogrammstinf• Optionsformetadatainclude

• Pre-preparedstation.info (sh_upd_stnfo,make_stnfo)• Mustset“xstinfo”insites.defaults

• RINEXheaders(sh_gamit default butmaychangesoon)• Updatestation.info unlessanentryalreadyexistsforthedaybeingprocessedor“stinf_unique”issetto“-u”inprocess.defaults andentryhasnotchanged

• Canbeusedwithnon-standardreceiverandantennanamesspecifiedinguess_rcvant.dat(ideallyyourRINEXfileshavetheIGSofficialreceiverandantennanames.Itiscriticalthatthisinformationiscorrect

• THISISAVERYIMPORTANTFILE!• Ifyoudonotgetthisfilecorrect(andverified)beforeprocessing,youmaylosealotoftimereprocessingphasedataattheGAMIT(slowest)stage


sestbl.(“sessiontable”)

• Controlsprocessingsetup• Observablestouse(e.g.LC,L1+L2,etc.)• Experiment(orbitsandEOPs)type

• “BASELINE”solvesforsitecoordinatesonlyusingfixedorbitalparameters[default]• “ORBIT”solvesfororbitalparametersonlyusingfixedsitecoordinates(from.apr-file)• “RELAX.”solvesforbothsiteandorbitalparameters

• Modelsused


sittbl.(“sitestable”)

• Controls:• Site-specificinformationforprocessing

• Constraint(accuracy)ofaprioricoordinatesinapr-file


sh_gamit internaloperation

Thefollowingprogramsarerunbythescript:• makexp andmakex preparethedata• fixdrv preparesthebatchcontrolfiles• arc integratesGPSsatelliteorbits• model calculatestheoretical(modeled)phaseandpartialderivativesofphasewithrespecttoparameters• autcln repairscycleslips,removesphaseoutliers,andresolvesthewide-laneambiguities• solve estimatesparametersvialeast-squares,resolvingthenarrow-laneambiguitiesandcreatinganh-fileforGLOBK(userconstraintsareremovedintheh-filetoallowreferenceframedefinition)


StepsinthestandardGAMITbatchsequence

• arc,model,autcln,solve forinitialsolution• 5-minutesampling,noambiguityresolution(GCRonly)• updatelfile.forcoordinatesadjusted>30cm• lookat:autcln.prefit.sum; q<expt>p.ddd

• model,autcln,solve forfinalsolution• 2-minutesampling,ambiguityresolution• Lookat-->autcln.post.sum,q<expt>a.ddd

• FinalsolutionrepeatedifNRMSreducedby>30%frominitialsolution,toassuregoodeditingandlinearadjustmentofparameters(originalfinal-solutionfilesoverwritten)


Whatsolve produces

• Printoutputistheq-file,whichrecordsindetail• A constrainedsolutionwithoutambiguitiesresolved(GCR)• A constrainedsolutionwithambiguitiesresolved(GCX)• Thesearethesolutionsyoushouldexamine,alongwiththeautcln summaryfiles,toassessthequalityofthesolution

andinsummaryonly• A loosesolutionwithoutambiguitiesresolved(GLR)• Aloosesolutionwithambiguitiesresolved(GLX)

• Updatedl-fileforsuccessiveiterationsordays• UsefuloutputforGLOBKistheh-file(analogoustotheIGS-standardSINEXfile),whichcontainstheparametersestimatesandfullcovariancematrix.• (Thereisalsoano-file,whichisjusttheq-filebutinmoremachine-readableform,andisseldomused;and,iforbitsadjusted,anupdatedg-file)


Aprioricoordinates(sh_gamit)

• Createl-fileindaydirectorybymergingexistinglfile.andapr_filefrom../tables(apr_file haspriority)• Ifsitenotfoundinl-file• UseRINEXheadercoordinates(use_rxc=Yinprocess.defaults,goodformodern(postSA,in2000)data.

or• Usepseudorange datainRINEXfiletoestimatepointpositionordifferentialpositionrelativetoasiteinsites.defaults (use_rxc=N,default)

• Duringthesh_gamit run,thecoordinatesareupdated(andcopiedto../tables/lfile.)iftheyareinerrorby>30cm


Ambiguityresolution

• (L2−L1)integersresolvedbyautcln andpassedtosolve inthen-file(“LC_AUTCLN”optioninsestbl.)• Weakdependenceongeometry• Needcurrentdifferentialcodebiasfiledcb.dat• Use“LC_HELP”forcodelessdata(before~1995)orifproblems(defaultmaxdistanceis500km)

• Narrow-lane(L1)resolvedbysolve• strongdependenceonphasenoiseandmodels• 5–10cmconstraintsonaprioricoordinatesusuallysufficient


sh_gamit_<DDD>.summary(alsoemail)

• Preamble

Input options -d 2002 30 31 32 33 -expt ncar -pres ELEV -yrext -netext aProcessing 2002 031 GPS week 1151 4 Raw 2 /data51/tah/SENH02/glob02/suomi/2002_031aDisk Usage: 12678.4 Free 76447.4 Mbyte. Used 15%

• SummaryStatistics(fromautcln)

Number of stations used 4 Total xfiles 4Postfit RMS rms, to and by satelliteRMS IT Site All 01 02 03 04 05 06 07 08 09 ...RMS 20 ALL 4.8 4 5 6 5 5 4 5 4 5 ...Best and Worst two sites:RMS 20 TMGO 3.2 3 3 4 4 4 3 3 3 4 ...RMS 20 SA09 4.6 4 4 5 4 5 4 4 4 5 ...RMS 20 PLTC 5.4 4 5 5 6 5 4 5 5 6 ...RMS 20 SA13 5.5 5 5 6 5 5 5 5 5 6 ...


sh_gamit_<DDD>.summary(alsoemail)

• Solutionstatisticsfromsolve

Double difference statisticsPrefit nrms: 0.31280E+03 Postfit nrms: 0.21324E+00 (Constrainedfree)Prefit nrms: 0.31185E+03 Postfit nrms: 0.21818E+00 (Constrainedfixed)Prefit nrms: 0.31272E+03 Postfit nrms: 0.20470E+00 (Loosefree)Prefit nrms: 0.31185E+03 Postfit nrms: 0.20756E+00 (Loosefixed)Number of double differences: 12447Numbers of WL and NL biases 120 Percent fixed 95% WL 85% NL

Anylargeadjustmentstopositions(>0.3m)...

• Thingstonote:• Numberofstationsmatchesexpectation• Sitepostfit RMSvalues3–10mm• NostationswithRMS=0(impliesnodataretainedbyautcln )• Postfit nrms fromsolve ~0.2forconstrainedandloosesolutions• “Most”ambiguitiesresolved(70–85%fornoisydays,>90%forbest)


Phaseresidualplots

• Setwith“-pres elev”insh_gamit command(requiresGMT)• Postscriptfilesindaydirectory,bydefaultconvertedtoPNGinfigs/directoryandthenerased(requiresGMT’spsconvert orps2raster)• Usetoassessmultipath,watervapor,andantennaphasecentermodel

2017/07/17 Batchprocessingwithsh_gamit 21“Skyplot” Phasevselevationangle

Highresidualsinthesameplaceatdifferenttimessuggestmulitpath

Highresidualsappearinginagivenplaceonlyatonetimesuggestwatervapor

Skyplots


Phasevselevationangle


Normalpattern:bandsarehigh-frequencymultipath;redissmoothingofindividualvalues,showingnostrongsystematics.Mid-elevationanglenoisecouldbeatmosphericdelayerrors?

Badpattern:systematicsignatureofsmoothedvaluesindicatesapoormodeloftheantennaphasepattern(perhapsamisidentifiedantennainstation.info)

(Greenlinesshowtheelevation-dependentnoisemodelshownattopandusedtoreweightthedatainsolve)

Whatcangowrong?

• Sitemissing(notlisted)• noRINEXdatawithinsessionspan:checkRINEXfileand/ormakex.expt.infor• toofewdata,x-filetoosmallandnotused:checkRINEXfilesize,change“minxf”inprocess.defaults

• Siteinsolutionbutnodataoradjustment• aprioricoordinates>10moff:checkrangerms inautcln.prefit.sum,

• runsh_rx2apr differentiallyforseveralRINEXfiles• badreceiver:examineRINEXfilesorinitialc-fileswithcview

• q-filenrms >0.2• solutionover-constrained:checkGCXvsGLXnrms,rerunwithonlyonesiteconstrainedinthesittbl.


Problemswithaprioricoordinates

• Needtobegoodto<10mtogetthroughautcln• Safestsourceisaprevioussolutionorapseudorange solutionusingsvpos/svdiff (sh_rx2apr)• Rangerms andbiasflagsaddedfromautcln summaryfileareausefulcheck• Convergenceis1:100to1:1000(1merrorin.apr-filecanleadto1–10mmerrorinadjustment),henceautomaticupdateofl-fileforiterationofsecondGAMITsolution• Watchforrepeatedupdatesinemailsummaryasasignofbaddata


ConstrainingtheGAMITsolution

• Minimal(single-station)constraintisallthat’sneededforambiguityresolution,butsittbl.canlistseveraltoassureone• Orbitscanbefixedortightlyconstrained(.005ppm)forIGSorbitssinceatleast1996• Useof“BASELINE”modewithIGSForbitsfixednowrecommendedforprocessingregionsuptoatleast6000km

• Lookforgood(~0.2)loose(GLR/GLX)nrms butelevatedconstrainednrms (GCR/GCX)asindicationofanover-constrainedsolution


Moresubtleproblems

• Sitewithhighrms inautcln.post.sum• highmultipathing orwatervapor:checkskyplotsofphase• badreceiver:examineRINEXfilesorinitialc-fileswithcview

• Phasevselevationangleplotlargeandsystematic• misidentifiedantenna(wrongPCVmodel)• couplingbetweenantennaandmount

• GAMITresultswithinnormalrangebuttimeseriesshowsoutlier• survey-mode:antennanotleveledandcenteredovermark• changeinmultipath(water,objects)orwatervapor• snowonantenna• incorrectambiguityresolution(eastcomponentexceptforhighlatitudes)


Exampleofunderstandingoutliers

autcln RMS:• Day2019.6mm• Day2026.0mm• Noticeheightoutlieronday201


ALBH2003Day201

ALBH2003Day202


Documents

Batch processing with - Massachusetts Institute of Technologygeoweb.mit.edu/~floyd/courses/gg/201707_EOS/pdf/14-sh_gamit.pdf · Batch processing with sh_gamit M. A. Floyd Massachusetts