Presented to: By: Date: Federal Aviation Administration Wide Area Augmentation System (WAAS) Operations Team AJW-1921 Offline Monitoring B. J. Potter Brad

Presented to:

By:

Date:

Federal AviationAdministration

Wide Area Augmentation System (WAAS) Operations Team

AJW-1921

Offline Monitoring

B. J. Potter

Brad Dworak

Chad Sherrell

March 7, 2011

WIPP

Offline Monitoring

2Federal AviationAdministration

2011-03-07

Introduction

• This presentation covers the 4th Quarter of 2010– (2010-10-01 – 2010-12-31)

• Next Steps– Analyze data for entire quarter– Transition all OLM analysis to SOS

Offline Monitoring


2011-03-07

Clock Runoff

• AssertionThe a priori probability of a GPS satellite failure resulting in a rapid change in the GPS clock correction is less than 1.0x10-4 per satellite.

• Monitoring Approach– Events typically result in a fast correction that exceeds 256

meters– When this occurs, the satellite is set Do Not Use until the

correction reaches a reasonable size– Events where the satellite is set Do Not Use from excessively

large fast corrections while the satellite is healthy are recorded

Offline Monitoring


2011-03-07

Clock Runoff

• No Clock Runoff Events between 2010-10-01 – 2010-12-31

Offline Monitoring


2011-03-07

Ephemeris

• AssertionThe CDF of GPS ephemeris errors in a Height, Cross-track, and Along-track (HCL) coordinate system is bounded by the CDF of a zero-mean Gaussian distribution along each axis

whose standard deviations are osp-ephh, osp-ephc, and osp-ephl. The probability that a satellite’s position error is not characterized by this a priori ephemeris model is less than 10-4 per hour.

• Monitoring Approach– Compare broadcast vs precise in HCL to ensure sigmas are

less than 1m, 2.5m, 7.5m for Radial, Cross Track, and In Track

Offline Monitoring


2011-03-07

Ephemeris – Radial

PRN: 112010-10-15 00:00:00-2.16619174736

PRN: 052010-10-23 23:45:00-1.25456410331

PRN: 312010-11-20 03:45:001.44416876193

PRN: 292010-12-16 02:45:00-1.28223089084

PRN: 252010-12-23 04:45:00-1.40505207557

PRN: 252010-12-24 06:00:00-1.29037947749

Offline Monitoring


2011-03-07

Ephemeris – In TrackPRN: 112010-10-15 04:00:0014.0549074655

PRN: 252010-12-24 07:45:00 14.6629813657

PRN: 242010-12-30 01:30:00 -15.7248821707

PRN: 302010-12-30 04:15:00 8.86779042477

PRN: 272010-11-07 20:15:008.39537775327

Offline Monitoring


2011-03-07

Ephemeris – Cross Track

PRN: 172010-12-01 06:45:00-2.73555408015

PRN: 212010-12-31 00:00:00-2.57918952043

Offline Monitoring


2011-03-07

RIC OutliersPRN R I C PRN R I C

1 0 0 0 17 0 0 92 0 0 0 18 0 0 03 0 0 0 19 0 0 04 0 0 0 20 0 0 05 1 0 0 21 0 0 16 0 0 0 22 1 0 07 3 0 0 23 0 0 08 2 0 0 24 110 18 09 6 0 0 25 42 56 0

10 8 0 0 26 0 0 011 9 24 0 27 1614 35 012 1 0 0 28 0 0 013 0 0 0 29 1 0 014 0 0 0 30 7 25 015 0 4 0 31 3 1 016 0 0 0 32 0 8 0

Offline Monitoring


2011-03-07

Ionospheric Threat Model Monitoring

• AssertionThe values of and iono adequately protect against worst case undersampled ionosphere over the life of any ionospheric correction message, when the storm detectors have not tripped.

• Monitoring Approach– Monitor for Chi^2 values greater than 1 in the four

regions • CONUS > 1%• Alaska > 2%• Caribbean > 10%• Other > 3%

decorrundersamp

Offline Monitoring


2011-03-07

Monitoring Regions

Offline Monitoring


2011-03-07

Offline Monitoring


2011-03-07

Offline Monitoring


2011-03-07

Offline Monitoring


2011-03-07

Total Chi2 values ≥ 1 from all regions for 2010 at ZLA (35.07% zeros)

Offline Monitoring


2011-03-07

2010 Total Chi2 Values Over 1

Offline Monitoring


2011-03-07

Antenna Monitoring

• Assertion

The position error (RSS) for each WAAS reference station antenna is 10cm or less when measured relative to the ITRF datum for any given epoch. (Mexico City is allowed 25cm). The ITRF datum version (realization) is the one consistent with WGS-84 and also used for positions of the GPS Operational Control Segment monitoring stations.

Offline Monitoring


2011-03-07

Purpose

• Accurate antenna positions needed to support DGPS applications

• Correct for Time Dependent Process– Tectonic Plate Movement– Subsidence

• Correct for Shift Events– Seismic– Maintenance

• WIPP Review for integrity issues– Greater than 10 cm WIPP should review– Greater than 25 cm WIPP must review– Special case for Mexico City (25 cm for review)

• Project the need for a WAAS Antenna Coordinate Update

Offline Monitoring


2011-03-07

Survey Details

• Survey Date– 2011-01-28

– Cross Compared Against– CSRS-PPP– WFO-R2– Coordinates Projected to six months beyond WFO

WFO Release 3– 2012-05-01

Offline Monitoring


2011-03-07

Results• Against CSRS-PPP

– All sites less than 5 cm.

• Against WFO-R2– All sites less than 5 cm.

Offline Monitoring


2011-03-07

Code Carrier Coherence

• AssertionThe a priori probability of a CCC failure is less than 1x10-4 per set of satellites in view per hour for GPS satellites and 1.14x10-4 for GEO satellites.

Offline Monitoring


2011-03-07

CCC monitoring approach Anik, Galaxy 15 and all GPS satellites are monitored for CCC trips for Q4

2010 (last data for CCC data for Galaxy 15 was on 2010-12-15). AMR is not currently monitored (not used as ranging source, UDRE

floor=50m) All CCC monitor trips are investigated whenever a trip occurs to determine

source of trip Minimum data sources used in correlation and analysis:

• CCC test statistic

• UDRE threshold value

• CMCI measurements from NETS SQA

• WAAS Iono calculation

• L1/L5 Iono GUST calculation

• published planetary Kp and Ap values

• Chi2 values

Offline Monitoring


2011-03-07

Reported CCC trips for Q4 2010

Date GEO PRN C&V2010-10-24 06:38:25 138 ZLA 2010-10-24 06:38:30 138 ZDC 2010-10-25 19:23:54 138 ZTL2010-10-25 19:24:05 138 ZLA 2010-12-08 11:55:40 138 ZDC ZTL2010-12-08 14:03:56 138 ZDC ZLA 2010-12-11 22:55:44 138 ZDC ZTL2010-12-12 02:45:52 138 ZDC ZTL2010-12-12 15:24:27 138 ZDC ZTL

Offline Monitoring


2011-03-07

CCC plots

Offline Monitoring


2011-03-07

CCC plots

Offline Monitoring


2011-03-07

Signal Quality Monitor

• AssertionThe a priori probability of a signal deformation (SD) failure is less than 2.4x10-5 per set of satellites in view per hour for GPS or GEO satellites.The worst-case range errors due to nominal signal deformations are more than 25cm on any satellite signal relative to the other satellites in view.

• Monitoring Approach– All SQM Trips will be monitored for and investigated– Max and Median data for each metric will be plotted by Requested

UDRE• Monitoring for discrepancies between satellite• Plots are for the first 4 days of every week for the entire quarter• Plots were made using the tools from HMI Build 299

Offline Monitoring


2011-03-07

SQM max plot

Offline Monitoring


2011-03-07

GEO Signal Quality

• AssertionThe WAAS SIS satisfies the requirements for code-carrier coherence and fractional coherence stated in sections 3.1.4.2 and 3.1.4.3 of the [draft] system specification FAA-E-2892c

• Monitoring Approach– Collect WAAS SIS data from each GEO using GUST receivers

connected to dish antennas– Compute and plot the metrics outlined in sections 3.1.4.2 and

3.1.4.3 of FAA-E-2892c– Examine plots, tabulate max metric values and pass/fail states,

analyze failures in further detail to identify possible causes

Offline Monitoring


2011-03-07

29

Performance Summary

L1 CCC L5 CCC CC

short-term

long-term

short-term

long-term

short-term

long-term

PRN 135

CRW

max

mean

PRN 138 CRE

max

mean

regularly below spec limit

near spec limitregularly above

spec limit

Offline Monitoring


2011-03-07

SummaryOnly issue: fly-by of CRW apparently affected measurements of both

CRE and CRW

Twice-daily elevated noise on some days

When CRW most N and S; bigger effect as CRE, CRW got close

Different sites saw effects of different magnitude

Additional periods of noise on Nov 12 (closest approach)

PR oscillation correction now included in processing

Mitigates systematic error in GUST receiver

L1 and L5 PR corrections mapped using GUST receiver and prototype SIGGEN in Zeta lab

Applicable to any WAAS GUST/G-II receiver

Dependent on PRN code, PR(t) and PR(t-1)

Allows more accurate evaluation of received signal

30

Offline Monitoring


2011-03-07

31

Example of Elevated Noise: 19 Nov 2010

• Elevated noise at approx. 2:21 and 14:19 UTC (N and S extremes of CRW orbit) -- close to but not at zero Doppler at either OKC or LTN (or APC Primary, which showed little to no effect at either time this day) caused higher than normal max CCC values

• Effect worse when CRW was close to CRE (effect also seen on CRE)

• Since different sites show different effects, probably not on SIS; will monitor as CRW returns to nominal orbit slot

OKC PRN 135 LTN PRN 135

Offline Monitoring


2011-03-07

32

Example of PR Correction Effect: 19 Nov 2010

• PR oscillation correction generally benefits L1 more than L5 and PRN 138 more than PRN 135 (oscillation signatures have different magnitudes)

• CRE performance marginal without correction but well below spec limits with correction (nominally; not including CRW fly-by effects)

• Since oscillations are a systematic receiver effect, mitigation allows better evaluation of received signal

OKC PRN 138 before correction OKC PRN 138 after correction

Offline Monitoring


2011-03-07

33

PRN 135 Short-term CCC

Note: missing values indicate days with switchovers or incomplete data

CRW passes CRE Nov 12

Offline Monitoring


2011-03-07

34

PRN 135 Long-term CCC


Offline Monitoring


2011-03-07

35

PRN 135 Short-term CC


Offline Monitoring


2011-03-07

36

PRN 135 Long-term CC


Offline Monitoring


2011-03-07

37

PRN 138 Short-term CCC


Offline Monitoring


2011-03-07

38

PRN 138 Long-term CCC


Offline Monitoring


2011-03-07

39

PRN 138 Short-term CC


Offline Monitoring


2011-03-07

40

PRN 138 Long-term CC


Offline Monitoring


2011-03-07

Code Noise and Multipath (CNMP) OverboundingAssertion

The Code Noise and Multipath (CNMP) error bound is sufficiently conservative such that the error in linear combinations of L1 and L2 measurements is overbounded by a Gaussian distribution with a sigma described by the Root Sum Square (RSS) of L1 and L2 CNMP error bounds except for biases, which are handled separately.3

Monitoring Approach Bounding for L1, IFPR, DelayAggregate and WRE SlicesAll bounding failures analyzed in further detail

Offline Monitoring


2011-03-07

Equations Used•Cumulative distribution function (CDF):

•For examining the behavior at larger values of x:

•Pass is Δx > 0 for all |x|>0.25

x

tR dtex 22

2

1)(

0

)(1

)(1)(1

0)(

)()(

)(

xx

xx

xx

xx

x

Rtheory

Rdata

Rtheory

Rtheory

Rdata

Rtheory

Offline Monitoring


2011-03-07

Aggregate Plot of CNMP Delay

Offline Monitoring


2011-03-07

Aggregate Plot of CNMP IFPR

Offline Monitoring


2011-03-07

Aggregate Plot of CNMP RDL1

Offline Monitoring


2011-03-07

CNMP Tabular Results from Poor Performing WRE Slices

Sliced by WRE Legend:

- = passed

X = did not pass

WRE #, WRE Name

L1 IFPR Delay

μ σ |max|

pass/ fail

μ σ |max|pass/ fail

μ σ |max|

pass/ fail

29, Houston C 0.032 0.46 3.09 - 0.058 0.42 3.31 - -0.064 0.40 3.10 -

44, Salt Lake C -0.071 0.27 1.69 - -0.026 0.99 2.15 - 0.002 0.26 2.20 -

07, Anchorage B -0.096 0.33 2.39 - -0.038 0.33 2.03 - 0.012 0.33 2.18 -

99, Goose Bay A 0.029 0.21 1.81 - 0.040 0.17 1.34 - -0.041 0.15 1.01 -

79, Bethel B -0.014 0.18 1.27 - -0.033 0.22 1.83 - 0.021 0.21 1.53 -

*This is a subset of sites as an example

Offline Monitoring


2011-03-07

Summary• Quarterly monitoring results continue to support

specific assertions called for in the HMI document.

• All antenna positions are within 5 cm.

• The CCC Test Statistic for the GEOs is ????

Offline Monitoring


2011-03-0748

Offline Monitoring DocumentReport format is separated into 3 hierarchical reading levels:

Level 1: Executive summary2-3 page overview of the events

Level 2: Main body~30 pages of technical briefings, limited number of graphs

Level 3: Materials and MethodsSupplemental information, including:

Additional Figures

Details of the tool configuration (build no, flag settings, etc.)

Data filenames and location (to possibly re-run in the future)

OLM coding standards and guidelines

First draft is scheduled to be released on March 31st

Offline Monitoring


2011-03-0749

Offline MonitoringData Types and Standards

Standards:

Slicing requirements – data from different sources are examined separately and not aggregated

UDRE index

PRN

Binning requirements – different bin sizes are used for different analyses (0.01, 0.001, etc.)

4 File Formats:

(1) Histogram files – histogram of raw counts of the metric (not probabilities), can be compiled together

Offline Monitoring


2011-03-0750

Offline MonitoringData Types and Standards

(2) Statistics files – each column in histogram file has a list (rows) of 15 descriptive statistics associated with it:

Counts

Mean

Standard deviation

Minimum, Maximum, Absolute maximum

Sigma over-bound (zero centered), Sigma over-bound (mean centered)

1st quartile, Median, 3rd quartile

Mean and standard deviation of absolute value

RMS

Variance

(3) Time series files: variable data over time

Time represented in WAAS time, UTC time (HHMMSS) and seconds into the day

Files can be concatenated together to form multi-day sets

(4) Quantity files: two-dimensional slices of any particular quantity (ex. UDREI/GPS PRN of |CCC metric|)

Documents

Presented to: By: Date: Federal Aviation Administration Wide Area Augmentation System (WAAS) Operations Team AJW-1921 Offline Monitoring B. J. Potter Brad