LEADS/EMSOVERVIEW. 1 - 3 IPS MeteoStar February 22,2007 WHAT IS LEADS Leading Environmental Analysis and Display System Development Began At Lockheed

LEADS/EMSLEADS/EMS

OVERVIEWOVERVIEW

1 - 3 IPS MeteoStarFebruary 22,2007

WHAT IS LEADS

• Leading Environmental Analysis and Display System

• Development Began At Lockheed In Late 1989 To Produce A New Commercial Weather Analysis Product

• LEADS Grew Out Of Two Military Weather Contracts

– TESS(3) -- A Navy Program Which Uses Surface Observations And Numerical Forecast Models To Provide Tactical Weather Forecasts For Carrier Fleets

– Mark IVB -- An Air Force Program Which Uses Satellite Imagery And Analysis Tools To Provide Long-Range Strategic Forecasts

• In 1994 Lockheed Was Brought Under Contract With TCEQ To Add Pollution

• In 1997 Information Processing Systems (IPS) Acquired The MeteoStar Product Line


SYSTEM CAPABILITIES

• Near Real-time Data Acquisition– Pollution Sensor Data– Water Probe Data– Conventional Meteorological Surface Data– Weather Satellite Imagery (Geostationary)– Automated Gas Chromatograph (AutoGC)– Upper Air Data– Radar Profiler And Acoustic Sounder

• Visualization• Long-term Data Archival• EPA Reporting• Analysis

– Ozone Action Day Prediction– Transport Studies– Pollution Source Determination


LEADS DATA PYRAMID

DataCustomers

Review & ValidateData

Data Presentation & Analysis

Automatic QA/QC

Data Collection

EMS Configuration

LEADS/EMS System

Manual Validation EditorData Report Web Pages

AIRNowWeb Pages

EPA AQS SubmittalPolicy MakersPublic


SYSTEM INTEGRATION

• The Entire Data Collection System Is Closely Integrated• The Datalogger And Automatic Gas Calibrator Communicate

With Each Other Via A Serial Interface– The Interface Was Completely Defined By IPS– This Interface Results In Preliminary Data Flagging At The Monitoring Site

Based On What The Calibrator Is Doing

• The MeteoStar LEADS Software That Collects Data From The Datalogger Uses The Native Datalogger Computer-To-Computer Language (cc-sail)

• The Datalogger Keeps Track Of Which Data Records Have Been Retrieved

– Simplifies Retrieval Software– Retrieval Software Queries Datalogger For Latest Unsent Data Records– Data Collection Is Very Forgiving Of Missed Phone Calls, Busy Signals,

Network Outages, Etc.


DEFINITION OF A SITE

• A Monitoring Site Is Associated With A Fixed Location• LEADS/EMS Uses A Nine-Digit Number To Uniquely Identify

Each Monitoring Site– These Are Known As EPA Site Numbers Or AQS Numbers Or AIRS

Numbers– The First Two Digits Identify The State And The Next Three Identify The

County Where The Site Is Located; The Last Four Are Assigned By The Monitoring Entity And Are Often Not Unique Across A Network

– The EPA Has Tables Of All States And Counties

• At Each Monitoring Site, There Can Be One Or More Data Loggers

• Each Data Logger Requires A Unique Identifier– This Is The CAMS (Continuous Ambient Monitoring Station) Number– This Is An informal Site Identifier– There Can Be A Many-To-One Mapping Of CAMS To EPA Sites


POLLUTION DATA COLLECTION

• LEADS Uses A Multi-Tier Data Collection System– Reduces Load On Any Single Computer– Able To Rapidly Reconfigure the Communications Network In The Event Of A

Hardware Failure– Multi-Stage Archival Of Raw Measurements– Possible To Retrieve Data From Any Of The Archive Locations And

Reintroduce It Into The System

• The Datalogger Installed At Each Site Is Capable Of Storing One To Three Weeks Of Data Based On Number Of Instruments

• The Comms Front-End Processor Communicates With The Dataloggers And Retrieves Data From Them

– Preliminary Data Checks Are Performed– The CFEP Forwards Collected Data To The Main Or Central Processor– The CFEP Computers Store Three Months Of Data

• Once Data Arrives At The Central Processor, It Is Decoded, Checked For Errors, Processed And Stored In A Database


GETTING DATA FROM THE SITES

• Monitoring Sites Are Serviced By The CFEP• Communications Is A Balance Of Cost And Data Retrieval

– MeteoStar LEADS Nominally Collects Data From Every Site In The System Every 15 Minutes

– That Means 96 Connections To Every Site Each Day– For IDEM, Most Sites Are Connected Via Aircards

• Data Is Generally Available Within 20 - 22 Minutes From The Time It Is Collected (Based On A 15-Minute Comms Cycle)

• Data Generally Made Available To The Public Within An Hour

• Public Data Has Gone Through Automatic QA/QC Checks, But May Change Later Based On Human Review


DATA STORAGE

• The LEADS/EMS Data Base Uses A Flat, Packed-Binary Structure

• There Are Tables For Site Definitions, Parameter Definitions, Etc.

• The Binary Data Files Are Physically Segregated By Date And Monitoring Site

• Each Binary Data File Contains All The Measurements From A Monitoring Site For A Specific Day

• There Are Five Elements That Are Used To Construct A Unique Key For Data Storage And Retrieval

– Date The Sample Was Collected (In UTC)– Time The Sample Was Collected (In UTC)– The AQS Number– A Five-Digit Parameter Identifier– The Parameter Occurrence Code (POC)


LEADS QA/QC PHILOSOPHY

• Standardize Data Handling– Insure Consistency Across The Entire Network– Minimize Subjective Data Quality Judgements

• Automate As Much As Possible– Instrument Calibrations– Periodic Instrument Challenges– Data Quality Flagging

• Rigorously Check The Instruments In Each Station And The Calibration System

– Perform Automatic Instrument Challenges On A Schedule– Augment With Periodic Station Visits To Perform Independent Audits

• Provide A Set Of Tools For Operators, Validators, And Other Data Customers To View Not Only The Data But QA/QC Information And System Performance As Well


LEADS QA/QC IMPLEMENTATION

• TCEQ, IPS, Coastal (Datalogger), And Dasibi (Gas Calibrator) Developed Interfaces Between The Various Components And Software

– The Gas Calibrator Communicates Its Status To The Datalogger Which In Turn Sets The Initial Data Quality Flag

– IPS-Developed Software Examines The Flags Coming From The Datalogger And Determines Whether Or Not To Interpret The Data As Ambient Or Some Type Of Calibration Or Instrument Challenge

• MeteoStar LEADS Uses Established Methods For Challenging The Various Instruments

– Five-Point Calibration– Three-Point Span Check– Span-Zero Check– Limit Checks

• Your Existing Quality Assurance Project Plan (QAPP) Will Most Likely Need To Be Updated To Incorporate LEADS


OTHER QA/QC CONSIDERATIONS

• The Introduction Of LEADS Will Not Affect Some Things– Documentation Of Procedures And Processes– Traceability Of Measurements Back To A Basic Standard – Independent Instrument Calibration And Certification– Independent Station Audits– Instrument Maintenance And Upkeep Procedures

• Major Impacts Of LEADS– Ability To Expand Monitoring Network Without Increase In Manpower– Higher Data Return– High Confidence Level In Quality Of Data – Ability To Rapidly Furnish Data To The General Public


• There Are Several Types Of Automatic Quality Checks• Calibration Sequences Are Run Periodically Or When

Equipment Is Changed To Establish A New Slope And Intercept

• Span Check Sequences Are Run Weekly To Ascertain Whether Or Not A Particular Instrument Is Drifting Or Malfunctioning - May Also Set A New Intercept

• Span-Zero Check Sequences Are Run Nightly To Ascertain Whether Or Not A Particular Instrument Is Drifting Or Malfunctioning

• Any Of These Sequences Can Be Automatically Scheduled On Either The Calibrator Or The Datalogger

• Any Of These Sequences Can Be Manually Initiated As Necessary

QUALITY CHECKS


AUTOMATIC TESTS

Test 5-Point 3-Point 2-PointCompleteness X X XVoltage Outlier X X XConcentration Outlier X X XConcentration Spacing X X XSlope XIntercept XPrecision XLinearity XZero X X XSpan X X XNOx Balance Test X XNOx Converter Eff. Test X XH2S Converter Eff. Test X XH2s Scrubber Eff. Test X X


OTHER DATA CHECKS

• Automatic Met Data Checks

• Optional Upper And Lower Limits

• Optional NEG Test

• Optional MUL Test


OPERATIONAL DOCUMENTATION

• There Is An Electronic Operator Log On Each Datalogger– Web Interface For Viewing Operator Logs– Can Also Add Log Entries Via The Web– Certain Web Pages Also Trigger Automatic Operator Log Entries

• Any Change Made By A Data Validator Results In An Automatic Validator Note Entry

– Web Interface For Viewing Validator Notes– Certain Web Pages Also Trigger Automatic Validator Note Entries

• There Is A Web-Based Contact Manager That Can Be Used To Record Who Operates Sites And/Or Particular Instruments As Well As Who Validates The Data

• Web-Based Data Reports


WEB INTERFACE

• Web Pages Are Dynamic And Written In Perl• Web Pages Run Off A Combination Of Derived

Files And The Master Pollution Database– Derived Files Automatically Updated By Background

Processes– Derived Files Generally Contain Data Averages Or

Information That Is Not Stored In The LEADS Pollution Database

– Some Derived Files May Literally Take Hours To Update - The Derived Files Speed Delivery Of Information To Web Users

• Many Of The Web Pages Display System Health And Status Information


POLLUTION RETRIEVAL/DISPLAY

• Also Known As Manual Validation Or ManVal

• Lets You Examine 5-Minute Data As Well As Hourly Averages

• Data Graphically Displayed

– Up To One Month Of Data Can Be Displayed

• Data Validators Use This Interface To Edit Data– Requires Specific Privileges In Order To Make Any Changes

To Data– Can Change Data Flags– Can Change Slope And Intercept

• Wind And Pollution Roses Included In This Interface


MANUAL VALIDATION WINDOW


• Information About Each Data Point Is Displayed At The Bottom Of The Window

• The Information Displayed Is Dynamic And Is Based On The Data Point Located Under The Cursor

• Available Information Includes:– Date And Time Sample Was Collected

– Sample Value In Engineering Units

– EPA Data Flag

– The Slope And Intercept Applied To The Measured Voltages

– Whether Or Not The Data Point Has Been Validated (Two Indicators)

– Whether Or Not There Are Any Validator Notes Attached To The Data

• There Are Also Data Entry Areas For Manually Entered Slopes And Intercepts

STATUS INFORMATION


WHAT’S IN THE DISPLAY


• There Are Four Types Of Roses Available– Each Of The Roses Summarizes Data Over The Time Span You Select

– Data Is Accumulated Into Each Of The Direction Bins

• Wind Rose– This Is A Standard Wind Rose Plot Which Illustrates Wind Speed As A Function Of

Direction

• Average Concentration Rose– This Pollution Rose Will Display The Average Concentration Of A Particular Pollutant Or

Other Parameter Measured At A Monitoring Site As A Function Of Wind Direction

• Maximum Concentration Rose– This Pollution Rose Will Display The Maximum Concentration Of A Particular Pollutant Or

Other Parameter Measured At A Monitoring Site As A Function Of Wind Direction

• Percentile Rose– This Pollution Rose Allows You To Filter Pollution Concentrations In Order To Reject

Abnormally High Or Low Readings - The Smaller The Percentile You Select, The Less Data Is Included In The Pollution Rose

– Selecting A Percentile Of 100 Is The Same As Selecting A Maximum Concentration Rose

ROSE TYPES


WIND ROSE


POLLUTION ROSE


• Sequences Are Composed Of Levels• A Level Consists Of A Set Concentration From The

Calibrator Introduced Into A Monitor For A Set Number Of 5-Minute Sample Periods

• Each Level Is Assigned A Letter Code (M, R, S, T, Or G) By The Datalogger -- These Correspond To Various Percentages Of The Instrument Full-Scale

• A Set Number Of 5-Minute Samples (Usually 1 - 3) In Each Level Are Allowed For Instrument Stabilization

• The Remaining Samples (Usually 3 - 4) Are Processed By LEADS.

QC SEQUENCES


FIVE-POINT CALIBRATION


THREE-POINT SPAN CHECK


TWO-POINT SPAN-ZERO CHECK


• Incomplete– The Data Did Not Arrive In The Proper Sequence Or There Were Either

Too Few Or Too Many Samples At Each Level

• Invalid– The Calibrator Did Not Deliver The Challenge Gas At Correctly Spaced

Intervals

• Failed– One Or More Of The Automatic QC Checks Exceeded A Pre-Set Failure

Limit -- Data Is Automatically Rejected Accordingly

• Warning– One Or More Of The Automatic QC Checks Exceeded A Pre-Set Warning

Limit -- The Calibration Or Span Check Is Still Valid And Is Still Used

• Passed– All Automatic QC Checks Were Passed

POSSIBLE QC CHECK FLAGS


• Completeness Test

– Tests To See That Data Arrives In Correct Order And The Correct Number Of Updates For Each Level Are Present

• Voltage Outlier

– Tests Are Based On The Measured Voltages From The Instrument

– Tests To See That No Single Voltage Measurement For A Level Is More Than An Allowed Deviation From The Average Voltage

• Concentration Outlier

– Tests To See That No Single Concentration Measurement For A Level Is More Than An Allowed Deviation From The Average Concentration

– This Test Was Specifically Designed To Catch Calibration System Problems

AUTOMATIC TESTS


• NO2 Concentration Outlier

– Based Entirely On NO Channel Performance On Previous Non-Titration Level (M-Level) And Current Titration Level (M*-Level) -- NO2 Voltages Are Not Part Of This Test

– Tests To See That No Single Concentration Measurement For A Level Is More Than An Allowed Deviation From The Average Concentration

• Concentration Spacing

– Tests To See That The Calibrator Is Delivering Evenly Spaced Concentrations Throughout The Instrument’s Range

– No Concentration For A Level Can Be More Than An Allowed Deviation From The Ideal Concentration

AUTOMATIC TESTS (Cont.)


• NO2 Concentration Spacing

– Based Entirely On NO Channel Performance On Previous Non-Titration Level (M-Level) And Current Titration Level (M*-Level) -- Differs From Normal Concentration Spacing Test

– No Concentration For A Level Can Be More Than An Allowed Deviation From The Ideal Concentration

• Slope

– Tests To See That The Instrument Is Responding Correctly. All Instrument Responses Should Be Linear And The Calculated Slope Should Be Near The Ideal

– Ideal Slopes:

• CO = 20• Other Pollutants = 1000 (0 - 1000 ppb Instrument Range)• Other Pollutants = 2000 (0 - 500 ppb Instrument Range)



• Intercept

– Tests To See That The Instrument Is Responding Correctly And That The Calculated Intercept Is Near The Ideal

– Ideal Intercept = 0

• Precision

– Tests To See That The Instrument Has A Stable, Linear Response

– The Voltage Average For Each Level Is Compared To The Calculated Regression Line And Must Be Within Limits

• Linearity

– Tests To See That The Instrument Is Holding A Stable, Linear Response

– The Voltage Average For The T-Level Is Compared To The Line Drawn Between The M-Level And The G-Level And Must Be Within Limits



• Zero

– Tests To See That The Instrument Calibration Is Holding Since The Last Good Calibration Or Span

– The Voltage Average For The G-Level Is Compared To The G-Level From The Last Good Calibration Or Span And Must Be Within Limits

• Span

– Tests To See That The Instrument Calibration Is Holding Since The Last Good Calibration Or Span

– The Voltage Average For The M-Level Is Compared To The M-Level From The Last Good Calibration Or Span And Must Be Within Limits



• NOx Balance Test

– Tests The Electronic And Flow Balance Between The NO and NOx Channels. Compares The Ratio Of The Average NO2 Span Response To The Average NO Span Response During Non-Titration Steps. Sensitive To NO2 Impurities In NO Cylinder. Impurities Give Positive Errors.

– Only Affects The Validity Of The NO2 Channel Calibration

• NO2 Converter Efficiency Test

– Tests The Efficiency Of the NO2 NO Converter

– Affects The Validity Of The NO2 And NOx Channel Calibrations



• H2S Converter Efficiency Test

– Tests The Efficiency Of the H2S SO2 Converter

– Based On The T-Level And The T1-Level. Test Is Sensitive To Agreement Of The Calibrations Of The H2S Cylinder To The SO2 Cylinder.

• SO2 Scrubber Efficiency Test

– Tests The Efficiency Of the SO2 Scrubber



TEST LIMITS

• All QC Tests Performed On Calibration,Span Check, Or Span-Zero Data Have Both Warning And Failure Limits Except The Outlier Tests

• Each Outlier Test Uses Only One Limit But The Test Is Repeated If An Outlier Is Detected.

• Each Warning Limit Is Chosen Statistically To Represent The 3rd Standard Deviation Value About The Mean Error Of A Test

• There Should Be Only A 0.27% Probability Of Exceeding A Warning Limit If The Monitoring System Is Working Properly

• Each Failure Limit Is Intended To Represent The Maximum Error That Will Be Tolerated Without Invalidation Of The Affected Data


DATA VALIDATION RULES

• Automatic Invalidation Of Data Is Based Only On Whether A QC Test Passes Or Fails

• Warnings Are Not Considered In This Processing• Failure Of A Concentration Outlier Test Or A Concentration

Spacing Test Indicates A Problem With The Calibration System But Not With The Monitor (Except NO2)

• The Calibration Or Span Check Event Involved Is Considered Invalid And The Ambient Pollution Data Is Unaffected

• In The Event Of A Test Failure, Data Is Automatically Rejected (Flagged LIM)

– To Stop Rejecting Data Forward, A Successful Five-Point Calibration Must Be Run

– Data Is Rejected Back To The Last Successful Calibration, Span Check, Or Span-Zero Check


SPAN SOURCE AUDITS

• Automatic Quality Control Should Be Augmented By Quarterly Manual Span Source Audits

• Span Source Audits Evaluate The Accuracy Of The Calibrator At Each Site

• Corrective Action Is Required If Audit Limits Are Exceeded• If the Audit Passes, Then the Pollutant Monitor Should Be

Adjusted to Agree With the M-level Concentration Produced by the Calibrator

• This Sets the Slope of the Monitor’s Response to the Ideal• If a Slope Test Warning Is Reported Thereafter, Then Action

Must Be Taken to Determine If the Error Was Caused by Monitor Drift or Span Source Drift

• A Span Source Audit May Be Needed If There Are No Obvious Instrument Problems


UPPER AND LOWER LIMITS

• Limits Can Be Defined For A Specific Instrument At A Site

• Any Parameter Can Have Limit Checks Defined

• Can Define Either An Upper Or Lower Limit Or Both

• System Automatically Flags Any Data That Falls Outside Specified Limits As LIM

• LIM Data Is Automatically Excluded From Averaging And Automatic Alerts


AUTOMATIC MET CHECKS

Wind Speed Limits: Less than 0 or greater than 25 m/s (55.925 mph) Does not vary by .1 m/s (.2237 mph) over 3 hours Does not vary by .5 m/s (1.1185 mph) over 12 hours

Wind Direction Limits: Less than 0 or greater than 360 degrees Does not vary by 1 degree over 3 hours Does not vary by 10 degrees over 18 hours

Outdoor Temperature Value changes 15 C (27 F) degrees in one hour Value does not vary by .5 C (.9 F) degrees over 12 hours

Dewpoint Temperature Value is greater than ambient temperature Value changes 15 C (27 F) degrees in one hour Value does not vary by .5 C (.9 F) degrees over 12 hours

Precipitation Total is more than 25mm in one hour Total if more than 100mm in 24 hours

Barometric Pressure Value is less than 940 mb or greater than 1060 mb Value changes more than 6 mb in 3 hours

Solar Radiation Value is greater than 0.005 ly/min between 10 pm and 4 am Value greater than 1.94 ly/min anytime


NEG AND MUL TESTS

• Only Performed On Hourly Averages - Does Not Affect Five-Minute Averages

• NEG Test Can Be Defined For Any Parameter That Is Rolled Up Into An Hourly Average

– Test Will Automatically Set Slightly Negative Averages To 0.0

– If Data Is “Too” Negative, It Will Be Flagged NEG– Data Flagged NEG Is Automatically Excluded By

Other Displays And Analysis Algorithms

• MUL Test Designed Specifically For NOx Family– Tests To Make Sure That Sum Of NO And NO2

Measurements Are Consistent With NOx Measurement

– If Data Is Not Consistent, All Three Parameters Will Be Flagged MUL

– Data Flagged MUL Is Automatically Excluded By Other Displays And Analysis Algorithms


ROLES AND ACTIONS

• There Are Five Basic User Roles– System Administrator– EMS General Administrator– Data Validator– Field Operator– Weather Forecaster

• Everyone Has Access To The Same Basic Tools (Reviewing Data, System Operations, Etc.)

• Certain Roles Have Expanded Capabilities (Such As Changing System Configurations, Etc.)


SYSTEM ADMINISTRATOR

• This Is Administration At The Operating System Level

• Responsible For System User Accounts– Creation– Maintenance– File Ownership– File Permissions

• Creates Scheduled Tasks Related To Operating System

• Responsible For Data Backups And Restores


EMS GENERAL ADMINISTRATOR

• Administration At The EMS System Level• Adding New Sites• Monitoring Site Maintenance• Log Maintenance• General System Health• Scheduled Tasks Related To EMS• Setting Up And Maintaining Synchronization• Communications• Data Reloads• User Accounts For Web Pages And Manual

Validation


DATA VALIDATOR

• Primary Job Is To Ensure Data Meets Quality Objectives Prior To Full Release

• Manual Validation Tool Designed Around Data Validation

– Review Of Data– Data Flagging– Data Recovery

• Creates Reports For Submission To AQS (Used To Be Called AIRS) Database

• This Is The Only Role With ManVal Write Permissions


FIELD OPERATOR

• Responsible For Keeping The Monitoring Sites And Instruments Running

• Performs Physical Site Maintenance– Filter Replacement– Scheduled Instrument Maintenance– Instrument Removal/Replacement/Repair

• Should Be Using Available Tools To Review Data And System Performance

• Use The Operator Log To Keep A Record Of Everything You Do At The Site


WEATHER FORECASTER

• Has Access To Full LEADS Weather Toolkit• Can Create Products Overlaying Measured

Data From The Sites With National Weather Service Data

• Can Use The LEADS Tools To Create Custom Pollution Forecast Products

– Ozone Potential Zones– Pinpoint Weather Forecasts

• Should Be Involved With The Validation Of The Meteorological Data Measured At The Monitoring Sites


SYSTEM ARCHITECTURE


LEADS1 MACHINE

• Central Processor And Control Machine

• Master Data Repository

• Performs Majority Of Calculations Involving Data

• Synchronization Master

• Data Product Export

• Hosts Manual Validation Sessions

• Hosts Weather UI Sessions


NOAAPORT MACHINE

• Connected To The NOAAPort Antenna

• Ingests An Almost Constant Stream Of Data From The NOAAPort Satellite

• Data Products Ingested Include:

– Surface Observations From Around The World

– Multiple Forecast Models

– Satellite Imagery

– Radar Imagery

• The NOAAPort Machine Acts As A Server To Distribute Weather-Related Data To Clients


CFEP MACHINE

• COMMS Front-End Processor

• Handles All Communications Directly To The Sites Related To Data Collection

• Performs A Variety Of Pre-Processing On The Data From The Monitoring Sites

• Capable Of Distributing Data To Multiple Servers


DSR MACHINE

• The Internal Web Server

• Also Hosts The Same Web Pages Available To The General Public

• Majority Of The Internal Pages Are Technical And Are Dedicated To Some Aspect Of System Performance

• Users Of This Machine Are Assumed To Be Trusted (This Machine Can Only Be Reached From Inside The Agency Firewall)


WWW MACHINE

• The External Web Server

• Hosts The Web Pages Available To The General Public

• These Pages Are A Subset Of All Available Pages

• These Pages Have More Explanatory Text Than The Internal Counterparts

• Restricted To Hourly Averages Only

• Users Of These Pages Are Assumed To Be Unsavory At Best


FINI

This Concludes LEADS/EMS Overview

Documents

LEADS/EMSOVERVIEW. 1 - 3 IPS MeteoStar February 22,2007 WHAT IS LEADS Leading Environmental Analysis and Display System Development Began At Lockheed