1

QUALITY MANAGEMENT, CALIBRATION, TESTING AND COMPARISON OF INSTRUMENTS AND OBSERVING SYSTEMS

WMO TECHNICAL CONFERENCE ON METEOROLOGICAL AND ENVIRONMENTAL INSTRUMENTS AND METHODS OF OBSERVATION TECO-2005 C. Bruce Baker, NOAA USA

2

Note the stability

The Backbone

QUALITY MANAGEMENT, CALIBRATION, TESTING AND COMPARISON OF INSTRUMENTS AND OBSERVING SYSTEMS

3

• Infrastructure in Place for Quality measurements

• Collects open access data and provides consistent quality assurance and control

• Distributes data and information (via multiple paths) in real time (varies with parameter) and ensures archival

• Abides by national / international standards and fosters the implementation of standards by local and regional observing systems

Functions of an International/National Backbone

4

• Management of Network Change

• Parallel Testing• Meta Data• Data Quality and Continuity• Integrated Environmental

Assessment• Complementary Data• Continuity of Purpose• Data and Meta Data Access

Key Components

5

MANAGEMENTDocumentation, Performance Measures, and Requirements

PROGRAM POLICY

Determined by Internationalor National Policy

and ScienceDriven Directives

QUALITY MANAGEMENT

SYSTEMPersonnel, Hardware, Ingest, and

Dissemination

QUALITY MANUAL

Requirements Documents

RESEARCHTesting, Intercomparisons, Transfer functions

Overlapping Measurements

IMPLEMENTATIONProgram Infrastructure

VOCABULARY

QUALITY CONTROL

Automated, Manual, Maintenance

QUALITY ASSURANCE

Documented Metadata, Performance Measures

6

• Functional Requirements– Systems - parameters, ranges, accuracies,

resolutions, expandability, design life, maintainability

– Program - number of systems, cost and schedule targets, communications

• Commissioning– Defines decision point – when data are official – Sustained operation, data from each site 95% of

the time within one hour and/or successful entry into the archives within 30 days

7

Configuration Management

• Change management of hardware and software items, metadata management responsibilities and procedures for CCB

8

Test and Evaluation Phase

• Conducted by Evaluation Team

• Reviewed by Ad Hoc Science Working Group

• Six areas Evaluated– Site Selection– Site Installation– Field Equipment and Sensors– Communications– Data Processing and Quality Control– Maintenance

9

5 Components of Data Quality Assurance (QA)

• Laboratory Calibration• Routine Maintenance and In-Field

Comparisons• Automated Quality Assurance• Manual Quality Assurance• Metadata, Metadata, Metadata• Ability to Integrate New technology

10

Laboratory Calibration

• Every sensor is calibrated before deployment to verify and improve upon the manufacturer’s specifications

• Sensors are routinely rotated back into the lab from the field to be re-calibrated

11

Routine Maintenance and In-Field Comparisons

Site Maintenance PassesThree visits scheduled annually

Trouble Ticket or Emergency Repairs

Malfunctioning SensorLightning StrikeCommunication ProblemsTheft and Vandalism

12

Site Maintenance Passes

Sensor Inspection

Air Temperature and Humidity sensors are inspected for dust accumulation, spider webbing and wasp nests. The radiation shields of these sensors are also cleaned.

13

Trouble Ticket or Emergency Repairs

Trouble Tickets •Issued by the Data QA Manager

•Priorities range from 2 to 30 business days (based on sensor)

•QA Manager provides a description of the problem

•Technicians complete the form with time of fix, serial numbers of sensors and a description of the repairs made

•Technicians may also generate tickets in the field and submit them to the QA Manager

14

Quality Assurance of Instruments

Documented in Anomaly Tracking System Users ManualReports of Incidences collected, evaluated, maintenance as neededMetadata records updated

Quality Control Data

Documented in Data Management – Ingest to AccessData ingest

Tests for proper message form, communication errors, etc.Automated

Limits - Gross limits checkVariance - Limits for individual parametersRedundancy - Data inter-comparison relies on multiple sensors

Manual -- Handbook of Manual Monitoring

15

Metadata ManagementSurvey to Operations

16

Communications Network

Field Sites

Instrument Suite

Communications Device

Processing Unit

Internet

User Community

Access

Ingest

Processing

Quality Control

Maintenance Notification

Maintenance Provider

Flagged-Data Archive

Raw-Data Archive

online

offline

17

114 CONUS Geographic Locations Required

• Captures 98% of variance in monthly temperature,

95% in annual precipitation for CONUS.• Average annual error <0.1ºC for temperature,

<1.5% for precipitation• Trend “errors” <0.05ºC per decade• IPCC: projects warming of 0.1-0.3ºC/decade and

precipitation changes of 0–2%/decade for CONUS.

Performance Measures

18

Determine the Actual Long-term Changes in Temperature and Precipitation of the Contiguous U.S. (CONUS)

FY2005 Target: Capture more than 96.9% and 91.1% of the temperature and precipitation trends.

19

RESEARCH

20

Cumulative Precipitation Gauge Comparison Sterling, VA

0.00

4.00

8.00

12.00

16.00

20.00

24.00

Geonor #2

Ott-704

Ott-754

Geonor #1

TB#1

TB#2

Frise-C1

Frise-D3

8"S

8"N

Ott-706

Ott-705

8" Std

8" DFIR

21

Tretyakov Shield with OttTretyakov Shield with Ott

22

Double Alter with GeonorDouble Alter with Geonor

23

24

Air Temperature & RH MonitoringAt

High Plains Regional Climate Center(Lincoln)

DewTrack MET2010 Standard RMY

New ASOS

ASOS

USCRN Shield

MMTS CRS Gill

Standard HMP243

PMT

25

26

Network Integration

27

Cross-Network Transfer Functions

Cooperative Observer Network (~10,000 Stations)

28

Planned USCRN Stations at end of 2008 (114* stations)

* Does not Include Alaska, Canada, Hawaii, & GCOS stations

Installed Paired Locations

Installed Single Locations

As of April 26, 2005

29

Experimental Product

30

Siting Standards Documents Representativeness

• Network Plan• Site Acquisition Plan• Site Information Handbook• Site Survey Plan• Site Survey Handbook• Site Survey Checklist• Site Acquisition Checklist

31

Major Principles of Station Siting

• Site is representative of climate of region.• Minimal microclimatic influences.• Long-term (50-100 year) land tenure• Minimal prospects for human development• Avoids agriculture, major water bodies, major forested

areas, basin terrain.• Accessible for calibration & maintenance.• Stable Host Agency or Organization.• Follows WMO Climate Station Siting Guidelines

32

Objective Site Scoring

• An objective scoring sheet was developed based on the Leroy method. The score for a station becomes part of the metadata for the station

• Re-scoring of stations is part of the annual maintenance visit; allows tracking time change in representativeness of station meteorology

33

34

35

• U.S Representative on the Canadian National Monitoring Change Management Board

• Canadian Reference Climate Network program participates on the USCRN Science Review Panel

• USCRN hardware architecture incorporated into Canadian Climate Monitoring Network

• Two nations will exchange and co-locate reference climate stations FY04

International Cooperation ,Collaboration and Partnerships

First step in international cooperation to have commonalityestablished for surface observing systems to monitor climate change

36

QUESTIONS

• How do we continue to expand International and National Partnerships??

• What is the best way for the exchange of information??

• How do we glue the system of systems together??

E-Mail: Bruce.Baker@noaa.gov

URL: http://www.ncdc.noaa.gov/oa/climate/uscrn/index.html

37

38

Network Characteristics

• Benchmark Network for temperature and precipitation• Anchor points for USHCN and full COOP network• Long-Term Stability of Observing Site (50+ years) likely to

be free from human encroachment • Sensors Calibrated to Traceable Standards• Planned redundancy of sensors and selected stations• Network Performance Monitoring - Hourly and Daily • Strong Science & Research Component