Quality management, calibration, testing and comparison of instruments and observing systems M....

Quality management, calibration, testing and comparison of instruments and observing

systems

M. Leroy,

CIMO ET on SBII&CM

Quality management, some aspects

Explicit identification of uncertainty objectives Representativeness of observations Calibration and maintenance Selecting instruments : knowing their characteristics

– Testing– Intercomparisons

Composite observing systems Value analysis ISO 9001-2000 is an international frame

Quality management process

Setting and documenting objectives Defining associated indicators Regular survey of indicators Regular survey of users’ (customers) needs and

satisfaction. Correcting and improving

Identification of uncertainty objectives

Identification of users of observation data– Forecasters and climatologists– Few direct users of observation : special case is aeronautic

users, with needs clearly defined in Annex 3 of ICAO.

Getting the users’ needs is not always easy– They are not always aware of the possible uncertainty and

associated cost. – A value analysis is necessary to get a good compromise.

The CIMO guide (WMO doc n° 8) is a guide.– Annex 1B indicates users’ needs and achievable measurements

(best state of the art).

The measurement uncertainty must be demonstrated. So realistic choices must be done.

An example

For his proprietary Radome network, mainly dedicated for real-time observation, Météo-France has set up the following objectives; some are less ambitious than the achievable measurement uncertainty stated by the CIMO guide.– 0.5 hPa for pressure– 6% for relative humidity– 0.5°C for air temperature– 10% for wind speed– 5-10% for precipitation amount– 5% for daily amount of solar radiation.

For other objectives, additional care can be necessary

Example : US Climate Reference Network – Redundancy and cross check of measurements.– 3 independent air temperature measurements.– Dual fence rain gauge.– Associated measurement of wind speed.

Such efforts cannot be made in each station of a climatological cooperative network.

Quality indicators

Availability of data in the (final) data base– Within a given limit of time : example 95% of expected surface

observations available within 10 minutes.– Availability of data in the climatological data base : example 98%

for the Radome network. Measurement uncertainty

– % of sensors calibrated in a stated delay : example 12 months nominal, 15 months maximum, objective is more than 90%.

– % of sensors found outside an Acceptable User Limit : example hygrometers found with one (or more) control point more than 5% from the reference. These sensors are flagged as having drifted.

– % of sensors for which the periodicity of preventive maintenance is respected.

– % of large errors when compared to numerical models (monitoring)

Some examples

Knowing the characteristics of an instrument

Technological survey : reading publications, participating to conference and instrument exhibitions.

Testing of instruments : please, publish results and make them available.

Intercomparisons, WMO intercomparisons. Read the manufacturers’ technical

documentation, having in mind the common characteristics of instruments, for example, thru the CIMO guide.

Recent WMO intercomparisons

Radiosonde intercomparisons WMO Laboratory Intercomparison of Rainfall

Intensity gauges WMO Field Intercomparison of RI gauges : May

2007-May 2008, Vigna di Valle, Italy WMO Intercomparison of Thermometer

Screens/Shields in conjunction with Humidity Measurements : 2007, Ghardaïa, Algeria

WMO Field Intercomparison of Rainfall Intensity Instruments

The laboratory Intercomparison was a first step, easier to handle than a field test.

Not all instrument types available on the market were tested in laboratory.

Looking for a possible site, Vigna di Valle, Italy (Italian Meteorological Service, ReSMA) was selected.

August 2007 August 2008

Field Intercomparison

Call for instruments (March 2006) 24 instruments of different measuring principles were selected (54

proposed !). Reference based on a set of high quality devices, inserted in a pit

gauge. Draft Rec. 4.2/2 : Procedure and reference instruments for field

rainfall intensity intercomparisons

WMO Combined Intercomparison of Thermometer Screens/Shields, in conjunction with Humidity Measuring Instruments

Looking for a possible site in harsh conditions, Ghardaïa, Algeria was selected.

Desert region Jan. 2007 Jan. 2008 Instruments’ calibration before and after

Screens & Hygrometers Intercomparison

Call for instruments (March 2006) 16 screens/shields’ types (29 proposed) and 11 hygrometers’

types(17 proposed) were selected. Reference : Thygan Data analysis :

– International Organizing Committee

– ISO standard 17714 (draft)

Conclusion

There are many other subjects and actions concerning quality measurement :– Quality control directly at the level of the measurement systems.– Quality control using spatial and temporal cross-check.– Representativeness of observations, due to the nearby

environment of the measuring site.– …

Quality management and ISO 9001-2000 certification force us to explicit the objectives of observing networks and the proofs demonstrating their achievement.