Quality Audits

Supporting documentation

Bernhard, G., and G, Seckmeyer, “Uncertainty of measurements of spectral solar UV irradiance”, J. Geophys. Res., Vol. 104,

No. D12, 14,321-14,345 (1999).

Gardiner, B.G., “QA in ultraviolet spectroradiometry: the state of the art”, WMO QC/QC working group.

Heikkilä, A., et al., “SUVDAMA data storage Edition 4”, Final SUVDAMA Report (1999).

Seckmeyer, G., et al., “Instruments to measure solar ultraviolet radiation Part 1: Spectral instruments”, version 18 from the Final SUVDAMA Report (1999).

Webb, A., et al., “Guidelines for site quality control of UV monitoring”, WMO/GAW Rep. 126, World Meteorological Organization, Geneva, WMO TD No. 884, 39 pp.

Test audit

Objectives (1)

To provide assurance to data users that a certain level of quality has been achieved

“The site operator should be able to persuade an interested visitor that the measurement and collection of data are carried out diligently and to a standard that justifies any claims made to the quality of the data.”

Objectives (2)

To provide site operators with information that will allow them to improve the standard of their QC and QA procedures, and thus the value of their data

The final report will attempt to identify and promote best practice

Requisites (1)

Mission statement

The site operator should give a description of the site objectives in relation to the quantity and quality of data collected.

Requisites (2)

The site operator should be able to present a detailed and comprehensive description of the calibration, measurement and data analysis procedures, together with the results of their customary application.

The site operator should be able to make a quantitative statement about the accuracy of the measurements and be able to justify this assessment.

Checklist

1.    Location1.1.   the longitude, latitude and altitude of site;1.2.  the source and accuracy of these coordinates;1.3.  a description of the local environment;1.4. availability & accuracy of a horizon/alt map;1.5.  a description of nearby and tall objects;1.6.  site representative to local environment;1.7.  the local climate;1.8. photographs of horizon and measuring site;

2.    Operational matters2.1. site ‘mission statement’,;2.2. details of who is responsible for what;2.3. time dedicated to measurements;2.4. % time spent on maintenance etc.;2.5. % of operator’s working week on measurements;2.6. the experience of staff;2.7. how often staff change;2.8. the training of new staff;2.9. details of how often procedures have changed;2.10. description of instrument characteristics;2.11. the records of past intercomparisons;2.12 description of any publications read/adopted;2.13.  photographs of the operators;

3.    Instrumentation3.1.     list & description of instruments;3.2.     documentation of instrument characteristics;3.3.     Freq. of regular maintenance & characterisation;3.4.     maintenance record;3.5.     past changes in instrumentation;3.6.     planned changes to instrumentation;3.7.     measurement of angular response;3.8.     measurement procedure for slit function(s);3.9.     photographs of instruments; 4.    Calibrations4.1.     documentation;4.2.     certificates;4.3.     frequency of calibrations; 4.4.     definition of fiducial plane;4.5.     measurement of lamp distance;4.6.     method for wavelength calibration;4.7.     typical instrument drift between calibrations;4.8.     evidence that the calibration is reliable

at the quoted level of accuracy;4.9.     age of lamps;4.10.  level of agreement between lamps

(internal consistency);4.11.  level of repeatability/consistency in calibration;4.12.  steps taken to eliminate stray light;4.13.  evaluation with the JRC calibration equipment;

5.    Measurement regime5.1.     standard operating procedures;5.2.     type and quantity of data collected;5.3.     resolution and freq. of spectral measurements;5.4.     typical downtime for repairs/maintenance;5.5.     documentation;5.6.     demonstrable repeatability;

6.    Data Analysis6.1.    documentation;6.2.    analysis of errors; 6.3.   correction for errors; 6.4.     use of ancillary measurements;6.5.     estimates of relative/absolute errors;6.6.     application of WMO guidelines;6.7.     measurements of slit functions;6.8.     procedures for the application of

retrospective corrections;6.9.     data storage;6.10.  QC tools available and applied;

7.    Quality control7.1.     documentation;7.2.     standard procedures;7.3.     details of past/future instrument intercomparisons;7.4.     details of data version control;

Presentation of results (1)

.1 .2 .3 .4 .5 .6 .7 .8 .9 .10 .11 .12 .13 .14

1

2

3

4

5

6 A A A B B B A

7 C B B D C A A B

8A Exemplary

B Satisfactory

C Could be improved

D Should be improved

Presentation of results (2)

Formal statement (1)

Brief summary of the audit Containing: an overview (i.e., site objectives and practices), major

findings, conclusions The institute will have the opportunity to comment on this

statement, and to append further information or comments they consider appropriate, before it is circulated

Blank sections left for future updates

The formal statement will be in a format that can be used within the final report on the quality audits.

Formal statement (2)

IntroductionThis document is intended to provide an informal summary of the quality audit performed at RIVM over the course of the three days Monday 19 to Wednesday 21 March 2001. This audit was the first of some one dozen site visits that will be conducted over the next 18 months within the EDUCE project. It was planned as a ‘test’ audit, so that the audit procedures and timetable could be considered and refined. While I hope that the notes presented below will prove useful to both RIVM and the EDUCE coordinator, the trial nature of this first visit and the incompleteness of the audit mean that impressions and conclusions are necessarily preliminary.

Tim Martin

OverviewThis document is intended to provide an informal summary of the quality audit performed at RIVM over the course of the three days Monday 19 to Wednesday 21 March 2001. This audit was the first of some one dozen site visits that will be conducted over the next 18 months within the EDUCE project. It was planned as a ‘test’ audit, so that the audit procedures and timetable could be considered and refined. While I hope that the notes presented below will prove useful to both RIVM and the EDUCE coordinator, the trial nature of this first visit and the incompleteness of the audit mean that impressions and conclusions are necessarily preliminary.

In February, it was agreed that it would not be possible to observe a full calibration of a spectroradiometer during the course of the first visit to RIVM. For this reason, the sections of the audit report and summary relating to instrument calibration are incomplete. A calibration is planned for the return visit in October 2002, when RIVM also hope to be able to make use of the JRC travelling calibration kit.Three full days were provided for this visit. In future, only two days will be allowed.

I am most grateful to all at RIVM for their time, hospitality and helpful advice during the course of the visit.This document is sent to:

Best practiceThis document is intended to provide an informal summary of the quality audit performed at RIVM over the course of the three days Monday 19 to Wednesday 21 March 2001. This audit was the first of some one dozen site visits that will be conducted over the next 18 months within the EDUCE project. It was planned as a ‘test’ audit, so that the audit procedures and timetable could be considered and refined. While I hope that the notes presented below will prove useful to both RIVM and the EDUCE coordinator, the trial nature of this first visit and the incompleteness of the audit mean that impressions and conclusions are necessarily preliminary.

In February, it was agreed that it would not be possible to observe a full calibration of a spectroradiometer during the course of the first visit to RIVM. For this reason, the sections of the audit report and summary relating to instrument calibration are incomplete. A calibration is planned for the return visit in October 2002, when RIVM also hope to be able to make use of the JRC travelling calibration kit.Three full days were provided for this visit. In future, only two days will be allowed.

Room for improvementThis document is intended to provide an informal summary of the quality audit performed at RIVM over the course of the three days Monday 19 to Wednesday 21 March 2001. This audit was the first of some one dozen site visits that will be conducted over the next 18 months within the EDUCE project. It was planned as a ‘test’ audit, so that the audit procedures and timetable could be considered and refined. While I hope that the notes presented below will prove useful to both RIVM and the EDUCE coordinator, the trial nature of this first visit and the incompleteness of the audit mean that impressions and concl

I am most grateful to all at RIVM for their time, hospitality and helpful advice during the course of the visit.This document is sent to:

ConclusionsThis document is intended to provide an informal summary of the quality audit performed at RIVM over the course of the three days Monday 19 to Wednesday 21 March 2001. This audit was t=he first of some one dozen site visits that will be conducted over the next 18 months within the EDU CE project. It was planned as a ‘test’ audit, so that the audit procedures and timetable could be considered and refined. While I hope that the notes presented below will prove useful to both RIV EDUCE coordinator, the trial nature of this first visit and the incompleteness of the audit mean that impressions and conclusions are necessarily preliminary.

In February, it was agreed that it would not be possible to observe a full calibration of ort and summary relating to instrument calibration are incomplete. A calibration is planned for the return visit in October 2002, when RIVM also hope to be able to make use of the JRC travelling calibration kit.Three full days were provided for this visit. In future, only two days will be allowed.

Quality Audit at Institute – April 2002

Final report

The report will be circulated to all EDUCE PIs

The report will be made available at the database website (accessible only to registered users).

Some non-contentious information about the visit will also be placed at the public pages of the EDUCE web site.