Unlocking the Mystery of the Data Quality Act: What it Means for NOAA Research

Jamie Krauk,

Office of Scientific Support

August 6, 2002

Overview Section 515 – Data Quality Act Directives from OMB What is “Information”? The Three Elements of Quality: Utility, Integrity,

and Objectivity NOAA’s Information Standards Compliance Procedures Administrative Mechanism Implications for NOAA Research Discussion

Section 515 – Data Quality Act

Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554):– Directs the Office of Management and Budget

(OMB) to issue government wide guidelines that “provide policy and procedural guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information disseminated by Federal agencies”

Directives from OMB

OMB issued guidelines directing all Federal agencies to:– Create and issue agency-specific standards to

ensure Information Quality and develop a pre-dissemination review process

– Establish administrative mechanisms allowing affected persons to seek and obtain correction of information

– Report periodically to OMB on the number and nature of complaints received

What is “Information”?

Any communication or representation of knowledge such as facts or data, in any medium or form, including textual, numerical, cartographic, narrative, or audiovisual forms (includes web dissemination but not hyperlinks to information that others disseminate; does not include clearly stated opinions)

The Three Elements of Quality Utility

– The usefulness of the information to its intended users, including the public

Integrity– The protection and security of information from

unauthorized access or revision to prevent corruption or falsification

Objectivity– The presentation of accurate, reliable, unbiased

information in an accurate, clear, complete and unbiased manner within the proper context

NOAA’s Information Standards

Ensuring Utility NOAA strives to continually improve the

usefulness of its data and information products NOAA interacts with broad range of customers

through workshops, surveys, product reviews to assess and improve product utility and accessibility

NOAA disseminates products in a manner that allows them to be accessible and understandable to a broad range of users via print, diskettes, CD-ROM, internet, and broadcast media using consistent format and attribution

Ensuring Integrity Information is safeguarded from improper access

prior to dissemination to a degree commensurate with the risk associated with the information if it was compromised

All electronic disseminated information complies with “Security of Automated Information Resources”, Computer Security Act, and Government Information Systems Reform Act

Confidentiality of information is protected under the Privacy Act

Additional protections are provided as appropriate

Ensuring ObjectivityA) Original Data

B) Synthesized Products

C) Interpreted Products

D) Hydro-meteorological, Hazardous Chemical Spill, and Space Weather Warnings, Forecasts, and Alerts (Time Sensitive Products)

E) Experimental Products

F) Natural Resource Plans

G) Corporate and General Information

Objectivity – Original Data Data are collected according to documented

procedures or in a manner that reflects standard practices accepted by the relevant scientific and technical communities

Original data undergo quality control prior to being used by the agency, including:– Gross error checks, comparisons with other independent

sources of measurement, examination of statistical summaries, visual inspection

NOAA strives for transparency regarding data collection procedures, level of quality, and limitations

NOAA’s quality control process is ongoing

Objectivity – Synthesized Products Data and information sources are identified or made

available upon request All data is of known quality or from sources

acceptable to the relevant community Products are created using methods in standard

manuals, documented in an accessible format, or generally accepted by the relevant community

All synthesized products, or the procedures used to create them, are reviewed

All methods are included with product or made available upon request

Objectivity – Interpreted Products Data and information sources are properly

referenced or made available upon request Products are created using methods in standard

manuals, documented in an accessible format, or generally accepted by the relevant community

All products are put into context by additional information that demonstrates quality and limitations of products

All products are reviewed ranging from internal peer review to external, independent review

All methods are included with product or made available upon request

Objectivity – Time Sensitive Products

Data of known quality is used to the extent possible under tight time constraints

Data and information sources are identified or made available upon request

To the extent possible, information is produced using methods and techniques that are documented in accessible formats or are generally accepted

Accuracy and skill score of warnings, forecasts, and advisories is tracked as a mechanism for evaluation

Objectivity – Experimental Products Products are either disseminated for experimental

use, evaluation or feedback, or used in urgent, time-critical cases when qualified scientists believe risk will be lessened if the product is used

The best science and supporting studies available are used in accordance with sound and objective practices, evaluated in relevant communities, and peer reviewed if feasible

Provisional documentation of theory and methods are prepared, including assumptions

When products are used, they are accompanied by explicit limitations and used in conjunction with non-experimental methods and tools if possible

Objectivity – Natural Resource Plans Plans are developed according to published

standards and are often used as basis for policy Plans are a composite of several types of scientific

information from both internal and external sources and are based on the best information available

All supporting materials and data are properly referenced

Plans are reviewed by technically competent individuals ranging from internal to external independent review commensurate with importance

Objectivity – Corporate Information This is non-scientific, non-financial, non-

statistical information Sources of materials are noted as appropriate

and information is reliable and accurate to an acceptable degree of error

Product review is incorporated into the normal product formulation process, including:– Personal or supervisory review, process design

and monitoring, use of quality check lists, use of management controls

Compliance Procedures Read and understand all standards carefully and

ensure compliance by October 1, 2002 Identify what research products and information

fall into each category, and the corresponding standard for each

Prepare a current and updateable inventory of all scientific procedures and limitations for each type of information and have available upon request, including:– Methods, materials, assumptions, techniques– Process by which information was last reviewed– Date of last review, names of reviewers, actions taken

Administrative Mechanism Any affected person may request timely correction

of disseminated information Requests must be directed to NOAA Section 515

Officer (NOAA Executive Secretariat) for recording and forwarding to Lab/Program

Requests must contain all relevant information or the request is denied or returned to be amended and resubmitted

If the request is complete the Lab/Program director determines whether the request states a claim

Either way, a response is required within 60 days from original receipt to NOAA Section 515 Officer

Administrative Mechanism A claim is stated if the information was based on

misapplication or non-application of the appropriate standard

All supporting documents pertaining to the information in question must be gathered and filed as potential future administrative record

Lab/Program decides whether the information complies with the applicable standard– If yes, an explanatory response is drafted within 60 days

and requestor has 30 days to file an appeal to DAA– If no, Lab/Program decides whether the information can

be corrected, how, and when and then Lab/Program drafts a response within 60 days as above

Implications for NOAA Research

Paradigm shift in NOAA’s research enterprise?

Opens door for court cases and limits flexibility and scientific freedom

Required to keep impeccable notes Web community needs to be aware and

well-versed in standards in order to be the eyes and ears for NOAA Research

Questions?