Upload
milo-wilson
View
219
Download
1
Tags:
Embed Size (px)
Citation preview
Meteorological Measurement Error Analysis
based on ANSI/ANS-3.11 (2005)
Kenneth G. WastrackKenneth G. Wastrack
Tennessee Valley AuthorityTennessee Valley Authority
NUMUG 2005NUMUG 2005
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
22
IntroductionIntroduction
Presentation TopicsPresentation Topics
Error Calculation HistoryError Calculation History ANSI/ANS-3.11 (2005) Method for Calculating ANSI/ANS-3.11 (2005) Method for Calculating
AccuracyAccuracy Example CalculationExample Calculation
Accuracy is the primary indicator of meteorological is the primary indicator of meteorological system performance. Observed accuracy must be system performance. Observed accuracy must be calculated in a consistent manner to permit calculated in a consistent manner to permit comparison with specified values. One of the key comparison with specified values. One of the key changes in ANSI/ANS-3.11 (2005) addresses this changes in ANSI/ANS-3.11 (2005) addresses this issue.issue.
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
33
Error Calculation HistoryError Calculation History
Section 4, "Instrument Accuracy," states plus or minus Section 4, "Instrument Accuracy," states plus or minus (±) accuracy values for different meteorological (±) accuracy values for different meteorological variables, but does not indicate if the accuracy applies variables, but does not indicate if the accuracy applies just to the sensors or to the entire data channel.just to the sensors or to the entire data channel.
No calculation methodology is defined.
Safety Guide 1.23, "Onsite Meteorological Programs," 1972
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
44
Error Calculation HistoryError Calculation History
Section 3, “System Accuracy”:Section 3, “System Accuracy”: System accuracy refers to the composite channel System accuracy refers to the composite channel
accuracy.accuracy. System accuracy defined as the root sum of the squares.System accuracy defined as the root sum of the squares. For time-averaged values, random errors may be For time-averaged values, random errors may be
decreased by considering the number of samples.decreased by considering the number of samples. The system accuracies similar to Safety Guide 1.23 (1972).The system accuracies similar to Safety Guide 1.23 (1972).
NRC defines calculation methodology.
Proposed Revision 1 to Regulatory Guide 1.23, "Meteorological Measurement Programs for Nuclear Power Plants," ~1981 (never issued)(never issued)
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
55
Error Calculation HistoryError Calculation History
Section 6.1, “System Accuracy”:Section 6.1, “System Accuracy”: System accuracy refers to the composite accuracy. System accuracy refers to the composite accuracy. System accuracy defined as the root sum of the System accuracy defined as the root sum of the
squares (squares (references “Brooks and Caruthers” methodsreferences “Brooks and Caruthers” methods). ). For time-averaged values, random errors may be For time-averaged values, random errors may be
decreased by considering the number of samples.decreased by considering the number of samples. The system accuracies are similar to those in RG 1.23 The system accuracies are similar to those in RG 1.23
(1972) and Proposed Revision 1 to RG 1.23 (1981).(1972) and Proposed Revision 1 to RG 1.23 (1981).
ANSI/ANS-2.5 (1984) fills gap in guidance.
ANSI/ANS-2.5 (1984), "Standard for Determining Meteorological Information at Nuclear Power Sites," 1984
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
66
Error Calculation HistoryError Calculation History
Not a guidance document, but provides insight into Not a guidance document, but provides insight into how ANSI/ANS-2.5 (1984) guidance was interpreted how ANSI/ANS-2.5 (1984) guidance was interpreted and used.and used.
Section 3, “Potential Sources of Error”:Section 3, “Potential Sources of Error”: System accuracy refers to the composite accuracy. System accuracy refers to the composite accuracy. Distinguishes between “bias” and “random” errors.Distinguishes between “bias” and “random” errors.
Illustrates implementation of ANSI/ANS-2.5 (1984) and provides example calculations.
NUMUG Presentation, "A Methodology for Calculating Meteorological Channel Accuracies," by Brad Harvey, 1999
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
77
Error Calculation HistoryError Calculation History
Section 7.1, “System Accuracy”:Section 7.1, “System Accuracy”: Calculation methodology is described as "root-mean-Calculation methodology is described as "root-mean-
square" (RMS) rather that "root sum of the squares" square" (RMS) rather that "root sum of the squares" (RSS).(RSS).
Does not clearly state that the calculation methodology Does not clearly state that the calculation methodology has changed--has changed--many users don’t realize the many users don’t realize the calculation methodology has changedcalculation methodology has changed. .
No longer distinguishes between instantaneous and No longer distinguishes between instantaneous and time-averaged values.time-averaged values.
Not certain changes were really intended.
ANSI/ANS-3.11 (2000), "Determining Meteorological Information at Nuclear Facilities," 2000
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
88
Error Calculation HistoryError Calculation History
Root-mean-square (RMS)
RMS =RMS =
Root sum of the squares (RSS)
RSS = (rRSS = (r11))22+ (r+ (r22))2 2 +. . . + (r+. . . + (rxx))22
ANSI/ANS-3.11 (2000), "Determining Meteorological Information at Nuclear Facilities," 2000 (continued) (continued)
222
21 xrrrRSS
(r(r11))22+ (r+ (r22))2 2 +. . . + (r+. . . + (rxx))22
xx
r1, r2 . . . rx are random error components.
x is the number of components.
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
99
Error Calculation HistoryError Calculation History
Section 7.1, “System Accuracy” has been revised and Section 7.1, “System Accuracy” has been revised and Exhibit 1, “Method for Calculating System Accuracy” has Exhibit 1, “Method for Calculating System Accuracy” has been added.been added.
ANSI/ANS-3.11 (2005) is intended as an ANSI/ANS-3.11 (2005) is intended as an officialofficial source source of guidance. of guidance.
ANSI/ANS-3.11 (2005) defines what is required by ANSI/ANS-3.11 (2005) defines what is required by including the actual equations and step-by-step including the actual equations and step-by-step instructions to ensure a consistent approach.instructions to ensure a consistent approach.
ANSI/ANS-3.11 (2005) reverts back to RSS. ANSI/ANS-3.11 (2005) reverts back to RSS. ANSI/ANS-3.11 (2005) distinguishes between bias errors ANSI/ANS-3.11 (2005) distinguishes between bias errors
and random errors.and random errors.
ANSI/ANS-3.11 (2005), "Determining Meteorological Information at Nuclear Facilities," 2005
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
1010
Error Calculation HistoryError Calculation History
Section 7.1 was principally written by Ken Wastrack.Section 7.1 was principally written by Ken Wastrack. Significant input from Paul Fransiloi (SAIC), Brad Significant input from Paul Fransiloi (SAIC), Brad
Harvey (NRC), and Matt Parker (Westinghouse Harvey (NRC), and Matt Parker (Westinghouse Savannah River).Savannah River).
Reviewed by about 30 members of the ANS-3.11 Reviewed by about 30 members of the ANS-3.11 working group.working group.
Comments from Stan Krivo (EPA) and Walt Schalk Comments from Stan Krivo (EPA) and Walt Schalk (NOAA).(NOAA).
ANSI/ANS-3.11 (2005), "Determining Meteorological Information at Nuclear Facilities," 2005 (continued) (continued)
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
1111
Error Calculation HistoryError Calculation History
Section 7.1 - System AccuracySection 7.1 - System Accuracy
Accuracy values shall reflect the performance of the total system, and Accuracy values shall reflect the performance of the total system, and shall be based on the more stringent of the individual facility shall be based on the more stringent of the individual facility requirements or the minimum system accuracy and resolution requirements or the minimum system accuracy and resolution requirements given in requirements given in Table 1Table 1 (which provides values for a monitoring (which provides values for a monitoring system using typical tower-mounted sensors and digital data processing system using typical tower-mounted sensors and digital data processing systems).systems).
System accuracy should be estimated by performing system calibrations, System accuracy should be estimated by performing system calibrations, or by calculating the overall accuracy based on the system's individual or by calculating the overall accuracy based on the system's individual components. Accuracy tests involve configuring the system near to components. Accuracy tests involve configuring the system near to normal operation, exposing the system to multiple known operating normal operation, exposing the system to multiple known operating conditions representative of normal operation, and observing results. conditions representative of normal operation, and observing results. Data channels may be separated into sequential components, as long as Data channels may be separated into sequential components, as long as results from each component are directly used as input for the next results from each component are directly used as input for the next component in sequence. component in sequence. Exhibit 1Exhibit 1 provides a method that should be used provides a method that should be used to calculate system accuracy from individual component accuracy values.to calculate system accuracy from individual component accuracy values.
ANSI/ANS-3.11 (2005), "Determining Meteorological Information at Nuclear Facilities," 2005 (continued) (continued)
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
1212
Method for Calculating System Method for Calculating System AccuracyAccuracy
[from ANSI/ANS-3.11 (2005) Exhibit 1][from ANSI/ANS-3.11 (2005) Exhibit 1]
1.1. Identify the individual components that Identify the individual components that contribute to system accuracy.contribute to system accuracy. SensorSensor Installation error (e.g., sensor alignment)Installation error (e.g., sensor alignment) Operational error (e.g., solar heating of temp. Operational error (e.g., solar heating of temp.
sensors)sensors) Data processing equipmentData processing equipment Computer (e.g., conversion equations)Computer (e.g., conversion equations) CalibrationsCalibrations etc.etc.
SA
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
1313
Method for Calculating System Method for Calculating System AccuracyAccuracy
[from ANSI/ANS-3.11 (2005) Exhibit 1][from ANSI/ANS-3.11 (2005) Exhibit 1]
2.2. Classify the error type for each component.Classify the error type for each component. Bias errors (b (b11, b, b22, . . . b, . . . bxx).).
Bias (or systematic) errors consistently affect Bias (or systematic) errors consistently affect the system accuracy in a known manner.the system accuracy in a known manner.
For example: Solar heating only For example: Solar heating only increasesincreases apparent air temperatures during daytime.apparent air temperatures during daytime.
Random errors (r (r11, r, r22, . . . r, . . . rxx).).
Random errors are independent and can Random errors are independent and can fluctuate within the range between the fluctuate within the range between the extreme maximum and minimum values.extreme maximum and minimum values.
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
1414
Method for Calculating System Method for Calculating System AccuracyAccuracy
[from ANSI/ANS-3.11 (2005) Exhibit 1][from ANSI/ANS-3.11 (2005) Exhibit 1]3.3. Estimate the values of the component errors Estimate the values of the component errors
based on engineering analysis, vendor based on engineering analysis, vendor specifications, accuracy tests, or operational specifications, accuracy tests, or operational experience.experience. If a bias applies to only a portion of the sampling If a bias applies to only a portion of the sampling
period, multiple calculations of the system accuracy, period, multiple calculations of the system accuracy, using different bias values, are necessary. using different bias values, are necessary.
The random errors of the individual components The random errors of the individual components should represent ±2should represent ±2 values or 2 times the standard values or 2 times the standard deviation of errors based on component testing deviation of errors based on component testing (95.5%). Unless otherwise stated, manufacturer's (95.5%). Unless otherwise stated, manufacturer's data for random errors can normally be assumed to data for random errors can normally be assumed to be ±2be ±2 values. values.
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
1515
Method for Calculating System Method for Calculating System AccuracyAccuracy
[from ANSI/ANS-3.11 (2005) Exhibit 1][from ANSI/ANS-3.11 (2005) Exhibit 1]4.4. Perform time-average adjustments for each Perform time-average adjustments for each
random error component.random error component.a = r/( n)a = r/( n)
Where:Where: r is the unadjusted random error r is the unadjusted random error
component.component. n is the number of samples.n is the number of samples. a is the adjusted random error component.a is the adjusted random error component.
Note:Note: For instantaneous values (where n=1), a = For instantaneous values (where n=1), a = r.r.
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
1616
Method for Calculating System Method for Calculating System AccuracyAccuracy
[from ANSI/ANS-3.11 (2005) Exhibit 1][from ANSI/ANS-3.11 (2005) Exhibit 1]
5.5. Calculate "root sum of the squares" (RSS) Calculate "root sum of the squares" (RSS) for the adjusted random error components.for the adjusted random error components.
RSS = (RSS = (aa11))22+ + ((aa22))2 2 +. . . + +. . . + ((aaxx))22
Where:Where: aa11, a, a22, . . . a, . . . axx are adjusted random error are adjusted random error
components.components.
x is the number of components.x is the number of components.
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
1717
Method for Calculating System Method for Calculating System AccuracyAccuracy
[from ANSI/ANS-3.11 (2005) Exhibit 1][from ANSI/ANS-3.11 (2005) Exhibit 1]6.6. Add bias errors to obtain system accuracy Add bias errors to obtain system accuracy
((SASA).).
SASA = RSS + b = RSS + b11 + b + b22 +. . . + b+. . . + bxx
Where:Where:bb11, b, b22, . . . b, . . . bxx are bias error components. are bias error components.
x is the number of components.x is the number of components.
Note:Note: Repeat as necessary with different bias values to determine extreme values.
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
1818
Method for Calculating System Method for Calculating System AccuracyAccuracy
[from ANSI/ANS-3.11 (2005) Exhibit 1][from ANSI/ANS-3.11 (2005) Exhibit 1]
7.7. Compare the extreme system accuracy (Compare the extreme system accuracy (SASA) ) values with applicable requirements to values with applicable requirements to evaluate system performance.evaluate system performance.• Table 1 in ANSI/ANS-3.11 (2005) for most cases.Table 1 in ANSI/ANS-3.11 (2005) for most cases.• Table 1 values are both system (channel) Table 1 values are both system (channel)
accuracy and sensor accuracy values.accuracy and sensor accuracy values.o System accuracy encompasses all channel components System accuracy encompasses all channel components
impacting system accuracy (sensors, data processing impacting system accuracy (sensors, data processing equipment, computer, calibrations, etc).equipment, computer, calibrations, etc).
o Sensor accuracy applies to the manufacturer’s Sensor accuracy applies to the manufacturer’s instrument specification.instrument specification.
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
1919
Example CalculationExample Calculation[Wind Speed – Ultrasonic Sensor][Wind Speed – Ultrasonic Sensor]
1.1. Identify Sources of Error.Identify Sources of Error.• SensorSensor
• Input – Laboratory Standard Input – Laboratory Standard • Input – Transfer StandardInput – Transfer Standard• Input – Test Position (placement in wind Input – Test Position (placement in wind
tunnel)tunnel)• Input – Comparison ApparatusInput – Comparison Apparatus• Output – ToleranceOutput – Tolerance
• Sampling (placement in field)Sampling (placement in field)• Signal Conditioning and Data LoggerSignal Conditioning and Data Logger
• Final rounding of hourly average valueFinal rounding of hourly average value
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
2020
Example CalculationExample Calculation[Wind Speed – Ultrasonic Sensor][Wind Speed – Ultrasonic Sensor]
2.2. Classify Error Type.Classify Error Type.• SensorSensor
• [Input] Sensor placement is [Input] Sensor placement is biasbias• [Input] Other sensor components are [Input] Other sensor components are randomrandom• [Output] Tolerance is [Output] Tolerance is randomrandom
• Sampling (placement) is Sampling (placement) is randomrandom• Final rounding is Final rounding is randomrandom
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
2121
Example CalculationExample Calculation[Wind Speed – Ultrasonic Sensor][Wind Speed – Ultrasonic Sensor]
3.3. Estimate the values of the component errors.Estimate the values of the component errors.[0-100 mph range][0-100 mph range]• Input – Laboratory Standard Input – Laboratory Standard [± 0.20 mph][± 0.20 mph]• Input – Transfer Standard Input – Transfer Standard [± 0.10 mph][± 0.10 mph]• Input – Test Position Input – Test Position [± 0.02 mph][± 0.02 mph]• Input – Comparison Apparatus Input – Comparison Apparatus [± 0.08 mph][± 0.08 mph]• Output – Tolerance Output – Tolerance [± 0.30 mph][± 0.30 mph]
• Sampling Sampling [± 0.04 mph][± 0.04 mph]• Final rounding of hourly average value Final rounding of hourly average value [± 0.05 [± 0.05
mph]mph]
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
2222
Example CalculationExample Calculation[Wind Speed – Ultrasonic Sensor][Wind Speed – Ultrasonic Sensor]
4.4. [Sensor only] Perform time-average [Sensor only] Perform time-average adjustments for each random error adjustments for each random error component.component.
• Only 1 sample is used for calibrationOnly 1 sample is used for calibration
• Adjusted values equal initial valuesAdjusted values equal initial values
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
2323
Example CalculationExample Calculation[Wind Speed – Ultrasonic Sensor][Wind Speed – Ultrasonic Sensor]
5.5. [Sensor only] Calculate "root sum of the [Sensor only] Calculate "root sum of the squares" (RSS) for the adjusted random squares" (RSS) for the adjusted random error components.error components.
RSS = (0.20)RSS = (0.20)22+ (0.10)+ (0.10)22+(0.08)+(0.08)22+ (0.30)+ (0.30)22 mphmph
RSS = RSS = ±0.38 mph±0.38 mph
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
2424
Example CalculationsExample Calculations[Wind Speed – Ultrasonic Sensor][Wind Speed – Ultrasonic Sensor]
6.6. [Sensor only] Add bias errors to obtain [Sensor only] Add bias errors to obtain system accuracy (system accuracy (SASA).).
SASA = RSS + b = RSS + b
SASA = 0.38 + 0.02 = 0.40 mph = 0.38 + 0.02 = 0.40 mph
SASA = 0.38 – 0.02 = 0.36 mph = 0.38 – 0.02 = 0.36 mph
SASA = = ±± 0.40 mph0.40 mph
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
2525
Example CalculationExample Calculation[Wind Speed – Ultrasonic Sensor][Wind Speed – Ultrasonic Sensor]
4.4. [Field Measurements] Perform time-average [Field Measurements] Perform time-average adjustments for each random error adjustments for each random error component.component.
• Sensor output is random (time-averaged)Sensor output is random (time-averaged)
0.03 = 0.40/( 180) mph0.03 = 0.40/( 180) mph• Sampling is random (time-averaged)Sampling is random (time-averaged)
0.00 = 0.04/( 180) mph0.00 = 0.04/( 180) mph• Rounding is random (1 sample)Rounding is random (1 sample)
0.05 = 0.05/( 1) mph0.05 = 0.05/( 1) mph
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
2626
Example CalculationExample Calculation[Wind Speed – Ultrasonic Sensor][Wind Speed – Ultrasonic Sensor]
5.5. [Field Measurements] Calculate "root sum [Field Measurements] Calculate "root sum of the squares" (RSS) for the adjusted of the squares" (RSS) for the adjusted random error components.random error components.
RSS = (0.03)RSS = (0.03)22+ (0.00)+ (0.00)22 +(0.05) +(0.05)22 mph mph
RSS = RSS = ±0.06 mph±0.06 mph
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
2727
Example CalculationExample Calculation[Wind Speed – Ultrasonic Sensor][Wind Speed – Ultrasonic Sensor]
6.6. [Field Measurements] Add bias errors to [Field Measurements] Add bias errors to obtain system accuracy (obtain system accuracy (SASA).).
• No bias term:No bias term:
SASA = RSS + b = RSS + b
SASA = 0.06 + 0.00 = 0.06 mph = 0.06 + 0.00 = 0.06 mph
SASA = = ±± 0.06 mph0.06 mph
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
2828
Example CalculationExample Calculation[Wind Speed – Ultrasonic Sensor][Wind Speed – Ultrasonic Sensor]
7.7. Compare the extreme system accuracy (Compare the extreme system accuracy (SASA) ) values with applicable requirements to values with applicable requirements to evaluate system performance.evaluate system performance.• Table 1 specifies WS accuracy as:Table 1 specifies WS accuracy as:
0.2 m/s +5% of observation0.2 m/s +5% of observation == 0.45 mph at 0 0.45 mph at 0 mphmph
5.45 mph at 100 5.45 mph at 100 mphmph
• Applies to both sensor and channel accuracies.Applies to both sensor and channel accuracies.• Sensor accuracy of Sensor accuracy of ±± 0.40 mph 0.40 mph and channel and channel
accuracy ofaccuracy of ± ± 0.06 mph0.06 mph both meet specification. both meet specification.
Measurements satisfy ANSI/ANS-3.11 (2005).Measurements satisfy ANSI/ANS-3.11 (2005).
ANSI/ANS-3.11 (2005) Error AnalysiANSI/ANS-3.11 (2005) Error Analysiss
2929
Final CommentsFinal Comments
ANSI/ANS-3.11 (2005) provides specific ANSI/ANS-3.11 (2005) provides specific guidance for calculating meteorological guidance for calculating meteorological measurements accuracy.measurements accuracy.
The ANSI/ANS-3.11 (2005) methodology is The ANSI/ANS-3.11 (2005) methodology is consistent with past practices--prior to consistent with past practices--prior to ANSI/ANS-3.11 (2000), so calculations should be ANSI/ANS-3.11 (2000), so calculations should be comparable with historical information.comparable with historical information.
Hopefully, ANSI/ANS-3.11 (2005) will be Hopefully, ANSI/ANS-3.11 (2005) will be adopted as an adopted as an officialofficial source of guidance source of guidance regarding meteorological measurements regarding meteorological measurements accuracy.accuracy.