22Mitch-UncertaintiesinDosimetry 26

Treatment of UncertaintiesTreatment of Uncertainties

in Radiation Dosimetryy

Michael G Mitch Ph D 1Michael G. Mitch, Ph.D.

Larry A. DeWerd, Ph.D.2

Ronaldo Minniti, Ph.D.1o do , . .

Jeffrey F. Williamson, Ph.D.3

1Physics Laboratory, National Institute of Standards and Technology (NIST)2Deptartment Of Medical Physics, University of Wisconsin-Madison

3Department of Radiation Oncology, Virginia Commonwealth University

Why is Uncertainty Analysis Important?

1. Assessment of the quality of a measurement or calculation

2. Quantitative comparison of results from different investigators

3. Critical analysis of measurement or calculation method

“Have I thought about all possible factors that influence the result ofmy measurement or calculation?”my measurement or calculation?

D = 14 28 mGy / s.

Dw = 14.28 mGy / s

D = 14 28 mGy / s.

Dw = (14.28 ± 0.12) mGy / s.

Dw = 14.28 mGy / s

w

D = 14 28 mGy / s.

Dw = (14.28 ± 0.12) mGy / s.

Dw = 14.28 mGy / s

Uncertainty Component Type A Type B(%) (%)

w

(%) (%)

Heat defect 0.30Reproducibility of measurement groups 0.15Beam attenuation from glass wall 0.10Beam attenuation from calorimeter lid 0.05Field size 0.23Vessel positioning 0.02Thermistor calibration 0.01Water density 0 02Water density 0.02

Quadratic sum 0.16 0.39

Relative combined standard uncertainty 0.42 %

Relative expanded uncertainty (k = 2) 0.84 %

Error vs. Uncertainty

Error = Difference between a measured or calculated value of a quantity and the “true” value (unknowable)

Uncertainty = An interval about the average value of a series of measurementsl l ti hi h ithi t i l l f fid i b li dor calculations which, within a certain level of confidence, is believed

to contain the “true” value of a quantity

Error vs. Uncertainty

Error = Difference between a measured or calculated value of a quantity and the “true” value (unknowable)

Uncertainty = An interval about the average value of a series of measurementsl l ti hi h ithi t i l l f fid i b li dor calculations which, within a certain level of confidence, is believed

to contain the “true” value of a quantity

NOTE: A measurement or calculated result with a low uncertainty is notnecessarily a result of high quality.

Method of Classifying Uncertainties

Type A Uncertainty = calculated by statistical methods

Type B Uncertainty = evaluated by other means

1981 – CIPM (Comité International des Poids et Mesures)

1993 – GUM (Guide to the Expression of Uncertainty in ) SO ( i l O i i fMeasurement), ISO (International Organization for

Strandardization)

1994 NIST (National Institute of Standards and Technology)1994 – NIST (National Institute of Standards and Technology) Technical Note 1297

Method of Classifying Uncertainties

Type A Uncertainty = calculated by statistical methods

Type B Uncertainty = evaluated by other means

Random Effect = the variation in the results of measurements or calculationsthat averages to the (true value ± bias) over many iterations

Systematic Effect = an error that is constant for each iteration = bias (unknown)

Precision = random effects only

Accuracy = random and systematic effects

Method of Evaluating Uncertainties

Type A Uncertainty = standard deviation of the mean, uA = s

2/1

1

2)()1(

1

n

ii zz

nns

n

iiz

nz

1

1

Type B Uncertainty = scientific judgment, uB

1 instrument manufacturer’s specifications1. instrument manufacturer s specifications

2. investigator’s knowledge and experience

32

aauB

a- a+

Combined Standard Uncertainty, uc

),...,,( 21 Nxxxfy

2/112 N N N fff

1

1

1 1

2 ),(2)(

N

i

N

i

N

ijji

jii

ic xxu

xf

xfxu

xfu

2 2 2( ) ( ) ( )i A i B iu x u x u x

n

kjjkiikji xxxx

nxxu

1))((

11),(


),...,,( 21 Nxxxfy

2/112 N N N fff

1

1

1 1

2 ),(2)(

N

i

N

i

N

ijji

jii

ic xxu

xf

xfxu

xfu

2 2 2( ) ( ) ( )i A i B iu x u x u x

n

kjjkiikji xxxx

nxxu

1))((

11),(

THE LAW OF PROPAGATION OF UNCERTAINTY


2/12

If all variables xi are independent, then u(xi, xj) = 0

2/1

1

22

)(

N

ii

ic xu

xfu

21 xxy

Sums and differences Products and quotients

21xxy

21 xxy 121 xxy

2/12

22

21

21

2 )()()()( xuxuxuxuu BABAc

2/12

2

2

2

2

2

2

1

1

2

1

1 )()()()(

xxu

xxu

xxu

xxu

yu BABAc

2/12222 2/12

22

21

21

2 )(%)(%)(%)(%% xuxuxuxuu BABAc

Interpretation of y ± uc

• For a normal distribution with mean and standard deviation , the interval ± contains 68.27 % of the distribution.

A i h h di ib i i d i h h l f• Assuming that the distribution associated with the results from our measurementsor calculations is approximately normal (and we perform enough iterations), thenthe interval y ± uc contains about 68 % of the distribution, and we state that the “true” value is believed to lie within this interval with a 68 % level of confidencetrue value is believed to lie within this interval with a 68 % level of confidence.

Expanded Uncertainty, V

• When the results of measurements and calculations are to be used wherehealth and safety are a concern (such as in medical physics), an expandeduncertainty is used.

V = kuV = kuc

k is the coverage factor

• NIST primary standards for all dosimetric quantities in medical physics• NIST primary standards for all dosimetric quantities in medical physicsuse k = 2, corresponding to an interval with a 95 % level of confidence.

2

Student’s t

• If the number of measurements is small, one should consider using the t valueto calculate a confidence interval.

y ± tuc

Deg. offreedom

( = n – 1)68.27 %(k = 1)

95.45 %(k = 2)

No. ofmeas.

(n)

2 1 1.84 13.97

10 9 1 06 2 3210 9 1.06 2.32

20 19 1.03 2.14

∞ 1.00 2.00

Uncertainty Budget, NIST SK Standard for 125I seeds

jj

iidetdr

effairairK QKKQKMKK

Vd

eWdQKS )(),()()(

22

Net current, sI 0.06

Value Type A (%) Type B (%)

Net current, s 0.06),(det QKM

eW / 33.97 J / C - 0.15Air density, ρair 1.196 mg / cm3 - 0.03Aperture distance, d - 0.24Effective chamber volume, Veff 0.11 0.01Decay correction, K1 T1/2 = 59.43 d - 0.02Recombination < 1 004 0 05)(KK Recombination, < 1.004 - 0.05Attenuation in filter, K3(Q) 1.0295 - 0.61Air attenuation in WAFAC, K4(Q) 1.0042 - 0.08Source-aperture attenuation, K5(Q) 1.0125 - 0.24Inverse-square correction, K6 1.0089 - 0.01Humidity, K7(Q) 0.9982 - 0.07

)(KKdr

Humidity, K7(Q) 0.9982 0.07In-chamber photon scatter, K8(Q) 0.9966 - 0.07Source-holder scatter, K9 0.9985 - 0.05Electron loss, K10 1.0 - 0.05Aperture penetration, K11(Q) 0.9999 - 0.02External photon scatter, K12(Q) 1.0 - 0.17

Combined standard uncertainty, uc (s2 + 0.7622)1/2

Expanded uncertainty, V 2uc

AAPM TG-138: Photon Brachytherapy Source Dosimetric Uncertainty Analysis

Larry DeWerd (Chair), Geoffrey Ibbott, Ali Meigooni, Michael Mitch, Mark Rivard, Kurt Stump, and Bruce Thomadsen

1 M t d M t C l t i ti1. Measurement and Monte Carlo uncertainties

2. Uncertainty in TG-43 formalism parameters

3. Transfer of NIST SK standard to ADCLs

4. Uncertainty in clinical measurements

Measurement Traceability for Brachytherapy Sources

sourcessourcesSK

Manufacturersecondary standardADCL

verification forwell-ionization

chamberssources

verification fortreatment planning

Clinic SKClinic

Measurement Traceability for Brachytherapy Sources – Uncertainties

seedSK (± 0.8 %)

seed

ManufacturerADCL

SK / IADCL (± 0.9 %)

Clinic


ManufacturerADCLd

SKADCL (± 1.1 %)

seed

WICK ( )

SKADCL / IClinic (± 1.2 %)

Clinic


ManufacturerADCL

WICseed (SK

M)

Clinic SKClinic (± 1.3 %)


Step in chain

Measurement Description Quantity (Units) Relative Propagated Uncertainty (%)

1 NIST WAFAC calibration SK (U) 0.80

2 ADCL well-ion chamber calibration SK / IADCL (U / A) 0.94

3 ADCL calibration of seed from manufacturer SKADCL (U) 1.06

4 ADCL calibration of Clinic well-ion chamber SKADCL / IClinic (U / A) 1.17

5 Clinic measures seed air-kerma strength SKClinic (U) 1.27

Expanded uncertainty (k = 2) SKClinic (U) 2.54

Step in chain

Measurement Description Quantity (Units) Relative Propagated Uncertainty (%)y ( )

(1) NIST WAFAC calibration SK (U) 0.80

6 Manufacturer well-ion chamber calibration SK / IM (U / A) 0.94

7 Manufacturer calibration of QA seed SKM (U) 1.06

8 Manufacturer calibration of QA well-ion chamber

SKM / IM (U / A) 1.17

9 Manufacturer calibrates seed for Clinic SKM (U) 1.27

10 Manufacturer places seed in 2 % bin SKM (U) 1.40

Expanded uncertainty (k = 2) SKM (U) 2.80

Does SKClinic Agree With SK

M ?

SKClinic = (1.034 ± 0.026) U

SKM = (1.000 ± 0.028) U

1.08

1.04

1.06

1.02

S K (U

)

Cli i

0.98

1.00 Clinic

0.96Manufacturer

Degree of Equivalence

SKClinic – SK

M < V 2Clinic + V 2M – V 2NIST

0.08

0.06

(U)

0.02

0.04

Clin

ic -

SK

M

0.00

SK

C

-0.02

AAPM

Board of Directors

Science Council

Therapy Physics Committee

Brachytherapy SC Calibration Laboratory Accreditation SC

Low Energy Brachytherapy Source Dosimetry WG ADCLs

High Energy Brachytherapy Source Dosimetry WG

Brachytherapy Source Registry WG

Special Brachytherapy Modalities WG

Robotic Brachytherapy WG

AAPM

Board of Directors

Science Council








Recommendations of the Calibration Laboratory Accreditation SC:

New source

1 5 sources are sent to NIST for S calibration well chamber1. 5 sources are sent to NIST for SK calibration, well chamber measurements (SK / I), and spectrum analysis

2 If (S / I) for each source is within ± 1 00 % of average 3 sources are2. If (SK / I) for each source is within ± 1.00 % of average, 3 sources are sent to the ADCLs, and 2 sources are returned to the manufacturer or sent to a dosimetry investigator for measurement of D(r, )

.

3. If (SK / I) is out of tolerance for one or more sources, another set of 5 sources is sent by the manufacturer to NIST

f M d Ph 31 (3) M h 2004 675 681ref: Med. Phys. 31 (3), March 2004, pp. 675-681.

Measurement Traceability for Brachytherapy Sources – New Source

5 sourcesSK

ManufacturerADCL

Clinic

Measurement Traceability for Brachytherapy Sources – New Source

2 sources3 sourcesSK

Manufacturersecondary standardADCL1 ADCL2 ADCL3

(SK / I)0 SK / I ?

ADCL calibration date

Clinic

Measurement Traceability for Brachytherapy Sources - Clinics

ManufacturerADCL

well-ionizationchambers

Clinic(SK / I)ADCL

Measurement Traceability for Brachytherapy Sources - Clinics

ManufacturerADCL

verification forsources (SK

M)verification for

treatment planning

Clinic SKClinic(SK / I)ADCL

Recommendations of the Calibration Laboratory Accreditation SC:

QA for sources with established NIST SK standard

1 3 sources sent to NIST (preferably within 6 months but not exceeding1. 3 sources sent to NIST (preferably within 6 months but not exceeding 1 year) for SK calibration and (SK / I) evaluation

2 If (S / I) for each source is within ± 2 00 % of established2. If (SK / I) for each source is within ± 2.00 % of established (SK / I) at NIST or the ADCLs, no action needs to be taken

3 If (S / I) is out of tolerance the cause should be investigated3. If (SK / I) is out of tolerance, the cause should be investigated, and another set of 3 sources is sent by the manufacturer to NIST and the ADCLs

4. If (SK / I) remains out of tolerance for the second set of source measurements, discrepancies among the ADCLs and NIST should be resolved quicklybe resolved quickly

Measurement Traceability for Brachytherapy Sources – Annual QA

3 sources3 sourcesSK

ManufacturerADCL1 ADCL2 ADCL33 sources

(SK / I)t

± 2.00 %vs.

(SK / I)0

Clinic

Well-ionization Chambers

Note that due to the use of well chambers of different designs by NISTand the 3 ADCLs, discrepancies in tolerance level achievement do occur.

Control Chart, I / SK, seed “E”

5.5

5.7

5.9

A /

U)

Jul02 Aug02 Oct04 Jan05 Oct05 May06 Sep06 Nov06 May07 Jan08

4 9

5.1

5.3

I / S

K (p

A

4.9


5.5

5.7

5.9

A /

U)


4 9

5.1

5.3

I / S

K (p

A

4.9

1.05

Manufacturer vs. NIST (SKM / SK

NIST)

1.02

1.03

1.04

1.05

/SK

NIS

T

Oct04 Jan05 Oct05 May06 Sep06 Nov06 May07 Jan08

0.99

1.00

1.01

S KM

/

Fluorescence K / Decay K, seed “E”

0.6

0.8

1.0

x

100


0 0

0.2

0.4

0.6

F K

/

D K

0.0


NIST)1.05

1.02

1.03

1.04

1.05

/SK

NIS

T


0.99

1.00

1.01

S KM

/


5.5

5.7

5.9

A /

U)


4 9

5.1

5.3

I / S

K (p

A

ADCLreset

4.9

1.05


NIST)

1.02

1.03

1.04

1.05

/SK

NIS

T


0.99

1.00

1.01

S KM

/


5.5

5.7

5.9

A /

U)


4 9

5.1

5.3

I / S

K (p

A

ADCLreset

4.9

1.05


NIST)

1.02

1.03

1.04

1.05

S KN

IST


0.99

1.00

1.01

SKM

/


NIST)

1.031.041.05

4 5 %

0 991.001.011.02 3.0 % 3.0 %

4.5 %

0.960.970.980.99

Uncertainty of S M from calibration certificate

3 %5 %

0.95103Pd 125I

Uncertainty of SK from calibration certificate

Overall Average = 1.001, = 0.008

Source manufacturers have generally been successful in transferring the NIST SK standard to their g y g Kfacilities. However, there is much variation with respect to the magnitude and precision of reported uncertainties on calibration certificates, if uncertainties are reported at all.

Uncertainty in Secondary Standards based onWell-Ionization Chamber MeasurementsWell Ionization Chamber Measurements

To minimize uncertainty:

• Maintenance of secondary standards at ADCLs (AAPM recommendations)

1) NIST receives a batch of 3 seeds of each design annually

2) NIST characterization measurements detect normal manufacturing variability and anomalous sources

To quantify uncertainty:To quantify uncertainty:

• Utilize control charts for results of characterization measurements

1) C l l d d d i i ( ) d f l f I / S f1) Calculate standard deviation (s) and range of values of I / SK for seeds with a significant calibration history at NIST (includes 3 103Pd and 8 125I source models), s k = 1 uncertainty component

2) Study variations in measured spectra and anisotropy (A) to distinguish normal manufacturing variability from design change

1 4%

Standard Deviation and Range of (I / SK)

0 8%

1.0%

1.2%

1.4%max = 1.3 %103Pd 125I

0 2%

0.4%

0.6%

0.8%, I / SK

min = 0.5 %

0.0%

0.2%

6.0%

1 2 3 Model # 4 5 6 7 8 9 10 11

4.0%

5.0%

6.0%103Pd 125I

1.0%

2.0%

3.0%Range, I / SK ± 2 %AAPM

tolerance

0.0%

%level

1 2 3 Model # 4 5 6 7 8 9 10 11

Range of (Ag K / ) and (I / SK)

20%

25%

30%125I

Note wide variationin admixture of Agfluorescence x rayscausing range of

/ dRange, Ag K /

5%

10%

15%I / SK to exceedAAPM tolerancelevel (Model # 11)

hi d d l i

6 0%

0%

5% This seed model isno longer produced

1 2 3 Model # 4 5 6 7 8 9 10 11

4.0%

5.0%

6.0%103Pd 125I

1.0%

2.0%

3.0% ± 2 %AAPM

tolerance

Range, I / SK

0.0%

1.0%level

1 2 3 Model # 4 5 6 7 8 9 10 11

American Association of Physicists in Medicine (AAPM)

Board of Directors

Science Council








American Association of Physicists in Medicine (AAPM)

Board of Directors

Science Council








Low Energy Brachytherapy Source Dosimetry WG

TG-43 Report (1995)1. Dosimetry formalism introduced2. Consensus datasets for 2 125I, 1 103Pd, and 1 LDR 192Ir seeds

TG-43U1 (2004)1. Dosimetry formalism updated (includes NIST WAFAC SK standard)2 C d t t f 6 125I d 2 103Pd d2. Consensus datasets for 6 125I and 2 103Pd seeds3. Recommended dosimetry methodology (TLD, Monte Carlo)

TG-43U1S1 (2007)( )1. Consensus datasets for 7 125I and 1 103Pd seeds2. Interpolation and extrapolation methods

TG-43U1S2 (in preparation)1. Consensus datasets for 2 125I, 2 103Pd, 1 131Cs seeds…2. Experimental method evaluation (TLD powder in water, photon spectrometry radiochromic film)spectrometry, radiochromic film)

Update of AAPM Task Group No. 43 Report: A Revised AAPM Protocol for Brachytherapy Dose Calculations

(TG-43U1)(TG 43U1)

Mark Rivard, Bert Coursey, Larry DeWerd, William Hanson, Saiful Huq, Geoffrey Ibbott, Michael Mitch, Ravinder Nath, and Jeffrey Williamsony y

• Specification of measurement and Monte Carlo calculation methodologiesincludes a comprehensive uncertainty analysis

• Good practice for Monte Carlo calculations includes:Type A uncertainty component ≤ 2 % at r ≤ 5 cm for dose rateType A uncertainty component ≤ 1 % for air-kerma

TG-43 Formalism

KSrD ),( 00

),()(),(

),(),(

00

rFrgrGrG

SrD LL

LK

Dose rate in water

)(rG 122 )4/()0( LrrG

Geometry Function

Dose rate constant (NIST-traceable SK)

sin

),(Lr

rGL )4/()0,( LrrGL

Radial Dose Function 2D Anisotropy FunctionD )( 1

),(),(

),(),(

)(0

00

00

0

rGrG

rDrD

rgX

XX

Radial Dose Function

),(),(

),(),(),( 0

0

rGrG

rDrDrF

L

L

2D Anisotropy Function

KSrD ),( 00 r0 = 1 cm

0 = / 2

Uncertainty of Dose Rate Constant

EXP MC

Component Type A Type B C T A T BComponent Type A Type B(%) (%)

Repeated TLD measurements 1.3

Component Type A Type B(%) (%)

Statistics 0.2

TLD calibration (inc. linac cal.) 1.8

Absorbed dose energy dep. and 0.7PMMA-to-liquid water conv.

Photon cross sections 0.7

Seed geometry 0.75

Seed and TLD positioning 1.2

Intrinsic energy dep. corr. 5

Source energy spectrum 0.2

Combined std. unc., uc 1.1

NIST-traceable SK meas. 1

Combined std. unc., uc 5.7

Dolan and Williamson, 2006

Uncertainty of Consensus Dose Rate Constant

MCEXP

EXP = (0.980 ± 0.056) cGy h-1 U-1 (5.7 %)

2CON

MC = (0.950 ± 0.010) cGy h-1 U-1 (1.1 %)

2/12

2

22

2

BIAS

MCEXPc uuuu

CON = (0.965 ± 0.028) cGy h-1 U-1 (2.9 %)

C

(without bias term)

32

MCEXPBIASu

CON = (0.965 ± 0.030) cGy h-1 U-1 (3.1 %)

(including bias term)

Summary

• Uncertainty analysis is a critical element of the science of metrology

• All factors that could possibly influence the result of a measurementor calculation should be considered

• An uncertainty budget quantifies Type A and Type B components

• Expanded uncertainties (k = 2) should be used in clinical dosimetry

Documents

22Mitch-UncertaintiesinDosimetry 26