Precision Metrology 1078-01

7/25/2019 Precision Metrology 1078-01

1/54

SCOPE OF ACCREDITATION TO ISO/IEC 17025:2005& ANSI/NCSL Z540-1-1994

PRECISION METROLOGY, INC.9

7350 North Teutonia AvenueMilwaukee, WI 53209

Carol Shipley Phone: 414 351 7420

CALIBRATION

Valid To: July 31, 2016 Certificate Number: 1078.01

In recognition of the successful completion of the A2LA evaluation process, accreditation is granted tothis laboratory to perform the following calibrations1, 9:

I. Acoustical Quantities

Parameter/Equipment Range CMC2, 8() Comments

Sound Level (0 to 140) dB

(0 to 140) dB

0.16 dB + 0.032 %

0.32 dB

Piston phone

Sound level calibrator

Sound Level3 (0 to 140) dB 0.6 dB Sound level calibrator

II. Chemical


Conductivity Meters 10 S/cm100 S/cm1000 S/cm1400 S/cm

0.68 S/cm + 0.6R1.2 S/cm + 0.6R4.5 S/cm + 0.6R6.9 S/cm + 0.6R

Conductivity solutions


2/54

III. Dimensional


Angle Blocks 0to 90 5 + 10 parts in 106 Measuring machine

Angle Plates3 Up to 30 in (120 + 6H) in Gage amp. probe, indi-square and test

indicator

Arbor Straightness Up to 24 in (35 + 9L) in Gage amp w/probe

Bench Center

Center Parallelism

Base Flatness & BaseParallelism

Up to 24 in

Up to 24 in

(68 + 4L) in

(62 +2.5L) in

Gage amp w/probe

Benchmics3 Up to 2 in 30 in Gage blocks

Bore Gages3 Up to 12 in 10 in/in + 0.6R Gage blocks

Box Parallels

ParallelismSquareness

5 in 10 in 10 in (12 + 20H) in(18 + 20H) in

Gage amp w/probe,cylindrical square

Calipers3(Including

Intertest, Oditest, SnapJaw and other ODCalipers)

Resolution 0.0001 in Up to 2 in(2 to 12) in

78 in70 in + 4 in/in

Gage blocks


3/54

Parameter/Equipment Range CMC

2, 4

() Comments

Chamfer Gages/HoleGages3

Up to 12 in 20 in + 0.6R Cylindrical rings

Clinometers andInclinometers

Up to 360 16 + 0.6R Sine bar/gage blocks

CMMs3

Length Error (EL)

Linear

Volumetric

Length Error (E150)

Repeatability Range

(R0)

Length

Volume

Probing Analysis

Bi-directional

Up to 52 in

Up to 52 in

----

Up to 114 in on each

axis

Up to 36 in for

shortest axis

---

38 in + 5.2 in/in

58 in + 8.1 in/in

38 in + 5.2 in/in

---

38 in + 5.2 in/in

82 in + 13 in/in

2 m

(40 + 10L) in

ASME

B89.4.1.10360.2

ASME B89.4.1.2000

Ball bar

Test probe

Coating ThicknessShims/Precision Shims

25 in to 0.5 in 19 in + 82 in/in Heidenhain-CertoCT 6001

Concentricity Gage N/A 130 in + 8 in/in Indicator, bench center

Cylindrical Plugs3 Up to 4 in(4 to 20) in

4 in + 5 in/in4 in + 5.4 in/in

ULM


4/54


Cylindrical Squares Up to 12 in (13 + 9H) in Gage amp w/probe;His the height of thecylinder from the basein inches.

External Spline Gages3

Measurement Over Pins

Circular ToothThickness

Major Diameter

Up to 8 in (170 + 28D) in

(110 + 16L) in

(28 + 12D) in

UMM

Optical comparator

UMM

Feeler/Thickness Gage3 Up to 0.2 in 36 in UMM

Fixed Gaging

Screw Pitch Gages, DrillGages, Taper Gages,

Center Gages, Sheet andWire Gages, Angle Gages

---

---

(110 + 16L) in (lengthmeasures)

54

Measuringmachine/opticalcomparator

Gage Amplifier andProbe(s)3

Single ProbeDual Probe

------

8.9 in2.1 in

Gage blocksGage blocks

Gage Balls Up to 4 in (32 + 1.5D) in Universal measuringmachine (UMM)

Gage Blocks Up to 4 in(4 20) i

(3.1 + 3.2L) in(3 3 3 2L) i

Gage blocks and


5/54


Height Measures3 Up to 24 in 13 in/in + 94 in Height gage

Indicators3

Dial

Test

Up to 8 in

Up to 0.060 in

10 in/in + 0.6R

0.0001 in

Indicator calibrator

Indicator Calibrator3 ---- 12 in + 0.6R Gage blocks, ampw/probe

Length Standards3

CROBLOX

Ceramic

Steel

Up to 4 in(4 to 20) in



(20 to 120) in

4 in + 2.2 in/in4 in + 2.4 in/in

4 in + 2.7 in/in4 in + 3 in/in

4 in + 5 in/in4 in + 5.4 in/in

3.7 in/in + 14 in

ULM, gage blocks

ULM

Levels (Machinist)3 Up to 96 in 0.00019 in/ft Sine plate

Linear Measure3 (Up to 24) in 94 in + 13 in/in Height gage

Linear Measure (Up to 30) in (330 + 5.8L) in CMM

Linear Measurement ofMachine Tools3

Up to 20 ft 1.5 in/in + 0.58 in Laser


6/54

Parameter/Equipment Range CMC2, 4, 8() Comments

Linear Scales/Reticlesand Stage Micrometers

2D

1D

Up to 12 in

Up to 30 in

(115 + 15L) in

9.9 in + 1.5 in/in

Vision system

Gitterperiodeninterferometer

Micrometers3

InsideDepthTri-BoresOutsideGroove

Up to 294 inUp to 12 inUp to 6 inUp to 42 inUp to 4 in

10 in/in + 0.6R10 in/in + 0.6R10 in/in + 0.6R10 in/in + 0.6R18 in/in + 0.6R

Gage blocks

Microscopes3

ReticuleMagnification

Up to 25 mmUp to 1000x

23 m2.4 %

Stage micrometer

Optical Comparator3

LinearAngleMagnification

Up to 12 in0 to 36010x, 20x, 50x,62.5x, 100x, 200x

0.0002 in20.00012 in

Glass scaleCalibration sphereCalibration sphere

Optical Flats Up to 8 in 2.8 in Reference flat andmonochromatic light

source

Parallels3

Steel 1.5 in 6 in (10 + 6L) in Gage amp w/probe;Listhe distance between


7/54


Pitch MicrometerStandards

Length

Angle

Up to 12 in

Up to 60

(22 + 6L) in

54

UMM

Vision system

Precision Diameter Tapes Up to 48 in(48 to 96) in(96 to 780) in

200 in + 4 in/in560 in + 1.4 in/in0.0013 in + 6 in/in

Setting discs, ULM

Profilometers3 120 in Ra 2.8 in Roughness specimen

Protractor Digital and

Mechanical3

0to 180 0.0076 Sine bar and gageblocks

Radius Gage Up to 5 in 0.00015 in Optical comparator

Riser Blocks and Stands Up to 24 in (13 + 8H) in Gage amplifier with

probe

Roundness Testers3

Coning Error

Axial Error

Radial Accuracy

Z-Axis straightness

---

---

---

(0.1 to 12) in

11 in

11 in

11 in

(14 + 12H) in

Test ball and risercylinder

Test ball

Test ball

Cylindrical square; Inthe CMC,His theheight of the cylinderfrom the base in inches


8/54


Sine Plates/Bars3

Flatness/Parallelism &Parallelism Cylinder toBase

Angle CalibrationCenter of Rolls

Up to 20 in

0 to 90Up to 20 in

50 in

0.0004516 in/in

Gage amp w/probe,angle blocks and gageblocks

Snap Gages3 Up to 20 in 88 + 13 L Box parallel with gageamp and probe

Squares3

Master

Granite

Steel

Up to 30 in

Up to 30 in

Up to 30 in

(56 + 5H) in

(42 + 6H) in

(42 + 6H) in

Gage amp and probe

Indi-square and testindicator

Master square and gageblocks

Straightness andStraight Edges3

Up to 60 in (10 + 14L) in Gage amplifier w/probe

Step Gages Up to 50 in 9.7 in + 3.1 in/in Gage amp and probe,gage blocks

Surface Plates3

Flatness

Repeatability

18 in x 24 in to72 in x 144 in

Up to 72 in x 144 in

8Din

34 in

Electronic levelsDis the length of theplate diagonal ininches.

Repeat o meter


9/54


Surface Roughness3 (15 to 150) in Ra 3.4 in Verification ofspecimens per ASMEB46.1 Type C usingprofilometers; Ra is theresolution of theprofilometers inmicroinches.

Tape Measures3 Up to 300 ft (0.026 + 0.6R) in UMM

Tapered Plugs3

Simple Pitch Diameter

Notch Height

Up to 18 in

Up to 18 in

(100 + 12D) in

13 in/in + 94 in

Thread wires/UMM

Height measuringmachine

Tapered Rings3

Simple Pitch Diameter

Taper

Standoff and Thickness

Functional Diameter

Up to 18 in

Up to 18 in

Up to 18 in

Up to 2 in

300 in

400 in

120 in

(300 + 8D) in

NPT master plug,UMM

NPT master plug

NPT master plug andheight measuringmachine

Thickness Gages3 Dial

and Digital

Up to 1 in 19 in + 0.6R Gage blocks

Thread Micrometers3(Screw Thread, Pitch,Point)

Up to 4 in 10 in + 0.6R Pitch standard


10/54


Thread Rings3

Simple Pitch DiameterMinor Diameter

Functional DiameterMinor Diameter

Up to 18 inUp to 2 in


300 in(70 + 13D) in

(300 + 8D) in(300 + 11D) in

UMM

w/ set plug

Thread Wires Up to 0.5 in (8.5 + 7D) in UMM, gage blocks

Universal MeasuringMachines3(UMMs)

English

Metric


Up to 100 mm

1.5 in + 1.2 in/in1.3 in + 1.8 in/in

0.04 m + 0.0015 m/mm

Gage blocks

V-Blocks

Parallelism Side

V

Squareness

8 in 8 in 8 in 14 in

20 in

(37 + 11H) in

Gage amplifierw/probe

Cylindrical square

Cylindrical plug

Vision Systems3

X-Y Linear Accuracy

Z-Axis Linear

Accuracy

Up to 24 in

Up to 20 in

(110 + 13L) in

(74 + 11L) in

Glass scale

Gage blocks

Wire Crimpers3

Crimp Height

Crimping Chamber

--- 0.0024 in

(10 + 19D) in

Micrometer

Pin; D is the diameter


11/54

IV. Electrical DC/Low Frequency

Parameter/Range Frequency CMC2, 7() Comments

AC Current Generate& Measure

(10 to 300) A

(0.3 to 3) mA

(3 to 10) mA

10 Hz20 Hz

30 Hz40 Hz55 Hz100 Hz300 Hz400 Hz500 Hz1 kHz3 kHz5 kHz

10 kHz20 kHz30 kHz

10 Hz20 Hz30 Hz40 Hz55 Hz100 Hz

300 Hz400 Hz500 Hz1 kHz3 kHz5 kHz10 kHz20 kHz30 kHz

10 Hz20 Hz40 Hz400 Hz1 kHz5 kHz10 kHz

0.018 % + 30 pA67 A/A + 30 pA

42 A/A + 30 pA37 A/A + 30 pA37 A/A + 30 pA38 A/A + 30 pA38 A/A + 30 pA38 A/A + 30 pA38 A/A + 30 pA38 A/A + 30 pA46 A/A + 30 pA46 A/A + 30 pA

46 A/A + 30 pA63 A/A + 30 pA80 A/A + 30 pA

0.017 % + 300 pA61 A/A + 300 pA30 A/A + 300 pA30 A/A + 300 pA30 A/A + 300 pA30 A/A + 300 pA

30 A/A + 300 pA30 A/A + 300 pA30 A/A + 300 pA30 A/A + 300 pA30 A/A + 300 pA31 A/A + 300 pA30 A/A + 300 pA45 A/A + 300 pA48 A/A + 300 pA

0.017 % + 1 nA69 A/A + 1 nA38 A/A + 1 nA36 A/A + 1 nA35 A/A + 1 nA36 A/A + 1 nA36 A/A + 1 nA

Fluke 5790A,5700A/EP,

A40, standard resistors


12/54


AC Current Generate& Measure (cont)

(10 to 20) mA

(20 to 30) mA

(30 to 50) mA

(50 to 100) mA

(100 to 200) mA

10 Hz20 Hz40 Hz400 Hz1 kHz5 kHz10 kHz20 kHz50 kHz

10 Hz20 Hz40 Hz400 Hz

1 kHz5 kHz10 kHz20 kHz50 kHz

10 Hz20 Hz40 Hz400 Hz


10 Hz20 Hz40 Hz400 Hz


10 Hz

0.017 % + 2 nA69 A/A + 2 nA38 A/A + 2 nA36 A/A + 2 nA35 A/A + 2 nA36 A/A + 2 nA36 A/A + 2 nA36 A/A + 2 nA64 A/A + 2 nA

0.017 % + 3 nA69 A/A + 3 nA38 A/A + 3 nA36 A/A + 3 nA

35 A/A + 3 nA36 A/A + 3 nA36 A/A + 3 nA36 A/A + 3 nA64 A/A + 3 nA

0.017 % + 5 nA69 A/A + 5 nA38 A/A + 5 nA36 A/A + 5 nA


0.017 % + 60 nA70 A/A + 60 nA38 A/A + 60 nA36 A/A + 60 nA


0.017 % + 120 nA

Fluke 5790A,5700A/EP,A40, standard resistors


13/54


AC Current Generate &Measure (cont)

(200 to 300) mA

(300 to 500) mA

(0.5 to 1) A

(1 to 2) A

(2 to 3) A

10 Hz20 Hz40 Hz400 Hz1 kHz5 kHz10 kHz20 kHz50 kHz

10 Hz20 Hz40 Hz400 Hz


10 Hz20 Hz40 Hz400 Hz


10 Hz20 Hz40 Hz400 Hz


10 Hz

0.017 % + 180 nA72 A/A + 180 nA38 A/A + 180 nA36 A/A + 180 nA35 A/A + 180 nA36 A/A + 180 nA36 A/A + 180 nA40 A/A + 180 nA72 A/A + 180 nA

0.017 % + 300 nA72 A/A + 300 nA38 A/A + 300 nA36 A/A + 300 nA


0.018 % + 6 A72 A/A + 6 A42 A/A + 6 A40 A/A + 6 A

39 A/A + 6 A40 A/A + 6 A40 A/A + 6 A52 A/A + 6 A0.011 % + 6 A

0.018 % + 12 A75 A/A + 12 A42 A/A + 12 A40 A/A + 12 A

40 A/A + 12 A40 A/A + 12 A40 A/A + 12 A53 A/A + 12 A0.011 % + 12 A

0.019 % + 18 A



14/54


AC Current Generate &Measure (cont)

(3 to 5) A

(5 to 10) A

(10 to 20) A**

50 Turn Coils

(Effective CurrentTransfer Ratio)

10 Hz20 Hz40 Hz400 Hz


10 Hz20 Hz40 Hz

400 Hz1 kHz5 kHz10 kHz20 kHz50 kHz

10 Hz20 Hz

40 Hz400 Hz1 kHz5 kHz10 kHz20 kHz50 kHz

25 Turn Amps, 50 Hz,

400 Hz500 Turn Amps, 50 Hz,400 Hz800 Turn Amps, 400 Hz1000 Turn Amps, 50 Hz

0.019 % + 30 A85 A/A + 30 A58 A/A + 30 A57 A/A + 30 A

57 A/A + 30 A57 A/A + 30 A57 A/A + 30 A71 A/A + 30 A0.017 % + 30 A

200 A/A + 60 A110 A/A + 60 A73 A/A + 60 A

72 A/A + 60 A72 A/A + 60 A72 A/A + 60 A72 A/A + 60 A90 A/A + 60 A0.013 % + 60 A

0.022 % + 120 A0.013 % + 120 A

86 A/A + 120 A85 A/A + 120 A85 A/A + 120 A85 A/A + 120 A85 A/A + 120 A0.011 % + 120 A0.016 % + 120 A

0.11 % of ratio

0.12 % of ratio

0.14 % of ratio0.14 % of ratio


** Measure Only

Comparison tostandard coil


15/54

Parameter/Range Frequency CMC2, 5, 6() Comments

AC Current Generate3

(cont)

(11 to 20.5) A (45 to 100) Hz

100 Hz to 1 kHz

(1 to 5) kHz

0.094 % + 3.9 mA

0.12 % + 3.9 mA

2.4 % + 3.9 mA

Fluke 5520A

AC Current Measure3

(0 to 100) A

(0.1 to 1) mA

(1 to 10) mA

(10 to 100) mA

(0.1 to 1) A

(10 to 20) Hz(20 to 45) Hz(45 to 100) Hz100 Hz to 5 kHz

(10 to 20) Hz

(20 to 45) Hz(45 to 100) Hz100 Hz to 5 kHz(5 to 20) kHz(20 to 50) kHz(50 to 100) kHz

(10 to 20) Hz(20 to 45) Hz(45 to 100) Hz

100 Hz to 5 kHz(5 to 20) kHz(20 to 50) kHz(50 to 100) kHz

(10 to 20) Hz(20 to 45) Hz(45 to 100) Hz100 Hz to 5 kHz(5 to 20) kHz(20 to 50) kHz(50 to 100) kHz

(10 to 20) Hz(20 to 45) Hz(45 to 100) Hz100 Hz to 5 kHz

0.27 % + 20 nA0.11 % + 20 nA0.042 % + 20 nA0.042 % + 20 nA

0.27 % + 0.14 A

0.11 % + 0.14 A0.042 % + 0.14 A0.023 % + 0.14 A0.042 % + 0.14 A0.27 % + 0.27 A0.37 % + 1 A

0.27 % + 1.4 A0.11 % + 1.4 A0.042 % + 1.4 A

0.023 % + 1.4 A0.042 % + 1.4 A0.27 % + 2.7 A0.37 % + 10 A

0.27 % + 14 A0.11 % + 14 A0.042 % + 14 A0.023 % + 14 A0.042 % + 14 A0.27 % + 27 A0.37 % + 100 A

0.27 % + 140 A0.11 % + 140 A0.057 % + 140 A0 070 % + 140 A

Agilent/HP 3458A


16/54


AC Current Measure3(cont)

(1 to 2) A

(2 to 20) A

50 A to 1 kA

10 Hz to 2 kHz(2 to 10) kHz(10 to 30) kHz

10 Hz to 2 kHz(2 to 10) kHz

60 Hz

0.057 % + 190 A0.068 % + 190 A0.25 % + 1.4 A

0.072 % + 1.9 mA0.20 % + 1.9 mA

2.0 %

Fluke 8508

Fluke 8508

Fluke 337

AC Power3 PF = 1

(29 to 330) A(0.33 to 3.3) mA

3.3 mA to 3.3 A

(29 to 330) A(0.33 to 3.3) mA(3.3 to 330) mA330 mA to 3 A

(29 to 330) A(0.33 to 3.3) mA(3.3 to 330) mA

330 mA to 1.1 A(1.1 to 3) A(3 to 11) A(11 to 20.5) A

(29 to 330) A(0.33 to 3.3) mA(3.3 to 33) mA(33 to 330) mA330 mA to 1.1 A

(1.1 to 3) A(3 to 11) A(3 to 20.5) A

(29 to 330) A(0.33 to 3.3) mA(3.3 to 33) mA

(10 to 20) Hz1 mV to 33 V

(20 to 45) Hz1 mV to 33 V

(45 to 100) Hz1 mV to 1020 V

100 Hz to 1 kHz1 mV to 1020 V

(1 to 5) kHz1 mV to 1020 V

0.19 %0.17 %

0.15 %

0.15 %0.11 %0.080 %0.15 %

0.13 %0.088 %0.041 %

0.052 %0.054 %0.065 %0.12 %

0.13 %0.088 %0.041 %0.041 %0.052 %

0.054 %0.095 %0.14 %

0.28 %0.17 %0.070 %

Fluke 5520A


17/54

Parameter/Range Frequency CMC2, 6, 7() Comments

AC Power3 PF = 1(cont)

(1.1 to 3) A(3 to 11) A(11 to 20.5) A

(29 to 330) A(0.33 to 3.3) mA(3.3 to 33) mA(33 to 330) mA330 mA to 1.1 A(1.1 to 3) A

(29 to 330) A(0.33 to 3.3) mA(3.3 to330) mA

(1 to 5) kHz1 mV to 1020 V

(5 to 10) kHz1 mV to 1020 V

(10 to 30) kHz1 mV to 330 V

0.26 %2.4 %2.4 %

0.67 %0.40 %0.17 %0.18 %2.3 %2.1 %

1.3 %0.79 %0.32 %

Fluke 5520A

AC Voltage Measure& Generate3

(0.6 to 2.2) mV

(2.2 to 7) mV

(7 to 22) mV

(10 to 20) Hz(20 to 40) Hz40 Hz to 20 kHz(20 to 50) kHz(50 to 100) kHz

(100 to 300) kHz(300 to 500) kHz500 kHz to 1 MHz


(300 to 500) kHz500 kHz to 1 MHz

(10 to 20) Hz(20 to 40) Hz40 Hz to 20 kHz(20 to 50) kHz

0.14 % + 1.1 V0.062 % + 1.1 V0.040 % + 1.1 V0.067 % + 1.6 V0.097 % + 2.0 V

0.19 % + 3.2 V0.19 % + 6.3 V0.31 % + 6.3 V

0.068 % + 1.1 V0.032 % + 1.1 V0.021 % + 1.1 V0.034 % + 1.6 V0.049 % + 2.0 V0.096 % + 3.2 V

0.11 % + 6.3 V0.19 % + 6.3 V

0.024 % + 1.1 V0.016 % + 1.1 V0.0096 % + 1.1 V0.018 % + 1.6 V

Fluke 5790A, 5720A


18/54


AC Voltage Measure& Generate3(cont)

(22 to 70) mV

(70 to 220) mV

(220 to 700) mV

(0.7 to 2.2) V

(2.2 to 7) V

(10 to 20) Hz(20 to 40) Hz40 Hz to 20 kHz(20 to 50) kHz(50 to 100) kHz(100 to 300) kHz(300 to 500) kHz500 kHz to 1 MHz




(300 to 500) kHz500 kHz to 1 MHz

(10 to 20) Hz(20 to 40) Hz40 Hz to 20 kHz(20 to 50) kHz(50 to 100) kHz(100 to 300) kHz(300 to 500) kHz

500 kHz to 1 MHz


0.022 % + 1.2 V0.012 % + 1.2 V0.0071 % + 1.2 V0.012 % + 1.6 V0.025 % + 2.0 V0.048 % + 3.2 V0.063 % + 6.3 V0.096 % + 6.3 V

0.017 % + 1.2 V0.0076 % + 1.2 V0.0037 % + 1.2 V0.0058 % + 1.6 V0.013 % + 2.0 V

0.022 % + 3.2 V0.031 % + 6.3 V0.079 % + 6.3 V

0.017 % + 1.2 V0.0061 % + 1.2 V0.0029 % + 1.2 V0.0046 % + 1.6 V0.0063 % + 2.0 V0.017 % + 4.3 V

0.025 % + 6.3 V0.074 % + 6.3 V

0.018 %0.0061 %0.0022 %0.004 %0.0057 %0.015 %0.023 %

0.074 %

0.016 %0.0054 %0.0024 %0.0041 %0.0065 %

Fluke 5790A, 5720A


19/54


AC Voltage Measure& Generate3(cont)

(7 to 22) V

(22 to 70) V

(70 to 220) V*

(220 to 700) V

(700 to 1000) V**

(10 to 20) Hz(20 to 40) Hz40 Hz to 20 kHz(20 to 50) kHz(50 to 100) kHz(100 to 300) kHz(300 to 500) kHz500 kHz to 1 MHz




(300 to 500) kHz


(10 to 20) Hz(20 to 40) Hz

40 Hz to 20 kHz(20 to 50) kHz(50 to 100) kHz

0.016 %0.0057 %0.0030 %0.0039 %0.0065 %0.018 %0.033 %0.097 %

0.016 %0.0058 %0.0034 %0.0050 %0.0075 %

0.016 %0.032 %0.09 %

0.016 %0.0058 %0.0034 %0.0056 %0.0080 %0.017 %

0.039 %

0.016 %0.0080 %0.0038 %0.011 %0.040 %

0.016 %0.0080 %

0.0036 %0.011 %0.040 %

Fluke 5790A, 5720A

* Subject to 2.2 x 107V-Hz limitation

Fluke 5790A, 5720A,5725A

** Measure only


20/54


AC High Voltage Measure3

(1 to 5) kV(5 to 35) kV(35 to 75) kV

(50 to 60) Hz 0.2 % + 1 V0.4 % + 65 V0.4 % + 49 V

Vitrek 4700A

AC High Voltage Generate3

(1 to 5) kV 60 Hz 0.24 % + 1 V Assoc. res. 3565D

AC Voltage Measure32 MHz

(0 to 10) mV

(10 to 100) mV

(0.1 to 1) V

(1 to 10) V

(1 to 40) Hz40 Hz to 1 kHz(1 to 20) kHz(20 to 50) kHz(50 to 100) kHz(100 to 300) kHz

(1 to 40) Hz40Hz to 1kHz(1 to 20) kHz

(20 to 50) kHz(50 to 100) kHz(100 to 300) kHz300 kHz to 1 MHz(1 to 2) MHz

(1 to 40) Hz40 Hz to 1kHz(1 to 20) kHz(20 to 50) kHz

(50 to 100) kHz(100 to 300) kHz300 kHz to 1 MHz(1 to 2) MHz

(1 to 40) Hz40 Hz to 1 kHz

0.023 % + 2 V0.018 % + 0.74 V0.026 % + 0.74 V0.069 % + 0.74 V0.34 % + 0.74 V2.7 % + 1.4 V

0.033 % + 2.7 V0.014 % + 1.4 V0.019 % + 1.4 V

0.037 % + 1.4 V0.067 % + 1.4 V0.22 % + 6.7 V0.68 % + 6.7 V1.1 % + 6.7 V

0.0047 % + 27 V0.0047 % + 14 V0.0094 %+ 14 V0.020 % + 14 V

0.054 % + 14 V0.20 % + 67 V0.67 % + 67 V1.1 % + 67 V

0.0077 % + 0.27 mV0 0062 % + 0 14 mV

Agilent/HP 3458A


21/54

Parameter/Range Frequency CMC2, 5, 7, 8() Comments

AC Voltage Measure3(cont) 2 MHz

(10 to 100) V

(100 to 700) V

(1 to 40) Hz40 Hz to 1 kHz(1 to 20) kHz(20 to 50) kHz(50 to 100) kHz(100 to 300) kHz300 kHz to 1 MHz

(1 to 40) Hz40 Hz to 1 kHz(1 to 20) kHz(20 to 50) kHz(50 to 100) kHz

0.025 % + 2.7 mV0.016 % + 1.4 mV0.014 % + 1.4 mV0.025 % + 1.4 mV0.082 % + 1.4 mV0.27 % + 6.7 mV1.1 % + 6.7 mV

0.033 % + 27 mV0.027 % + 14 mV0.041 % + 14 mV0.081 % + 14 mV0.21 % + 14 mV

Agilent/HP 3458A

Capacitance Measure3

1 pF10 pF100 pF to 1 F

(0.7 to 110) mF

10 pf to 0.1 F

1 kHz

DC

12 Hz to 2 MHz

10 F/F13 F/F14 F/F

0.012 %

See Table IV.a

AH2500A

5700A w/ 3458A,charge method

1689M , Quad tech 7600

Capacitance Generate3,Fixed Points

100 pF0.001 F0.002 F(0.01, 0.02, 0.05) F(0.1, 0.5, 1) F

100 Hz, 1 kHz 0.054 %0.068 %0.056 %0.055 %0.055 %

Standard capacitors


22/54

Table IV.a: CMC (in %) for Capacitance measured with 1689M and 7600 LCR Meter, Fixed Points

10 pF 100 pF 1 nF 10 nF 100 nF 1 F 10 F 100 F 0.001 F 0.01 F 0.1 F12 Hz 3.1 5 1.5 0.79 0.93 0.14 0.11 0.46 4.5

20 Hz 1.9 3 0.63 0.48 0.49 0.11 0.11 0.76

50 Hz 3.8 0.39 0.47 0.25 0.11 0.16 0.058 0.11 0.94

100 Hz 1.4 0.16 0.21 0.15 0.047 0.067 0.047 0.16 1.4

200 Hz 0.71 0.081 0.12 0.095 0.047 0.047 0.047 0.29 2.8

500 Hz 1.9 0.2 0.035 0.042 0.058 0.035 0.035 0.058 0.48 4.4

1 kHz 0.48 0.058 0.024 0.024 0.024 0.024 0.024 0.058 0.48 4.7

2 kHz 0.71 0.081 0.047 0.035 0.035 0.041 0.035 0.17 1.2

5 kHz 0.66 0.083 0.056 0.047 0.047 0.056 0.08 0.32 2.810 kHz 0.42 0.07 0.056 0.055 0.055 0.06 0.11 0.6

20 kHz 0.035 0.024 0.048 0.057 0.058 0.068 0.18 1.3

50 kHz 0.024 0.024 0.066 0.066 0.07 0.11 0.46 4.1

100 kHz 0.024 0.03 0.08 0.081 0.091 0.2 1.2

200 kHz 0.27 0.11 0.11 0.12 0.15 0.46 3.7

500 kHz 0.43 0.4 0.2 0.38 0.38 2.1

1 MHz 0.36 0.35 0.35 0.41 1.1

2 MHz 0.65 0.64 0.66 0.9 3.4


DC Power3

33 mV to 1020 V 330 A to 330 mA330 mA to 3 A(3 to 20.5) A

0.068 %0.07 %0.12 %

Fluke 5520A

Parameter/Equipment Range CMC2, 6, 8 () Comments

DC Current Generate3 (10 to 20) A

(20 to 100) A(100 to 300) A

31 A/A + 12 A

27 A/A + 18 A/C0 011 % + 18 A/C

Fluke 5520A,3458A/HFL, andstandard resistor

HP power supply,shunt 3458A/HFL


23/54

Parameter/Equipment Range CMC2, 5, 6, 8 () Comments

DC Current Measure3 (0 to 100) nA(0.1 to 1) mA(1 to 10) mA

(10 to 100) mA(100 to 1) mA(1 to 10) mA

(10 to 100) mA(0.1 to 1) A(1 to 2) A(2 to 10) A(10 to 20) A

(20 to 100) A(100 to 300) A(300 to 1000) A

30 A/A + 0.27 nA19 A/A + 0.27 nA17 A/A + 0.067 nA

5.2 A/A + 0.01 nA4.9 A/A + 0.1 nA5.1 A/A + 1 nA

5.6 A/A + 60 nA7.3 A/A + 6 A8.9 A/A + 12 A31 A/A + 6 A31 A/A + 12 A

27 A/A + 8.7 A/C0.011 % + 8.7 A/C0.077 % + 8.7 A/C

Agilent/HP 3458A(90day cal)

Agilent/HP3458A/HFLFluke 742 resistor

standards

Fluke 3458A withshunts

DC Voltage Generate,Fixed Points

100 mV

1 V

10 V

19 V

100 V

1000 V

0.82 V/V

0.63 V/V

0.55 V/V

1.1 V/V

0.63 V/V

0.82 V/V

Fluke 732B, 752A,HP 34420A

Fluke 732B, 752A,HP 34420A

Fluke 732B

Fluke 732B, 720A,

752A,HP 34420A

Fluke 732B, 752A,HP 34420A

Fluke 732B, 752A,HP 34420A

DC Voltage Generate (0 to 220) mV220 mV to 2.2 V(2.2 to 11) V(11 to 22) V(22 to 220) V(220 to 1100) V

5.6 V/V + 0.39 V3.2 V/V + 0.63 mV2.4 V/V + 2.4 V2.4 V/V + 3.9 V3.2 V/V + 39 V4.8 V/V + 390 V

Fluke 5700A/EP


24/54


DC Voltage Measure3 Up to 100 mV100 mV to 1 V(1 to 10) V(10 to 100) V(100 to 1000) V

3.0 V/V + 0.21 V2.1 V/V + 0.21 V2.1 V/V + 0.34 V3.5 V/V + 21 V3.7 V/V + 74 V

Agilent/HP 3458A

DC High Voltage

Measure3

(1 to 6) kV

(6 to 35) kV(35 to 100) kV

0.2 % + 1 V

0.4 % + 26 V0.4 % + 140 V

Vitrek 4700A

DC High Voltage Generate3

(1 to 6) kV 0.21 % + 1.1 V Assoc. res. 3565Dmonitored with Vitrek4700A

Edge Characteristics

Amplitude (0 to 100) mV

(0.1 to 1) V

(1 to 10) V(10 to 100) V

3.0 V/V + 0.21 V2.1 V/V + 0.21 V2.1 V/V + 0.34 V3.5 V/V + 21 V

Agilent/HP 3458A/HFL

Electrical Calibration of

RTD IndicatingDevices3

Pt 385, 100

Pt 3926, 100

-200 C to 0 C0 C to 100 C100 C to 300 C300 C to 400 C400 C to 630 C630 C to 800 C

-200 C to 0 C0 C to 100 C100 C to 300 C300 C to 400 C400 C to 630 C

0.05 C0.07 C0.09 C0.1 C0.12 C0.14 C

0.05 C0.07 C0.09 C0.1 C0.26 C

Fluke 5520A


25/54

Parameter/Equipment Range CMC2() Comments

Electrical Calibration ofRTD IndicatingDevices3(cont)

Pt 385, 200

Pt 385, 500

Pt 385, 1000

Pt 3916, 100

-200 C to 100 C100 C to 260 C260 C to 300 C

300 C to 400 C400 C to 600 C600 C to 630 C

-200 C to -80 C-80C to 100 C100 C to 260 C260 C to 400 C400 C to 600 C

600 C to 630 C

-200 C to 0 C0 C to 100 C100 C to 260 C260 C to 300 C300 C to 600 C600 C to 630 C

-200 C to -190 C-190 C to -80 C-80 C to 0 C0 C to 100 C100 C to 260 C260 C to 300 C300 C to 400 C400 C to 600 C600 C to 630 C

0.04 C0.05 C0.12 C

0.13 C0.14 C0.16 C

0.04 C0.05 C0.06 C0.08 C0.09 C

0.11 C

0.03 C0.04 C0.05 C0.06 C0.07 C0.23 C

0.25 C0.04 C0.05 C0.06 C0.07 C0.08 C0.09 C0.10 C0.23 C

Fluke 5520A

Electrical Calibration ofThermocoupleIndicating Devices3

Type E -250 C to -100 C-100 C to -25 C

0.5 C0 16 C

Fluke 5520A


26/54


Electrical Calibration ofThermocoupleIndicating Devices3(cont)

Type J

Type K

Type S

Type T

-210 C to -100 C-100 C to -30 C-30 C to 150 C150 C to 760 C760 C to 1200 C

-200 C to -100 C-100 C to -25 C-25 C to 120 C120 C to 1000 C1000 C to 1372 C

0 C to 250 C250 C to 1000 C1000 C to 1400 C1400 C to 1767 C

-250 C to -150 C-150 C to 0 C0 C to 120 C120 C to 400 C

0.28 C0.16 C0.14 C0.17 C0.23 C

0.34 C0.18 C0.16 C0.26 C0.40 C

0.47 C0.36 C0.37 C0.46 C

0.63 C0.24 C0.16 C0.14 C

Fluke 5520A


27/54


High FrequencyCapacitance, FixedPoints3

1 pF

10 pF

100 pF

1000 pF

1 kHz1 MHz2 MHz

3 MHz4 MHz5 MHz10 MHz13 MHz

1 kHz(1, 2) MHz3 MHz

4 MHz5 MHz10 MHz13 MHz

1 kHz1 MHz2 MHz3 MHz


1 kHz1 MHz2 MHz3 MHz


73 parts in 1060.01 %0.023 %

0.041 %0.063 %0.088 %0.26 %0.38 %

62 parts in 10662 parts in 10664 parts in 106

67 parts in 106

72 parts in 1060.013 %0.017 %

61 parts in 10662 parts in 10668 parts in 10682 parts in106

0.01 %0.014 %0.034 %0.049 %

63 parts in 10680 parts in 1060.016 %0.028 %

0.046 %0.064 %0.2 %0.29 %

Agilent/HP 16380Aseries capacitors


28/54

Parameter/Range Frequency CMC2, 8 () Comments

High FrequencyCapacitance, FixedPoints3 (cont)

0.01F

0.1 F

1 F

(0.12, 1, 10, 100) kHz

(0.12, 1, 10, 100) kHz

0.12 kHz(1, 10) kHz100 kHz

63 parts in 106

63 parts in 106

69 parts in 10663 parts in 10685 parts in 106

Agilent/HP 16380Aseries capacitors

Inductance3

Measure (12 Hz to

2 MHz)

Generate FixedPoints

0.1 H to 10 H

100 H(1, 5, 10, 100) mH(1, 2, 5) H

See Table IV.b

0.026 %0.011 %0.011 %

LCR meters

Standard inductors

Table IV.b: Inductance Measure CMC (in %) measured with 1689M and 7600 LCR Meters

Freq. 0.1 H 1 H 10 H 100 H 1 mH 10 mH 100 mH 1 H 5 H 10 H

12 Hz 0.81 1.3 1.1 1.1 1.1

20 Hz 4.8 0.49 0.71 0.58 0.56 0.56

50 Hz 0.97 0.11 0.17 0.26 0.26 0.26

100 Hz 3.6 0.37 0.048 0.069 0.11 0.16 0.16

200 Hz 1.8 0.19 0.048 0.048 0.058 0.11 0.085

500 Hz 4.8 0.49 0.060 0.036 0.036 0.036 0.051 0.036

1 kHz 1.2 0.13 0.024 0.024 0.024 0.024 0.024 0.024

2 kHz 0.90 0.13 0.036 0.042 0.036 0.036 0.048 0.0485 kHz 3.2 0.37 0.084 0.048 0.062 0.062 0.055 0.079 0.096

10 kHz 1.7 0.22 0.079 0.064 0.062 0.064 0.069 0.097 0.26

20 kHz 0.94 0.15 0.074 0.066 0.066 0.067 0.081 0.030 0.048

50 kHz 4.7 0.54 0.12 0.080 0.075 0.075 0.16 0.50 0.058 0.11

100 kHz 3.4 0.42 0.13 0.095 0.091 0.092 0.21 1.9 0.11 0.20


29/54

Parameter/Range Frequency CMC2 () Comments

Phase Generate

5 Vrms (Voltage Ratio= 1)

50 m to 100 Vrms

(100 to 120) Vrms

(0.1 to 100) Vrms

1 Hz to 1 kHz(1 to 6.25) kHz(6.25 to 50) kHz(50 to 200) kHz

1 Hz to 1 kHz

(1 to 6.25) kHz(6.25 to 50) kHz(50 to 200) kHz

1 Hz to 1 kHz(1 to 6.25) kHz(6.25 to 50) kHz(50 to 200) kHz

1 Hz to 1 kHz(1.001 to 6.25) kHz(6.25 to 50) kHz(50.01 to 100) kHz

6.6 m5.2 m13 m21 m

(6.5 + (0.05 Ratio))m

(11 + (0.1 Ratio))m(19 + (0.15 Ratio))m(41 + (0.4 Ratio))m

(15 + (0.1 Ratio))m(21 + (0.2 Ratio))m(31 + (0.3 Ratio))m(81 + (1 Ratio))m

13 m13 m20 m42 m

Clark-Hess 5500-2Ratio= ratio of thelarger voltage dividedby the smallervoltage

Phase Measure

(0.01 to 0.1) Vrms

(0.1 to 120) Vrms


(50 to 100) kHz


0.210.36

0.71

0.0540.036

KH 6620 phase meter


30/54


Oscilloscope Calibration3

Squarewave Signal50 at 1 kHz Source

Squarewave Signal1 Mat 1 kHz Source

Leveled Sine WaveFlatness (relative to 50kHz)

Period

Rise Time Generate

Rise Time Measure

(1 to 110) mV110 mV to 2.2 V(2.2 to 11) V(11 to 1100) V

(1 to 110) mV110 mV to 2.2 V(2.2 to 11) V(11 to 1100) V

50 kHz to 100 MHz(100 to 300) MHz(300 to 600) MHz(600 to 1100) MHz

1 ns to 20 ms50 ms100 ms200 ms500 ms1 s2 s5 s

12.8 ps

> 5.4ps

0.27 % + 42 V0.27 % + 130 V0.27 % + 1.2 mV0.27 % + 12 mV

0.15 % + 42 V0.15 % + 130 V0.15 % + 1.2 mV0.15 % + 12 mV

1.5 % + 110 V2 % + 110 V4 % + 110 V5 % + 110 V

2.0 parts in 10659 parts in 10697 parts in 106180 parts in 106410 parts in 106800 parts in 1061600 parts in 1063900 parts in 106

3.9 ps

5.4 ps

Fluke 5520A/SC1100scope option

Tektronix 067-1338-00

Tektronix TDS8000with 80E03

Resistance Generate,Fixed Points3

100

1 m 10 m 0.1

1 10 100

29 /

14 /9.5 /6.5 /

4.6 /4.2 /2 3 /

Norma 80

L&N 4222BL&N 4223B

L&N resistors

Fluke 742A, standard


31/54

Parameter/Equipment Range CMC2, 8 () Comments

Resistance Generate,Fixed Points3(cont)

10 G100 G1 T10 T

0.13 %0.22 %0.34 %1.2 %

IET high resistancestandard

Resistance Measure,

DC

100

100 to1 m(1 to 10) m(10 to100) m(0.1 to 1)

(0 to 2) (2 to 20) (20 to 200) 200 to 2 k(2 to 20) k

(20 to 200) k200 kW to 2 M(2 to 20) M(20 to 200) M200 Mto 2 G(2 to 20) G

100 Mto 1 G(1 to 10) G(10 to 100) G100 Gto 1 T(1 to 10) T(10 to 100) T

85 /+ 0.48 n

12 /+ 4.8 n9.2 /+ 4.8 n5.4 /+ 48 n2.6 /+ 0.48

3.4 /+ 4.4 2.1 /+ 15 2.6 /+ 33 2.5 /+ 0.33 m1.5 /+ 3.3 m

2.3 /+ 33 mW4.4 /+ 1.1 4.4 /+ 11 8.3 /+ 110 28 /+ 110 k0.21 % + 1.1 M

0.081 %0.12 %0.24 %

0.35 %0.59 %1.2 %

Stable current source,

3458A/HFL, standardresistors

Fluke 8508A andstandard resistors

6500A

Resistance Measure,AC

10 Hz to 2 MHz See Table IV.c LCR meters


32/54

Table IV.c: AC Resistance Measure CMC (in %) measured with LCR Meters

0.1 1 10 100 1 k 10 k 100 k 200 k 500 k 1 M12 Hz 0.60 0.11 0.11 0.11 0.11 0.11 1.720 Hz 0.60 0.11 0.11 0.11 0.11 0.11 0.85 2.8 4.650 Hz 3.1 0.31 0.058 0.058 0.058 0.058 0.058 0.38 1.3 2.0

100 Hz 2.3 0.23 0.047 0.047 0.047 0.047 0.047 0.23 0.70 1.2200 Hz 2.3 0.23 0.047 0.047 0.047 0.047 0.047 0.15 0.44 0.70

500 Hz 1.5 0.16 0.035 0.035 0.035 0.035 0.035 0.10 0.29 0.441 kHz 0.77 0.09 0.024 0.024 0.024 0.024 0.024 0.082 0.24 0.35

2 kHz 1.0 0.15 0.035 0.035 0.035 0.035 0.047 0.075 0.21 0.315 kHz 0.90 0.14 0.047 0.047 0.047 0.047 0.063 0.072 0.20 0.30

10 kHz 1.0 0.14 0.062 0.055 0.055 0.055 0.063 0.073 0.21 0.3020 kHz 1.1 0.16 0.066 0.057 0.057 0.057 0.067 0.077 0.22 0.3350 kHz 1.4 0.19 0.077 0.066 0.065 0.066 0.31 0.37 1.1 1.7

100 kHz 1.9 0.26 0.10 0.081 0.079 0.081 0.40 0.47 1.5 2.2200 kHz 3.0 0.39 0.14 0.11 0.11 0.12 0.56 0.68 2.2 3.4500 kHz 0.79 0.26 0.20 0.20 0.26 1.1 1.4 4.21 MHz 1.5 0.45 0.35 0.35 0.70 1.9 2.42 MHz 2.8 0.85 0.65 0.64 1.4 3.5 4.5


33/54

V. Electrical RF/Microwave


Amplitude Modulation3

AM Accuracy:

(0.15 to 10) MHz

5 % to 40 % AM Depth


(10 to 1300) MHz



(1.3 to 26.5) GHz



AM Distortion:

150 kHz to 1.3 GHz

(1.3 to 26.5) GHz

AM Flatness:(0.01 to 26.5) GHz


Rate:50 Hz to 10 kHz

Rate: 20 Hz to 10 kHz

Rate 50 Hz to 10 kHz










< 50 % AM Depth


< 50 % AM Depth



2.3 % + 0.012 % AM

3.5 % + 0.012 % AM

2.3% + 0.12 % AM

3.5 % + 0.12 % AM

1.2 % + 0.012 % AM

3.5 % + 0.012 % AM

1.2 % + 0.12 % AM

3.5 % + 0.12 % AM

1.2 % + 0.012 % AM

3.5 % + 0.012 % AM

1.2 % + 0.12 % AM

3.5 % + 0.12 % AM

1.1 dB

2.1 dB

1.1 dB

2.1 dB

0.31 %

Measuring

receiver

Measuring

receiver, down

converter and

local oscillator

Measuring

receiver and

distortion analyzer

Measuring

receiver

Di t ti A


34/54


Frequency Modulation3

FM Accuracy(0.15 to 10) MHz< 4 kHz FM peak Dev.< 40 kHz FM peak Dev.

(10 to 1300) MHz< 400 kHz FM pk Dev.

(1.3 to 26.5) GHz< 400 kHz FM pk Dev.

FM Distortion

400 kHz to 10 MHzRate: 20 Hz to 10 kHz

10 MHz to 26.5 GHzRate: 20 Hz to 100 kHz

Residual FMCarrier Frequency ()

< 100 MHz

(100 to 1300) MHz(1.3 to 6.2) GHz(6.2 to 12.4) GHz(12.4 to 18) GHz

Rate:

20 Hz to 10 kHz20 Hz to 10 kHz



< 10 kHz FM Deviation

< 100 kHz FM Deviation

Bandwidth:50 Hz to 3 kHz

50 Hz to 3 kHz50 Hz to 3 kHz50 Hz to 3 kHz50 Hz to 3 kHz

2.4 % + 1.2 Hz2.4 % + 12 Hz

1.3 % + 120 Hz5.8 % + 240 Hz

1.3 % + 120 Hz5.8 % + 100 Hz

0.15 %

0.15 %

1 Hz (rms)

0.4 Hz + (6 x 10

-9

) 17 Hz33 Hz49 Hz

Measuring receiver

Measuring receiverand distortionanalyzer

Measuring receiver

Phase Modulation3

(0.15 to 10) MHz< 40 Radians (peak)

< 400 Radians (peak)

(10 to 1300) MHz< 4 Radians (peak)< 40 Radians (peak)< 400 Radians (peak)



4.7 % + 0.012 rad

4.7 % + 0.12 rad

5.2 % + 0.0012 rad3.5 % + 0.012 rad3.5 % + 0.12 rad

Measuring receiver


35/54


Range Calibration (PowerMeters)3

Zero SetInstrument Accuracy 3 W

10 W30 W(100, 300) W

(1, 2) mW(10, 30, 100) mW

0.25 % + 6.2 nW1.6 %0.46 %0.28 %0.26 %

0.28 %0.30 %

Range calibrator,Agilent 11683A orAgilent 8477A

RF Power Level3Generate

Absolute

Relative

(Power SensorCalibration Factors)

Relative

(Power SensorCalibration Factors)

0 dBm @ 50 MHz

+20 to - 20 dBm0.1 MHz to 26.5 GHz

-20 to -127 dBM0.9 MHz to 1.3 GHz

-20 to -100 dBm(1.3 to 26.5) GHz

100 kHz

300 kHz1 MHz3 MHz10 MHz30 MHz100 MHz300 MHz500 MHz

1 GHz

2 GHz3 GHz4 GHz5 GHz6 GHz7 GHz8 GHz

0.11 dB +M

0.12 dB +M

0.18 dB +M

0.18 dB +M

0.84 % + M

0.53 % + M0.47 % +M0.47 % +M0.45 % +M0.45 % +M0.55 % +M0.62 % +M0.62 % +M

0.67 % +M

0.78 % +M0.95 % +M1.1 % +M1.2 % +M1.2 % +M1.2 % +M1 2 % + M

Signal generator,power splitter,

attenuator, standardsensors, and powermeter

Signal generator,

power splitter,attenuator, standardsensors, and powermeter. Referencedto, 1 mW

UUT Mismatchuncertainty (M)

Signal generator,

power splitter,attenuator, standardsensors, and powermeter. Referencedto, 1 mW

UUT Mismatch


36/54


RF Power Level3Generate (cont)

Relative(Power SensorCalibration Factors)

16 GHz17 GHz18 GHz19 GHz20 GHz21 GHz

22 GHz23 GHz24 GHz25 GHz26 GHz26.5 GHz

2.2 % +M1.8 % +M2.9 % +M4.1 % +M5.2 % +M4.7 % +M

3.5 % +M2.8 % +M3.1 % +M3.9 % +M3.9 % +M3.9 % +M

Signal generator,power splitter,attenuator, standardsensors, and powermeter. Referencedto, 1 mW


RF Power Level3 Measure

Absolute 0 dBm @ 50 MHz

+45 to 20 dBm0.1 MHz to 4.2 GHz

+20 to - 20 dBm0.1 MHz to 26.5 GHz

-20 to -127 dBM0.9 MHz to 1.3 GHz

-20 to -100 dBm(1.3 to 26.5) GHz

0.04 dB +M

0.06 dB +M

0.05 dB +M

0.15 dB +M

0.15 dB +M

8478B, 432A

8482B, powermeter

8482A, 8481A,8485A, powermeter

11722A, 8902A

11792A, 8902A11793A, E8244A



37/54


RF Power Level3Measure, Relative (cont)

0 dB

-10 dB-20 dB-30 dB-40 dB-50 dB-60 dB-70 dB-80 dB-90 dB-100 dB

-110 dB-120 dB

-127 dB

+ 10 to -100 dBm0.9 MHz to 26.5 GHz

-100 to -127dBm0.9 MHz to 1.3 GHz

0.01 dB +M

0.04 dB +M0.04 dB +M0.04 dB +M0.08 dB +M0.08 dB +M0.08 dB +M0.08 dB +M0.10 dB +M0.10 dB +M0.10 dB +M

0.10 dB +M0.12 dB +M

0.19 dB +M

11722A, 11792A8902A, 11793A,E8244A

11722A, 8902A


Power Sensors, FixedPoints3

100 kHz300 kHz1 MHz(10, 30, 50) MHz100 MHz(0.3, 0.5, 1) MHz2 GHz

3 GHz4 GHz5 GHz6 GHz7 GHz8 GHz9 GHz10 GHz11 GHz(12, 12.4) GHz

13 GHz14 GHz15 GHz16 GHz17 GHz18 GHz19 GHz

2.8 %0.98 %0.93 %0.92 %0.77 %0.83 %0.85 %

0.88 %0.87 %1.1 %1.1 %1.1 %1.1 %1.3 %1.4 %1.7 %1.4 %

1.3 %1.4 %1.7 %1.7 %1.4 %2.1 %3.2 %

Signal generator,power splitter,attenuator, standardsensors, and powermeter referenced to50MHz, 1W


38/54


S-Parameters3 ReflectionS11/22 Magnitude andPhase

3.5 mm (0.01 to 1.0) lin(0 to 0.01) lin(0.01 to 0.1) lin(0.01 to 0.5) lin

(0.5 to 1) lin

Magnitude(0 to 1) lin

Phase(0 to 0.2) lin

(0.2 to 1) lin

7 mm Magnitude

(0 to 1) lin

Phase(0 to 0.2) lin

(0.2 to 1) lin

N-Type Magnitude(0 to 1) lin

Phase

(1.5 to 26.5) GHz

300 kHz to 1.3 GHz(1.3 to 3.0) GHz(3.0 to 6.0) GHz

300 kHz to 1.3 GHz(1.3 to 3.0) GHz

(3.0 to 6.0) GHz





100 kHz to 0.3 GHz(0.3 to 1.3) GHz(1.3 to 3.0) GHz(3.0 to 6.0) GHz

(0.012 to 0.040) lin(7.4 to 180) deg(0.095 to 20) deg(0.54 to 2.3) deg

(0.50 to 1.1) deg

(0.005 to 0.015) lin(0.008 to 0.002) lin(0.015 to 0.032) lin

(180 to 2) deg(180 to 3) deg

(180 to 4.5) deg

(2 to 1) deg(3 to 1.5) deg(4.5 to 2) deg

(0.001 to 0.006) lin(0.002 to 0.01) lin(0.005 to 0.018) lin

(180 to 1) deg(180 to 1.5) deg(180 to 2) deg

(1 to 0.5) deg(1.5 to 0.5) deg(2 to 1) deg

0.0085 lin(0.005 to 0.018) lin(0.005 to 0.025) lin(0.01 to 0.05) lin

Network analyzerwith S-parametertest set, calibrationkit, and cables


39/54

Parameter/Range Frequency CMC2

() Comments

S-Parameters3Transmission S12/S21Magnitude andPhase

3.5 mm(0 to 20) dB

(20 to 40) dB

(10 to -90) dB

7 mm(10 to -90) dB

N-Type(10 to -90) dB

(1.5 to 26.5) GHz

(1.5 to 26.5) GHz

300 kHz to 1.3 GHz

(1.3 to 3.0) GHz

(3.0 to 6.0) GHz

300 kHz to 1.3 GHz

(1.3 to 3.0) GHz

(3.0 to 6.0) GHz

300 kHz to 1.3 GHz

(1.3 to 3.0) GHz

(3.0 to 6.0) GHz

(0.11 to 0.29) dB

(1.2 to 12) deg

(0.13 to 0.68) dB(1.5 to 19) deg

(0.3 to 6.0) dB(0.25 to 80) deg

(0.04 to 6.0) dB(0.35 to 90) deg

(0.06 to 9.0) dB(0.67 to 180) deg

(0.03 to 5.0) dB(0.26 to 80) deg

(0.03 to 5.0) dB(0.26 to 90) deg

(0.05 to 10) dB(0.54 to 180) deg

(0.03 to 6.0) dB(0.2 to 80) deg

(0.04 to 6.0) dB(0.44 to 90) deg

(0.1 to 9.0) dB(0.67 to 90) deg

Network analyzer

with S-parametertest set, calibrationkit, and cablesSystem verificationwith verification kit


40/54

VI. Fluid Quantities


Dynamic Viscosity3 (0 to 100 000) cP 0.83 % Standard viscosityfluids, PRT

Flow3 2 sccm to 10 slm

(10 to 1000) slm

0.26 %

0.51 %

Mass flow system

Hydrometers

BaumeLighter than Water

(80.5 to 57)API (81 to 57)

(57 to 10)API (57 to 10)

Heavier than Water

(0.72 to 21)API (10 to -11)

(0.72 to 67)

Specific Gravity:

1.00

(0.66 to 0.75)

(0.75 to 1)

(1 to 1.17)

(1 to 1.85)

0.0058 % + 0.6R

0.0072 % + 0.6R

0.006 % + 0.6R

0.0058 % + 0.6R

0.0058 % + 0.6R

Distilled water

Petroleum basedsolutions

Alcohol solutions

Sulfuric acid and/orglycerine solutions

Sulfuric acidsolutions

Kinematic Viscosity3 Zahn cups (1 to 5)Shell cups (1 to 6)

0.84 s + 0.24 %0.84 s + 0.24 %

Temperature bath,viscosity oils,

stopwatch

Volumetric Calibration (0 to 200) mL200 mL to 1 L(1 to 25) L

0.24 % + 130 nL0.24 % + 9 L0.24 % + 900 L

Gravimetric method


41/54

VII. Foundry-Industry Specific Calibrations


AFS Clay Tester3 --- 0.42 s Stopwatch

Cone Jolt ToughnessTester3

Cam MeasurementRise In CamCone Weight

0.030 in0.030 in(470 to 480) g

180 in200 in0.14 g

Micrometer feelergagescale

Friability Tester3 60 s 0.42 s Stopwatch

Hot Tensile Tester3

LoadTemperature

Gap Measure

130 psi450 F

0.030 in

0.38 psi0.7 F

200 in

Dead weightTemperaturemeasuring device

Feeler gage

Melt Point Apparatus3 (200 to 220) F 2.8 F Temperaturemeasuring device

Mold Strength Tester3 (0 to 50) psi 0.2 % + 0.6R Balances andweights

Moisture Teller3 (0 to 300) F 2 F Temperaturecalibrator

Permmeter3 (0 to 200) perms 6.2 perms Perm standard

Sand Rammer3(Compactability Tester)

--- 0.0066 in + 0.6R Impact rings


42/54


43/54


Cable Tensiometers/Wire Tension Meters

(0 to 1000) lb 1.1 lb + 0.6R Deadweights

Deadweight Testers

Piston and CylinderArea

Mass

Performance Test(Cross-Float)

(0 to 10 000) psi (12 + 3D) in

0.2 g

0.04 %

UMM

Class F weights andscales

Pressure calibrator withtransducer

Durometers3

Indenter Shape andExtension:

Durometer Spring

Types A, B, C, D, E,O and DO

Pressure foot orificediameter

Indentor extensionlength

Cone angle

Tip radius

Indenter thickness

Up to 45 N

3.2 m

3.2 m

1 arcmin

3.2 m

3.2 m

36 mN

ASTM D2240

Vision system

Durometer calibrator

Durometer Calibrator Scales A and D 0.2 % Scale and CMM


44/54


Dynamometer andForce Rings3

(0 to 10 000) lbf 0.11 % + 0.6R Load cells

Force Gages (0 to 500) lbf(200 to 10 000) lbf

0.0024 % + 0.007 lbf0.6 % + 1.2 lbf

DeadweightsLoad cell

Force Gages3 (0 to 250) lbf 0.12 % + 0.6R Deadweights

Indirect Verification ofBrinell HardnessTesters at TestCondition(s)3

(3000, 1500, 500) Kg (125 to 260) HBW 0.008d

ASTM E10

dis the mean of the n

mean test diameter inmillimeters.

Indirect Verification ofLeeb Hardness Testers

Up to 800 HLD 7.4 HLD Standard test block

Indirect Verification of

MicroindentationHardness Testers(Knoop and Vickers)3

Repeatability under

forces P(gf):

1 P< 500

100 HK 940100 HV 940

20 HK + 1.2 %13 HV + 3 %

ASTM E384


45/54


Indirect Verification ofRockwell Hardness andRockwell SuperficialHardness Testers3

HRA:(22 to 63) HRA(64 to 73) HRA(74 to 83) HRA

HRBW:(0 to 40) HRB(41 to 60) HRB

(61 to 80) HRB

HRC:(25 to 35) HRC(40 to 55) HRC(60 to 70) HRC

HRE:(57 to 63) HRE(64 to 81) HRE(82 to 99) HRE

HRF:(57 to 63) HRF(64 to 80) HRF(81 to 97) HRF

HR15N:(72 to 78) HR15N(80 to 88) HR15N(90 to 94) HR15N



HR15T:(61 to 67) HR15T(70 to 77) HR15T(80 to 91) HR15T

HR30T:(15 to 28) HR30T

0.7 HRA0.7 HRA0.7 HRA

0.7 HRB0.7 HRB

0.7 HRB

0.7 HRC0.7 HRC0.7 HRC

0.7 HRE0.7 HRE0.7 HRE

0.7 HRF0.7 HRF0.7 HRF

0.7 HR15N0.7 HR15N0.7 HR15N

0.7 HR30N0.7 HR30N0.7 HR30N

0.7 HR45N0.7 HR45N0.7 HR45N

0.7 HR15T0.7 HR15T0.7 HR15T

0.7 HR30T


46/54


Load Cells (0 to 500) lbf

(500 to 10 000) lbf

0.034 % + 0.00044 lbf

0.05 % + 0.024 lbf

Dead weights,comparison to masterload cell

Load Cells3 (0 to 10 000) lbf 0.1 % + 0.002 lbf Dead weights,comparison to master

load cell

Manometers (0 to 24) in H2O 0.00017 in H2O Hook gage

Mass 1 mg to 50 g(50 to 220) g(220 to 400) g

400 g to 1.2 kg(1.2 to 8.2) kg(8.2 to 30) kg

17 g35 g0.14 mg

3.7 mg12 mg0.11 g

By comparison

Mass3 (2 to 30) kg 0.2 g By comparison

Pressure/Vacuum

Gauges

3

(-15 to 1000) psi

Pneumatic(1000 to 5000) psi

(5000 to 10 000) psi

0.005 psi + 0.012 % fs

0.32 psi + 0.026 % fs

1.8 psi + 0.026 % fs

Pressure calibrator

Pressure transducers

Refractometers3 (0 to 100) % 0.02 % + 0.6R Standard solutions

Scales3 (1 to 50) mg51 mg to 10 g(11 to 50) g(51 to 300) g(301 to 1000) g(1 to 5) kg

0.016 mg + 0.58R0.03 mg + 0.58R0.054 mg + 0.58R0.24 mg + 0.58R3.8 mg + 0.58R11 mg + 0 58R

Verification with massstandards


47/54


Scales3(cont) Up to 0.5 lb(0.5 to 1) lb(1 to 2) lb(2 to 5) lb(5 to 10) lb(10 to 20) lb(20 to 50) lb

(50 to 100) lb(100 to 500) lb(500 to 1000) lb

0.000 0015 oz + 0.58R0.000 0030 oz + 0.58R0.000 011 oz + 0.58R0.000 14 oz + 0.58R0.000 34 oz + 0.58R0.000 68 oz + 0.58R0.000 93 oz + 0.58R

0.0043 oz + 0.58R0.0086 oz + 0.58R0.043 oz + 0.58R

Verification withmass standards

Scales ProportionalTesting3

Up to 750 lb(750 to 1500) lb(1500 to 3000) lb(3000 to 4500) lb

(4500 to 6000) lb

0.22 lb + 0.6R0.28 lb + 0.6R0.36 lb + 0.6R0.42 lb + 0.6R

0.48 lb + 0.6R

NIST Class Fweights

Spring Testers3

Force Scale

Deflection Accuracy

(0 to 1000) lbf

Up to 20 in

0.02 % + 0.6R

26 in + 0.6R

Weights, load cells;Ris the resolution ofthe force scale of thespring tester.

Gage blocks;Risthe resolution of thedeflection scale.

Tachometers3

Non-Contact

Contact

Up to 180 000 rpm

Up to 5000 rpm(5000 to 15 000) rpm

0.8R

0.024 % + 1.4 rpm0.032 % + 0.84 rpm

Function generator,LED

Tachometercalibrator

Tensile Testers3 (0 to 30 000) lbf 0.4 % + 0.08 % fs Load cells


48/54


Torque Calibration ofTorque Meters andSensors

(0.1 to 2000) ftlbf 0.14 % Weights

Torque Watches3 (0.5 to 215) inozf 0.08 % + 0.52 inozf Torque calibrator

Ultrasonic ThicknessTesters3

(0 to 10) in (800 + 13L) in Standard thicknessspecimen

IX. Optical Quantities

Parameter/Equipment Range CMC

2

() Comments

Gloss Meters3 20

60, 85

0.73 GU

0.54GU

Gloss standards; GUrepresents gloss units.

X. Thermodynamics


Dew Point -20 C to 60 C -0.44 C + 0.6R Thunder Scientifichumidity chamber

Industrial PlatinumResistanceThermometers

-30 C to 75 C75 C to 200 C200 C to 250 C250 C to 300 C

0.024 C0.032 C0.041 C0.047 C

SPRT, temperaturebath, resistance bridge


49/54


Liquid-in-GlassThermometers andTemperature MeasuringDevices

-30 C to150 C150 C to 250 C250 C to 300 C

0.02 C + 0.6R0.03 C + 0.6R0.04 C + 0.6R

Constant temperaturebath, SPRT

Relative Humidity Measure

Hygrometer

Psychrometer

(10 to 95) % RH

5 C to 35 C

0.7 % RH + 0.6R

0.1 C + 0.84R

Humidity generator,Thunder Scientific2500ST

Comparison to SPRT intemp bath

Temperature Measuring Equipment3

-10 C to 50 C

50 C to 650 C

0.04 C + 0.6R

0.045 C + 0.6R

Dry-wells

Thermocouples andThermocouple Wire

-80 C to 650 C 0.42 C Comparison to SPRTwith Agilent 3458A

XI. Time & Frequency

Parameter/Equipment Range CMC2 () Comments

Frequency MeasuringEquipment

10 MHz

0.01 Hz to 200 kHz

200 kHz to 26.5 GHz

3 parts in 1011

3 parts in 1011+ 30 nHz

3 parts in 1011

GPS

Function generatorwith ext. timebase

Signal generator withext timebase


50/54


Stopwatches & Timers3 10 s to 72 hr

(2 to 960) s

28 ms + 0.6 s/s

0.037 s/day

Function generator,frequency counter

Timometer

______________________________________

1This laboratory offers commercial calibration service and field calibration service.

2Calibration and Measurement Capability Uncertainty (CMC) is the smallest uncertainty of measurementthat a laboratory can achieve within its scope of accreditation when performing more or less routinecalibrations of nearly ideal measurement standards or nearly ideal measuring equipment. CMCsrepresent expanded uncertainties expressed at approximately the 95 % level of confidence, usuallyusing a coverage factor of k = 2. The actual measurement uncertainty of a specific calibrationperformed by the laboratory may be greater than the CMC due to the behavior of the customers deviceand to influences from the circumstances of the specific calibration.

3Field calibration service is available for this calibration and this laboratory meets A2LAR104 GeneralRequirements: Accreditation of Field Testing and Field Calibration Laboratoriesfor these calibrations.Please note the actual measurement uncertainties achievable on a customer's site can normally beexpected to be larger than the CMC found on the A2LA Scope. Allowance must be made for aspectssuch as the environment at the place of calibration and for other possible adverse effects such as thosecaused by transportation of the calibration equipment. The usual allowance for the actual uncertaintyintroduced by the item being calibrated, (e.g. resolution) must also be considered and this, on its own,could result in the actual measurement uncertainty achievable on a customers site being larger than theCMC.

4In the statement of CMC,Lis the numerical value of the nominal length in inches;Ris the resolution ofthe unit under test; D is the diameter in inches; H is the height of the unit under test (except wherenoted); and fs represents full scale.Rais the numerical value of the nominal roughness of the surfacemeasured in micrometer roughness, except where noted.

5CMC for calibrations performed in the laboratory with the Agilent/HP 3458A/HFL is based upon 90-day specifications. CMC for calibrations performed field with the Agilent/HP 3458A is based upon 1-year specifications. The measurands stated are generated with the Agilent/HP 3458A. This capability issuitable for the calibration of the devices intended to measure the stated measurand in the rangesindicated. CMC are expressed as either a specific value that covers the full range or as a fraction of thereading plus a fixed floor specification. Unless otherwise noted, percentages are defined as percent ofreading.

6 CMC f lib ti f d ith th Fl k 5520A i b d 1 ifi ti CMC f

7 CMC f lib ti f d ith th Fl k 5790A i b d 1 ifi ti Th


51/54

7 CMC for calibrations performed with the Fluke 5790A is based upon 1-year specifications. Themeasurands stated are generated with the Fluke 5700 series of instruments. This capability is suitablefor the calibration of the devices intended to measure the stated measurand in the ranges indicated.CMCs are expressed as either a specific value that covers the full range or as a fraction of the readingplus a fixed floor specification. Unless otherwise noted, percentages are defined as percent of reading.

8In the statement of CMC, the value is defined as the percentage of reading, unless otherwise noted.

9This accreditation covers calibrations performed at the main laboratory listed above, and the followingsatellite laboratory located at 2591 S. Oakwood Road, Oshkosh, WI 54904.

Satellite Location:

PRECISION METROLOGY, INC.2951 S. Oakwood RdOshkosh, WI 54904

Carol Shipley Phone: 414 351 7420

Valid To: July 31, 2016 Certificate Number: 1078.01

I. Dimensional


Bore Gages3 Up to 12 in 10in/in + 0.6R Gage blocks

Calipers3

Resolution 0.0001 in

Resolution 0.0005 in

Resolution 0.001 in




78 in70 in + 4 in/in

450 in340 in + 3.7 in/in

840 in700 in + 3.4 in/in

Gage blocks

Chamfer Gages/HoleGages3

Up to 12 in (20 + 0.6R) in Cylindrical rings


52/54


Height Gage Up to 36 in 15 in/in + 0.6R Reference stack

Intertest/OditestCalipers


Micrometers3

DepthGroove

Up to 12 inUp to 4 in

20 in/in + 0.6R18 in/in + 0.6R

Gage blocks

Micrometer ODMicrometers

Up to 42 in 12 in/in + 0.6R Gage blocks

Protractor Digital and

Mechanical3

0to 180 0.0076 Angle blocks

Test Indicator Up to 0.5 in 0.0001 in Indicator calibrator

Thickness Gages3 Dialand Digital


Tri-Bore Micrometers Up to 3 in 15 in/in + 0.6R Cylindrical rings

II. Mechanical


Torque Wrenches Up to 1000 inlbfUp to 250 ftlbf

0.6 % + 0.6R0.6 % + 0.1 ftlbf

Torque calibrator


53/54

2Calibration and Measurement Capability Uncertainty (CMC) is the smallest uncertainty of measurementthat a laboratory can achieve within its scope of accreditation when performing more or less routinecalibrations of nearly ideal measurement standards or nearly ideal measuring equipment. CMCsrepresent expanded uncertainties expressed at approximately the 95 % level of confidence, usuallyusing a coverage factor of k = 2. The actual measurement uncertainty of a specific calibrationperformed by the laboratory may be greater than the CMC due to the behavior of the customers deviceand to influences from the circumstances of the specific calibration.

3Field calibration service is available for this calibration and this laboratory meets A2LAR104 GeneralRequirements: Accreditation of Field Testing and Field Calibration Laboratoriesfor these calibrations.Please note the actual measurement uncertainties achievable on a customer's site can normally beexpected to be larger than the CMC found on the A2LA Scope. Allowance must be made for aspectssuch as the environment at the place of calibration and for other possible adverse effects such as thosecaused by transportation of the calibration equipment. The usual allowance for the actual uncertaintyintroduced by the item being calibrated, (e.g. resolution) must also be considered and this, on its own,could result in the actual measurement uncertainty achievable on a customers site being larger than theCMC.

4In the statement of CMC,Ris the resolution of the unit under test.

5In the statement of CMC, the value is defined as the percentage of reading, unless otherwise noted.


54/54

A2LA has accredited

PRECISION METROLOGY, INC.Milwaukee, WI

for technical competence in the field of

CalibrationThis laboratory is accredited in accordance with the recognized International Standard ISO/IEC 17025:2005 General Requirements for

the Competence of Testing and Calibration Laboratories. This laboratory also meets the requirements of ANSI/NCSLI Z540-1-1994 and any

additional program requirements in the field of calibration. This accreditation demonstrates technical competence for a defined scope and theoperation of a laboratory quality management system(refer to joint ISO-ILAC-IAF Communiqu dated 8 January 2009).

Presented this 23rd

day of September 2014.

_______________________

President & CEO

For the Accreditation Council

Certificate Number 1078.01Valid to July 31, 2016

For the calibrations to which this accreditation applies, please refer to the laboratorys Calibration Scope of Accreditation.

Documents

Precision Metrology 1078-01