Nuclear Instruments and Methods in Physics Research B54 (1991) 1-6 North-Holland

Section I. Microprobe technology

Distributed computer control at the LLNL tandem *

M.L. Roberts, T.L. Moore, R.S. Homady and J.C. Davis Center for Accelerator Mass Spectrometty, Lawrence Livermore National Laboratory, Livermore, CA 94550, USA

Livermore is developing hardware and software tools to provide automated operation of an FN tandem accelerator. Automated operation is planned for several nuclear analytical techniques including hydrogen depth profiling and accelerator mass spectrometry. The control system utilizes a distributed computer system to provide 16 bit precision in control of accelerator power supplies.

1. Intruduction

The Physics Department of the Lawrence Livermore National Laboratory has operated for the past two years a new tandem Van de Graaff laboratory [l]. The facility utilizes a distributed computer system to control the accelerator and the facility radiation safety system. The laboratory was built primarily for multidisciplinary research with ion beams. Major design goals for the facility included low operating and staffing costs, preci- sion of operation, ability to shift rapidly from one mode of operation to another, and the ability to shut down for several months and restart efficiently if the research personnel shifted periodically to other assignments.

To meet the facility goals, a distributed control sys- tem operating through CAMAC interfaces to accelera- tor components was selected. This design has cost and schedule advantages in construction and in activation of geographically dispersed equipment, and allows a broad range of upgrade options. Varying degrees of automa- tion for standardized analytical measurement can be implemented within the system. In other large facilities at LLNL, previous versions of the system employed here have demonstrated much higher reliability and noise immunity than hard-wired systems. The intent of our work was to operate the accelerator in a computer- assisted fashion, i.e., all tuning and setting would be done by a scientist through the control system, with the system maintaining the machine precisely in the config- uration selected.

2. The multiuser tandem laboratory

The floor plan of the accelerator laboratory is shown in fig. 1. The accelerator is a High Voltage Engineering

* This work was performed under the auspices of the U.S. Department of Energy at the Lawrence Livermore National Laboratory under contract W-7405-Eng-48.

Corporation Model FN tandem, relocated from the University of Washington. As designed, the facility can accommodate up to five ion sources and nineteen ex- perimental beamlines. As most of the uses of the accel- erator produce little or no radiation, the facility was built as a single unshielded high bay with local shielding being used as required. Active radiation protection is provided by a combination of computer-monitored and hard-wired radiation sensors.

At present, three ion sources and seven research beamlines are installed and in use. A second switching magnet will be installed this year. Next year, replace- ment of the first inflection magnet will allow up to seven ion sources to be fielded permanently. The main control station is now placed near the low-energy end of the tandem; it will be relocated to a separate control room in the future.

The laboratory is supported by a consortium of users: LLNL’s Nuclear Physics and Nuclear Chemistry Divisions and the X-ray Laser Program, Sandia Na- tional Laboratories Physics Department, and the Re- gents of the University of California. The laboratory is operated by the Center for Accelerator Mass Spec- trometry of the LLNL Physics Department. Primary research missions are materials science with Sandia’s ion microprobe [2], hydrogen depth profiling in nuclear waste isolation materials, and experiments in nuclear physics, nuclear chemistry and nucleosyntheses. A sep- arate large effort is in the application of accelerator mass spectrometry to biomedical and environmental science and archaeology. Faculty and staff of the Uni- versity of California have access to the AMS spectrom- eter for research as a result of Regental co-funding of that spectrometer.

A unique characteristic of this facility is that most of its major components have been relocated and reconfig- ured from other uses. The accelerator was obtained from Washington, the switching magnets are from former cyclotrons (one from Duke University, the other from a radioisotope production facility), and the 90”

0168-583X/91/$03.50 8 1991 - Elsevier Science Publishers B.V. (North-Holland) I. MICROPROBE TECHNOLOGY

M.L. Roberts et al, / Distributed computer control

HE photon spectrometer

I r Scattering f ziz:Bne,” profi’ing

AMS source I-

Tritium source

> 3 Direct -I ’

I n extraction

LAMS or sputtering

source spectrometer

Ion -I L Gamma L Beta microprobe

RIB spectrometer spectrometer

spectrometer Fig. 1. Floor plan of the tandem laboratory. Components shown in black are operational at present; those in gray will be installed

this year.

magnets of the AMS spectrometer were reworked from bending magnets at Stanford’s High Energy Physics Laboratory. The majority of beamline components such as quadrupole and trim magnets and minutae of the vacuum system were recycled from the LLNL cyclograaff that was decommissioned in 1986. Magnet power supplies come from the mothballed MFTF-B facility at LLNL. A major task of the design was the coupling and integration of pre-existing components that were not optimally selected to be compatible.

Construction of the Laboratory building began in June 1986, with the structure erected around the pres- sure vessel by September 1986. LLNL staff had access to the building for system installation beginning in February 1987. The first beam was achieved in July 1987 with research operation of the ion microprobe (first beamline fielded) beginning in September 1987. Since that time, the laboratory has been in a simulta- neous construction and research mode. Beamlines have been added and integrated at average intervals of one every four or five months. A major computer upgrade and wiring reconfiguration has been performed since initial operation as well.

3. Control system

3.1. Design intent

The intended mode of operation for the control system was to hold accelerator supplies precisely at

selected states, monitoring them while having the capa- bility to accept requests for changes in that state from an operator or data acquisition system. To the maxi- mum extent possible, the system is designed to be passive: all control loops required for speed or safety (i.e., HV terminal stabilization, interlocks, radiation safety systems) are implemented outside the control system in hardware. The computers may adjust set- points as desired, but are not involved actively in the control loop. Control of the large magnets by feedback from Hall probes is the one exception to this philoso- phy. The overall system speed obtained is lo-15 cycles/s. Diagnostic devices producing high-frequency information (scanner and Faraday cup readouts) are kept outside of the control system; however, the oper- ation of such diagnostics is through the control system.

3.2. System structure and computers

Accelerator components are controlled through a two-tiered hierarchical system of computers connected through an Ethernet Local Area Network. Local com- puters are connected to individual power supplies and sensors through CAMAC instrumentation. All operator interaction is through a separate supervisory computer that provides operator displays and is the programming resource for the entire system. A separate and indepen- dent computer monitors radiation detectors and issues a permissive to the main computer for operation. Com-

M.L. Roberts et al. / Distributed computer control

HV Generator

- Beamlines

Fig. 2. Block diagram of the distributed control system. The fourth local computer will be added this year.

3

mercially available analog-to-digital converters (ADCs), digital-to-analog converters (DACs), and digital input and output registers comprise the bulk of the instru- mentation used. Analog measurements are made at either 12 bit or 16 bit precision as appropriate. All DAC channels are 16 bits. At present, there are approxi- mately 120 analog input channels, 80 analog output channels, and 500 digital input and output channels in the system. A schematic of the system is shown in fig. 2.

Originally, all computers were Hewlett-Packard 9000 series, Model 310. The original control software was written in BASIC. Approximately one year after initial operation, a computer and software upgrade was per- formed. Local computers are now HP 9000 series Model 319s, while the supervisory computer is a Model 350. The control system software now used is Thaumaturgic Automated Control Logic [3] (TACL) developed and utilized at the Continuous Electron Beam Accelerator Facility (CEBAF). LLNL is a co-developer of the soft- ware. Other users include Yale University, MIT and Brookhaven National Laboratory.

Notably absent from this hierarchy is any identified data acquisition computer. Each beamline or experi- ment provides its own data acquisition equipment. In- teraction with the accelerator control system may be through LAN, CAMAC, or not at all. Examples will be given for the ion microprobe, AMS, and hydrogen depth profiling lines.

3.3. Geographic organization

The geographic organization of the overall system was selected as an aid to rapid installation and debug-

ging. Individual racks were built up containing a local computer, CAMAC crate and cross connect blocks for wiring signal and control lines. These racks served as physical supports for beamline components. The com- puter coding and interfaces were debugged and in place before mechanical components were installed. Each lo- cal computer is capable of stand-alone operation of components through a keyboard and tabular parameter display. No human engineering of displays was pro- vided at this level as the intended usage of this capabil-

ity was just initial turn-on and maintenance. The geographic groupings are as follows: computer

1, ion sources; computer 2, low-energy transport and selected ion source supplies; computer 3, HV generator and HE transport and spectrometers. A fourth local computer will be added to control the second switching magnet and the beamlines connected to it.

4. Software system

The Thaumaturgic Automated Control Logic (TACL) software is capable of supporting multiple su- pervisory computers with a multitude of local com- puters. CEBAF has seven supervisory computers and more than forty local computers. In our configuration, TACL operates one supervisory and three local com- puters under HP-UX (UNIX) in a disc-less mode, com- mon file system environment. All programs and files are available to each local computer and to the supervisory computer. The operator interface is through a high- resolution 19 in. color monitor with mouse, keyboard, and nine-knob box. The knob box can have its knobs

I. MICROPROBE TECHNOLOGY

4 M. L Roberts et al. / distributed computer control

assigned and reassigned to any analog parameter in the system, providing a capability for continuous adjust- ment of those parameters. Once a specific accelerator tune is established, parameters can be saved and re- called at a later time. The mouse is the main method of selecting functions and parameters. Through mouse selection, for example, one can toggle certain devices or select an analog parameter to be entered through the keyboard.

Important features of the TACL software are the logic and display graphics editors. The logic editor allows the user to manipulate, in graphical schematic form, control function primitives to assemble complex control algorithms. All gains, functions and external connections are defined through the logic editor. Ad- ditional custom user-defined primitives and processes can be readily implemented with minimal difficulty,

enhancing the overall utility of the system. Since the algorithms are defined by the end user, no additional software specialist is required. Upon completion of the algorithm definitions, the editor compiles the algorithms and prepares files for execution.

The display editor is utilized to prepare operator control displays. The user generates and manipulates symbols and parameter fields to develop useful display configurations for the operator. Display parameters are automatically linked at run time to the operating al- gorithms through variable name references. There is no logical limit to the number of displays that can be developed. For example, each experimental beamline configuration at the laboratory has a unique display defined for it. The display for the AMS spectrometer is shown in fig. 3. For each parameter, requested and measured values are shown in physical units. An error

Fig. 3. Operator control screen for the tandem when running in the AMS mode. Both requested and measured values for optical components are displayed. The pink overlay on the quadrupole triplet in the AMS spectrometer indicates that it is operating out of

tolerance.

hf. L. Roberts et al. / Distributed computer control 5

band can be defined for each measured value and a color change of the display can be used to indicate

elements out of tolerance. At run time, each local computer executes two pro-

grams, Logic and Llan. Logic executes algorithms com- piled by the logic editor. Llan is used to communicate over the LAN. A third program, Run, is executed by the supervisory computer to provide operator interface using the control panels defined with the display editor. In the local computers, where there is no operator present, executing Run is unnecessary.

An additional advantage of TACL lies in its expan- sion capabilities. Expansion is truly an add-on process. When the second switching magnet and associated ex- perimental beamlines are added this year, an additional local computer and extra CAMAC resources will be assigned to the control of these new devices. Logic for the new local computer and additional unique beamline control panels will be defined without modification of existing control definitions, and algorithms. Because of the graphics editors within TACL, these additional con- trol algorithms and display functions can be imple- mented in less than a week, and can be completed prior to the installation of the actual hardware.

5. Radiation safety system

A separate HP 310 computer monitors an array of 15 radiation sensors (ten for photons, five for neutrons) spaced at suitable locations throughout the building. A file on hard disk maintains a continuous record of hourly integrated doses. The system is set to trip on both instantaneous rate of radiation and integrated dose in the previous hour. Trip points for individual detec- tors are adjusted depending on the level of access con- trol defined: fully open, restricted to source end, or wholly excluded from facility. The latter mode was defined to allow operation over a link from a neighbor- ing building in hypothetical high-radiation cases. In this mode, calculated radiation levels at the perimeter fence are compared to preset trip levels and the permissive to operate the machine is removed if the allowed radiation levels are exceeded.

The permissive signal for operation is a hardwired 24 V signal that the supervisory computer monitors but does not control. During the detector pollings, a se- quence that occurs at 6 s intervals, the computer sends a signal to a watchdog timer. Should the signal fail to arrive within 30 s, indicating computer lockup or other system failure, the timer terminates the radiological computer power and the operational permissive for the accelerator. The system is backed up by three hardwired monitors set to trip at 100 mrem/h. Manual system reset is necessary after either system trip.

6. Beamfine examples

6.1. Ion microprobe

This beamline is used for materials imaging down to micron scales by proton energy loss tomography, PIXE and other nuclear reaction methods [4]. The ion beam is focussed down a 12 m beamline to a Faraday cup controlled by the ion microprobe system. The cup, subsequent slit set, and the focal and steering elements of the microprobe itself are controlled by the experi-

menter from a separate data acquisition and control system. There is no connection between accelerator control system and the microprobe data acquisition system. The task of the tandem is to deliver a stable

beam to the probe entrance aperture. Changes in en- ergy, beam species, etc., are accomplished by communi- cation between humans. Beam spots as small as 2 urn in diameter have been achieved to date with the accelera- tor not being the principal limiting factor. Recall of a previously determined parameter file and re-establish-

ment of beam at the microprobe entrance cup has been accomplished in as little as 45 s. Customarily, another 15-20 min of fine tuningis necessary to obtain satisfac- tory spot size and stability.

6.2. Hydrogen isotope profiling

This beamline was built to perform measurements of hydrogen depth profiles in silicate candidate materials for nuclear waste cladding. The accelerator provides a 19F beam for depth profiling via the i9F(p, cuy)160

resonance technique. The beamline contains a reaction chamber that holds up to 24 unknown and standard samples. Control of the beamline is by a stand-alone HP 310 computer that executes an automatic excitation function sequence to produce a depth profile. After completion of each individual data point, the data acquisition computer requests a new energy from the tandem by inserting a word into memory of a CAMAC module. The tandem control system reads the new requested energy, slews switching magnet and terminal voltage appropriately, and scales high-energy transport elements as required.

6.3. Accelerator mass spectrometry

Design of the AMS system anticipated that this analytical tool might profitably be automated if poten- tial applications in the biomedical and climatological areas developed as anticipated. In this measurement mode, the tandem functions as a mass spectrometer (sufficiently stable and precise to measure isotope ratios to one part in 1014-i6 ) while multiple unknowns, calibration, and blank standards are cycled through an ion source. As implemented in this spectrometer, rare

I. MICROPROBE TECHNOLOGY

6 ML Roberts et al. / Distributed computer control

and abundant isotopes are sequentially injected into the accelerator by rapidly switching the voltage on an elec- trostatically isolated vacuum tank in the inflection mag- net. In this beamline, a data acquisition computer, an

HP 350, performs both the data acquisition and storage function and provides timing information for the iso-

tope switching system. The sputtering ion source used has a 60-sample cassette and sample insertion system that was originally processor-controlled. Low reliability of both electronic and mechanical components of the sample changing and insertion system led us to rebuild it completely. We switched to all air-operated compo- nents at high voltage. When the spectrometer achieves adequate precision and accuracy in operation, the source will be put under control of the data acquisition com-

puter. We anticipate adding computer readable diagnos- tics to this spectrometer in the future. The goal of the AMS activity is to progress to unattended operation for

routine measurements.

7. Operational experience

Overall, the control system and accelerator operation

with it have satisfied our expectations. Software bugs have occasionally caused massive system slowdowns involving memory allocation, requiring a 5-10 min sys-

tem restart to re-establish acceptable response for the operator. The desired precision of control and stability of elements can be achieved if care is taken in installa- tion, calibration and initial checkout. Specific problems have occurred, as expected, in a large distributed system

of low-level electronics in the high-noise environment of a megavolt system that occasionally arcs over and that contains high-current SCR switched supplies. Most vulnerable elements to date have been the DACs that control power supplies. Addition of isolation amplifiers as needed has solved these problems. Recurrence of such failures is anticipated as we raise the accelerator voltage from the present maximum of 5.5 MV to 10 MV with the change from N,-CO, to SF, insulating gas.

Acknowledgements

Successful implementation and activation of this sys- tem have required the efforts of many people. Tom Zimmerman has contributed greatly through circuit de- sign, installation and implementation and program- ming. Steve Homann has contributed system design, hardware selection and verification and programming

for the radiological system. Ivan Proctor, Dale Heik- kinen and John Southon have borne the brunt of devel- oping the accelerator tunes and of being the test rats for the control system software. The help of all is gratefully acknowledged.

References

[l] J.C. Davis, Nucl. Instr. and Meth. B40/41 (1989) 705. [2] A.E. Pontau, Nucl. Instr. and Meth. B40/41 (1989) 646.

[3] T.L. Moore, Nucl. Instr. and Meth. B40/41 (1989) 984.

[4] A.E. Pontau et al., these Proceedings (2nd Int. Conf. on Nuclear Microprobe Technology and Applications,

Melbourne, Australia, 1990) Nucl. Instr. and Meth. B54

(1991) 383.