What is SCADA?Axel Daneels, Wayne Salter , IT/CO

SCADA: Acronym for supervisory control and data acquisition, a computer system for gathering and analyzing real time data. SCADA systems are used to monitor and control a plant or equipment in industries such as telecommunications, water and waste control, energy, oil and gas refining and transportation. A SCADA system gathers information, such as where a leak on a pipeline has occurred, transfers the information back to a central site, alerting the home station that the leak has occurred, carrying out necessary analysis and control, such as determining if the leak is critical, and displaying the information in a logical and organized fashion. SCADA systems can be relatively simple, such as one that monitors environmental conditions of a small office building, or incredibly complex, such as a system that monitors all the activity in a nuclear power plant or the activity of a municipal water system. SCADA systems were first used in the 1960s. real time

Last modified: Wednesday, August 13, 2003 Occurring immediately. The term is used to describe a number of different computer features. For example, real-time operating systems are systems that respond to input immediately. They are used for such tasks as navigation, in which the computer must react to a steady flow of new information without interruption. Most general-purpose operating systems are not real-time because they can take a few seconds, or even minutes, to react. Real time can also refer to events simulated by a computer at the same speed that they would occur in real life. In graphics animation, for example, a real-time program would display objects moving across the screen at the same speed that they would actually move. computer

Last modified: Friday, January 04, 2002 

A programmable machine. The two principal characteristics of a computer are: It responds to a specific set of instructions in a well-defined manner.

It can execute a prerecorded list of instructions (a program). Modern computers are electronic and digital. The actual machinery -- wires, transistors, and circuits -- is called hardware; the instructions and data are called software. All general-purpose computers require the following hardware components:

memory : Enables a computer to store, at least temporarily, data and programs. mass storage device : Allows a computer to permanently retain large amounts of data. Common mass storage devices include disk drives and tape drives. input device : Usually a keyboard and mouse, the input device is the conduit through which data and instructions enter a computer. output device : A display screen, printer, or other device that lets you see what the computer has accomplished. central processing unit (CPU): The heart of the computer, this is the component that actually executes instructions.

In addition to these components, many others make it possible for the basic components to work together efficiently. For example, every computer requires a bus that transmits data from one part of the computer to another. Computers can be generally classified by size and power as follows, though there is considerable overlap:

personal computer : A small, single-user computer based on a microprocessor. In addition to the microprocessor, a personal computer has a keyboard for entering data, a monitor for displaying information, and a storage device for saving data. workstation : A powerful, single-user computer. A workstation is like a personal computer, but it has a more powerful microprocessor and a higher-quality monitor. minicomputer : A multi-user computer capable of supporting from 10 to hundreds of users simultaneously. mainframe : A powerful multi-user computer capable of supporting many hundreds or thousands of users simultaneously. supercomputer : An extremely fast computer that can perform hundreds of millions of instructions per second.

operating system Last modified: Friday, January 04, 2002 

The most important program that runs on a computer. Every general-purpose computer must have an operating system to run other programs. Operating systems perform basic tasks, such as recognizing input from the keyboard, sending output to the display screen, keeping track of files and directories on the disk, and controlling peripheral devices such as disk drives and printers. For large systems, the operating system has even greater responsibilities and powers. It is like a traffic cop -- it makes sure that different programs and users running at the same time do not interfere with each other. The operating system is also responsible for security, ensuring that unauthorized users do not access the system. Operating systems can be classified as follows:

multi-user : Allows two or more users to run programs at the same time. Some operating systems permit hundreds or even thousands of concurrent users. multiprocessing : Supports running a program on more than one CPU. multitasking : Allows more than one program to run concurrently. multithreading : Allows different parts of a single program to run concurrently. real time: Responds to input instantly. General-purpose operating systems, such as DOS and UNIX, are not real-time.

Operating systems provide a software platform on top of which other programs, called application programs, can run. The application programs must be written to run on top of a particular operating system. Your choice of operating system, therefore, determines to a great extent the applications you can run. For PCs, the most popular operating systems are DOS, OS/2, and Windows, but others are available, such as Linux.

As a user, you normally interact with the operating system through a set of commands. For example, the DOS operating system contains commands such as COPY and RENAME for copying files and changing the names of files, respectively. The commands are accepted and executed by a part of the operating system called the command processor or command line interpreter. Graphical user interfaces allow you to enter commands by pointing and clicking at objects that appear on the screen. 1. IntroductionOn 20 Sept. 2000, the Finance Committee approved the proposal to negotiate a contract with ETM A.G. (Eisenstadt, Austria) for the supply of PVSS - ETM's SCADA - for developing the control systems of ALICE, ATLAS, CMS and LHCb. In addition the SCADA Working Group, that was set up by the CERN Controls Board, recommends PVSS as one of the SCADA products for the development of future control systems at CERN.These decisions are the accomplishment of around thirteen person-years (FTE) of effort - spanning over more than three years - to identify and evaluate a proper industrial control system that copes with the extreme requirements of high energy particle physics experiments such as those of the LHC.Widely used in industry for Supervisory Control and Data Acquisition of industrial processes, SCADA systems are now also penetrating the experimental physics laboratories for the controls of ancillary systems such as cooling, ventilation, power distribution, etc. More recently they were also applied for the controls of smaller size particle detectors such as the L3 muon detector and the NA48 experiment, to name just two examples at CERN.SCADA systems have made substantial progress over the recent years in terms of functionality, scalability, performance and openness such that they are an alternative to in house development even for very demanding and complex control systems as those of physics experiments.2. What does SCADA MEAN?SCADA stands for Supervisory Control And Data Acquisition. As the name indicates, it is not a full control system, but rather focuses on the supervisory level. As such, it is a purely software package that is positioned on top of hardware to which it is interfaced, in general via Programmable Logic Controllers (PLCs), or other commercial hardware modules.SCADA systems are used not only in industrial processes: e.g. steel making, power generation (conventional and nuclear) and distribution, chemistry, but also in some experimental facilities such as nuclear fusion. The size of such plants range from a few 1000 to several 10 thousands input/output (I/O) channels. However, SCADA systems evolve rapidly and are now penetrating the market of plants with a number of I/O channels of several 100 K: we know of two cases of near to 1 M I/O channels currently under development.SCADA systems used to run on DOS, VMS and UNIX; in recent years all SCADA vendors have moved to NT and some also to Linux.3. ArchitectureThis section describes the common features of the SCADA products that have been evaluated at CERN in view of their possible application to the control systems of the LHC detectors [1], [2].3.1 Hardware Architecture One distinguishes two basic layers in a SCADA system: the "client layer" which caters for the man machine interaction and the "data server layer" which handles most of the process data control activities. The data servers communicate with devices in the field through process controllers. Process controllers, e.g. PLCs, are connected to the data servers either directly or via networks or fieldbuses that are proprietary (e.g. Siemens H1), or non-proprietary (e.g. Profibus). Data servers are connected to each other and to client stations via an Ethernet LAN. The data servers and client stations are NT platforms but for many products the client stations may also be W95 machines. Fig.1. shows typical hardware architecture.

Figure 1: Typical Hardware Architecture 3.2 Software Architecture The products are multi-tasking and are based upon a real-time database (RTDB) located in one or more servers. Servers are responsible for data acquisition and handling (e.g. polling controllers, alarm checking, calculations, logging and archiving) on a set of parameters, typically those they are connected to.

Figure 2: Generic Software Architecture

However, it is possible to have dedicated servers for particular tasks, e.g. historian, datalogger, alarm handler. Fig. 2 shows a SCADA architecture that is generic for the products that were evaluated. 3.3 Communications Internal CommunicationServer-client and server-server communication is in general on a publish-subscribe and event-driven basis and uses a TCP/IP protocol, i.e., a client application subscribes to a parameter which is owned by a particular server application and only changes to that parameter are then communicated to the client application.Access to DevicesThe data servers poll the controllers at a user defined polling rate. The polling rate may be different for different parameters. The controllers pass the requested parameters to the data servers. Time stamping of the process parameters is typically performed in the controllers and this time-stamp is taken over by the data server. If the controller and communication protocol used support unsolicited data transfer then the products will support this too.The products provide communication drivers for most of the common PLCs and widely used field-buses, e.g., Modbus. Of the three fieldbuses that are recommended at CERN, both Profibus and Worldfip are supported but CANbus often not [3]. Some of the drivers are based on third party products (e.g., Applicom cards) and therefore have additional cost associated with them. VME on the other hand is generally not supported.A single data server can support multiple communications protocols: it can generally support as many such protocols as it has slots for interface cards.The effort required to develop new drivers is typically in the range of 2-6 weeks depending on the complexity and similarity with existing drivers, and a driver development toolkit is provided for this.3.4 Interfacing Application Interfaces / Openness The provision of OPC client functionality for SCADA to access devices in an open and standard manner is developing. There still seems to be a lack of devices/controllers, which provide OPC server software, but this improves rapidly as most of the producers of controllers are actively involved in the development of this standard. OPC has been evaluated by the CERN-IT-CO group [4].The products also provide

an Open Data Base Connectivity (ODBC) interface to the data in the archive/logs, but not to the configuration database,

an ASCII import/export facility for configuration data, a library of APIs supporting C, C++, and Visual Basic (VB) to access data in the RTDB, logs and

archive. The API often does not provide access to the product's internal features such as alarm handling, reporting, trending, etc.

The PC products provide support for the Microsoft standards such as Dynamic Data Exchange (DDE) which allows e.g. to visualise data dynamically in an EXCEL spreadsheet, Dynamic Link Library (DLL) and Object Linking and Embedding (OLE).Database The configuration data are stored in a database that is logically centralised but physically distributed and that is generally of a proprietary format.For performance reasons, the RTDB resides in the memory of the servers and is also of proprietary format.The archive and logging format is usually also proprietary for performance reasons, but some products do support logging to a Relational Data Base Management System (RDBMS) at a slower rate either directly or via an ODBC interface.3.5 Scalability Scalability is understood as the possibility to extend the SCADA based control system by adding more process variables, more specialised servers (e.g. for alarm handling) or more clients. The products achieve scalability by having multiple data servers connected to multiple controllers. Each data server has its own configuration database and RTDB and is responsible for the handling of a sub-set of the process variables (acquisition, alarm handling, archiving). 3.6 Redundancy The products often have built in software redundancy at a server level, which is normally transparent to the user. Many of the products also provide more complete redundancy solutions if required.4. Functionality4.1 Access Control Users are allocated to groups, which have defined read/write access privileges to the process parameters in the system and often also to specific product functionality.4.2 MMI The products support multiple screens, which can contain combinations of synoptic diagrams and text.

They also support the concept of a "generic" graphical object with links to process variables. These objects can be "dragged and dropped" from a library and included into a synoptic diagram.Most of the SCADA products that were evaluated decompose the process in "atomic" parameters (e.g. a power supply current, its maximum value, its on/off status, etc.) to which a Tag-name is associated. The Tag-names used to link graphical objects to devices can be edited as required. The products include a library of standard graphical symbols, many of which would however not be applicable to the type of applications encountered in the experimental physics community.Standard windows editing facilities are provided: zooming, re-sizing, scrolling... On-line configuration and customisation of the MMI is possible for users with the appropriate privileges. Links can be created between display pages to navigate from one view to another. 4.3 Trending The products all provide trending facilities and one can summarise the common capabilities as follows:

the parameters to be trended in a specific chart can be predefined or defined on-line a chart may contain more than 8 trended parameters or pens and an unlimited number of charts can be

displayed (restricted only by the readability) real-time and historical trending are possible, although generally not in the same chart historical trending is possible for any archived parameter zooming and scrolling functions are provided parameter values at the cursor position can be displayed

The trending feature is either provided as a separate module or as a graphical object (ActiveX), which can then be embedded into a synoptic display. XY and other statistical analysis plots are generally not provided.4.4 Alarm Handling Alarm handling is based on limit and status checking and performed in the data servers. More complicated expressions (using arithmetic or logical expressions) can be developed by creating derived parameters on which status or limit checking is then performed. The alarms are logically handled centrally, i.e., the information only exists in one place and all users see the same status (e.g., the acknowledgement), and multiple alarm priority levels (in general many more than 3 such levels) are supported.It is generally possible to group alarms and to handle these as an entity (typically filtering on group or acknowledgement of all alarms in a group). Furthermore, it is possible to suppress alarms either individually or as a complete group. The filtering of alarms seen on the alarm page or when viewing the alarm log is also possible at least on priority, time and group. However, relationships between alarms cannot generally be defined in a straightforward manner. E-mails can be generated or predefined actions automatically executed in response to alarm conditions.4.5 Logging/Archiving The terms logging and archiving are often used to describe the same facility. However, logging can be thought of as medium-term storage of data on disk, whereas archiving is long-term storage of data either on disk or on another permanent storage medium. Logging is typically performed on a cyclic basis, i.e., once a certain file size, time period or number of points is reached the data is overwritten. Logging of data can be performed at a set frequency, or only initiated if the value changes or when a specific predefined event occurs. Logged data can be transferred to an archive once the log is full. The logged data is time-stamped and can be filtered when viewed by a user. The logging of user actions is in general performed together with either a user ID or station ID. There is often also a VCR facility to play back archived data. 4.6 Report Generation One can produce reports using SQL type queries to the archive, RTDB or logs. Although it is sometimes possible to embed EXCEL charts in the report, a "cut and paste" capability is in general not provided. Facilities exist to be able to automatically generate, print and archive reports.4.7 Automation The majority of the products allow actions to be automatically triggered by events. A scripting language provided by the SCADA products allows these actions to be defined. In general, one can load a particular display, send an Email, run a user defined application or script and write to the RTDB.The concept of recipes is supported, whereby a particular system configuration can be saved to a file and then re-loaded at a later date.Sequencing is also supported whereby, as the name indicates, it is possible to execute a more complex sequence of actions on one or more devices. Sequences may also react to external events.Some of the products do support an expert system but none has the concept of a Finite State Machine (FSM).5. Application Development5.1 Configuration The development of the applications is typically done in two stages. First the process parameters and associated information (e.g. relating to alarm conditions) are defined through some sort of parameter definition template and then the graphics, including trending and alarm displays are developed, and linked where appropriate to the

process parameters. The products also provide an ASCII Export/Import facility for the configuration data (parameter definitions), which enables large numbers of parameters to be configured in a more efficient manner using an external editor such as Excel and then importing the data into the configuration database.However, many of the PC tools now have a Windows Explorer type development studio. The developer then works with a number of folders, which each contains a different aspect of the configuration, including the graphics.The facilities provided by the products for configuring very large numbers of parameters are not very strong. However, this has not really been an issue so far for most of the products to-date, as large applications are typically about 50K I/O points and database population from within an ASCII editor such as Excel is still a workable option.On-line modifications to the configuration database and the graphics is generally possible with the appropriate level of privileges. 5.2 Development Tools The following development tools are provided as standard:

a graphics editor, with standard drawing facilities including freehand, lines, squares circles, etc. It is possible to import pictures in many formats as well as using predefined symbols including e.g. trending charts, etc. A library of generic symbols is provided that can be linked dynamically to variables and animated as they change. It is also possible to create links between views so as to ease navigation at run-time.

a data base configuration tool (usually through parameter templates). It is in general possible to export data in ASCII files so as to be edited through an ASCII editor or Excel.

a scripting language an Application Program Interface (API) supporting C, C++, VB a Driver Development Toolkit to develop drivers for hardware that is not supported by the SCADA

product. 5.3 Object Handling The products in general have the concept of graphical object classes, which support inheritance. In addition, some of the products have the concept of an object within the configuration database. In general the products do not handle objects, but rather handle individual parameters, e.g., alarms are defined for parameters, logging is performed on parameters, and control actions are performed on parameters. The support of objects is therefore fairly superficial.6. EvolutionSCADA vendors release one major version and one to two additional minor versions once per year. These products evolve thus very rapidly so as to take advantage of new market opportunities, to meet new requirements of their customers and to take advantage of new technologies.As was already mentioned, most of the SCADA products that were evaluated decompose the process in "atomic" parameters to which a Tag-name is associated. This is impractical in the case of very large processes when very large sets of Tags need to be configured. As the industrial applications are increasing in size, new SCADA versions are now being designed to handle devices and even entire systems as full entities (classes) that encapsulate all their specific attributes and functionality. In addition, they will also support multi-team development.As far as new technologies are concerned, the SCADA products are now adopting:

Web technology, ActiveX, Java, etc. OPC as a means for communicating internally between the client and server modules. It should thus be

possible to connect OPC compliant third party modules to that SCADA product. 7. EngineeringWhilst one should rightly anticipate significant development and maintenance savings by adopting a SCADA product for the implementation of a control system, it does not mean a "no effort" operation. The need for proper engineering can not be sufficiently emphasised to reduce development effort and to reach a system that complies with the requirements, that is economical in development and maintenance and that is reliable and robust. Examples of engineering activities specific to the use of a SCADA system are the definition of:

a library of objects (PLC, device, subsystem) complete with standard object behaviour (script, sequences, ...), graphical interface and associated scripts for animation,

templates for different types of "panels", e.g. alarms, instructions on how to control e.g. a device ..., a mechanism to prevent conflicting controls (if not provided with the SCADA), alarm levels, behaviour to be adopted in case of specific alarms, ...

8. Potential benefits of SCADAThe benefits one can expect from adopting a SCADA system for the control of experimental physics facilities can be summarised as follows:

a rich functionality and extensive development facilities. The amount of effort invested in SCADA product amounts to 50 to 100 p-years!

the amount of specific development that needs to be performed by the end-user is limited, especially with suitable engineering.

reliability and robustness. These systems are used for mission critical industrial processes where reliability and performance are paramount. In addition, specific development is performed within a well-established framework that enhances reliability and robustness.

technical support and maintenance by the vendor. For large collaborations, as for the CERN LHC experiments, using a SCADA system for their controls ensures a common framework not only for the development of the specific applications but also for operating the detectors. Operators experience the same "look and feel" whatever part of the experiment they control. However, this aspect also depends to a significant extent on proper engineering.REFERENCES

Note: this article is based on a very similar one that has been published in the Proceedings of the 7th International Conference on Accelerator and Large Experimental Physics Control Systems, held in Trieste, Italy, 4 - 8 Oct. 1999. [1] A.Daneels, W.Salter, "Technology Survey Summary of Study Report", IT-CO/98-08-09, CERN, Geneva 26th Aug 1998.[2] A.Daneels, W.Salter, "Selection and Evaluation of Commercial SCADA Systems for the Controls of the CERN LHC Experiments", Proceedings of the 1999 International Conference on Accelerator and Large Experimental Physics Control Systems, Trieste, 1999, p.353.[3] G.Baribaud et al., "Recommendations for the Use of Fieldbuses at CERN in the LHC Era", Proceedings of the 1997 International Conference on Accelerator and Large Experimental Physics Control Systems, Beijing, 1997, p.285.[4] R.Barillere et al., "Results of the OPC Evaluation done within the JCOP for the Control of the LHC Experiments", Proceedings of the 1999 International Conference on Accelerator and Large Experimental Physics Control Systems, Trieste, 1999, p.511.

How to choose SCADA equipmentby Janice Hungerford and Danetta YorkThere are several questions that need to be answered before a final decision is made on the actual components of a new Supervisory Control and Data Aquisition system.1. How versatile is it? For economic reasons, it is usually better to keep as much of the current equipment in the existing system as is realistically possible. The committee set up to design a system must then ask themselves what kind of connectivity a vendor has with the existing equipment. If connectivity is impossible or difficult then the question of customizing comes into play. If a system is customized, will that make future changes, upgrades, and maintenance even more costly?2. Is the vendor a recognized company? For most companies, the smart move is to look at nationally known, proven vendors of Supervisory Control and Data Aquisition equipment. It is important that they have a history of success stories behind them, yet also have local distribution and technical support. Once a vendor is chosen, listen to their experts on recommendations for customizing the system.3. Does it meet the requirements? Of course, a system can have all the buttons and toys any technical person could want, but if it doesn’t meet the needs that the evaluation committee has formulated, it isn’t worth having. Each department on the committee will be more concerned with one function over another. It is important that they rate, on a scale of one to 10, the importance of each feature or need so that an acceptable compromise can be reached.Available technologyReliable Supervisory Control and Data Aquisition systems are not only used for operations, but for measurement, forecasting, billing, historical analysis, and planning. Today’s marketplace is seeing a continued rise in technological innovations. The systems required today must meet a whole new level of automation, such as interface to equipment and a multitude of tools that were not available even five years ago. Along with that we are seeing hardware prices dropping, standardization of software, a narrowing supplier range, and the complexity of Supervisory Control and Data Aquisition systems skyrocketing.Whether you are designing a Supervisory Control and Data Aquisition for the first time or upgrading an existing system, it is important to review the system components before you decide on equipment. You will need to evaluate the following:1. The telemetry network. A telemetry network provides the communication pathway in a Supervisory Control and Data Aquisition system. It is made up of several components: topology, either point-to-point, point-to-multipoint, or multipoint-to-multipoint; transmission modes, the way information is sent and received, (network

topology determines your mode of data transmission); and link media — hard wire, fiber optics, or radio (micro wave).In order to determine which type is best for your system, ask the following questions:A. What are the data transmission needs of the application?B. Where will the remote sites and control center be located?C. What is the distance between sites?D. What link media services are available in these areas?E. What do you want to spend?After answering these questions, your system integrator can help you make the appropriate choice for your link media.Check protocol. In order for the host, or master, and the remote terminal units to communicate with each other there must be a common method of encoding and decoding the messages between them. This is referred to as the protocol. In order to choose the one best suited to your application, consider the following:A. Avoid proprietary protocols. A closed protocol leaves the end-user with fewer options for integrating equipment from various vendors.B. Select equipment that supports well-behaved, open protocols, that are well-documented and supported by many vendors, such as Modbus.C. Do you need to connect to existing equipment?Many Supervisory Control and Data Aquisition systems require the Modbus protocol, yet designers may, for various reasons, choose to install Allen-Bradley units which do not support Modbus. That is where third-party protocol suppliers may have the answer to this common problem. There are a number of third-party protocol suppliers available that allow existing protocols to communicate with equipment made by different manufacturers. For instance, ProSoft Technology, Inc., specializes in communication solutions for Rockwell Automation/Allen-Bradley. ProSoft has recently announced a new line of products called Multi Vendor Interface solutions. These product offerings will provide off-the-shelf base platforms with software tools to allow customized solutions for serial communication across Rockwell Automation’s PLC, SLC, ControlLogix, and FLEX platforms. With these modules, developing a custom serial communication application will mean an easy to use, well-supported, and full-feature development environment across all Rockwell Automation platforms.2. Data communication equipment. Whatever telemetry network you have chosen will determine the appropriate data communication equipment you will need. The equipment is simply the link between a transmission medium and the data terminal equipment, or it can be viewed as the data transport mechanism between the host computer system and the remote terminal unit. Data communication equipment can include telephone and radio modems, microwave, or satellite transmission equipment. No matter which communication method is chosen, Supervisory Control and Data Aquisition systems usually use a communication format called master-slave. This means that all conversations are initiated by the master station. The remote terminal unit, or slave, replies only when it receives a message. The master controls all conversations.An in-rack, or stand-alone, solution is usually more reliable than a PC-based system due to the greater reliability of PLC hardware vs. PCs.3. The master station. The master station gathers field data directly from the remote stations, or submaster, and provides monitoring and control over the entire system through its operator interface. There are several master station types available including:A. VAX or UNIX-based computer. Used in extremely large applications that may also require submaster stations to gather data, support local operator interface, support logging of alarms and events, communicate with remote stations, and interface with a larger master station.B. Personal computer. Used in small-to-medium-sized applications. There are many vendors of Supervisory Control and Data Aquisition software for the PC that allow acquisition of data, graphical interface, historical data storage, and alarming.C. Programmable controller. In order to determine whether a PLC should be used in your application, ask whether the master station needs to control local input/output, whether the application requires master station redundancy, and how many remote stations your application requires?4. Remote terminal units. An RTU is a microprocessor-based unit, specifically designed for real-time processing of input and output of data. remote terminal units also log alarms, report status to the master station, and carry out the commands from the master station. In order to choose the appropriate device for your application, ask the following questions:A. What protocol does your application require?B. Does it use analog input/output?C. How many input/output points does it require?D. Do you need the remote station to collect data without being told to by the master station?E. Do you need online programming, faster ladder logic speeds, and a built-in clock/calendar?

5. Data distribution. It is also important to evaluate the mechanisms available for data distribution in a Supervisory Control and Data Aquisition system.A. Who needs access to the information?B. How often?C. What network interfaces are possible with the system?D. Can the system function as a TCP/IP server to deliver archived data to other users as well as a client to deliver data to a server?Once you have gone through this check list and have the answers to these questions, you can begin your search for the appropriate vendor. Most will begin with the vendors they already use. But, don’t be afraid to do some homework on competitors’ products. Check out trade magazines, even if they aren’t in your particular industry. Keep in mind, for example, that a Supervisory Control and Data Aquisition system that has been installed in a water/waste water utility system may have some of the same needs and features that are needed for a gas pipeline.Check out the ads, read some articles, then make some phone calls to vendors who seem to fit the criteria you are looking for.6. Budget scope and bidding framework. At this point, the evaluation committee should have a firm grasp of the requirements and the technology available to put together an efficient Supervisory Control and Data Aquisition system. It is then important to communicate this understanding with the bidding companies involved. When preparing a formal bid package, instructions to bidders should specify, in detail, the functionality that is expected from the new system including: software specifications, hardware specifications, communication needs, performance standards, training, testing, and technical support.The bid package should be specific enough to allow bids to be evaluated on a level playing field. In other words, the committee should be able to compare apples to apples. It should also be remembered that the bids will also function as the framework for the final contract, so all project-specific requirements should be included to avoid problems during implementation. Open communication with all parties involved is the key to a successful bidding process.Buying Check listThere are some guiding principles to keep in mind when procuring a new Supervisory Control and Data Aquisition system:1. The new architecture must be based on open standards.2. It must have the flexibility to integrate new and existing systems.3. It must be reliable and supportable.4. Use non-proprietary equipment only.5. Use proven technologies when possible and be able to integrate new technologies when advisable.6. The ability to implement the new system without causing major disruptions to business operations.7. Selling the project to the various departments likely to be affected.8. Build communication and teamwork between the company and suppliers.Points to considerConsider the following communications-software questions before making a final decision:A. Can the system support multiple communications ports with the same field device protocol?B. Can it support different protocols?C. How many ports can be used for concurrent communications?D. Is the system capable of supporting concurrent interface to multiple types of communications media?E. Using the same communications port, can the system support multiple-vendor protocols to different types of equipment?F. Can the user fine tune communications through configuring command timeouts, retries, and polling frequencies at the command and device level?Once you have narrowed down your choice of data communication equipment, choose your master and remote station devices. Then, you may come back and finalize your transmission system.Measuring progressDuring the project implementation, it will be important to set measurable milestones for the company and the suppliers involved. These should be easy to evaluate through direct observation of the system in its various stages and should be written into the contract of all suppliers. These should include:1. Software design: all parties should have a good understanding of the project-critical requirements needed. Software demonstrations should be completed early in the process.2. Factory acceptance before system shipment.3. Site acceptance to ensure the system is ready for commercial operation.4. Final system acceptance.Once the new Supervisory Control and Data Aquisition system is up and running, it is important that a post-installation analysis be completed to identify project mistakes and the factors which contributed to the project’s

success. With the new technological advances becoming available each year, this analysis may be beneficial if and when another revision becomes necessary in the future.

Janice Hungerford and Danetta York represent ProSoft Technology, Inc. This article was adapted from a n ENTELEC paper and is the second in a three-part series.

Reprinted from Gas Utility Manager MagazineJuly 2001

SCADA HoneyNet Project: Building Honeypots for Industrial Networks Venkat Pothamsetty and Matthew Franz

Critical Infrastructure Assurance Group(CIAG) Cisco Systems, Inc.

Links Download Mailing List PLC Simulation Case Study Honeyd - a small daemon that creates virtual hosts on a network. The hosts can be configured to run arbitrary services, and their personality can be adapted so that they appear to be running certain operating systems News & Updates 6/01/04/(released version 0.2) - Fixed the bug regarding the absense of modbusHdrs.py, included sample nmap OS fingerprints of some PLCs, included a test file to generate custom Modbus packets to test the modbusSrve.py implementation 5/13/04 - Major cleanup of content 3/20/04 - PLC Simulation scripts available for down and PLC Simulation Case Study complete. Objectives The short-term goal of the project is to determine the feasibility of building a software-based framework to simulate a variety of industrial networks such as SCADA, DCS, and PLC architectures. We plan to document the requirements and release proof of concept code (in the form of honeyd scripts) so that a single Linux host can simulate multiple industrial devices and complex network topologies. Given the variety of deployments and the lack of standard, well-defined architectures for industrial networks, this project attempts to create the building blocks so that users can simulate their networks own networks--not make assumptions about what "real world" SCADA/DCS/PLC look like. Assuming deployment of "SCADA HoneyNets" ever reach critical mass, the longer term objective of the project is to gather information about general attack patterns and specific exploits that could be used to write signature for commercial and Open Source IDS products. Introduction There is still little information about SCADA vulnerabilities and attacks, despite the growing awareness of security issues in industrial networks. As is the case with IT security, owner-operators are often unwilling to release attack or incident data. However, unlike IT products and protocols, there are not the sort of public repositories of vendor advisories and vulnerabilities in industrial devices. Although some vulnerability research is being conducted in this area, very little has been released publically and no "SCADA security tools" (whatever that might mean) have been released to the public. To address these limitations, this goal of this project is to provide tools and to simulate a variety of industrial networks and devices. We see several uses for this project:

Build a HoneyNet for attackers, to gather data on attacker trends and tools Provide a scriptable industrial protocol simulators to test a real live protocol implementation Research countermeasures, such as device hardening, stack obfuscation, reducing application

information, and the effectiveness network access controls Feature Requirements Based on our knowledge of industrial network applications, products, and protocols, we identified the following requirements: Individual Device Simulation To simulate individual devices, the following functionality is needed:

Stack level: To simulate the TCP/IP stack of a Ethernet-based device device to a script kiddie type attacker who is scanning the network with OS detection tools such as Nmap and Xprobe.

Protocol level: To simulate industrial protocols for skilled attackers who have the tools which interrogate protocols and want to do something meaningful using the protocol features

Application level: To simulate various applications on a SCADA device such as web servers and management applications such as SNMP and Telnet.

Hardware level:Many of the SCADA devices use serial interfaces such as modems and RS232 interfaces for both SCADA protocol communication and for management purposes. An attacker who either "logs into" a SCADA device or has access to the serial network, needs to be presented with a serial device and/or a protocol communication over a serial device.

Simulate Network We need to simulate various entry points so that when an attacker encounters a perimeter device, he will be presented the same network as a real SCADA network at that particular network entry point Various network entry points that we need to simulate include:

1. A router directly connected to the Internet: Control system networks are typically not directly conne a control network is located inside a corporate network. Assuming the corporate network as Internet, we need to simulate the entry point of a router that seperates the control network and the corporate network. The devices that are normally connected to such routers would be Industrial Ethernet switches or industrial devices with an IP stack, such as some IP enabled PLCs and wireless access points.

2. Direct serial device:Some of the industrial devices have a modem that can be directly dialed into from a PSTN. We need to simulate a "modem server" that can take connections and behaves like a industrial device or is connected to a industrial device.

3. A Ethernet enabled industrial device directly connected to the Internet: Such a scenario should be the same as simulating the stack, the protocols and applications on that device and connecting that to Internet

4. An Ethernet serial gateway directly plugged into the Internet:An Ethernet serial gateway is a bridge between the IP network and the serial interface. The IP side of the device would be connected to the network, either a Industrial switch or a router to which other IP industrial devices are connected to. The serial side of the device would be connected to a serial device or a serial network.

5. Wireless: Wireless is one of the entry points into a Industrial network. Most of the Industrial wireless devices use proprietary wireless protocols and some of them use 802.1b standard. Typically the serial interface of the device would be connected to a wireless bridge.

6. Remote desktop access and HMIs:The Human Machine Interfaces and the software that communicates with Industrial devices usually run on a Windows machine. Administrators who want remote access to these devices would typically run a remote desktop viewer, such as VNC or PC anywhere. An attacker would normally find it through a port scan ' after he gets into the control network and might get to it using a VNC client. Simulating this would probably need a custom made VNC protocol simulation.

7. Remote Access Server (RAS):Another possible entry point into a control network is to dial into the network using PPP and use the PPP password to authenticate yourself to a Network Access Server and then directly access the Industrial device.

Capture the attacker tools and tracks Our scripts need to capture the attacker tools and tracks. That should include keystroke logging and facilities to capture the tools and binaries he might be up loading, if the attack. Our scripts also need to capture network traffic. Review of existing technologies and relavency Honeyd Honeyd has facilities for easy simulation of TCP/IP stacks and applications. Honeynet takes Nmap and Xprobe signatures through configuration files and sends packet responses to scans matching those signatures. Users can set up profiles, mapping IP addresses that Honeyd should respond to a corresponding device profile. When attackers Nmap or Xprobe scan the IP address which Honeyd is taking care of, he will be returned with packets matching the corresponding device profile. Therefore using Honeyd, it would be possible to simultaneously simulate stacks of multiple IP based Industrial devices, provided the corresponding scanning tools (Nmap or Xprobe) has the knowledge of the signature. As of now, there are no signatures of Industrial devices in Nmap's database. Honeyd allows the user to listen on a port and run a script on that particular port when anybody connects to that port. As of now, there are many scripts contributed to the project, which can simulate web pages, WSFTP servers and Cisco telnet servers. Using this feature on Honeyd, it is possible to write scripts that simulated various Industrial Ethernet protocols. For example, it would be possible to simulate a Modbus/TCP server on port 502 and EtherNet/IP on ports 44818/2222. Serial interface simulation Many industrial network devices use RS-232/485 for communication. Typically the serial port of a PC would be directly (or indirectly, via a serial Ethernet gateway) connected to the serial port of the device. There would be a software running on the PC, which sends commands to the device over the serial interface. By some accounts there are hundreds of serial protocols in use in SCADA networks. Some of the more common protocols are MODBUS and DNP. We need to simulate those protocols over the serial port, so as to present a protocol interface to an attacker who connects to the serial port. Many languages support serial interface programming including Python and Java. We

were able to achieve serial communication through a open source Python serial programming module (pyserial.sf.net). Simulating 802.11 The HostAP driver(http://hostap.epitest.fi/), replies for 802.1b management packets and converts a client adapter an access point. The driver can be used to simulate an access point which is inside a automation or a SCADA network Capturing attack tools and capturing the attackers' track Though not part of Honeyd, there are lots of keystroke loggers available. We need a mechanism to track the attacker on the web interface of the device. We do not know of any tools which can provide that functionality, however we explored some possibilities where the the Java applet (running on the "attackers" web browser) is able to comm Challenges Deployment and Testing An ideal deployment site for such a script would be a subnet close to a real Industrial/SCADA network or a phone number which belongs to a SCADA/Automation plant. We are not aware of any active and on-going SCADA specific attacks, it would be difficult to get a SCADA aware attacker into the honeypot. Send comments to scadahoneynet-talk@lists.sourceforge.net or ciag-tools@cisco.com

SCADA HoneyNet PLC Simulation Concepts, Design, and ImplementationVenkat Pothamsetty Cisco Systems, Inc. Critical Infrastructure Assurance Group (CIAG) BackgroundProgrammable Logic Controllers (PLCs) are common in some industrial applications (especially discrete manufacturing) and increasingly have network interfaces which support Ethernet and TCP/IP protocols as well as more traditional communication interfaces such as MODBUS, DeviceNet, ContrlNet, Foundation Fieldbus, etc. As is the case with any network device, different vendors implement their own shells on telnet and support various FTP commands, depending on their application requirements. The Ethernet communication module of the PLC typically runs an embedded operating system that includes standard network protocol as well as implementations of industrial network protocols such as Modbus/TCP or EtherNet/IP. For example, telnet and FTP servers are common and have identifying information which can be used to determine the vendor and version of software. Even on the industrial protocol side, we saw that not all PLCs support all commands of a given industrial protocol, so that implementations can be fingerprinted. Depending on the type (and capabilities) of the device there may be slight differences in the protocol. All of these characteristics make it possible for attackers to identify specific versions and vendors of device and allow us to be able simulate the devices as well. Implementation ApproachWe followed the following approach when simulating a PLC: Implement the generic features so that users can easily change them, add new features and re-submit them The feature implementation should be so that the code will serve as examples for the users to change it according to their own needs. Once the users submit enough code for more implementations, we visualize putting them into templates so that a generic configurable engine can load them according to a defined configuration Components Needed The following are the network components in a PLC that need to be simulated: The TCP/IP Stack of the PLC The simulation of the Modbus/TCP server implementation. The simulation of the FTP server, that is found on some PLCs. The simulation of the Telnetd server, that may be found on some PLCs. The simulation of the management HTTP server, which increasingly common on PLCs and other industrial network devices. Note that the scripts above can be used along with honeyd or standalone and need root permissions because they have to bind themselves to previleged ports below 1024. Simulating the TCP/IP Stack of the PLC Communication ModuleIn order for Honeyd to simulate the TCP/IP stack of a PLC, adding the TCP/IP signature of the device to the honeyd's nmap.prints file would be sufficient. But in oder for the attacker to feel that the stack is a PLC stack, the signature has to be in the database of the scanning tool that he is using. Though we tested that by putting the

signature in nmap-os-fingerprints file (which is Nmap's fingerprint database), we do not know of any scanner having the signature of any PLC at the time of writing this document. Simulation of the Modbus/TCP serverThe PLC can have multiple industrial protocol implementations which will listen at their corresponding ports for packets from the corresponding clients. We decided to simulate the Modbus/TCP server as a proof of concept because the protocol is simple. ModbusSrvr.py starts out with listen() method, which binds port 502 and waits for client connections. Once a client gets connected to it, it initiates a thread to serve the client and continues to listen on the port for further client connections. The thread calls the processData() method which extracts the top Modbus Header data and calls the method sendResponse() to send the correct response. In MODBUS protocol, the response depends on the function code of the query. Since we are not the experts of the protocol and we do not necessarily expect the users and developers of SCADA honeynet to be experts on industrial protocols, we followed and recommend the crude approach of observing and analyzing the communication packets between some clients and the PLCs. Based on the observation and analysis we chose to implement the "top responses" and to send an "error code" for the rest of the queries. The following are the observations:

There are two types of heavily used queries, the writes and the reads, and the target can be either coils or registers.

For responses to read requests, the response would be to give back data, equivalent to the number of bits requested in the read request. If its read multiple targets, then you usually give multiple modbus headers.

For responses to write requests, you give the bit/byte/word c ount of the data written We implemented the responses to read_coil (function code 1), write multiple registers (function code 16), diagnostics (function code 8 and the exception response with code 1(unknown function code). We welcome the users to study and analyze other responses and implement them. The script can be found in plc/modbusSrvr.py file. We also included our test file, modbusScanner.py, to send modbus packets towards the modbussrvr.py implementation. The users should manually go into modbusScanner.py and he can edit values of different modbus headers. Simulation of the FTP server The honeyd has shell script based FTP simulation, we decided to rewrite it in Python because it is an easier language for and we want to add more functionality to it. We implemented the following commands: The user command, when the user gives a username The password command, when the user gives the password The list command, when the user gives a ls command The syst command, which gives the user information about the system The port for transferring data for various commands It has various points where it writes information into the logfile and the user needs to change that to suite his or her own needs just by invoking writeLog method and passing it the string to write. The writeLog methos opens up "/var/log/ scadahoneynet.log" and appends information to it by default. The user should change the responses given as variables given at the top (such as ListCommandResponse and Syst CommandResponse of the file which suites their needs. The script can be found in plc/vxworks-ftpd.py Simulation of the Telnet serverWe decided to write a specific telnetd script because most of the PLCs run embedded systems and the shell has a unique set of commands. We implemented help and ls and cwd commands and the user gets the list of commands when he just hits return. The script can be found in plc/vxworks-telnetd.py Simulation of the Web serverFrequently the user connects to the web server of the device and a Java applet is downloaded to the client and runs within the web browser. In some cases, the applet will them make connection back to the PLC using protocols like Modbus/TCP and FTP for gathering data. The concept of an applet tracking the information of the downloader is new to the Honeynet world, we call them "Honey Applet". The problem with applets is that they would not be allowed to communicate with any other hosts other then the hosts that served the applet (http://java.sun.com/sfaq/). So make sure you have the hosIP variable as the exact hostname tof yours. Since each PLC has its own user interfaces, again our design and implementation goals are to write a generic enough proof of concept code which touches on most of the features in PLC applets. We used Java Swing classes to draw the Applet and we used Java SUN's 1.4.2 for development. The Button, and feature to track button clicks The Textfield, and feature to track text addition and removal To connect back to a host on port 502 and give information back. the user can run netcat like TCP listener to get the data back from the applet.

The connectBack() will connect back to the ip described statically in the code, encoded into the variable, host. We called connect back when the CA button is pressed or when the test is changed in TransmittButton for Demonstration purposes The use of threads and repaint with changing numbers in text fields The script can be found in plc/StatusApplet.java It needs to be debated whether the Applet can be replaced with a PHP script and how much would an attacker know about it if it were a PHP script Send coments to scadahoneynet-talk@lists.sourceforge.net or ciag-tools@cisco.com

SCADA software  designed by electricity SCADA experts,for electricity industry SCADA operations. 

Intuitive & safe operation iPower is designed specifically for electricity network control. The iPower GUI is consistent, clear and immediately intuitive to electricity SCADA Operators. Fast implementation Graphics configuration is typically the most time consuming implementation task. iPower commonly reduces 20 or more steps into one drag-and-drop, while delivering systems that are inherently consistent and easy to maintain.Reliable Client-server architecture, distribution of core applications and redundant networking add to a high-reliability SCADA system. Future proofed investment Running on top of iFIX (Intellution,USA) iPower is an investment in a state-of-the-art, open, standards-based system, delivering the best of the Windows environment, Internet technology and advanced data-sharing facilities

Operational benefits: Consistent display of information An intuitive interface Clear, precise operation Fast and safe operation

Management benefits Large reduction in configuration costs Far shorter system implementation time Consistency without micro-management Lower maintenance costs Maintenance without dependence on niche IT skills As you explore this site we trust you discover how iPower intelligently simplifies the installation, use,

ownership and rewards of SCADA for electricity industry businesses.  

Intuitive, Safe Operation iPower is developed specifically for monitoring and control of electricity systems and networks. The iPower GUI is consistent, clear and immediately intuitive to electricity SCADA Operators. iPower delivers:

Consistent display of information An intuitive interface Clear, precise operation Fast and safe operation

 Clicking on a breaker, switch, transformer or other device on your screen pops up precisely the right dialog....

Circuit Breaker dialog

Transformer Tap Position dialog

 ^               ^                ^              ^   click on the  tabs in these dialogs to learn more  ^                ^                ^             ^   

 Sample Display showing Operator dialogs

click to view (60kB)Intelligent Automated Configuration

 The single largest cost when installing a new SCADA is the time taken to configure the system.  iPower includes Intelligent Automation Software (IAS) that drastically speeds the configuration process. IAS has the additional benefit of producing highly consistent systems that are easy to maintain.Using IAS is simply a case of dragging the required electricity symbol and dropping it on to the SLD being built. This action triggers automation software that eliminates many database and display configuration steps. iPower reduces what might in some systems be 20-30 configuration steps to a single action.Consistent ConfigurationAll devices produced using iPower IAS look and operate the same way. The operational benefits are significant and obvious. Importantly this consistency is achieved by default rather than by careful management. This reduces the burden on ongoing management and training of staff, and separates the viability of the system from the expertise of key individuals. 

Purpose-built Operator dialogsAuto-generated Auxiliary Information ScreensAuto-generated Auxiliary Information Screens

 Many of the screens commonly required in a SCADA system are tabular displays of similar layout and content. For example every transformer may require an information screen to display the status of the many transformer alarms, plus other analogues and status tags related to the transformer. IAS further reduces system configuration time by completely automating the production of these Auxiliary Information Screens. Once again, these screens are entirely consistent throughout a system, helping to make the Operators job simpler and safer.

Circuit Breaker Auxiliary Information Screen

Open, Robust Architecture  

Distributed NetworkingiFIX’s networking design incorporates two basic principles: true distributed processing and on-demand data transfer.Distributed ProcessingMany systems operate in a hierarchical fashion that leave individual computers vulnerable to system failures anywhere on the network. The architecture of iPower allows you to distribute critical functions among nodes on the network.In a distributed processing network, each node independently executes the tasks assigned to it. One advantage of this strategy is that nodes can be taken off-line without bringing the whole network down. When a node looks for data from an off-line node, the networking application notifies the requesting node, so that the node handles the loss of data gracefully. Even though each node has integrity as an independent station, nodes can also access data anywhere on the network. For example, a View node can display a picture with links to many different SCADA nodes.SessionsWith iPower you can selectively configure which nodes can access data from SCADA nodes on the network. A communication link between two nodes over a network is called a session.Dynamic ConnectionsYou can also configure your node to automatically make connections online to remote SCADA nodes that are not specifically configured on your node. These connections are called dynamic connections.On Demand Data TransferSome control systems require every node that uses data from a SCADA node to have a copy of the entire database stored locally. The resulting network traffic can use significant system resources. To conserve system resources for local tasks, iPower reads and writes data on demand and only moves requested data over the network.File Storing and SharingUsing iPower and the built-in file sharing capabilities of Windows NT, you can store data that is needed by several nodes in one convenient location. Using the Windows Explorer, you can establish a networked drive connection to any other node in your local network. Once established, you have instant access to any shared files on that node, including databases, pictures, and other important files.

Centralized ProcessingSome applications only need one node to perform the required functions. It is easy to convert a distributed node to a stand alone node or a stand alone node to a distributed node. iPower operates just as smoothly in a single computer environment as it does in a distributed computer environment.RedundancyiPower includes a powerful redundancy feature that maximizes system performance by recognizing multiple paths to your data. Should a SCADA node or LAN connection become unavailable, iPower can switch from one path to another automatically. The process of switching from one connection to another is known as failover. Failover works the same whether you are using backup SCADA or LAN redundancy.Redundancy allows you to connect a View node to both primary and backup SCADA nodes that are connected to the same RTU/PLC. If the connection to the primary SCADA node is lost, iPower automatically fails over to the backup SCADA node. With LAN redundancy, you can establish two physical network connections between a View node and a SCADA node so that if one network path is lost, iPower automatically fails over to the other network path. These two features can be used together for the highest level of reliability.

Server Redundancy Module (SRM-2)SRM-2 is an additional redundancy module for either iFIX or iPower systems.

Open System StandardsiPower is a highly open system developed using the best industry standards and practices.

  High Tech Services

Production & Laboratory AutomationSystems Integration, Consulting, & Training

SCADA Factory & Laboratory Automation Software Our definition of SCADA or Supervisory Control and Data Acquisition software is:Computer based  Alarms Data acquisition Operator interface Non real-time control Database and Log FilesReports and Information Sharing

  Computer BasedWe feel that SCADA software must have all possible types of connectivity and integration.  This means serial ports, Ethernet, PCI slots, and the ability to run a wide variety of applications.  PLCs and simple operator interfaces (i.e. not based on the regular Windows operating system) are too limited in their functionality and capabilities.  Click for industrial computers or Visual Basic and C#.   Alarm and Event MonitoringA SCADA system must be able to detect, display, and log alarms and events.  When there are problems the SCADA system must notify operators to take corrective action.  Alarms and events must be recorded so engineers or programmers can review the alarms to determine what caused the alarm and prevent them happening again. 

Data AcquisitionSCADA must be able to read data from PLCs and other hardware and then analyze and graphically present that data to the user.  SCADA systems must be able to read and write multiple sources of data.  Click here for more on data acquisition.   Operator InterfaceA SCADA system collects all of the information about a process.  The SCADA system then needs to display this data to the operator so that they can comprehend what is going on with the process.  Click here for more on operator interfaces.     

 Non Real-Time ControlFor simple control requirements, the SCADA system should be able to perform control instead of a PLC.  However, for anything other than simplistic control we prefer a PLC or soft PLC to do the real-time control with SCADA doing the non-real-time control.  The SCADA system is the medium between the operator and the real-time controller.  It allows the operator to control the system, such as start a new batch, load a new recipe, etc.   Databases and Data LoggingMost applications require recipes, data logging, and other means of reading and writing databases.  The great thing about SCADA systems is that they can log incredible amounts of data to disk for later review.  This is helpful for solving problems as well as providing information to improve the process.  Many different methods should be available including, plain text, binary fixed column, Comma Separated Variable (CSV), XML, Excel, Access, SQL, SQL Server, ODBC, web services.   Reports and Information SharingWhat good is a SCADA system and all this information if you can't share it with others?  Some of the reports overlap with the previous description of databases and data logging.  For example, your users might prefer that you put the results into an Excel spreadsheet or database so that they can use their own tools for creating the report.  Or the users might want you to create reports in Microsoft Word format.  You also must share data with other users.  Such as windows sockets, web server, and web services.  These three methods would allow almost every computer around the world be able to access and use the information provided they have the correct permissions.     SCADA Software Links

Computer - PLC communicationsAfcon (P-CIM)  Citect  Genesis by IconicsIntellution  Think & Do   US Data Factory Link  Visual Basic and C#  Wonderware