Advanced Process Monitoring for Startups, Shutdowns & Switchovers Industrial Modeling Framework (APM-SUSDSO-IMF)

i n d u s t r IAL g o r i t h m s LLC. (IAL)

www.industrialgorithms.com March 2014

Introduction to Advanced Process Monitoring, UOPSS and QLQP Presented in this short document is a description of what is called “Advanced” Process Monitoring as described by Hedengren (2013) but related to Startups, Shutdowns and Switchovers-to-Others (APM-SUSDSO). APM is the term given to the technique of estimating or fitting unmeasured but observable variables or "states" using statistical data reconciliation and regression (DRR) in an off-line or real-time environment. It is also referred to as Moving Horizon Estimation (MHE) (Robertson et. al., 1996) in Advanced Process Control (APC) which goes beyond simply updating a bias to implement some form of measurement or parameter feedback (Kelly and Zyngier, 2008b). Essentially, the model and data define a simultaneous nonlinear and dynamic DRR problem where the model is either engineering-based (first-principles, fundamental, mechanistic, causal, rigorous) or empirical-based (correlation, statistical data-based, observational, regressed) or some combination of both (hybrid) (Pantelides and Renfro, 2012).

Figure 1. BP Texas City Oil-Refinery’s Raffinate Splitter with Blowdown System Flowsheet. Figure 1 depicts a relatively simple flowsheet problem in our unit-operation-port-state superstructure (UOPSS) (Kelly, 2004a, 2005, and Zyngier and Kelly, 2012) where this flowsheet is taken from the Interim Report on the Isomerization Unit Explosion (Raffinate Splitter) that

tragically occurred on March 23rd, 2005 at the BP Texas City Oil-Refinery. The flow diagrams used to construct this flowsheet can be found in Appendix A. The purpose of using this example is to highlight the importance of performing straightforward material, mass and/or volume balances on-line or in real-time during the periods of startup, shutdown, switchovers-to-others (i.e., from one mode to another) and switchovers-to-itself where switchovers-to-itself (Kelly and Zyngier, 2007) are what we call steady-state or stationary periods in a normal, standard or regular operation without upsets or significant disturbances (Kelly and Hedengren, 2013). Intervals of startups, shutdowns, switchovers-to-others including switchovers-to-itself with upsets, are typically defined as unsteady or non-stationary behavior. Appendix B, taken from the Final and Investigation Reports of the disaster, shows 25 specific citations to either material, mass or volume balances in the two documents. It was reported that failure to recognize that the entire Raffinate Splitter was quickly over-filling with liquid raffinate feed and consequently over-pressurizing the unit and rupturing the safety relief valves sending barrels of relatively light hydrocarbons (C5’s to 400 degree F) to the open-system blowdown was one of the main causes for the accident. The flowsheet shown in Figure 1 is a linear volume-based model given that we can neglect any gas or vapour traffic for this type of separation or distillation process in light of the fact that the splitter has a total condenser with minimal or negligible non-condensable components. If larger amounts of gases/vapours are interchanged comparatively with the liquid/solid exchanges then a bilinear or nonlinear mass-based model would be more appropriate provided sufficiently accurate density readings are available. A key difference between this flowsheet and others are the inclusion of material accumulators such as drums, vessels and tanks (triangles in Figure 1). This is useful to increase the amount of observability, redundancy and precision in the data reconciliation and regression model (Kelly, 1999 and 2000) as well as improving the diagnostic capability of the gross-error detection and identification by incorporating the level sensor readings and converting from level to volume or mass. Obviously for switchovers-to-itself operations the range or span of the level instruments are valid and can be used as measurements. However in abnormal, non-standard or irregular operations mentioned, these level readings will be declared as unmeasured and will be calculated or regressed in the DRR given that the accumulations may either be too low or too high to be sensed, measured or read. The diamond shapes or objects are the sources and sinks known as perimeters, the triangle shapes are the pools or tanks as previously mentioned and the rectangle shapes with the cross-hairs are continuous-process units. The circle shapes with no cross-hairs are in-ports which can accept one or more inlet flows and are considered to be simple or uncontrolled mixers. The cross-haired circles are out-ports which can allow one or more outlet flows and are considered to be simple or uncontrolled splitters. The lines, arcs or edges in between the various shapes are known as internal and external streams and represent in this context the flows of materials from one shape to another. In this example specifically, we have a feed surge drum (F-1101), a reboiler furnace with convection and radiant sections (B-1101), a raffinate splitter, fractionation column or distillation tower with a bottoms holdup (E-1101), a reflux drum (F-1102), heat exchangers C-1107-A/B, C-1104-A/B and C-1106-A/B, two tanks TK-103 (light raffinate) and T-538 (heavy raffinate) and an open-system blowdown system (F-20) that can be discharged to a stack, slop and/or sewer. All of the objects or shapes with dotted lines are excluded from the volume-based model given that they have no available sensors and/or their sensors are unknown at the time of the monitoring. If material does happen to flow to these destinations or dispositions but they are not included in

the model, then the gross-error statistics will detect and identify these situations or cases provided the unexpected flow or leak amounts have “significant-size” to be detectable given the uncertainty/tolerance and the number of included measurements. And finally, there are 7 flow sensors found on either the in-ports or out-ports i.e., F5001, F5003, F5000, F5005, F5007, F5015 and F5106 and there are 5 holdup sensors located on the pools i.e., L5007, L5100, L5008, T-538 and TK-103. Industrial Modeling Framework (IMF), IMPL and SIIMPLE To implement the mathematical formulation of this and other systems, IAL offers a unique approach and is incorporated into our Industrial Modeling Programming Language we call IMPL. IMPL has its own modeling language called IML (short for Industrial Modeling Language) which is a flat or text-file interface as well as a set of API's which can be called from any computer programming language such as C, C++, Fortran, Java (SWIG), C#, VBA or Python (CTYPES) called IPL (short for Industrial Programming Language) to both build the model and to view the solution. Models can be a mix of linear, mixed-integer and nonlinear variables and constraints and are solved using a combination of LP, QP, MILP and NLP solvers such as COINMP, GLPK, LPSOLVE, SCIP, CPLEX, GUROBI, LINDO, XPRESS, CONOPT, IPOPT, KNITRO and WORHP as well as our own implementation of SLP called SLPQPE (Successive Linear & Quadratic Programming Engine) which is a very competitive alternative to the other nonlinear solvers and embeds all available LP and QP solvers. In addition and specific to DRR problems, we also have a special solver called SECQPE standing for Sequential Equality-Constrained QP Engine which computes the least-squares solution and a post-solver called SORVE standing for Supplemental Observability, Redundancy and Variability Estimator to estimate the usual DRR statistics found in Kelly (1998 and 2004b) and Kelly and Zyngier (2008a). SECQPE also includes a Levenberg-Marquardt regularization method for nonlinear data regression problems and can be presolved using SLPQPE i.e., SLPQPE warm-starts SECQPE. SORVE is run after the SECQPE solver and also computes the well-known "maximum-power" gross-error statistics (measurement and nodal/constraint tests) to help locate outliers, defects and/or faults i.e., mal-functions in the measurement system and mis-specifications in the logging system. The underlying system architecture of IMPL is called SIIMPLE (we hope literally) which is short for Server, Interfacer (IML), Interacter (IPL), Modeler, Presolver Libraries and Executable. The Server, Presolver and Executable are primarily model or problem-independent whereas the Interfacer, Interacter and Modeler are typically domain-specific i.e., model or problem-dependent. Fortunately, for most industrial planning, scheduling, optimization, control and monitoring problems found in the process industries, IMPL's standard Interfacer, Interacter and Modeler are well-suited and comprehensive to model the most difficult of production and process complexities allowing for the formulations of straightforward coefficient equations, ubiquitous conservation laws, rigorous constitutive relations, empirical correlative expressions and other necessary side constraints. User, custom, adhoc or external constraints can be augmented or appended to IMPL when necessary in several ways. For MILP or logistics problems we offer user-defined constraints configurable from the IML file or the IPL code where the variables and constraints are referenced using unit-operation-port-state names and the quantity-logic variable types. It is also possible to import a foreign *.ILP file (row-based MPS file) which can be generated by any algebraic modeling language or matrix generator. This file is read just prior to generating the matrix and before exporting to the LP, QP or MILP solver. For NLP or quality problems we offer

user-defined formula configuration in the IML file and single-value and multi-value function blocks writable in C, C++ or Fortran. The nonlinear formulas may include intrinsic functions such as EXP, LN, LOG, SIN, COS, TAN, MIN, MAX, IF, NOT, EQ, NE, LE, LT, GE, GT and CIP, LIP, SIP and KIP (constant, linear and monotonic spline interpolations) as well as user-written extrinsic functions (XFCN). It is also possible to import another type of foreign file called the *.INL file where both linear and nonlinear constraints can be added easily using new or existing IMPL variables. Industrial modeling frameworks or IMF's are intended to provide a jump-start to an industrial project implementation i.e., a pre-project if you will, whereby pre-configured IML files and/or IPL code are available specific to your problem at hand. The IML files and/or IPL code can be easily enhanced, extended, customized, modified, etc. to meet the diverse needs of your project and as it evolves over time and use. IMF's also provide graphical user interface prototypes for drawing the flowsheet as in Figure 1 and typical Gantt charts and trend plots to view the solution of quantity, logic and quality time-profiles. Current developments use Python 2.3 and 2.7 integrated with open-source Dia and Matplotlib modules respectively but other prototypes embedded within Microsoft Excel/VBA for example can be created in a straightforward manner. However, the primary purpose of the IMF's is to provide a timely, cost-effective, manageable and maintainable deployment of IMPL to formulate and optimize complex industrial manufacturing systems in either off-line or on-line environments. Using IMPL alone would be somewhat similar (but not as bad) to learning the syntax and semantics of an AML as well as having to code all of the necessary mathematical representations of the problem including the details of digitizing your data into time-points and periods, demarcating past, present and future time-horizons, defining sets, index-sets, compound-sets to traverse the network or topology, calculating independent and dependent parameters to be used as coefficients and bounds and finally creating all of the necessary variables and constraints to model the complex details of logistics and quality industrial optimization problems. Instead, IMF's and IMPL provide, in our opinion, a more elegant and structured approach to industrial modeling and solving so that you can capture the benefits of advanced decision-making faster, better and cheaper. "Advanced” Process Monitoring for Startups, Shutdowns and Switchovers-to-Others Synopsis At this point we explore further, at least conceptually, the purpose of "advanced" process monitoring for startups, shutdowns and switchovers-to-others using a switchovers-to-itself volume-based linear model. Unfortunately there are no readily available data sets in the public-domain for the BP Texas City Raffinate Splitter incident although there is a “Sequence of Events” section in the Interim Report (pages 9 to 11) with some data referenced but not complete enough to perform DRR. There is no question that performing even relatively simple linear material balances in real-time (as opposed to more complicated nonlinear heat and momentum balances) at a frequency of every minute has tremendous value especially for safety reasons during significant operating transients and this was emphasized in the Investigation Report on page 78 i.e., “Start up of a unit

is a time when material balances should be calculated.” In addition, if manual or “mental” material

balances are expected to be performed by the operators during these often stressful transitions then expect inconsistencies in their outcomes as cited on page 86 of the Final Report i.e., “The

Day Shift Board Operator stated that he was performing a mental mass balance every few minutes, but the

actions taken are inconsistent with this.”

The notion of using the switchovers-to-itself or normal, standard or regular operating material balance model with explicit accumulators (pools) also for startups, shutdowns and switchovers-to-others is appropriate for one very important reason: there is only one model and only one set of data to configure, maintain, interface, understand and troubleshoot. There are two other related reasons: a) a linear material balance is simple and has little if any parameters or physical properties to be estimated and b) the simpler model can be solved quickly and reliably in an on-line environment where computing the sensitivity information (observability, redundancy and variability) from which to perform the gross-error diagnostics is simplified. When the data set is from a startup operation for instance and the model has been built based on the normal or steady operation then as the process moves from one transient to the next, valuable gross error statistics can be reviewed and displayed and if above their critical or threshold statistical values, appropriate signals and/or alarms can be raised in an automated and accurate fashion. This can provide useful and timely auxiliary information that may prove invaluable such as in the BP Texas City fatal accident when the Raffinate Splitter was entirely filled or flooded with liquid hydrocarbon whereby a simple material balance would have detected this minutes if not hours before over-pressuring the safety relief valves which then discharged significant amounts of hydrocarbons to the open blowdown system (and unfortunately was not vented downstream to a closed flare system). References Robertson, D.G., Lee, J.H., Rawlings, J.B., "A moving horizon-based approach for least-squares estimation", American Institute of Chemical Engineering Journal, 42, 2209, (1996). Kelly, J.D., "A regularization approach to the reconciliation of constrained data sets", Computers & Chemical Engineering, 1771, (1998). Kelly, J.D., "Practical issues in the mass reconciliation of large plant-wide flowsheets", AIChE Spring Meeting, Houston, March, (1999). Kelly, J.D., “The necessity of data reconciliation”, NPRA Computer Conference, Chicago, November, (2000). Kelly, J.D., "Production modeling for multimodal operations", Chemical Engineering Progress, February, 44, (2004a). Kelly, J.D., "Techniques for solving industrial nonlinear data reconciliation problems", Computers & Chemical Engineering, 2837, (2004b). Interim Report, “Fatal Accident Investigation Report: Isomerization Unit Explosion, Texas City, Texas, USA”, May, (2005). Final Report, “Fatal Accident Investigation Report: Isomerization Unit Explosion, Texas City, Texas, USA”, December, (2005). Kelly, J.D., "The unit-operation-stock superstructure (UOSS) and the quantity-logic-quality paradigm (QLQP) for production scheduling in the process industries", In: MISTA 2005 Conference Proceedings, 327, (2005).

Investigation Report, “Refinery Explosion and Fire (15 Killed, 180 Injured)”, U.S. Chemical Safety and Hazard Investigation Board, Report Number: 2005-04-I-TX, March, (2007). Kelly, J.D., Zyngier, D., "An improved MILP modeling of sequence-dependent switchovers for discrete-time scheduling problems", Industrial & Engineering Chemistry Research, 46, 4964, (2007). Kelly, J.D., Zyngier, D., "A new and improved MILP formulation to optimize observability, redundancy and precision for sensor network problems", American Institute of Chemical Engineering Journal, 54, 1282, (2008a). Kelly, J.D., Zyngier, D., "Continuously improve planning and scheduling models with parameter feedback", FOCAPO 2008, July, (2008b). Kalantarnia, M., Khan, F., Hawboldt, K., “Modelling of BP Texas City refinery accident using dynamic risk assessment approach”, Process Safety and Environmental Protection, 88, 191, (2010). Pantelides, C.C., Renfro, J.G., "The online use of first-principles in process operations: review, current status & future trends", FOCAPO/CPC 2012, January, (2012). Zyngier, D., Kelly, J.D., "UOPSS: a new paradigm for modeling production planning and scheduling systems", ESCAPE 22, June, (2012). Hedengren, J.D., "Advanced process monitoring", http://apm.byu.edu/pubs/springer.pdf (2013). Kelly, J.D., Hedengren, J.D., "A steady-state detection (SDD) algorithm to detect non-stationary drifts in processes", Journal of Process Control, 23, 326, (2013). Appendix A – Flowsheets of Raffinate Splitter and Blowdown System

Figure A1. Raffinate Splitter and Blowdown Flowsheet (Investigation Report, 2007).

Figure A2. Raffinate Splitter Flowsheet (Interim Report, 2005).

Figure A3. Blowdown System Flowsheet (Interim Report, 2005).

Appendix B – Cited Instances of “Material, Mass or Volume Balance” Terms Fatal Accident Investigation Report: Isomerization Unit Explosion, Final Report, Texas City, Texas, USA, December, 2005. There are ten (10) citations below ...

Page 24 – “Mass balance and dynamic modeling confirms that large volumes did indeed enter the

sewers and may have contributed to secondary fires.”

Page 25 – “A mass balance based upon DCS data performed by integrating the Splitter flow

meters into and out of the column over the startup period indicates that the vessel was substantially

overfilled.”

Page 27 – “The temperature crosses at these times are broadly as predicted by the simple volume

balance over the column which indicates that the liquid accumulation in the vessel reached tray 13 at

around 12:45 hrs.”

Page 37 – “Using an overall volume balance to corroborate shows: Estimated liquid inventory in the

Splitter at end of incident (balance of flow meters) (corresponds to liquid height of 131.8 ft—close to tray

13) …”

Page 39 – “Based on a simple single phase volume balance, it seems that the liquid level should only

have reached tray 13 (top tray =1).”

Page 40 – “This is fairly consistent with the volume balance and is corroborated by other modeling

performed.”

Page 47 – “These classes emphasized the importance of paying attention to all parameters (flow,

pressure, temperature, level and indirect measures, such as material and heat balances) to ensure

understanding of actual process conditions and to be able to discern if any instrument is providing

misleading indications due to process conditions.”

Page 52 – “The bottoms flow transmitter indicated a flow of 3-4 MBPD (thought to be a zero error)

(see Appendix 5), significantly lower than the 20 MBPD entering the tower. A material balance across the

tower would have shown that more feed was entering the Raffinate Splitter than was being removed, even

if the Day Shift Board Operator believed the zero error to be an actual flow. The control valve was

indicated closed and should have confirmed that it was an error. A mass balance calculation would have

provided clear indication of the overall process picture, i.e., that the Splitter had an abnormally high

internal liquid level.”

Page 58 – “Through most of the startup, alarms, mass balance and process instrumentation showed the

liquid level in the tower was too high and that no liquid was being removed from the tower through the

two pathways available.”

Page 86 – “The Day Shift Board Operator stated that he was performing a mental mass balance every

few minutes, but the actions taken are inconsistent with this.”

Investigation Report: Refinery Explosion and Fire (15 Killed, 180 Injured), U.S. Chemical Safety and Hazard Investigation Board, Report Number: 2005-04-I-TX, March, 2007. There are fifteen (15) citations below ...

Page 23 - “The control board display did not provide adequate information on the imbalance of flows in

and out of the tower to alert the operators to the dangerously high level. ”

Page 50 – “Material balance calculations conducted post-incident determined that the tower had

actually filled to 13 feet (4 m), four feet over the top tap of the level transmitter.”

Page 77/78 – “The procedure did not instruct the Board Operator to pay close attention to the

incoming/outgoing liquid raffinate flow readings or to calculate a material balance65 of the unit during

startup, nor did it explain how to make such a calculation.

Calculating material balance for a unit is a method used by operations personnel during times of potential

flow imbalances or level uncertainty; that is, when the amount of material coming into the unit does not

necessarily equate to the material being removed from the unit. Start up of a unit is a time when material

balances should be calculated.

A material balance calculation relies on knowledge of the flow rates into and out of a piece of equipment,

such as a tower, and the board operator’s understanding of how the net rate affects accumulation of liquid

within the equipment over time. ”

Page 83 – “On the day of the incident, however, the computerized control system display provided

either flow data in and out of the raffinate unit on the same display screen, nor a material balance

calculation, hindering the Board Operator’s ability to recognize the need to send liquid raffinate to

storage.

A material balance calculation is used to determine how much total liquid is in a given unit; it is

determined by comparing the amount of incoming feed to the amount of outgoing product, and requires

knowledge of how much liquid the total system needs to maintain and run the process smoothly. ”

Page 84 – “The control system did not calculate the material balance of the system and the operators

did not know how to make such calculations. The computerized control system was also configured to

display only portions of the unit in discrete detailed sections, and did not allow for a complete overview

of the process, just like the control system screens for the Texas City ISOM unit. The U.K. Health and

Safety Executive83 recommended that computerized control systems include a process overview and, as

appropriate, material balance summaries to ensure full process oversight by operators (HSE, 1997).”

Page 94 – “The ISOM unit operator training program did not include training for abnormal situation

management, the importance of material balance calculations, and how to avoid high liquid level in

towers; … ”

Page 96 – “The BP training program did not include specific instruction on the importance of

calculating material balances, and the startup procedures did not discuss how to make such calculations.

As a result, the Day Board Operator did not have a full understanding of the material balance for the

ISOM unit.”

Page 215 – “4-I-TX-R14 Evaluate your refinery process units to ensure that critical process equipment

is safely designed. At a minimum,

a. Ensure that distillation towers have effective instrumentation and control systems to prevent overfilling

such as multiple level indicators and appropriate automatic controls.

b. Configure control board displays to clearly indicate material balance for distillation towers.”

Page 258 – “Three complementary approaches were used to understand the overflow of the raffinate

splitter column:

modeling of column material balance,

modeling of liquid thermal expansion, and

qualitative evaluation of vapor accumulation.

The material balance was modeled in a spreadsheet based on flow data from the PI system.”

Page 260 – “Based on material balance, at 1:13 p.m., the level of cold liquid in the raffinate splitter

would have been 143 feet (44 m) above the lower tangent line on the column.”

Page 261 – “The amounts of sub-cooled hydrocarbon liquid lost from the raffinate splitter to the

atmosphere and sewer can only be estimated. A mass balance of the raffinate splitter, ancillary equipment,

blowdown drum, and piping using calculated flows from the safety relief valves on the raffinate splitter

overhead vapor line and flow to the sewer from the gooseneck piping at the bottom of the blowdown

drum was used to determine the flow from the blowdown drum stack to the atmosphere.”

Page 263 – “The overall mass balance of hydrocarbon flow from the raffinate splitter from the

computer simulation is shown in Table G-3. As much as 6,730 gallons of hydrocarbons could have been

released to the atmosphere prior to the explosion and an additional 855 gallons after the explosion.”

Page 300 – “The CSB found no evidence to suggest that the training covered material balance

calculations or the hazards of high liquid level in splitter towers.”

Page 301 – “The troubleshooting course was not unit-specific, nor did it discuss issues critical to unit

startup, such as calculating material balance or the hazards of high liquid levels.”

Page 313 – “The heat and material balance differential equations, which describe the transients, and

the fluid dynamic equations, which describe the flow capacity of an emergency relief system, were solved

by numerical integration.”

Appendix C - APM-SUSDSO-IMF.UPS (UOPSS) File