Embed Size (px)
Citation preview
Missouri University of Science and Technology Missouri University of Science and Technology
Scholars' Mine Scholars' Mine
Computer Science Technical Reports Computer Science
01 Oct 1992
A Process-Learning PID Controller Algorithm Utilizing an On-Line A Process-Learning PID Controller Algorithm Utilizing an On-Line
Iterative Improvement Technique Iterative Improvement Technique
Ryan Rosandich
Ralph W. Wilkerson Missouri University of Science and Technology, [email protected]
Follow this and additional works at: https://scholarsmine.mst.edu/comsci_techreports
Part of the Computer Sciences Commons
Recommended Citation Recommended Citation Rosandich, Ryan and Wilkerson, Ralph W., "A Process-Learning PID Controller Algorithm Utilizing an On-Line Iterative Improvement Technique" (1992). Computer Science Technical Reports. 23. https://scholarsmine.mst.edu/comsci_techreports/23
This Technical Report is brought to you for free and open access by Scholars' Mine. It has been accepted for inclusion in Computer Science Technical Reports by an authorized administrator of Scholars' Mine. This work is protected by U. S. Copyright Law. Unauthorized use including reproduction for redistribution requires the permission of the copyright holder. For more information, please contact [email protected].
A PROCESS-LEARNING PID CONTROLLER ALGORITHM UTILIZING AN ON-LINE ITERATIVE IMPROVEMENT TECHNIQUE
Ryan Rosandich* and Ralph Wilkerson
CSC-92-21
Department of Computer Science University of Missouri-Rolla
Rolla, Missouri 65401
*This report is substantially the M.S. thesis of the first author, completed Fall 1992.
Ryan George Rosandich ALL RIGHTS RESERVED
iiiABSTRACT
Three iterative improvement algorithms are presented for the determination of process controller gains. An offline algorithm is developed and tested as a basis for comparison, and a simple on-line algorithm is developed as an incremental step toward the final algorithm, proportional on-line iterative improvement. The algorithms are based on an Artificial Neural Network learning method, and this method is compared with other control optimization techniques. The performance of each of the algorithms was experimentally evaluated in numerous realistically simulated process control situations consisting of flow, level, and temperature control loops with various values of dead-time and process noise. The experimental results reveal that the learned process controller gains behave in a predictable and intuitive manner.
The final algorithm performed very well on most of the simulated processes, but it performed only marginally well on processes where both dead-time and noise were present in significant quantities. The algorithm's complexity and memory requirements do not preclude its application in microprocessor-based single and multiple loop controllers.
ivACKNOWLEDGEMENTS
First and foremost I would like to thank my parents, for throughout my life they have instilled in me a belief in the importance of higher education. Without their continued encouragement I would not now be pursuing a graduate degree. Next I would like to thank Dr. Wilkerson, my thesis advisor, and the members of my thesis committee Dr. Dagli and Dr. Ho, each of whom contributed significantly to my graduate education. I would also like to acknowledge the fact that, by granting me the Chancellor's Fellowship, the University of Missouri-Rolla has made my transition to full time academic life less financially difficult. Finally, I would like to thank my wife Vivian and my daughter Lacey for agreeing to endure an upheaval of their lives so that I could pursue my educational goals.
V
TABLE OF CONTENTSPage
ABSTRACT.................................................. iiiACKNOWLEDGEMENTS......................................... ivLIST OF ILLUSTRATIONS...... ...................... ...... viiLIST OF TABLES............................................ ixSECTION
I. INTRODUCTION..................................... 1A. SINGLE LOOP PROCESS CONTROL................ 1B. DIGITAL PID CONTROLLERS..................... 2C. SIMILARITY BETWEEN PID CONTROLLER
AND ARTIFICIAL NEURON....................... 4D. CHARACTERISTICS OF REAL PROCESSES.......... 7E . GOAL STATEMENT.............................. 8
II. SURVEY OF CONTROL OPTIMIZATION TECHNIQUES..... 10A. GENETIC ALGORITHMS. . ........... 10B. SIMULATED ANNEALING........ 11C. ARTIFICIAL NEURAL NETWORKS................. 12D. ITERATIVE IMPROVEMENT....................... 13
III. ALGORITHM DEVELOPMENT AND EVALUATION........... 16A. SIMULATED PROCESSES......................... 16B. CHOICE OF EVALUATION FUNCTION.............. 20C. CONTROLLER DESCRIPTION...................... 24D. CHARACTERISTICS OF ITERATIVE LEARNING..... 25E. SIMPLE OFF-LINE ITERATIVE LEARNING......... 27F. SIMPLE ON-LINE ITERATIVE LEARNING.......... 28G. PROPORTIONAL ON-LINE ITERATIVE LEARNING___ 30
IV. RESULTS AND DISCUSSION........................... 34A. EVALUATION FUNCTION PERFORMANCE............ 34B. ITERATIVE LEARNING PERFORMANCE CRITERIA.... 38C. SIMPLE OFF-LINE ITERATION.................. 39D. SIMPLE ON-LINE ITERATION.................... 47E. PROPORTIONAL ON-LINE ITERATION............. 53F. PROPORTIONAL LEARNING PERFORMANCE.......... 60
V. CONCLUSIONS AND RECOMMENDATIONS FORFUTURE RESEARCH.................................. 65
APPENDIX.................................................. 67BIBLIOGRAPHY.............................................. 82VITA....................................................... 84
vi
LIST OF ILLUSTRATIONSFigures Page1. Artificial Neuron.................. . 52. PID Controller....................................... 53. Hypothetical Process.....................................174. Flow Process Open Loop Response........................ 195 . Level Process Open Loop Response....................... 196. Temperature Process Open Loop Response.................207. Minimum Overshoot Response.............................. 228. Minimum Rise Time Response.............................. 239. Quarter-Amplitude Damping Response........ 2310. Controller Performance (Flow, DT=0, Absolute Error)...3511. Controller Performance (Flow, DT=0, Error Squared) .... 3512. Controller Performance (Flow, DT=0, Quadratic Min.)...3613. Controller Performance (Flow, DT=3, Absolute Error)...3614. Controller Performance (Flow, DT=3, Error Squared) ....3715. Controller Performance (Flow, DT=3, Quadratic Min.)...3716. Surface Plot of Evaluation Function....................3917. Controller Performance (Off-Line, PID, Flow/0/0) ..... .4318. Controller Performance (Off-Line, PID, Flow/0/5) ..... .4319. Controller Performance (Off-Line, PID, Flow/3/0)...... 4420. Controller Performance (Off-Line, PID, Flow/3/5)...... 4421. Controller Performance (Off-Line, PID, Level/0/0)..... 4522. Controller Performance (Off-Line, PID, Level/0/5)..... 4523. Controller Performance (Off-Line, PID, Level/3/0)..... 4624. Controller Performance (Off-Line, PID, Level/3/5)..... 4625. Controller Performance (Off-Line, PID, Temp./2 0/0) . . . . 47
vii
viii26. Controller Performance (Simple, PI, Flow/0/0)....... .4927. Controller Performance (Simple, PI, Flow/0/5)....... .4928. Controller Performance (Simple, PI, Flow/3/0)....... .5029. Controller Performance (Simple, PI, Flow/3/5) ........ . 5030. Controller Performance (Simple, PI, Level/0/0) ....... .5131. Controller Performance (Simple, PI, Level/0/5)...... .5132 . Controller Performance (Simple, PI, Level/3/0) ....... . 5233 . Controller Performance (Simple, PI, Level/3/5)...... .5234. Controller Performance (Prop., PID, Flow/0/0/0.10)... .5635. Controller Performance (Prop., PID, FIow/0/5/0.05)... .5636. Controller Performance (Prop., PID, Flow/3/0/0.10)... .5737. Controller Performance (Prop., PID, Flow/3/5/0.10)... .5738 . Controller Performance (Prop., PID, Level/0/0/8.0)... .5839 . Controller Performance (Prop., PID, Level/0/5/8.0)... .5840. Controller Performance (Prop., PID, Level/3/0/8.0)... .5941. Controller Performance (Prop., PID, Level/3/5/8.0) .5942. Controller Performance (Prop., PID, Temp./20/0/8.0).. .6043. Learning Performance (Dead-Time=0, Noise=0)........... 6344. Learning Performance (Dead-Time=0, Noise=5)........... 6345. Learning Performance (Dead-Time=3, Noise=0)........... 6446. Learning Performance (Dead-Time=3, Noise=5) ........... 64
LIST OF TABLESTables PageI. Simple Off-Line Iterative Improvement Tests..........29II. Simple On-Line Iterative Improvement Tests...........30III. Proportional On-Line Iterative Improvement Tests.... 33IV. Evaluation Function Performance Results.............. 34V. Simple Off-Line Iterative Improvement Results.........40VI. Simple On-Line Iterative Improvement Results..........48VII. Proportional On-Line Iterative Improvement Results...55
ix
I INTRODUCTION
A. SINGLE LOOP PROCESS CONTROLThe problem in single variable continuous process control
is that of controlling some quantity of interest, called the Process Variable (PV), to some desired value, called the Setpoint (SP), by changing the Controller Output (CO) thereby directly manipulating some part of the process, typically a valve opening. Since the 1930's this problem has been solved through the application of PI (Proportional-Integral) and PID (Proportional-Integral-Derivative) controllers. PID controllers have been implemented pneumatically, hydraulically, and with both analog and digital electronics. Presently, hundreds of thousands of PID controllers are implemented each year for the control of industrial processes [1 ] .
There are several reasons for the popularity of PID controllers in industry. First is their mathematical simplicity. PID controllers are based on the relatively simple concepts of multiplication, integration over time, and rate of change over time. A knowledge of Laplace transforms, Z-transforms, or modern control theory is not required in order to successfully apply the controller. Next is the fact that control engineers typically have little or no information on the dynamic response of the system to be controlled. PID controllers can be successfully adjusted through a trial-and- error process, so accurate a priori plant knowledge is not
2required. Finally, the PID algorithm is very simple, so controllers can be implemented inexpensively.
B. DIGITAL PID CONTROLLERSAnalog PID controllers usually implement some variation
of the following equation [2]:
m = Control Output (CO) e = Process Error (PV - SP) kp = Proportional Gain kt = Integral Gain k4 = Derivative Gain
When digital microprocessor-based controllers were developed, the discrete-time nature of the sampled error required that Equation 1 be converted into the following equation:
( 1 )
Where:
m=kp(e +kj£ e T Af) (2)
Where:T = sample period
3Controllers implemented in this manner are known as interacting- PID controllers, because the Proportional Gain (kp) affects the integral and derivative terms as well as the
proportional term. This interaction has been eliminated in some digital controllers to produce non-interacting PID control [3] , in which each of the three terms act independently. Controllers designed in this manner implement the following equation:
iv=kpe - '- k ^ e T A ? (3)
The primary difficulty in the application of PID controllers is the selection of the gain parameters kp, Jcf, and kd. This process is known as controller "tuning" and it is typically done through trial-and-error, or by observing the response of the process to a step change in the controller output and applying some technique such as the Ziegler-Nichols method to calculate the gain parameters [4] . Substandard controller performance often results from poor tuning due to lack of time or ability on the part of the control engineer. Some processes slowly change over time as well, resulting in slowly degrading controller performance unless periodic tuning is done.
In order to address the aforementioned problems, process control vendors have developed various types of self-tuning
4controllers [5]. These controllers employ some technique for determining the tuning parameters automatically on-line. Some self-tuning controllers attempt to estimate a model of the process based on process behavior, and calculate the gains based on the model. Others actually force a process disturbance and observe the response, and then determine the tuning parameters based on calculations similar to those used in the Ziegler-Nichols method. Still others use a pattern recognition approach, attempting to respond to certain process behavior in the same way that an expert control engineer would. Many control loops have been successfully implemented using self-tuning controllers. All of the systems mentioned are quite complicated, however, and therefore expensive to implement. Some of the systems also require considerable input from a control engineer in order to properly function.
C. SIMILARITY BETWEEN PID CONTROLLER AND ARTIFICIAL NEURON Artificial Neural Networks (ANN's) represent an attempt
to model the biological processes that take place in the human brain [6] . ANN's are typically highly interconnected networks of fairly simple individual neurons. Each neuron (Figure 1) usually processes a number of inputs by multiplying each input by its respective weight factor and then summing the results. This weighted sum of inputs is then passed through a transfer function and the result becomes the output of the neuron, known as it's activation level. This output is then passed to other neurons, or out to the real world as a network output.
5
Figure 1. Artificial Neuron
Kp
OUTPUT
Figure 2. PID Controller
6Much research has been recently done toward developing
the ability of ANN's to "learn" various functions [7]. These learning techniques result in the input weight factors evolving slowly toward values that allow the ANN to perform its function properly. In this way the weight factors do not have to be explicitly set or programmed [8].
A non-interacting PID controller behaves exactly like an artificial neuron (Figure 2). The process error, the integrated process error, and the time-derivative of the process error are the three inputs. The three controller gains kp, kt, and kd are the input weighting factors, and the transfer function is linear with clamps at 0% and 100% controller output. This similarity suggests that ANN learning techniques may be applicable to the determination of PID controller gains.
Although there are many ANN learning methods, only two basic classes will be described here. With error correction learning [9], an attempt is made at reducing some global cost function by adjusting each neuron's input weights slightly whenever an error is present in the output of the network. The amount of the change in each weight is usually proportional to its perceived level of contribution to the erroneous output. The actual output of the ANN is usually compared to the desired output in order to determine the output error.
Iterative learning works in a slightly different fashion [9]. Individual weights are changed a small amount, usually
7one at a time. The effect of that change on the ANN's performance is then evaluated, and the change is kept if the performance has improved. If the performance has deteriorated, the weight is changed back to its original value. In either case, another weight is then selected and the process is repeated.
D. CHARACTERISTICS OF REAL PROCESSESIn order to properly evaluate the performance of
controllers, tests must be conducted on realistically simulated processes. Real industrial processes exhibit four elements that make them difficult to control [2]. The first is known as dead-time. Dead-time occurs whenever there is a delay between the time when a process actually changes and the time when the controller first detects the effect of that change. Dead-time is typically a transportation delay due to the physical separation of the controlling element and the sensing device. For example, if a material being controlled by weight is being deposited on a conveyor, and the material is being weighed at some point further down the conveyor line, the dead time will be dependent on the distance between the addition point and the weighing point, and on the speed of the conveyor.
Process lag is also found in industrial process. Lag is the result of the fact that the state of energy of a mass cannot be changed instantaneously. For example, when fluid flow through a pipeline is increased, the mass of the fluid
8must be accelerated over time to the desired velocity. Also, a fluid that is being heated to a higher temperature must be heated for a period of time before the desired temperature will be achieved. Processes that exhibit lag respond to a step change in input with an inverse exponential time constant rather than with a step response.
Some industrial processes also contain capacity. Capacity is present whenever a process contains a variable volume and, during filling or emptying, that volume behaves like an integrator. For example, if the flow into a tank increases with no change in the flow out of the tank, the level in the tank will rise over time.
Finally, real industrial processes often exhibit measurement noise. Noise is often due to natural process variations. It can also be generated by the process measurement device, or it can be the result of electrical noise induced in the measurement signal as it is transmitted to the controller. Whatever the cause, process measurement noise is exhibited as a random variation of some magnitude in the measured signal about some average or actual value.
E . GOAL STATEMENTThe goal of this research was to develop an algorithm for
the on-line iterative improvement of PID controller gains, and to evaluate the performance of the algorithm under realistic process control conditions. It was also desired that the algorithm be easily understood by control engineers, and that
9
it's complexity or memory requirements not preclude its application in small microprocessor based single-loop and multiple-loop controllers.
10II. SURVEY OF CONTROL OPTIMIZATION TECHNIQUES
A. GENETIC ALGORITHMSGenetic algorithms have been used to optimize the
performance of ANN based controllers [10] . In this technique, a population of controllers is generated, each with different weights. Each of the controllers is then evaluated with respect to its performance on a given process. Additional controllers are then created through two processes, mating and mutation. The controllers that perform best have the highest probability of mating, and the controllers which perform the worst have the highest probability of mutating. During mating, two controllers randomly exchange some of their weights, creating two new individuals. During mutation, some of a controller's weights are randomly selected and changed by some amount, generating a new individual. All of the new individuals generated through the mating and mutation processes are then evaluated for performance. A number of controllers equal to the size of the original population is then selected from the total population, with the controllers with the highest level of performance having the highest probability of being selected. After several iterations of the evaluation-mating-mutation-selection cycle, one or more individual controllers that are well suited to the given control task usually emerge. The genetic algorithm essentially performs a stochastic optimization of the search for ideal weights.
11The advantage of the genetic approach is that it is
capable of discovering novel controllers that would not result from the traditional controller design process. The disadvantage is that many controllers must be created and tested during each iteration, some of which are sure to perform horribly. This process could obviously not be conducted on-line, so the algorithm would have to be allowed to operate on a process simulation, with the best resulting controller then being used to control the real process.
B. SIMULATED ANNEALINGSimulated annealing algorithms are also capable of
performing a stochastic search for a set of weights that minimizes some error function. With this technique, the physical process of heating a metal to a high temperature, and then allowing it to slowly cool under conditions of thermal equilibrium, is imitated [11] . When this is done with a metal, the metal eventually reaches a crystalline state of lowest possible energy. The corollary in controller development is that the controller gains would reach a state in which the process error is minimized and controller performance is maximized. During the development stage, a cooling schedule would be followed for the artificial temperature, which governs how much the gains are changed as well as the probability of the changes being kept under various conditions. When the "temperature" reaches some preselected low value, the cooling is said to be complete and
12the final controller gains are the result.
The advantage of simulated annealing is that, if the process is carried out slowly enough, it is guaranteed to find the set of gains that result in the minimum possible process error. The disadvantage is that, under these conditions, the development or training time for the controller would be extremely long. It is also possible that the noise present in real processes would disrupt the progress of the simulated annealing algorithm.
C. ARTIFICIAL NEURAL NETWORKSArtificial Neural Networks, as stated previously, have
demonstrated the ability to learn to perform control tasks. This fact has lead many researchers to apply ANN technology to difficult control problems [12]. The disadvantage of most ANN learning strategies, however, is that they require some previous knowledge of the desired control output in addition to the desired plant performance. This implies complete and accurate a priori knowledge of the plant to be controlled, something rarely present in industrial control situations. The popular backpropagation algorithm is one example of an ANN learning strategy that performs well when the desired control outputs are known and can be compared to the actual controller outputs [13]. This algorithm is difficult to apply, however, when no a priori plant knowledge is available, and when only a general evaluation of controller performance is provided as feedback to the learning process.
13Another ANN paradigm, the Adaptive Heuristic Critic
(AHC), has been successfully applied to control problems in which there is no a priori plant knowledge and where there is only a general evaluation of controller performance available[14] . The AHC actually consists of two sub-networks, the controller network and the critic network, each with it's own learning strategy. The critic network attempts to provide the controller network with an informative evaluation of controller performance, based on the controller inputs as well as the externally provided general evaluation signal. The controller network's learning process is dependent upon the performance information supplied by the critic network. Learning also takes place in the critic network, as it attempts to better interpret the general evaluation information in order to provide better specific performance information to the controller network, which in turn is attempting to learn to control the process more effectively. The AHC algorithm has been successfully applied to the problem of dynamically balancing an inverted pendulum, given only a general success/failure signal for performance evaluation[15] . The primary drawback of the AHC approach is the complexity introduced by the two sub-networks and the dual- level learning scheme.
D . ITERATIVE IMPROVEMENTThe iterative improvement approach to determining ideal
controller gains is relatively straightforward. First, some
14initial gain parameter values are selected, and the controller's performance with those gains is evaluated. Next, a change is made to one of the controller gain parameter values and the controller's performance is re-evaluated. If the performance has improved, the gain change is retained. If the performance has deteriorated, the gain parameter is changed back to it's original value and the process is repeated with another gain parameter. This process is either continued indefinitely, with changes being made to each of the gain parameters in rotation, or it is terminated when no further improvement is made in a complete pass through all of the gain parameters.
In this research, algorithms for both off-line and online iterative improvement were developed. In the off-line approach, controller response to a step change in a simulated process was evaluated, and that evaluation was used to determine whether or not gain changes were retained. The offline algorithm terminated when a complete pass was made through all of the gain parameters with no improvement in controller performance. In the on-line approach, a controller's response to a simulated process undergoing step changes at random times was evaluated over a fixed period of time. The result of this evaluation was used to determine •.whether or not gain changes were retained. The on-line algorithm was terminated after a preselected time interval, but in practice it could be active indefinitely.
The disadvantage of iterative improvement is that it can
15result in a local minimum error (possibly not representative of maximum achievable controller performance) being pursued rather than a global minimum error. The advantages of the algorithm are that it is simple, it requires no a-priori plant knowledge, it requires only a general evaluation of controller performance, and it results in continuous improvement of that performance.
16III. ALGORITHM DEVELOPMENT AND EVALUATION
A. SIMULATED PROCESSESThree separate processes were simulated in order to
evaluate the effectiveness of the iterative improvement algorithms, a flow process, a level process, and a temperature process. All three processes were derived from the hypothetical system shown in Figure 3. The system consists of a 24 gallon tank with a mixer, a line bringing water into the tank, and a line taking water out of the tank. The inflowing line contains a valve for adjusting the flow, and a flow meter for measuring the flow into the tank. The tank is equipped with an adjustable heating element, a level measurement device (transmitter) and a temperature measurement device (transmitter) . The flow of water out of the tank can be changed in order to simulate the changing demand for water downstream.
The simulated flow process represented the flow of water into the tank as measured by the flow meter and controlled by the valve. The dynamics of the process were designed to simulate a valve with 200 ft. of 1 in. line upstream, with an inlet pressure of 20 psig. The range of the flowmeter was simulated as 0-15 gal./min. The process contained both a fixed and variable lag, with the variable lag being dependent on the instantaneous flow rate. No capacity was present, and desired values for dead-time (in seconds) and noise (in percent) could be entered before each run. The open-loop
17
Figure 3. Hypothetical Process
18response of the simulated flow process with both dead-time and noise set to zero can be seen in the graph in Figure 4. The graph represents the response to a change in valve position from 16.6% open to 50.0% open after initial conditions of equilibrium were established at 25% flow.
The simulated level process represented the level of water in the tank as measured by the level transmitter and controlled by adjusting the inflow valve. The dynamics of the process were designed to simulate a tank with a capacity of 24 gal., with a variable inflow of 0-15 gal./min. and a variable outflow of 0-10 gal./min., and with a level transmitter range of 0-100%. The process contained the same fixed and variable lags as the flow process, since they are the result of the inflow valve. Again, desired values for dead-time and noise could be entered before each run. The open-loop response of the simulated level process with both dead-time and noise set to zero can be seen in the graph in Figure 5. The graph represents the response to a change in valve position from 16.6% open to 50.0% open after initial conditions of equilibrium were established at a tank level of 25%.
The simulated temperature process represented the temperature of the water in the tank as measured by the temperature transmitter and controlled by adjusting the variable heating element. Inflow water and ambient temperature were simulated as 50 deg.F., the adjustable heating element was given a range of 0-3.5 KW., and the temperature transmitter was given a range of 50-150 deg.F.
Per
cen
t
19
_JFigure 4. Flow Process Open Loop Response
Figure 5. Level Process Open Loop Response
20
Sample Periods (1 sec.)
Figure 6. Temperature Process Open Loop Response
The process contained a lag proportional to the mass of water in the tank. As before, desired values for dead-time and noise could be entered before each run. The open-loop response of the simulated temperature process with both deadtime and noise set to zero can be seen in the graph in Figure 6. The graph represents the response to a change in heater output from 6.2% on to 18.0% on after initial conditions of equilibrium were established at a temperature of 75 deg.F. The exact details of the coding of each of the simulated processes can be found in the program listing in the appendix.
B. CHOICE OF EVALUATION FUNCTIONThe purpose of the evaluation function ( is to
quantitatively evaluate controller performance. The
21difficulty is in finding some measure that correlates with a typical control engineer's judgement of performance. Most control engineers agree that performance that results in undamped oscillation is undesirable, as well as performance that results in some non-zero steady-state error. The definition of "good" control, which usually lies between these two extremes, is more difficult to quantify. Sometimes it is desired for the process to slowly approach the setpoint without crossing over it. This is known as minimum overshoot performance and usually results in slow response times. At other times it is desired to reach setpoint as quickly as possible, even if it means there will be considerable overshoot. This is known as minimum rise time response, and usually results in less stable control. Quarter-amplitude damping is a technique which results in a response somewhere between the previous two. With this method overshoot is allowed, as long as the second oscillation beyond the setpoint is damped to one-fourth of the amplitude of the original overshoot. These three responses are depicted graphically in Figures 7-9.
In order to numerically gauge performance, a controller's response to a given process change is usually measured over a fixed period of time, and some value is summed over the time period. The resulting sum is used as a measure of the particular controller's performance. The choice of Fe, then, is the choice of which value is to be summed. Three of the most commonly used approaches are Error Magnitude, Error
22Squared, and Quadratic Minimization. When Error Magnitude is used [2] , Fe is the sum of the absolute value of the process error over the time period (Equation 4). The result is
f.-Lc (4)
linear with total error magnitude. When Error Squared is used[2], the square of the process error is summed over the time period (Equation 5) . This method emphasizes large errors while de-emphasizing errors less than 1. When Quadratic
Figure 7. Minimum Overshoot Response
23
Figure 8. Minimum Rise Time Response
Figure 9. Quarter-Amplitude Damping Response
24Minimization is used [10], the error squared is summed along with the control output squared. The control output term is
usually multiplied by some factor (a < 1.0) in order to limit its effect (Equation 6). This method penalizes excess
control action. As part of this research, each of these Fe choices was judged for complexity and effectiveness, utilizing the evaluation of controllers on the simulated flow process as a guide. The results of this evaluation can be found in section IV.
C. CONTROLLER DESCRIPTIONThe same simulated PID controller was used throughout
this project, and it was based on Equation 3. The controller output was calculated using the following C++ code:
iv = iv + er;co = -1.0* (er*wt [0] + iv*wt[l] + der*wt[2]); if (co < 0.0)
co = 0.0; if (co > 100.0)
(5)
Fe= ^ t(e2 + a*m2} (6 )
co = 100.0;
25Where:
iv = the integration of the process errorer = process errorco = control outputwt[0] = proportional gain, kpwt [1] = integral gain, ktwt[2] = derivative gain, kdder = change in error over last sampling period
The sampling period used was one second for all situations. Note that the control output was not allowed to go below 0.0 or above 100.0.
Before each test, the controller was tuned by setting the gains, and initialized by setting values for the control output and integrator value. It is worth noting that the gains for the integral and derivative terms are based on the sample period, rather than on the one-minute interval that is typical of traditional PID controllers. This was done because, for the iterative improvement method to work properly, a change of the same magnitude in different gain parameters must result in a comparable effect on the controller response.
D. CHARACTERISTICS OF ITERATIVE LEARNINGFrom this point onward, the gains of the PID controller
are referred to as weights, taking advantage of the artificial neuron analogy presented in Section I. The task for the iterative learning procedure, then, is to somehow find the combination of weights that results in the lowest possible
26value of Fe. In order to clarify this, a PI controller (a PID controller with the derivative gain fixed at 0.0) is considered. Since a PI controller has only two weights, the space representing all possible weight combinations is a two- dimensional plane. Furthermore, if the performance of the controller is evaluated at each of the points in that plane, the resulting values could be plotted above the plane as a surface representing the value of Fe associated with each weight combination. The lowest point on this surface would then represent the best possible performance and would be found above the point in the plane representing the ideal weight combination.
In an attempt to find this ideal weight combination, the iterative learning algorithm progresses as follows:
1) Begins at some initial point (x,y) in the weight plane.
2) Evaluates the controller performance at the initial point in order to determine a baseline Fe value.
3) Investigates the "neighborhood" a fixed distance dfrom the initial point in each direction of each dimension by evaluating the controller's performance at the four points {(x+d,y) , , (x,y+d) ,( x,y-d)}. As soon as an Fe value is found that is less than the baseline, the point at which it occurred becomes the new initial point, the lower Fe value becomes the new baseline, and this step is repeated. If no improvement is found in the
27neighborhood, the low point has been found and the algorithm terminates.
If the Fe surface consists of several depressions, thisprocedure will find the one that is downhill from the initialpoint, not necessarily the global minimum. This makes theselection of the initial weights important. In making thatchoice, the following hypothesis was developed:
Hypothesis 1: There exists an iterative improvementalgorithm that, given initial weight values of zero or near zero, will converge on a weight set that, although it may not represent the global minimum evaluation function value, will result in desirable controller performance.
This research represents an attempt to experimentally test this hypothesis.
E. SIMPLE OFF-LIME ITERATIVE LEARNINGIterative learning was first done off-line in order to
develop baseline data which represented the goal weights for the on-line tests to follow. The distinction between off-line and on-line tests was in the controller evaluation. During off-line tests, the controller was evaluated over a fixed period of time with a certain step-change occurring at the beginning of the time period. During on-line iteration, the controller was tested over a fixed period of time, but step changes were allowed to occur randomly in the process. The following pseudo-code describes the off-line learning algorithm used:
Procedure: simple off-line iterative learninginput: initial weights;initialize controller;
28test controller, obtain initial Fe value; input: ch, amount of change per iteration; set a flag for each weight; while (any weight flag set)
for (each weight)increment weight by test controller, obtain if {Fel > Fe) then
decrement weight by ch; clear weight flag;
elseassign Fe = Fel;
if (weight flag not set) set weight flag; decrement weight by ch; test controller, obtain Pen if (Fel >Fe) then
increment weight by ch; clear weight flag;
elseassign Fe = FeI;
end for; end while;
The off-line iterative improvement algorithm was tested under the conditions shown in Table I. The results of these tests can be found in Section IV.
F. SIMPLE ON-LINE ITERATIVE LEARNINGThe on-line iterative learning tests were designed to
evaluate the iterative improvement algorithm's performance ina realistic control situation. The following pseudo-codedescribes the simple on-line learning algorithm used:
Procedure: simple on-line iterative learninginput: trial duration;input: initial weights;initialize controller;input: learning cycle time;input: ch, amount of change per iteration;initialize process;initialize Fe to max value;assign weight turn = max;clear change flag;while (duration not exceeded)
run random process and controller;
29accumulate Fel value; if (time for learning) then
if (Fel > Fe) and (change flag set) then decrement weight by ch; clear change flag;
elseif (ch < 0) then
rotate weight turn; assign ch = ch * -1.0; increment weight by ch; set change flag;
assign Fe = Fel; assign Fel = 0.0;
end while;The on-line iterative improvement algorithm increments and
Table I. Simple Off-line Iterative Improvement Tests
Controller Process Dead-Time Noise(sec.) (pet.)
PI Flow 0 0PID Flow 0 0 'jPI Flow 0 5PID Flow 0 5PI Flow 3 oPID Flow 3 0
PI Flow 3 5PID Flow 3 5PI Level 0 0PID Level 0 0
1 PI Level 0 5PID Level 0 5PI Level 3 0
PID Level 3 0PI Level 3 5PID Level 3 5PID Temperature 2 0 o II
30decrements each weight (by the change value in rotation, and the controller is tested after each change. If a change is undone, no further change is made until a new baseline Fe value has been obtained. The algorithm was tested under the conditions shown in Table II. The results of these tests can be found in Section IV.
Table II. Simple On-line Iterative Improvement Tests
Controller Process Dead-Time Noise(sec.) (pet.)
PI Flow 0 0PI Flow 0 5PI Flow 3 0PI Flow 3 5PI Level 0 0PI Level 0 5PI Level 3 0PI Level 3 5
G. PROPORTIONAL ON-LINE ITERATIVE LEARNINGThe previous algorithm was called simple on-line
iterative improvement because the magnitude of change (ch) made to the weights was input at the beginning of the trial and remained fixed throughout all iterations. As the tests progressed, however, it became apparent that this value would be process -dependent. Furthermore, it appeared that ch should start at some relatively high value to allow a coarser search of the weight space early in the learning process, and that it should be reduced as the Fe value decreases to allow a finer
31search as the learning process converges. This realizationled to the development of the proportional on-line iterativelearning algorithm that follows:
Procedure: proportional on-line iterative learninginput: trial duration;input: initial weights;initialize controller;input: It, learning cycle time;input: lr, learning rate;initialize ch, amount of change per iteration;initialize process;initialize Fe to max value;assign weight turn = max;clear change flag;while (duration not exceeded)
run random process and controller; accumulate Fel value; if (time for learning) then
if (Fel> Fe) and (change flag set) then decrement weight by ch; clear change flag;
elseassign ch = 0.9 + 0.1if {ch < 0) then
rotate weight turn; assign ch = -1.0; increment weight by ch; set change flag;
assign Fe = Fel; assign Fel = 0.0;
end while;The proportional on-line iterative improvement algorithm is identical to the simple algorithm except for the calculation that allows the change value (ch) to vary. The target value for ch is based on the learning rate (Ir) and on the current evaluation function value {Fel) . The evaluation function value is normalized by dividing it by the learning cycle time (It) , and the result is then multiplied by the learning rate (lr) to produce the new target value for ch. This value is not directly assigned to ch, however. Learning tests demonstrated that, due to the randomness of the process and the presence of
32process noise, considerable noise is present in the Fel value. In order minimize the effect of this noise, the smoothing function shown below was applied:
ch^c, (1.0-cJ-Ir-^ (7)
Where:ch0 = previous change value chj = new change value cm = momentum coefficient
This approach utilizes a portion of the previous value of as momentum in order to smooth the effect of the new target value [9] . The amount of momentum is controlled by the momentum coefficient (cm) . The learning rate (lr) proved to be process dependent, and its effect on the learning process is described in Section IV. The proportional on-line iterative learning algorithm was tested under the conditions shown in Table III. The results of these tests can be foundin Section IV.
Table III.Proportional On-line Iterative Improvement Tests
33
Controller Process Dead-Time (sec.)
Noise (pet.)
LearningRate
1 PI Flow 0 0 0.10PID Flow 0 0 0.10PI Flow 0 0 0.20PI Flow 0 5 0.05PID Flow 0 5 0.05PI Flow 0 5 0.10PI Flow 3 0 0.05PID 1 Flow 3 0 0.05PI Flow 3 0 0.10PID Flow 3 0 0.10PI Flow 3 5 0.05 1PID Flow 3 5 0.05PI Flow 3 5 0.10PID Flow 3 5 0.10
:| pi Level 0 0 2.00PID Level 0 0 i 4.00PID Level o 0 8.00PI Level 0 5 1.00PI Level 0 5 2.00PID Level o 5 4.00PID Level 0 5 8.00PI Level 3 0 1.00PI Level 3 0 2.00PID Level 3 0 8.00PI Level 3 5 2.00PID Level i 3 5 8.00PID Temperature 20 0 8.00
34IV. RESULTS AND DISCUSSION
A. EVALUATION FUNCTION PERFORMANCEIn order to judge evaluation function performance, off
line iteration tests were performed on the flow process with no noise and with dead-time values of zero and three seconds. The iteration was performed using each of the three candidate evaluation functions, and in each case it was allowed to continue until no further improvement in performance could be achieved. The results can be seen in Table IV. The
Table IV. Evaluation Function Performance Results
Process EvaluationFunction
RatioInitial/Final K ki K
Flow(DT=0)
AbsoluteError
15.70 0.47 0.67 0.00
Flow (DT=0)
ErrorSquared
12.12 0.45 0.67 0.01
Flow(DT=0)
QuadraticMinimization
3.98 0.55 0.47 0.01
Flow (DT=3)
AbsoluteError
3.32 0.27 0.14 0.26
Flow (DT=3)
ErrorSquared
3.32 0.29 0.16 0.40
Flow (DT=3)
QuadraticMinimization
2.28 0.32 0.13 0.32 1
performance of the controller with each set of final weights can be seen in Figures 10-12 for the case of zero dead-time and in Figures 13-15 for the case of three seconds dead-time.
Although the performance of all three evaluation functions was similar, the absolute error method resulted in
35
Figure 10. Controller Performance (Flow Process, DT=0, Absolute Error)
Figure 11. Controller Performance(Flow Process, DT=0, Error Squared)
36
Figure 12. Controller Performance (Flow Process, DT=0, Quadratic Minimization)
Figure 13. Controller Performance(Flow Process, DT=3, Absolute Error)
37
Figure 14. Controller Performance (Flow Process, DT=3, Error Squared)
Figure 15. Controller Performance(Flow Process, DT=3, Quadratic Minimization)
38
the best ratio of initial to final value, indicating that this method produced a steeper surface that would be easier to descend. For this reason, and because it was the least complicated to implement, the absolute error evaluation function was chosen for use in all further tests.
In order to illustrate the nature of the evaluation function surface, the value of the absolute error was plotted for a PI controller for each value of kp and k, over the range {0.10,1.00} with intervals of 0.05. This plot can be seen in Figure 16. The controller was evaluated on the simulated flow process with both noise and dead-time set to zero. Note that the apparent low point on the surface coincides with the point (0.45, 0.65) in the weight plane. The iterative improvement algorithm actually terminated at (0.47, 0.67), verifying this result.
B. ITERATIVE LEARNING PERFORMANCE CRITERIAIn order to judge the effectiveness of the iterative
learning techniques some measures of performance were developed. The most obvious was the relative change in the evaluation function value, since this was the primary objective of the algorithm. This measure was expressed as the ratio of the initial to the final evaluation function value. Next, and most importantly, the performance of the controller with the "learned" weights was evaluated by plotting the response of the controller to a process change similar to those simulated during the learning period. This response was
39then subjectively judged as excellent (Ex), good (Gd), fair (Fr) , or poor (Pr).
Figure 16. Surface Plot of Evaluation Function
C. SIMPLE OFF-LINE ITERATIONAs stated in Section III, the off-line iterative tests
were run primarily to provide a baseline for the on-line tests to follow, and the tests were run under the conditions shown in Table I. In each of these tests, the initial weights were kp=0.02, kt=0.02, and kd=0.0, and the magnitude of the weight
40change per iteration was fixed at 0.01. The results of these tests are shown in Table V, and the performance plots listed in the table (Fig.17 - Fig.25) follow this subsection.
These results present some interesting points. First, the off-line algorithm performed better when no noise was present than it did in the presence of noise. When noise was present the algorithm tended to terminate early, because the off-line algorithm terminates the first time that a cycle of
Table V. Simple Off-Line Iterative Improvement Results (DT = dead-time in seconds, NS = percent noise)
Conditions: Final Weights: Fe Perf. Perf.Process/DT/NS K K K Ratio Plot Rating
Flow/0/0 0.47 0.67 0.0* 15.58 none ExFlow/0/0 0.47 0.67 0.01 15.70 Fig.17 ExFlow/0/5 0.17 0.26 0.0* 5.61 none FrFlow/0/5 0.16 0.27 0.11 5.65 Fig.18 FrFlow/3/0 0.25 0.12 0.0* 2.85 none FrFlow/3/0 0.27 0.14 0.26 3.31 1 Fig.19 GdFlow/3/5 0.17 0.12 0.0* 2.69 none FrFlow/3/5 0.09 0.12 0.04 2.44 Fig.20 FrLevel/0/0 247.21 0.01 0.0* 41.89 none ExLevel/0/0 293.88 0.01 111.5 54.09 Fig.21 ExLevel/0/5 1.17 0.27 0.0* 1.09 none PrLevel/0/5 1.04 0.15 0.13 1.13 Fig.22 PrLevel/3/0 26.87 0.01 0.0* 7.27 none ExLevel/3/0 29.82 0.01 65.54 10.42 Fig.23 ExLevel/3/5 0.29 0.29 0.0* 1.05 none PrLevel/3/5 1.05 0.03 0.00 1.16 Fig.24 PrTemp./20/0 34.81 0.02 2021 13.34 Fig.25 Ex
* PI controller
41changes through all of the weights results in no improvement in performance, a condition which has a high probability of occurring in only a few iterations in the presence of process noise. It is for this reason that the on-line algorithms were designed to run indefinitely rather than to terminate arbitrarily.
The second observation is that the and kd gains behaved in exactly the manner that they are traditionally expected to. Conventional controller tuning wisdom states that, for a fast process with no capacity like the flow process, gains would generally be fairly low, the integral term would be higher than the proportional term, and derivative action would be unnecessary. These expectations are strongly supported by the data. Furthermore, a process with a longer response time and a large capacity like the level process would usually demand high proportional and derivative gains, while the integrating effect of the tank level would reduce the need for integral action in the controller. Again, the data shows that the proportional and derivative gains were indeed higher and that integral action was virtually eliminated when the level process was being controlled. Also, it is usually thought that the job of derivative action is to deal with process dead-time. Again, the data shows that for all three processes (without noise), the final derivative gain was drastically higher when deadtime was present. This was particularly true in the case of the temperature process, where the combination of long lags
42and 20 seconds of process dead-time resulted in a very high value of derivative gain.
Finally, an observation on derivative action is warranted. It has been traditionally thought that the presence of any derivative action in loops that "don't need it", like the flow loop simulated here, would only lead to control instability. For this reason, derivative action was usually shut off in these situations resulting in PI rather than PID control. The data does not support this belief. In every case but one, the PID controller outperformed the PI controller. This would suggest that the difficulty with derivative action lies not in the fact that it is inherently bad, but rather that manually tuning the derivative gain is very difficult.
43
Sample Periods (1 sec.)
Figure 17. Controller Performance (Simple Off-Line, PID, Flow/0/0)
Figure 18. Controller Performance(Simple Off-Line, PID, Flow/0/5)
44
Figure 19. Controller Performance (Simple Off-Line, PID, Flow/3/0)
45
Figure 22. Controller Performance(Simple Off-Line, PID, Level/0/5)
46
Figure 23. Controller Performance (Simple Off-Line, PID, Level/3/0)
Figure 24. Controller Performance(Simple Off-Line, PID, Level/3/5)
47
Sample Periods (1 sec.)
Figure 25. Controller Performance (Simple Off-Line, PID, Temp./20/0)
D. SIMPLE ON-LINE ITERATIONThe simple on-line iteration tests were run in the manner
described in Section III and under the conditions shown in Table II. Each test on the flow process was run for a simulated time of 60 hours, and learning was conducted on one- minute intervals. Each test on the level process was run for a simulated time of 300 hours, and learning was conducted on five minute intervals. For all of the tests the initial weights were set at kp= 0.02, 0.02, and kd=0.0, and the magnitude of the weight change per iteration was fixed at 0.01. The results of these tests are shown in Table VI, and the performance plots listed in the table (Fig.26 - Fig.33) follow this subsection.
Table VI. Simple On-Line Iterative Improvement Results (DT = dead-time in seconds, NS = percent noise)
48
Conditions: i Process/DT/NS
FinalK
Weights:K K Ratio
Perf. Plot
Perf. Rating
Flow/0/0 0.56 0.60 0.0* 10.64 Fig.26 ExFlow/0/5 0.42 0.30 0.0* 3.03 Fig.27 GdFlow/3/0 0.28 0.12 0.0* 2.45 Fig.28 FrFlow/3/5 0.42 0.08 0.0* 1.57 Fig.29 PrLevel/0/0 0.70 0.03 0.0* 4.98 Fig.30 PrLevel/0/5 0.09 0.11 0.0* 1.22 Fig.31 ! Pr 1Level/3/0 0.59 0.07 0.0* 3.19 Fig.32 PrLevel/3/5 0.34 0.06 0.0* 2.07 Fig.33 Pr
* PI controller
It became obvious during these tests that the simple iterative technique utilizing the fixed weight change per iteration would not achieve the high gains required for some processes. Once this was established, no further tests of this algorithm were performed. This failure was primarily due to the fact that the random nature of the process, as well as the presence of process noise, created a very noisy evaluation surface that was difficult to descend with small weight changes of constant size. For this reason the proportional algorithm, which allowed for a process-dependent learning rate to be selected and for weight changes to vary in size, was developed.
49
Figure 26. Controller Performance (Simple On-Line, PI, Flow/0/0)
Figure 27. Controller Performance(Simple On-Line, PI, Flow/0/5)
50
Sample Periods (1 sec.)
Figure 28. Controller Performance (Simple On-Line, PI, Flow/3/0)
Figure 29. Controller Performance(Simple On-Line, PI, Flow/3/5)
51
Figure 30. Controller Performance (Simple On-Line, PI, Level/0/0)
Figure 31. Controller Performance(Simple On-Line, PI, Level/0/5)
52
Figure 32. Controller Performance (Simple On-Line, PI, Level/3/0)
Figure 33. Controller Performance(Simple On-Line, PI, Level/3/5)
53E. PROPORTIONAL ON-LINE ITERATION
The proportional on-line iteration tests were run in the manner described in Section III and under the conditions shown in Table III. Each test on the flow process was run for a simulated time of 60 hours, and learning was conducted on one- minute intervals. Each test on the level process was run for a simulated time of 300 hours, and learning was conducted on five minute intervals. The test on the temperature process was run for a simulated time of 6000 hours and learning was conducted on 200 minute intervals. For all of the tests the initial weights were set at kp=0.02, k,=0.02, and kd=0.0, and the magnitude of the weight change per iteration was proportional to the evaluation function value, as described in Section III. The results of these tests are shown in Table VII, and the performance plots listed in the table (Fig.34 - Fig.42) follow this subsection.
For each case where the PID controller was used the algorithm learned a set of weights that resulted in at least good controller performance, except for the cases where both noise and dead-time were present. Those cases resulted in fair performance, and they represent the most difficult cases, i.e. situations where the evaluation surface had the least slope and the most superimposed noise.
The weights again behaved predictably, that is the characteristics of each of the simulated processes were reflected in the final learned weights with the relationships described in Subsection C. Also, although the presence of
54noise generally resulted in poorer final controller performance, on-line learning proceeded properly in noisy situations. Finally, the choice of learning rate for each process situation obviously affected the eventual performance of the algorithm, a fact that is explored further in the following subsection.
55Table VII.
Proportional On-Line Iterative Improvement Results (DT = dead-time, NS = noise, LR = learning rate)
Conditions: Proc./DT/NS/LR
FinalK
Weights: K kd
KRatioPerf. Plot
Perf. Rating
Flow/0/0/0.10 0.31 0.45 0.0* 5.50 none GdFlow/0/0/0.10 0.17 0.42 0.07 4.39 Fig.34 GdFlow/0/0/0.20 0.79 0.54 0.0* 11.03 none GdFlow/0/5/0.05 0.29 0.29 0.0* 3.67 none Gd |jFlow/0/5/0.05 0.15 0.26 0.03 4.04 Fig.35 GdFlow/0/5/0.10 0.07 0.45 0.0* 3.13 none GdFlow/3/0/0.05 0.52 0.12 0.0* 1.81 none PrFlow/3/0/0.05 0.53 0.01 0.24 1.59 none PrFlow/3/0/0.10 1.77 0.03 0.0* 0.80 none i PrFlow/3/0/0.10 0.28 0.13 0.38 2.53 Fig.36 GdFlow/3/5/0.05 0.49 0.09 0.0* 2.06 none PrFlow/3/5/0.05 0.10 0.01 1.47 1.04 none PrFlow/3/5/0.10 0.49 0.08 0.0* 1.69 none PrFlow/3/5/0.10 0.26 0.25 0.60 1.80 Fig.37 FrLevel/0/0/2.00 5.77 0.32 0.0* 9.14 none Gd 1Level/0/0/4.00 1.71 0.03 10.9 3.94 none FrLevel/0/0/8.00 9.53 0.02 38.5 7.87 Fig.38 GdLevel/0/5/1.00 2.94 0.02 0.0* 3.93 none FrLevel/0/5/2.00 9.97 0.03 0.0* 8.07 none GdLevel/0/5/4.00 6.67 0.02 22.2 4.44 none FrLevel/0/5/8.00 6.68 2.39 22.3 7.16 Fig.39 GdLevel/3/0/1.00 6.44 0.02 0.0* 7.77 none GdLevel/3/0/2.00 3.31 5.27 0.0* 1.03 none PrLevel/3/0/8.00 10.28 0.67 1.95 20.40 Fig.40 GdLevel/3/5/2.00 11.24 0.06 0.0* 7.98 none GdLevel/3/5/8.00 8.17 0.07 31.9 4.19 Fig.41 FrTemp/20/0/8.00 47.92 0.02 1201 776.7 Fig.42 Gd
* PI controller
56
Figure 34. Controller Performance (Proportional On-Line, PID, Flow/0/0/0.10)
57
Figure 36. Controller Performance (Proportional On-Line, PID, Flow/3/0/0.10)
Figure 37. Controller Performance(Proportional On-Line, PID, Flow/3/5/0.10)
58
Figure 38. Controller Performance (Proportional On-Line, PID, Level/0/0/8.00)
Figure 39. Controller Performance(Proportional On-Line, PID, Level/0/5/8.00)
59
Figure 40. Controller Performance (Proportional On-Line, PID, Level/3/0/8.00)
Figure 41. Controller Performance(Proportional On-Line, PID, Level/3/5/8.00)
60
Figure 42. Controller Performance (Proportional On-Line, PID, Temp./20/0/8.00)
F. PROPORTIONAL LEARNING PERFORMANCETwo factors in the proportional on-line learning
algorithm were adjusted in order to influence learning performance, learning cycle time and learning rate. The learning cycle time defined the time that the algorithm gathered information after a weight change in order to assess the effect of that weight change. It appears that this should be at least two to three times the overall response time of the process. In a real process this response time can usually be measured with a simple step response test, and this was the method used to determine learning cycle times for the various proportional algorithm trials. If process response time cannot be directly determined, however, a longer learning
61cycle time appears to be the safer choice. The longer learning cycle time will slow down the learning process, however, since weight changes are made only once per learning cycle.
The learning rate also had an important effect on the algorithm's performance. If the learning rate was too low, little or no change in the weights was observed during the trial, while a learning rate that was too high often resulted in the algorithm bypassing the optimum weight set in favor of a less desirable set of higher weights. The optimum learning rate is related to the slope of the evaluation surface. A surface with a steep slope requires a lower learning rate, while a surface with very little slope requires a higher learning rate. The learning rates used for the proportional algorithm trials were determined using this fact, and through trial and error.
Finally, no investigation of a learning algorithm would be complete without some evaluation of convergence. In this context, convergence is defined as the ability of the algorithm to iteratively change the weights in such a way as to result in the continual reduction of the evaluation function value until some lower bound of that value is approached. In order to observe the convergence properties of the proportional iterative improvement algorithm, the evaluation function value was plotted during the learning process for all four variations of the simulated flow process, with the PID controller. For the case where neither noise nor
62dead-time were present this plot can be seen in Figure 43. For the case with only noise (5%) present, the plot can be seen in Figure 44. For the case with only dead-time (3 sec.) present, the plot can be seen in Figure 45, and for the case where both noise (5%) and dead-time (3 sec.) were present the plot can be seen in Figure 46. For the first three cases, the evaluation function value descends rather rapidly and appears to asymptotically approach some minimum. Each of these cases generated final weight sets that resulted in controller performance that was rated good or better, as was shown in the previous subsection. In the fourth case the evaluation function value was reduced, but not in as deliberate or direct a fashion. Also, this case generated weights which resulted in only fair controller performance, as shown in the previous subsection. These results suggest that when the algorithm converges, the resulting weights generate desirable controller characteristics, and when the algorithm does not converge well the resulting weights do not perform well. From this it directly follows that the iterative learning procedure performs only marginally well when both dead-time and noise are present in significant quantities in the controlledprocess.
63
Learning Time (min.)
Figure 43. Learning Performance (Dead-Time = 0 sec., Noise = 0%)
2018
300 600 900 1 2 0 0 1 5 0 0 1 8 0 0 2 1 0 0 2 4 0 0 2 7 0 0 3 0 0 0 3 3 0 0 3 6 0 0 Learning Time (min.)
Figure 44. Learning Performance(Dead-Time = 0 sec., Noise = 5%)
64
Learning Time (min.)
Figure 46. Learning Performance(Dead-Time = 3 sec., Noise = 5%)
65V. CONCLUSIONS AND RECOMMENDATIONS FOR FUTURE RESEARCH
Three algorithms were developed for the iterative improvement of PID controller gains, and the performance of each of the algorithms was experimentally evaluated under realistically simulated process control conditions. The offline algorithm was used as a baseline case and did not perform well in the presence of process noise. The simple on-line algorithm was found to be limited due to the fixed nature of the weight changes. The final algorithm, proportional on-line iterative improvement, is relatively uncomplicated, and as such should be easily understood by control engineers. Also, it's complexity or memory requirements do not preclude its application in small microprocessor based single-loop and multiple-loop controllers. This algorithm performed well under a wide variety of simulated process conditions, as it almost always found a set of weights that resulted in good controller performance. The algorithm performed only marginally well, however, on processes where both dead-time and noise were present in significant quantities. The algorithm is capable of producing on-line continuous improvement of PID controller performance.
As this research represents the original development of this algorithm, much remains to be investigated. Learning times were often long, and modifications could be made to the algorithm in order to improve this facet of its performance. Also, the exact convergence properties of the learning process
66need to be quantified, including the identification of specific situations under which the algorithm will or will not converge. Although the algorithm was tested on realistically simulated processes, those processes were fairly simple. Testing on more complex processes would be necessary in order to determine all of the algorithm's limitations. Finally, the evaluation of the algorithm in a digital PID controller controlling a real industrial process would be the ultimate test of its usefulness.
67APPENDIX
C++ Program Listing
//---------------------------------------------- ----------------------// CS-490 Research Student: Ryan Rosandich// Master program - simulates processes and controllers.// Conducts performance tests, off-line iteration and on-line // learning./ / ----- ----------- ---------- --------------------------- ------#include <iostream.h>#include <iomanip.h>#include <conio.h>#include <fstream.h> ftinclude <math.h> ftinclude <stdlib.h>ofstream outfile;double d t [21]; // global deadtime arrayint proc, dead, pctn; // global process characteristics//— !==============================:===—class valve{
double pos; // valve positiondouble kg; // process gaindouble tf; // fixed time constantdouble tvr; // variable time constant ratiodouble fp; // flow percentage outpublic:
// constructorvalve() {kg=1.50; tf=1.0; tvr=0.50; fp=0.0; pos=0.0;}void init(double co) {pos=co; fp=pos*kg;} // initialize
flow outdouble update(double co); // returns flow pet
};// class valve member functionsdouble valve::update(double co)
{double tv; pos=co;tv= (fp/100.0)*tvr;fp=fp + (kg*pos - fp)*(1.0 - exp(-1.0/(tf + tv))); return fp;
//=— ===== ===== ------- ----------------- ---===..........class tank{
double size; double level; public:
tank() {size=240; level=0;} // constructor
68void init(double 1) {level=l;} // initialize tank level double update(double fi, double fo) ; // returns level pet
} i// class tank member functionsdouble tank::update(double fi, double fo)
level=level + (fi - fo)/size; if (level < 0)
level=0.0; if (level > 100)
level=100.0; return level;
class heater{double qint; // internal energydouble tf; // fixed time constantdouble tmp; // temperature at transmitterdouble fp; // flow percentage out of heaterpublic:
// constructorheater() {qint=0.0; tf=1500.0; tmp=50.0; fp=0.0;} void init(double t) {tmp=t; qint=(t-50.0)*200.0;} //
initialize heater tempdouble update(double co, double fo); // returns temp pet
};// class heater member functionsdouble heater::update(double co,double fo)
double tint,qout; fp=fo;qout=(fp*0.832*(tmp-50.0))/60.0 + (tmp-50.0)*0.15;qint=qint-qout+(co*2.0);tint=50.0 + (qint/200.0) ;tmp=tmp + (tint-tmp)* (1.0-exp(-1.0/tf));return tmp-50.0;
//=..............................................................class controller{double wt[3]; // weights static double iv; // integrator value double co; // controller output public:
controller (void) {wt[0]=0; wt[l]=0; wt[2]=0; iv=0; co=0;} // constructor
void init(double aco);// NOTE: should tune before initializing due to dependence
on weightsvoid tune(double *awt) {wt[0]=awt[0]; w t [1]=awt[1] ;
wt [2] =awt [2] ;}double update(double er, double der); // returns co pet
}»// class controller member functions
69void controller::init(double aco)
{co=aco;iv=-l.O*co/wt[1];}double controller::update(double er, double der) iv=iv + er;co=-l.0*(er*wt[0] + iv*wt[l] + der*wt[2]); if (co < 0.0)
co=0.0;if (co >100.0)
co=100.0; return co;
class process{protected:
int dt; // dead time seconds int pn; // percent noise
public:process(int d, int p) {dt=d; pn=p;} // constructor
class flow : public process{ double pv; // flow out pet valve vl;public:
flow(int d, int p) : process(d,p) {pv=0.0;} // constructor void init(double co); // initialize process double update(double co); // returns flow out pet
};// class flow member functionsvoid flow::init(double co)
{vl.init(co); pv=vl.update(co);}double flow::update(double co)pv=vl.update(co); return pv;
class level : public process{ double pv; // level pet valve vl; tank tl; public:
level (int d, int p) : process (d,p) {pv=0.0;} //constructor
void init(double co, double apv); // initialize process double update(double co, double fo); // returns level pet
};
70// class level member functionsvoid level::init(double co, double apv)
{pv=apv;tl.init(pv);vl.init(co) ;}double level::update(double co, double fo)double fi;fi=vl.update(co); pv=tl.update(fi,fo) ; return pv;
/ / ™ - ----------- ----- --------- — -----=====----------------class temp : public process{ double pv; // temp pet heater hi; public:
temp(int d, int p) : process(d,p) {pv=0.0;} // constructorvoid init(double t); // initialize processdouble update(double co,double fo); // returns temp out
pet};// class temp member functions
void temp::init(double t)hi.init(t); pv=t-50.0;}double temp::update(double co,double fo)pv=hl.update(co,fo); return pv;
// global function prototypesdouble rotate (double); void performance(void); void surface(void); void off_line_iter(void);int initialize (flow &fl, level &11, temp &tl, double &fo, double &pv, double &pvd);double test(controller &cl, flow &fl, level &11, temp &tl) ;void on_line_learning(void);void open_loop(void);double noise(int);void dead_time(double);// main functionmain()
int choice; char name [20];choice=l; while (choice)
{clrscr () ;cout<<"Procedure choices:\n\n"; cout«" O..QUIT\n";cout«" 1..Generate a performance plot\n" cout<<" 2..Generate a surface plot\n"; cout<<" 3..Perform off-line iteration\n"; cout<<" 4..Perform on-line learning\n"; cout<<" 5..Generate open-loop response\n" cout<<"\nEnter a choice (0-5): "; cin>>choice;if (choice)
{cout<<"Process choices:\n\n"; cout«" l..Plow process\n"; cout«" 2.. Level Process\n" ; cout<<" 3..Temperature Process\n"; cout<<"\nEnter a choice (1-3): "; cin>>proc;if (proc < 1) proc=l; if (proc > 3) proc=3;cout<<"\nEnter dead-time (0-20 seconds): cin»dead;cout<<"\nEnter noise (percent): "; cin>>pctn;cout<<"\nEnter output filename.ext: ";cin»name;outfile.open(name);
switch(choice){case 0: break;case 1: performance() ;
outfile.close();cout<<"\npress any key to continue.getch();break;
case 2: surface();outfile.close(); break;
case 3: off_line_iter(); outfile.close();cout<<"\npress any key to continue.
getch(); break;
72
case 4: on_line_learning(); outfile.close(); break;
case 5: open_loop(); outfile.close(); break;
default;break;} // end switch
} // end while choice } II end main
// flinetion definitions// ================--- _ _ _ _ _double rotate (double t)
{if (++t>2) return 0;
elsereturn t;
void dead_time(double pv)int i;for (i=19; i>=0; i--)
dt[i+l]=dt ti]; dt [0] =pv;
//}---------------- ------double noise(int petn)
double n;n = ((double)(petn*(random(lOl)-50)))/100.0; return n;
void performance(void)double etime,pv,pvd,co,er,der,sp,etotal,fo;double wt[3];int i,t,tmax,tout;controller cl; flow fl(dead, petn); level 11(dead, petn); temp tl(dead, petn);
73randomize();cout <<"Enter kp, ki, kd: cin >>wt [0] >>wt [1] >>wt [2] ; cl.tune(wt); switch(proc)
case 1: // initialize flow processcout <<"Enter new setpoint (25/75): "jcin >>sp;tmax=2 0;tout=l;if (sp==25.0)
pv=75.0;cl.init(50.0) ;f1.init(50.0);}else{sp=75.0; pv=25.0; cl.init(16.6); f1.init(16.6);}break;
case 2: // initialize level processcout <<"Enter new flow out (25/75): ",cin >>fo;tmax=200;tout=5;sp=75.0;pv=75.0;if (f0==25.0)
{cl.init(50.0);11.init(50.0,75.0);}else{f0=75.0; cl.init(16.6);11.init(16.6,75.0);
break;case 3: // initialize temp process
fo=25.0; tmax=1200; tout=30; sp=75.0 pv=75.0]Cl.init(44.6) ; tl.init(125.0) ; break;
74} // end switch
pvd=pv;for (i=0; i<21; i++)
dt[i]=pvd + noise(petn); // initialize deadtime array er=etime=etotal=0.0;for (t=0;t<=tmax;t++)
{der=(pvd-sp)-er; er=pvd-sp;cout<<setw(9)<<t<<setw(12)<<co<<setw(12)<<pvd<<"\n"; co=cl.update(er,der);if ((t % tout)==0) // write to file during trial
outfile << setw(9)<<t<<setw(12)<<co <<setw (12) <<pvd«"\n";
switch(proc)case 1: pv=fl.update(co) ;
break;case 2: pv=ll.update(co,fo);
break;case 3: pv=tl.update(co,fo);
break;} // end switch
dead_time(pv + noise(petn)); pvd=dt[dead]; etime++;etotal=etotal + fabs(er/100.0);} // end for t
cout <<"\n"<<setw(12)<<wt[0]<<setw(12)<<wt[1]<< setw(12) << etotal <<"\n";
int initialize (flow &fl, level &11, temp &tl, double &sp, double &fo, double &pv, double &pvd)
int i,tmax;switch(proc)
case 1: // initialize flow process tmax=20; fo=0.0; sp=75.0; f1.init(16.6); pv=25.0; break;2: // initialize level process tmax=100; pv=75.0; f0=25.0; sp=75.0;11.init(50.0,75.0) ; break;
case
75case 3: // initialize temp process
tmax= 8000 ; fo=25.0; sp=25.0; pv=25.0; tl.init(75.0); break; end switch0; i<21; i++)=pvd + noise(petn); // initialize deadtime array tmax;
} // pvd=pv;for (i=
dt [i] return}
//'void open_loop(void)double pv,pvd,co2,fo,sp; int i,t,tmax; flow f1 (dead, petn); level 11(dead, petn); temp tl(dead, petn);randomize();tmax=initialize(f1,11,tl,sp,fo,pv,pvd); switch(proc)
case 1: co2=50.0; break;
case 2: co2=50.0 break;
case 3: co2=18.0; break;
} // end switch for (t=0;t<=tmax;t++)
{cout<<setw(9)<<t<<setw(12)<<co2<<setw(12)<<pvd<<"\n"; if ((t % (tmax/40))==0) // write to file 40 times during
trialoutfile « setw(9)<<t<<setw(12)<<co2
<<setw(12) <<pvd«"\n" ; switch(proc)
case 1: pv=f1.update(co2) ; break;
case 2: pv=ll.update(co2,fo); break;
case 3: pv=tl.update(co2, fo) ; break;
} // end switch dead_time(pv + noise(petn)); pvd=dt[dead];} // end for t
void surface(void){double etotal,wt[3],wtOO,wtOstep,wtlO,wtlstep;int p ,i;controller cl;flow fl(dead, petn);level 11(dead, petn);temp tl(dead, petn);randomize() ;w t [2]=0.0;cout<<"\nEnter initial kp: cin>>wt00;cout<<"Enter kp step size: "; cin>>wtOstep;cout<<"\nEnter initial ki: "; cin>>wtl0;cout<<"Enter ki step size: "; cin>>wtlstep;for (p=l;p<21;p++)
for (i=l;i<21;i++){// initial conditions wt[0]=wt00 + (double)p*wtOstep; w t [1]=wtlO + (double)i*wtlstep; cl.tune(wt); cl.init(16.6); etotal=test(cl,fl,11,tl);
cout <<setw(9)<<wt[0]<<setw(9)<<wt[1] <<setw(12)<<etotal<<"\nn;
outfile <<setw(12)<<etotal;} // end for i outfile <<"\n";
} IIend for p
void off line iter(void){ “ “double etime,wt[3],etotal,etemp,einit,change;int kpflag,kiflag,kdflag;controller cl;flow fl(dead, petn);level 11(dead, petn);temp tl(dead, petn);clrscr(); etotal=1000000.0; cout<<"Enter initial kp, ki, kd: cin>>wt[0]>>wt[1]>>wt[2] ; cl.tune(wt);
cl.init(16.6); einit=test(cl,fl,11,tl);cout<<"Enter change per iteration cin >> change;kpflag=kiflag=kdflag=l; while (kpflag||kiflag||kdflag)
kpflag=kiflag=kdflag=l;// increment and test kp w t [0]+=change; cl.tune(wt); cl.init(16.6); etemp=test(cl,f1,11,tl);if (etemp > etotal)
w t [0]-=change; cl.tune(wt); kpflag=0;
elseetotal=etemp;
if ((!kpflag)&&(wt[0]>=change)) kpflag=l;// decrement and test kp w t [0]-=change; cl.tune(wt); cl.init(16.6); etemp=test(cl,fl,11,tl); if (etemp > etotal)
w t [0]+=change; cl.tune(wt); kpflag=0;else
etotal=etemp;
// increment and test ki wt[1]+=change; cl.tune(wt); cl.init(16.6); etemp=test(cl,f1,11,tl); if (etemp > etotal)wt[1]-=change; cl.tune(wt); kiflag=0;
78}elseetotal=etemp;
if ((!kiflag)&&(wt[1]>=change)) kiflag=l;// decrement and test ki w t [1]-=change; cl.tune(wt); cl.init(16.6); etemp=test(cl,fl,11,tl); if (etemp > etotal)
wt [1]+=change; cl.tune(wt); kiflag=0;}elseetotal=etemp;
// increment and test kd wt [2]+=change; cl.tune(wt); cl.init(16.6); etemp=test(cl,f1,11,tl); if (etemp > etotal)
wt [2]-=change; cl.tune(wt); kdflag=0;}elseetotal=etemp;
if ((!kdflag)&&(wt[2]>=change)) kdflag=l;// decrement and test kd wt [2]-=change; cl.tune(wt); cl.init(16.6); etemp=test(cl,fl,11,tl); if (etemp > etotal)
wt [2]+=change; cl.tune(wt); kdflag=0;}elseetotal=etemp;
79cout <<setw(9) <<wt [0] <<setw(9) «wt [1]
<<setw(9)<<wt[2]<<setw(16)<<etotal<<"\n"; } // end while
cout <<"\nlnitial error: "<<setw(12)<<einit<<" Final error: "<<setw(12)<<etotal<<endl;
double test(controller &cl, flow &fl, level &11, temp &tl){double etime,pv,pvd,co,er,der,sp,etotal, fo; int t,tmax;fo=0.0;tmax=initialize(fl,11,tl,sp,fo,pv,pvd); sp=75.0;etime=er=etotal=0.0;for (t=0;t<=tmax;t++)
{der=(pvd-sp)-er; er=pvd-sp;co=cl.update(er,der); switch(proc)
case 1: pv=fl.update(co); break;
case 2: pv=ll.update(co,fo); break;
case 3: pv=tl.update(co,fo) ; break;
} // end switch dead_time(pv + noise(petn)); pvd=dt[dead]; etime++;etotal=etotal + fabs(er/100.0);} // end for t
return etotal;
void on_line_learning(void)d o u b l e
pv,pvd,co,er,der,change,chval,sp,ernew,erold,fo,Irate; double wt[3];int t,tmax,trialdur,turn,chmade,mind,maxd,tlearn;long etime,dur;controller cl;flow fl(dead, petn);level 11(dead, petn);temp tl(dead, petn);cout<<"\nEnter the total duration in hours: ";cin >> dur;dur*=3600; // convert to seconds
80cout<<"\nEnter initial weights kp, ki, kd: ";cin >>wt[0]>>wt[1]>>wt[2];cl.tune(wt);chval=0.01;change= -1.0;cout<<"\nEnter the minimum trial duration in seconds: "; cin>>mind;cout<<"\nEnter the maximum trial duration in seconds: "; cin>>maxd;cout<<"\nEnter the learning cycle time in minutes: cin>>tlearn;tlearn*=60; // convert to seconds cout<<"\nEnter the learning rate: "; cin>>lrate; randomize();// initial conditions cl.init(16.6);tmax=initialize(f1,11,tl,sp,fo,pv,pvd);tmax+=0; // dummy stetementetime=er=ernew=0.0;erold=1000000.0;turn=l;chmade=0;while (etime < dur)
{trialdur=mind + random(maxd-mind+1); for (t=0;t<=trialdur;t++)
{der=(pvd-sp)-er; er=pvd-sp;co=cl.update(er,der); switch(proc)
case 1: pv=fl.update(co) ; break;
case 2: pv=ll.update(co,fo); break;
case 3: pv=tl.update(co,fo); break;
}dead_time(pv + noise(petn));pvd=dt[dead];etime++;ernew=ernew + fabs(er/100.0);
{if (etime % tlearn == 0) // do learning
if ((ernew > erold)&&chmade) // undo the last changewt[turn]-=change*chval; cl.tune(wt); chmade=0;
81}else // make a new change if within bounds{chval=0.90*chval + 0.10*(lrate*ernew/tlearn); //update
change valueif ((change > 0)&&(wt[turn] < change*chval)) // avoid
negative weightsturn=rotate(turn);
else {if (change < 0) turn=rotate(turn); change*= -1.0;}wt[turn]+=change*chval;
cl.tune(wt); chmade=l;} // end else
// output to screen every learning cycle cout <<setw(8)<<etime/60<<setw(12)<<wt[0]
<<setw(12)<<wt[1]<<setw(12)<<wt[2]<<setw(12)<<ernew<<setw(12)«chval<<"\n";
// write to file every 10 learning cycles if (etime%(10*tlearn)==0)
outfile <<setw(8)<<etime/60<<setw(12)<<wt[0] <<setw(12)<<wt[1]<<setw(12)<<wt[2]<<setw(12) «e r n e w « " \ n " ;
erold=ernew; ernew=0.0;} // end learning } // end for t
switch(proc)case 1: // flow process swap
if (sp==25.0) sp=75.0;else
sp=25.0; break;
case 2; // level process swap if (fo==25.0)
fo=75.0;else
fo=25.0; break;
case 3: // temp process swap if (fo==25.0)
fo=75.0;else
fo=25.0; break;
} // end switch } IIend while
// end program
82BIBLIOGRAPHY
[1] Astrom, K. J. , "Toward Intelligent Control," ControlSystems Magazine (April, 1989): 60-64.
[2] Shinskey, F.G., Process Control Systems. 2nd ed. McGraw-Hill, 1979.
[3] Cadzow, J.A. and H.R. Martens, Discrete-Time and Computer Control Systems. Prentice-Hall, 1970.
[4] Ziegler, J.G. and N.B. Nichols, "Optimum Settings for Automatic Controllers," Transactions of the (November, 1942): 759-768.
[5] Astrom, K.J., "Where is the Intelligence in Intelligent Control?" IEEE Control Systems Magazine (January, 1991): 37-39.
[6] Rumelhart, D.E. and J.L. McClelland, Parallel DistributedProcessing. Cambridge, MA: MIT Press, 1986.
[7] Wasserman, P.D., Neural Computing: Theory and Practice. New York: Van Norstrand Reinhold, 1989.
[8] Freeman, J.A. and D.M. Skapura, Neural Networks: Algorithms, Applications, and Programming Techniques. Addison-Wesley, 1991.
[9] Simpson, P.K., Artificial Neural Systems: Foundations, Paradigms, Applications,and Implementations. Pergamon Press, 1990.
[10] Ichikawa, Y. and T. Sawa, "Neural Network Application for Direct Feedback Controllers, " IEEE Transactions on Neural Networks, Vol. 3, No. 2 (March, 1992): 224-231.
[11] van Laarhoven, P.J.M. and E.H.L. Aarts, Simulated Annealing: Theory and Applications. Dordrecht, Holland: D. Reidel Publishing, 1988.
[12] Miller, T.W., R.S.Sutton, and P.J. Werbos, Neural Networks for Control. Cambridge, MA: MIT Press, 1990.
[13] Khalid, M. and S. Omatu, "A Neural Network Controller for a Temperature Control System," IEEE Control Systems Magazine (July, 1992): 58-64.
[14] Barto, A.G., R.S. Sutton, and C.W. Anderson, "Neuronlike Adaptive Elements Solve Difficult Learning Control Problems," IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-13, No. 5 (1983): 834-846.
83[15] Anderson, C.W., "Learning to Control an Inverted Pendulum
Using Neural Networks," IEE Control Systems Magazine (April, 1989): 31-37.
[16] Husain, A., Chemical Process Simulation. Halstad Press, 1986.
[17] Iiguni. Y., H. Sakai, and H. Tokumaru, "A Nonlinear Regulator Design Using Multilayered Neural Networks," IEEE Transactions on Neural Networks, Vol. 2, No. 4 (July, 1991): 410-417.
[18] Kooi, S.B.L., and K. Khorassni, "Control of a Woodchip Refiner Using Neural Networks," TAPPI Journal, Vol.75, No. 6 (June, 1992): 156-162.
[19] Kuperstien, M. and J. Rubenstien, "Implementation of an Adaptive Neural Controller for Sensory Motor Control," IEEE Control Systems Magazine (April, 1989): 44-48.
[20] Nguyen, D.H., and B. Widrow, "Neural Networks for Self Learning Control Systems," IEEE Control Systems Magazine (April, 1990): 18-23.
[21] Palay, A.J., Searching With Probabilities. Marshfield, MA: Pitman Publishing, 1985.
[22] Pearl, J., Heuristics: Intelligent Search Strategies for Computer Problem Solving. Addis on-Wes ley, 1984.
[23] Psaltis, D., A. Sideris, and A. Yamamura, "A Multilayered Neural Network Controller," IEEE Control Systems Magazine (April, 1989): 44-48.
[24] Rabelo, L.C. and J.R. Avula, "Hierarchical Neurocontroller Architecture for Robotic Manipulation," IEEE Control Systems Magazine (April, 1992): 37-41.
[25] Saerens, M. and A. Soquet, "A Neurocontroller Based on the Backpropagation Algorithm," Proceedings of the First IEE International Conference on Artificial Neural Networks, London, (1989): 211-215.
[26] Wilde, D.J., Optimum Seeking Methods. Prentice-Hall, 1964.
