IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5 ... · 2540 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5, SEPTEMBER 2014 Fig. 1. Four-machine–two-area benchmark system

IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5, SEPTEMBER 2014 2539

Energy-Storage-Based Low-Frequency OscillationDamping Control Using Particle Swarm Optimization

and Heuristic Dynamic ProgrammingXianchao Sui, Yufei Tang, Student Member, IEEE, Haibo He, Senior Member, IEEE, and Jinyu Wen, Member, IEEE

Abstract—Low-frequency oscillation is one of the main barrierslimiting power transmission between two connected power sys-tems. Although power system stabilizers (PSSs) have been provedto be effective in damping inner-area oscillation, inter-area oscil-lation still remains a critical challenge in today’s power systems.Since the low-frequency oscillation between two connected powersystems is active power oscillation, power modulation throughenergy storage devices (ESDs) can be an efficient and effectiveway to maintain such power system stability. In this paper, weinvestigate the integration of a new goal representation heuristicdynamic programming (GrHDP) algorithm to adaptively controlESD to damp inter-area oscillation. A particle swarm opti-mization (PSO)-based power oscillation damper (POD) has alsobeen proposed for comparison. Various simulation studies withresidue-based POD controller design, the proposed PSO optimizedcontroller design, and the GrHDP-based controller design overa four-machine-two-area benchmark power system with energystorage device have been conducted. Simulation results havedemonstrated the efficiency and effectiveness of the GrHDP-basedapproach for inter-area oscillation damping in a wide range ofsystem operating conditions.

Index Terms—Energy storage device (ESD), goal representationheuristic dynamic programming (GrHDP), particle swarm opti-mization (PSO), power oscillation damper (POD), power systemstability.

I. INTRODUCTION

T HE August 2003 blackout in the northeast United Statesand the July 2012 India blackout that affected over 620

million people are two of the widely publicized examples inwhich power outages affected many millions of users. From a

Manuscript received October 10, 2013; revised January 31, 2014; acceptedFebruary 03, 2014. Date of publication March 03, 2014; date of current ver-sion August 15, 2014. This work was supported in part by the National Sci-ence Foundation (NSF) under Grant ECCS 1053717, the Army Research Of-fice under Grant W911NF-12-1-0378, NSF-DFG Collaborative Research on“Autonomous Learning” (a supplement grant to CNS 1117314), and the Na-tional Natural Science Foundation of China under Grant 51228701. Paper no.TPWRS-01299-2013.X. Sui is with Dalian Power Supply Company, Dalian 116033, China. (e-mail:

[email protected]).Y. Tang and H. He are with the Department of Electrical, Computer and

Biomedical Engineering, University of Rhode Island, Kingston, RI 02881 USA(e-mail: [email protected]; [email protected]).J. Wen is with the College of Electrical, Electronic and Engineering,

Huazhong University of Science and Technology, Wuhan 430074, China.(eemail: [email protected]).Color versions of one or more of the figures in this paper are available online

at http://ieeexplore.ieee.org.Digital Object Identifier 10.1109/TPWRS.2014.2305977

broader perspective, such power outage events underscore thecomplex issues associated with the generation and use of elec-tricity: the reliability of the grid, the increased deployment of re-newable energy, and the development of electric vehicles (EVs)to decrease dependence on traditional resources [1]. Among theefforts to address these problems, recent development of en-ergy storage devices (ESDs) offers a well-established approachto improve grid reliability and utilization. While the transmis-sion and distribution systems are responsible for moving elec-tricity over distances to end users, an ESD system involves atime dimension to provide electricity when it is needed and in-crease the power system operation and control margin. A re-cent EPRI study identified a number of high-value opportunitiesfor energy storage, including wholesale energy services, inte-gration of renewables, commercial and industrial power qualityand reliability, transportable systems for transmission and distri-bution, and grid management [2]. In this paper, we focus on theESD-based controller design for power system damping control.In traditional power system stability controller design, such

as power system stabilizers (PSSs), a linearized power systemmodel near the operating point is used [3]. However, we need torelax this assumption as modern power systems become moreand more nonlinear, time-variant, and uncertain with the con-tinuously increased deployment of flexible alternating currenttransmission system (FACTS), renewable energy, and EVs. Assystem state parameters and operating conditions are changing,power system modeling becomes a very complex and time-consuming task for the electrical engineers and operators. Insuch a situation, two major drawbacks of the traditional con-trol methods are the lack of robustness and online learning ca-pability. Meanwhile, as an inherent phenomenon, inter-area os-cillation in connected power systems is mainly due to the dy-namic power imbalance between synchronous machines causedby disturbances, and, for most of the cases, this imbalance be-haves as low-frequency oscillation (0.1 Hz to 0.8 Hz). In recentyears, high-voltage direct current (HVDC) devices and FACTShave been adopted for inter-area oscillation damping control [4].However, PSSs are still the first choice for the suppression oflow-frequency oscillation. A PSS provides supplementary con-trol signal to an excitation system of synchronous machines,and this supplementary control signal is generated using localmeasurements, which limits its effectiveness for system-widedamping control [5].Meanwhile, ESDs hold the advantage of providing flexible

active or reactive power to the power grid to compensate for

0885-8950 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

2540 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5, SEPTEMBER 2014

Fig. 1. Four-machine–two-area benchmark system with energy storage devices.

the power imbalance caused by disturbances, which could be apowerful tool in power system stability control [6]–[8]. Previousstudies have shown that the flywheel energy storage systemusing independent active and reactive power decoupling con-trol strategy can effective suppress the low-frequency oscilla-tion in the system [9]. In [10], an energy-storage-based dampingcontroller (ESDC) considering anti-windup to improve the sat-uration-dependent stability has been proposed. The anti-windupfeedback loop is augmented to the ESDC, along with modelreduction technique and linear matrix inequality (LMI) tech-nique design, to improve the system damping under both normaland saturation operating conditions. Many other ESDs applica-tions, such as transient stability enhancement by fuzzy logic-controlled super-conducting magnetic energy storage (SMES)[11], inter-area oscillation damping by unified power flow con-trollers using ultra-capacitors [12], and wind farm fluctuationsmitigation by a battery energy storage system [13] have beenintensively investigated in the society.Inspired by the aforementioned discussions, in this paper, a

real-time wide-area control framework using the reinforcementlearning (RL) technique has been used to design an ESD-baseddamping controller, which can provide effective control to in-crease the power system stability margin. The main contribu-tions of this paper are summarized as follows.• A particle swarm optimization (PSO)-based power oscilla-tion damper (POD) for ESD control has been investigatedin this paper. The PSO algorithm has been employed totune the control parameters in POD using a time-domainsimulation mechanism.

• A new heuristic dynamic programming (HDP) algo-rithm, namely goal representation heuristic dynamicprogramming (GrHDP), has been introduced in this paperto adaptively control the ESD in a real-time manner.Different from the classical HDP algorithm with twonetworks (i.e., action network and critic network), theGrHDP introduced an additional network (i.e., goal net-work) to provided adaptive internal goal signal to facilitatethe learning ability. Under this GrHDP framework, the

ESD can interact with the benchmark system and learn toadaptively adjust its active power output to damp systemoscillations.

• Comparative studies of GrHDP, PSO, and residue methodhave been performed under three different scenarios. Otherissues, such as real-time data acquisition using phase mea-surement units (PMUs), impact of signal delay, and coor-dination with PSSs, have also been discussed for practicalconsiderations.

The remainder of this paper is organized as follows. Section IIbriefly describes the benchmark system, the POD model, andthe ESD model used in this paper. Section III presents the de-tailed oscillation damping controller design using the aforemen-tioned three methods. In particular, Section III-A provides theresidue-based controller design, Section III-B provides the PSOalgorithm-based controller design, and Section III-C providesthe GrHDP-based controller design. Section IV illustrates theeffectiveness of the GrHDP-based control for improving the sta-bility of the benchmark system by comparing against conven-tional method-based control and PSO algorithm-based control.Section V concludes this paper and provides some discussionson practical application in real power systems.

II. BENCHMARK SYSTEM AND POD MODULE

Fig. 1 demonstrates the structure of Kundur’s four-ma-chine–two-area benchmark system [3], which includes twoareas and four synchronous machines. The ESD is applied tothis benchmark power system to damp inter-area oscillations.From the system’s controllability and observability perspective[3], the optimal location of ESD is different from the reactivepower compensators. As indicated in [8], ESD has betterperformance to damp the inter-area oscillation when its locatedat the end-side of the tie-line rather than at the middle of thetie-line. Thus, in this paper, the ESD is placed at bus 7 toinject/absorb active power to/from the system. The capacityof the ESD is limited to 40 MW, which is about 10% of thetransmission power from area one to area two.

SUI et al.: ENERGY-STORAGE-BASED LOW-FREQUENCY OSCILLATION DAMPING CONTROL USING PSO AND HEURISTIC DYNAMIC PROGRAMMING 2541

Fig. 2. The schematic diagram of POD controller.

The controllable current resource is used to represent the ESDas follows:

(1)

where the real and imaginary part of the current is calculated bythe equation as follows:

(2)

In simulation, the current is transferred to a polar coordinatepresentation, and the magnitude and angle are the orders givento the controlled current source. The model of ESD can repre-sent different kinds of energy storage devices in real power sys-tems, including super capacitors energy storage (SCES), super-conducting magnetic energy storage (SMES), flywheels energystorage (FES), and advanced batteries energy storage (ABES)[14], [15], among others.A classical POD controller is shown in Fig. 2 [16]. Part 1 is a

measurement unit, where the measured transmission-line activepower is compared with the steady-state value to generate activepower deviation. Part 2 is a POD amplifier/gain unit. Part 3 is adirect current (dc) blocking unit to filter out the smooth compo-nent in input signal. Parts 4 and 5 are two time constants of thelead-lag blocks to provide necessary phase compensation. Part6 simulates time lag of energy storage devices. Each phase com-pensation block is recommended to compensate less than 60 .The value of is set as 5–10. The number of phase compensa-tion block is , then the phase compensation of the POD is .The parameters of the lead-leg block are calculated as follows:

(3)

POD controllers are effective in contribution to the dampingof poorly damped inter-area modes, while PSSs are an efficienttool for damping the local modes. These two kinds of controllerscould be properly coordinated to ensure that the power systemoperated with adequate damping over a wide range of operatingconditions and system configurations. In this paper, we focus oninter-area oscillation damping, where only the POD controllerparameters , , and are optimized.

III. OSCILLATION DAMPING CONTROLLER DESIGN

A. Residue-Based Design

In general, there are two kinds of methods to tune the PODcontroller parameters. One is residue method based on tradi-tional control theory. The design procedure is similar to thatof the FACTS-based damping controller design and POD de-sign in [8], [10], and [17]–[19], which will be briefly introduced

TABLE IMODE ANALYSIS OF THE BENCHMARK SYSTEM

here. The other is using an optimization algorithm to search thecontrol parameters in the solution space [20], which will be de-scribed in the next section.The benchmark system is linearized around a nominal op-

erating point, and small-signal analysis is shown in Table I. Itis shown that mode 1 is the inter-area oscillation mode, whichneeds to be damped with the POD controller. Using the equa-tions in [18], and can be obtained. ThePOD controller gain should be carefully selected to in-crease the inter-area damping, while not deteriorating the otherinner-area modes. In this paper, the controller gain is setto 5, as suggested in [19].

B. PSO Algorithm-Based Design

Here, PSO algorithm is used to search the optimal , ,and parameters in a POD controller.PSO was first brought forward by Kennedy and Eberhart

by representing the movement organisms in a bird flock or afish school in 1995 [21]. It is an evolution algorithm basedon the swarm’s behavior. The main idea, through constructinga number of swarm particles and setting the fitness function,is to make a judgment of the adaptability of each particle ineach generation. Then, the fitness value of each particle ineach generation is compared to obtain the global best particleand the local best particle. Finally, based on the informationsharing among particles, direction and velocity of each particleis updated. The effectiveness and searching capability of thismethod are related to the group size, generation number, andfitness function design. The procedure can be generalized inthe following four steps.1) Step One: Initialization: The position of each particle in

the swarm contains three dimensions, corresponding to the threecontrol parameters in the POD controller. The original speedvalue of the PSO method is formed by experience. The par-ticle number and iteration times are both set to 20, the searchingrange of gain is set as , where the searchingrange of time constants and are set as . The ve-locities for the position updating are initialized as follows:

(4)

where and are the upper and lower bounds of par-ticle . The initial velocities are generated randomly between

, where the other parameters are initialized asfollows:

(5)

where and are the accelerating constants, and andare the initial and final weights.


2) Step Two: Updating Individual Best and Global Best:After initialization, the position of each particle is sent to thePOD controller as parameters values and run the simulation toobtain the fitness value using the following function:

(6)

where the optimization goal is to minimize the fitness function.is the time range of oscillation, and is the instant re-

inforcement signal. The later time’s oscillation magnitude canbetter reflect the effect of decay and therefore it is more impor-tant. The individual and global best particles will be determinedaccording to the fitness of each particle.3) Step Three: Position Updating: The velocity and position

updating of each particle is formulated as follows:

(7)where and are uniformly distributed numbers in , andand are the individual and global best solution in the

current generation.4) Step Four: Determining Whether to Finish the Proce-

dure: If one of the following termination criteria is satisfied,the process of optimization will be finished.1) The iteration number has reached the maximum generationnumber.

2) The fitness value of global best solution is smaller than theset value, which is called iteration converge.

If neither of the two situations is satisfied, then the procedurewill jump to step two.The optimization goal/fitness function of PSO is to minimize

the reinforcement signal as follows [22]:

(8)

where , and are rotor speed de-viations corresponding to different oscillation modes as follows:

(9)

where , , 2, 3, 4 is the rotor speed of the th gener-ator. By adjusting the weights , 1, 2, 3, the most possibledestabilizing oscillation mode will be suppressed. From the en-ergy point of view, there are several oscillation modes after asystem fault, and is viewed as an index of the kinetic energyof the entire system oscillation. The flow chart of the proposedPSO-based parameters tuning is shown in Fig. 3.

C. HDP-Based Design

Before presenting the HDP-based damping controller design,we will first briefly introduce the goal representation heuristicdynamic programming (GrHDP) algorithm. GrHDP is a newreinforcement learning mechanism from the family of adaptivedynamic programming (ADP) designs in recent years [23],[24]. It requires three function approximation networks: a

Fig. 3. Flow chart of the proposed PSO-based parameters tuning.

goal network, a critic network, and an action network. Thecritic network learns to approximate the cost-to-go functionin Bellman’s equation, the action network learns to generatethe control policy that minimizes the cost-to-go approximatedby the critic network, while the goal network provides anadaptive internal reinforcement signal in addition to the pri-mary reinforcement signal to the critic network for improvedgeneralization and learning capability [23]–[25]. Specifically,the cost-to-go function is defined as follows:

(10)

where is the state vector of the system, is the con-trol action, is the utility function, and is a discount factor.In this paper, all three networks are implemented in neural net-works of a three-layer nonlinear architecture with one hiddenlayer. However, the learning principles can also be generalizedto any arbitrary function approximator by properly applying thebackpropagation rule. The comparison between different imple-mentations is beyond the scope of this paper.1) Goal Network Training: As indicated in (10), the system

performance cost is expressed in a compact form. The objectiveof dynamic programming is to choose the control sequenceso the cost function is minimized as follows:

(11)


This is the foundation for implementation dynamic program-ming by working backward in time. In this structure, can beestimated by minimizing the following error over time:

(12)

When for all , (12) indicates

(13)

Putting one time step backward, we can obtain

(14)

From (13) and (14), the objective function to be minimized inthe goal network is [26]

(15)

and the high-level conceptual backpropagation path is

(16)

Since the three-layer neural network is used in this paper, theweight adjustments for the hidden to the output layer and forthe input to hidden layer are as follows:

(17)

2) Critic Network Training: Once the goal network outputsthe signal, it will be used as an input to the critic networkand also be used to define the error function to adjust the param-eters of the critic network as follows:

(18)

and the backpropagation path is

(19)

The weight adjustments for the hidden to the output layer andfor the input to hidden layer in the critic network are as follows:

(20)

3) Action Network Training: The procedure of adapting theaction network in this architecture is similar to the classic ADPapproach to implicity backpropagate the error between the de-sired ultimate object and the approximate function fromthe critic network [27]. is in accordance with the signal of thereinforcement when the state conducted by the action implies a

success. Therefore, the error function to adjust the parametersof the action network is

(21)

Since the action network is connected with both goal networkand critic network, the backpropagation path will formed in twoparts as follows:

(22)

and the weight adjustments for the hidden to the output layerand for the input to hidden layer in the action network are asfollows:

(23)

4) GrHDP-BasedDamping Controller: The configuration ofthe GrHDP-based controller with the power plant is shown inFig. 4. The utility function is set equal to zero to representsuccess. Since contains the informationof inter-area and inner-area oscillation, they are chosen as theinputs of the GrHDP controller. The output of the GrHDP con-troller is the injected active power by the ESD and limited to 40MW. The reinforcement signal is the same as in the PSOfitness function and is rewritten as follows:

(24)

The controller works in the following procedures.• The action network receives the measured plant stateand uses it to generate the control signal to the ESD.

• The goal network uses the external reinforcement signaland plant state to generate the internal reinforce-

ment signal .• The critic network uses the internal reinforcement signal

, plant state , and control signal to estimate thecost function .

• The goal network will update its weights according to(15)–(17) until the stop criterion is satisfied.

• The critic network will update its weights according to(18)–(20) until the stop criterion is satisfied.

• The action network will update its weights according to(21)–(23) until the stop criterion is satisfied.

• These steps are repeated at each simulation time step.

IV. SIMULATION RESULTS

Here, we focus on comparison of the residue-based PODcontroller, the PSO optimized controller, and the GrHDP-basedcontroller to damp system oscillation resulting from a variety ofdisturbances applied at different system operating conditions.The parameter setting in the GrHDP controller is shown in


Fig. 4. Proposed configuration of the GrHDP-based controller with power plant.

TABLE IIPARAMETER USED IN GRHDP CONTROLLER

Table II. The weights of the neural networks in GrHDP arerandomly initialized only in the first trial. The controller doesnot know the proper control strategy before training. It is wellknown that, in neural network, the initial weights contributesignificantly to the performance of the controller. Therefore, weshould save the weights of the controller and carry them on asthe initial weights for the next trial, regardless of the simulationperformance. This trial-and-error methodology [28]–[30] isused in the following three different scenarios.

A. Case 1: Disturbance With Original Benchmark System

In case 1, the structure of the benchmark power system isillustrated in Fig. 1. The excitation of synchronous machineG3 experiences a 0.2-s-long, 5% step disturbance at time 1 s.Without an ESD, the system would have lost stability after thissmall disturbance. Then, the GrHDP controller is activated inthe benchmark power system to control ESD, and the simula-tion result of the first trial is shown in Fig. 5. Because of therandom initial weights, we can observe that the GrHDP con-troller does not generate proper control signal during the earlystage of the simulation (1–5 s) in the first trial. After about 10s, the GrHDP controller learned to damp the line active powerswing by adapting the weights of the neural networks.The weights in the first trial are saved as the initial weights in

the second trial. Results of the second trial are shown in Figs. 6

Fig. 5. Comparison between line active power and energy storage output in thefirst trial.

and 7. Specifically, Fig. 6 shows the ESD output active powerwith conventional POD control, PSO optimized control, andGrHDP control, and Fig. 7 shows the active power of the trans-mission line. From both figures, we can see that, with all threeapproaches, the system can become stable after about 6 s. Using

as an index of the inter-area oscillation mode, Fig. 8shows the comparison of the three controllers for inter-area os-cillation mode damping. It can be observed that, without ESD,the system will become unstable after the disturbance. The per-formance of residue-based POD controller and PSO optimizedcontroller are similar to that of the GrHDP-based controller inthis case.

B. Case 2: Disturbance With Line Cut-Off

In case 2, the benchmark power system configuration ischanged. In addition to the same disturbance in case 1, we alsoassume that one transmission line between buses 7 and 8 isout of service. Under a new operating condition, the GrHDPcontroller keeps adjusting the weights in the neural network toobtain an optimal control performance. Figs. 9–11 show the


Fig. 6. Comparison of ESD output with POD, PSO, and GrHDP controllers incase 1.

Fig. 7. Comparison of line active power with POD, PSO, and GrHDP con-trollers in case 1.

Fig. 8. Comparison of inter-area oscillation with POD, PSO, and GrHDP con-trollers in case 1.

results of ESD output active power, transmission-line activepower, and inter-area rotor speed deviation with residue-basedPOD controller, PSO optimized controller, and GrHDP con-troller. We can observe that the PSO method and the residuemethod have similar performances under this operating condi-tion. The simulation results also indicate that, under a differentoperating point, the conventional method-based POD controllerand the PSO optimized controller can no longer maintain itsdesired performance. We should notice from Fig. 10 that,with the GrHDP controller, the first swing of the transmittedactive power has deteriorated. This is because the weightsin the neural network are adjusted for the changed operating

Fig. 9. Comparison of ESD output with POD, PSO, and GrHDP controllers incase 2.



condition. However, the GrHDP-based method still has betterrobustness and optimization capability as it obtains the bestdamping performance in the post-fault period. The reason forthis is that the learning ability of the GrHDP controller keepsdriving the controller to the optimal control point.


Fig. 12. Comparison of ESD output with POD, PSO, and GrHDP controllersin case 3.



C. Case 3: Disturbance With Load Changing

In case 3, we modify the benchmark power system configura-tion with load 2 active power decreased from 1767 to 1567MW.Under this system operating point, the GrHDP controller canstill adapts to this new situation. Figs. 12–14 show the results ofESD output active power, transmission-line active power, andinter-area oscillation represented by rotor speed deviation with

residue-based POD controller, PSO optimized controller, andGrHDP controller. In order to focus on the comparison of thesethree methods, we did not show the system dynamics withoutESD in these results. The simulation results indicate that, underthis new operating point, the POD controller design based onthe conventional residue method has the worst performance.Meanwhile, with the continue learning ability, the GrHDP con-troller performs slightly better than the PSO optimized con-troller. These results also demonstrate that damping enhance-ment can be achieved over a wide range of operating points withthe proposed GrHDP method.

V. CONCLUSION AND DISCUSSIONS

This paper analyzed the power system low-frequencyoscillation supplementary damping control using heuristicdynamic programming. A classical four-machine–two-areasystem with ESDs has been applied for the comparative studyof residue-based POD control design, PSO-based control de-sign, and the GrHDP control design. The simulation resultsunder different operation conditions and system configurationsdemonstrated the effectiveness of the GrHDP method overthe other two methods. From this study, we can see that theGrHDP controller has the potential of more robust performancethan the conventional POD design and the PSO optimal designover a wide range of system conditions. Also, we shouldnotice that the benchmark power system used in this papercould easily be replaced by a large power system for a morecomprehensive study of the proposed controller. Consideringthe adaptive ability of damping power system oscillation ofGrHDP controller, it may be interesting to apply it to otherpower system oscillation problems, such as subsynchronizeoscillation problems in the future studies.The proposed ESD damping controller design based on HDP

can be utilized for system-wide damping control or local modeenhancement. We should notice that the reinforcement signal

in (8) and (9) requires real-time rotor speed signals in re-mote areas. However, this is no longer a hurdle in modern powersystems because of large-scale installation of PMUs [31], [32].Since the proposed real-time HDP damping controller is basedon instant interactions and learning between the power system,the signal transmission delay in a real power system will impactits performance. However, it has been shown that the neural net-work-based control can successfully compensate for communi-cation delays [33], [34]. In practice, the HDP-based controllercould service as a supplementary control for the traditional PIDcontrollers. Since PSSs have been widely used in power systemdamping control, especially for local oscillation mode, the pro-posed controller could be coordinated with the local PSSs toachieve a better system operating stability.

REFERENCES[1] B. Dunn, H. Kamath, and J.-M. Tarascon, “Electrical energy storage

for the grid: A battery of choices,” Science, vol. 334, no. 6058, pp.928–935, 2011.

[2] Electr. Power Res. Inst., “Electrical energy storage technologyoptions,” Tech. Rep., Dec. 2010 [Online]. Available: http://www.epri.com

[3] P. Kundur, Power System Stability and Control. New York, NY,USA: Mc Graw-Hill, 1994.


[4] S. P. Azad, R. Iravani, and J. Tate, “Damping inter-area oscillationsbased on a model predictive control (MPC) HVDC supplementary con-troller,” IEEE Trans. Power Syst., vol. 28, no. 3, pp. 3174–3183, Aug.2013.

[5] D. Molina, G. Venayagamoorthy, J. Liang, and R. Harley, “Intelligentlocal area signals based damping of power system oscillations usingvirtual generators and approximate dynamic programming,” IEEETrans. Smart Grid, vol. 4, no. 1, pp. 498–508, 2013.

[6] B. Pal, A. Coonick, and D. C. Macdonald, “Robust damping con-troller design in power systems with superconducting magnetic energystorage devices,” IEEE Trans. Power Syst., vol. 15, no. 1, pp. 320–325,Feb. 2000.

[7] S. Cheng, G. Li, H. Sun, and J. Wen, “Energy storage application inpower system and its prospects,” Grid and Clean Energy, no. 2, pp.1–8, 2009.

[8] J. Wu, J. Wen, H. Sun, and S. Cheng, “Feasibility study of segmentinglarge power system interconnections with ac link using energy storagetechnology,” IEEE Trans. Power Syst., vol. 27, no. 3, pp. 1245–1252,Aug. 2012.

[9] S. Cheng, W. Yu, J. Wen, H. Sun, and H. Wang, “Energy storage andits application in power system stability enhancement,” Power Syst.Technol., vol. 31, no. 20, pp. 98–108, Oct. 2007.

[10] J. Fang, W. Yao, Z. Chen, J. Wen, and S. Cheng, “Design of anti-windup compensator for energy storage-based damping controller toenhance power system stability,” IEEE Trans. Power Syst., vol. PP, no.99, pp. 1–11, 2013.

[11] M. Ali, T. Murata, and J. Tamura, “Transient stability enhancement byfuzzy logic-controlled SMES considering coordination with optimalreclosing of circuit breakers,” IEEE Trans. Power Syst., vol. 23, no. 2,pp. 631–640, May 2008.

[12] M. Zarghami,M. Crow, J. Sarangapani, Y. Liu, and S. Atcitty, “A novelapproach to interarea oscillation damping by unified power flow con-trollers utilizing ultracapacitors,” IEEE Trans. Power Syst., vol. 25, no.1, pp. 404–412, Feb. 2010.

[13] Q. Jiang, Y. Gong, and H. Wang, “A battery energy storage systemdual-layer control strategy for mitigating wind farm fluctuations,”IEEE Trans. Power Syst., vol. 28, no. 3, pp. 3263–3273, Aug. 2013.

[14] P. Ribeiro, B. Johnson,M. Crow,A. Arsoy, andY. Liu, “Energy storagesystems for advanced power applications,” Proc. IEEE, vol. 89, no. 12,pp. 1744–1756, Dec. 2001.

[15] M. G. Molina, “Dynamic modelling and control design of advancedenergy storage for power system applications,” Universidad Nacionalde San Juan, Argentina.

[16] R. Sadikovic, “Use of FACTS Devices for power flow control anddamping of oscillations in power systems,” Ph.D. dissertation, Dept.Inf. Technol. Electr. Eng., Swiss Federal Inst. of Technol., Zurich,Switzerland, 2006.

[17] P. S. Dolan, J. Smith, and W. Mittelstadt, “A study of TCSC optimaldamping control parameters for different operating conditions,” IEEETrans. Power Syst., vol. 10, no. 4, pp. 1972–1978, Nov. 1995.

[18] N. Yang, Q. Liu, and J. McCalley, “TCSC controller design fordamping interarea oscillations,” IEEE Trans. Power Syst., vol. 13, no.4, pp. 1304–1310, Nov. 1998.

[19] J. Sun, X. Zhao, D. Li, M. Li, X. Li, W. Lin, and J. Wen, “Study onenergy storage in damping tie line power oscillations in power system,”Power Syst. Protection and Control, no. 17, pp. 1–8, 2013.

[20] Y. Tang, P. Ju, H. He, C. Qin, and F. Wu, “Optimized control of DFIG-based wind generation using sensitivity analysis and particle swarmoptimization,” IEEE Trans. Smart Grid, vol. 4, no. 1, pp. 509–520,2013.

[21] J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proc.IEEE Int. Conf. Neural Networks, 1995, vol. 4, pp. 1942–1948.

[22] C. Lu, J. Si, and X. Xie, “Direct heuristic dynamic programming fordamping oscillations in a large power system,” IEEE Trans. Syst., Man,Cybern. B, Cybern., vol. 38, no. 4, pp. 1008–1013, Aug. 2008.

[23] H. He, Self-Adaptive Systems for Machine Intelligence. Hoboken,NJ, USA: Wiley, 2011.

[24] H. He, Z. Ni, and J. Fu, “A three-network architecture for on-linelearning and optimization based on adaptive dynamic programming,”Neurocomputing, vol. 78, no. 1, pp. 3–13, 2012.

[25] Z. Ni, H. He, and J. Wen, “Adaptive learning in tracking control basedon the dual critic network design,” IEEE Trans. Neural Netw. LearningSyst., vol. 24, no. 6, pp. 913–928, Jun. 2013.

[26] Z. Ni, H. He, J. Wen, and X. Xu, “Goal representation heuristic dy-namic programming on maze navigation,” IEEE Trans. Neural Netw.Learning Syst., vol. PP, no. 99, pp. 1–1, 2013.

[27] J. Si and Y.-T. Wang, “Online learning control by association and re-inforcement,” IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 264–276,Mar. 2001.

[28] Y. Tang, H. He, and J. Wen, “Comparative study between HDP andPSS on DFIG damping control,” in Proc. IEEE Symp. ComputationalIntell. Applicat. in Smart Grid, 2013, pp. 59–65.

[29] Y. Tang, H. He, Z. Ni, J. Wen, and X. Sui, “Reactive power control ofgrid-connected wind farm based on adaptive dynamic programming,”Neurocomputing, vol. 125, no. 1, pp. 125–133, 2014.

[30] Y. Tang, H. He, and J. Wen, “Adaptive control for an HVDC trans-mission link with FACTS and a wind farm,” in Proc. IEEE InnovativeSmart Grid Technol. Conf., Washington, DC, USA, Feb. 2013, pp. 1–6.

[31] I. Kamwa, R. Grondin, and Y. Hebert, “Wide-area measurement basedstabilizing control of large power systems-A decentralized/hierarchicalapproach,” IEEE Trans. Power Syst., vol. 16, no. 1, pp. 136–153, Feb.2001.

[32] B. Chaudhuri, R. Majumder, and B. Pal, “Wide-area measurement-based stabilizing control of power system considering signal transmis-sion delay,” IEEE Trans. Power Syst., vol. 19, no. 4, pp. 1971–1979,Nov. 2004.

[33] W. Qiao, G. K. Venayagamoorthy, and R. G. Harley, “Optimal wide-area monitoring and nonlinear adaptive coordinating neurocontrol of apower system with wind power integration and multiple facts devices,”Neural Netw., vol. 21, no. 2, pp. 466–475, 2008.

[34] N. Chaudhuri, B. Chaudhuri, S. Ray, and R. Majumder, “Wide-areaphasor power oscillation damping controller: A new approach to han-dling time-varying signal latency,” IET Gener., Transm. Distrib., vol.4, no. 5, pp. 620–630, 2010.

Xianchao Sui received the B.S. and M.S. degrees inelectrical engineering from Huazhong University ofScience and Technology (HUST), Wuhan, China, in2008 and 2011, respectively.He is currently with Liaoning Dalian Power

Supply Company, Dalian, China. His researchinterests include smart grids, renewable energy, andpower system control.

Yufei Tang (S’13) received the B.Eng. and M.Eng.degrees in electrical engineering from Hohai Univer-sity, Nanjing, China, in 2008 and 2011, respectively.He is currently working toward the Ph.D. degreeat the Department of Electrical, Computer, andBiomedical Engineering, University of RhodeIsland, Kingston, RI, USA.His research interests include power system

modeling, power system stability control, windenergy generation and integration, smart grids,power system cyber security, and the application of

computational intelligence in power systems.

Haibo He (SM’11) received the B.S. and M.S.degrees from Huazhong University of Science andTechnology, Wuhan, China, in 1999 and 2002,respectively, and the Ph.D. degree from Ohio Uni-versity in 2006, all in electrical engineering.He is currently the Robert Haas Endowed Pro-

fessor in Electrical Engineering with the Universityof Rhode Island, Kingston, RI, USA. From 2006 to2009, he was an Assistant Professor with the De-partment of Electrical and Computer Engineering,Stevens Institute of Technology, Hoboken, NJ,

USA. He has published one research book, edited one research book and six


conference proceedings, and authored and coauthored over 130 peer-reviewedjournal and conference papers. His research has been covered by nationaland international media. His research interests include smart grids, renewableenergy, power system cyber security, cyber-physical systems, computationalintelligence, machine learning, data mining, and various application fields.Prof. He is an associate editor of the IEEE TRANSACTIONS ON NEURAL

NETWORKS AND LEARNING SYSTEMS and the IEEE TRANSACTIONS ON SMARTGRID. He was a recipient of the IEEE Computational Intelligence SocietyOutstanding Early Career Award (2014), the National Science FoundationCAREER Award (2011), and the Providence Business News Rising StarInnovator Award (2011).

JinyuWen (M’10) received the B.Eng. and Ph.D. de-grees in electrical engineering from Huazhong Uni-versity of Science and Technology, Wuhan, China, in1992 and 1998, respectively.He was a Visiting Student from 1996 to 1997 and a

Research Fellow from 2002 to 2003 at the Universityof Liverpool, Liverpool, U.K., and a Senior VisitingResearcher with the University of Texas at Arlington,TX, USA, in 2010. From 1998 to 2002, he was a Di-rector Engineer with XJ Electric Company Ltd. inChina. In 2003, he joined Huazhong University of

Science and Technology, Wuhan, China, where he is now a Full Professor. Hiscurrent research interests include renewable energy integration, power systemcontrol, energy storage application, multi-terminal HVDC, and power systemoperation and control.

Documents

IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5 ... · 2540 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5, SEPTEMBER 2014 Fig. 1. Four-machine–two-area benchmark system