Modeling and Nonlinear Optimal Control of Weak ... - IEEE Xplore

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY

1

Modeling and Nonlinear Optimal Control of Weak/Islanded Grids Using FACTS Device in a Game Theoretic Approach Hamidreza Nazaripouya, Student Member, IEEE, and Shahab Mehraeen, Member, IEEE Abstract— A nonlinear discrete-time model along with an optimal stabilizing controller using a unified power quality conditioner (UPQC) is proposed for weak/islanded grids in this paper. An advanced stabilizing controller greatly benefits islanded medium-sized grid and microgrid due to their relatively small stored energy levels, which adversely affect their stability, as opposed to larger grids. In addition, a discrete-time grid model and controller are preferred for digital implementation. Here, the discrete-time Hamilton–Jacobi–Isaacs optimal control method is employed to design an optimal grid stabilizer. While UPQC is conventionally utilized for power quality improvement in distribution systems in the presence of renewable energy, here, the stabilizing control is added and applied to the UPQC series voltage in order to mitigate the grid’s oscillations besides UPQC’s power conditioning tasks. Consequently, the UPQC can be employed to stabilize a grid-tie inverter (GTI) or a synchronous generator (SG) with minimum control effort. When controlling the GTI associated with renewable energy sources, a reduced UPQC structure is proposed that only employs the series compensator. Next, a successive approximation method along with neural networks is utilized to approximate a cost function of the grid dynamical states, the UPQC control parameters, and disturbance, in a two-player zero-sum game with the players being UPQC control and grid disturbances. Subsequently, the cost function is used to obtain the nonlinear optimal controller that is applied to the UPQC. Simulation results show effective damping behavior of the proposed nonlinear controller in controlling both GTI and SG in weak and islanded grids. Index Terms— Discrete-time optimal control, Flexible Alternating Current Transmission System (FACTS) devices, Hamilton– Jacobi–Isaacs (HJI), microgrid, neural networks (NNs), power system stability, virtual synchronous generator (VSG).

I. I NTRODUCTION

R

ECENTLY, the application of a unified power quality conditioner (UPQC) has been proposed for improving power quality in distribution systems, for unlike in large systems factors such as load imbalance and harmonics due to renewable sources are more severe in smaller grids [1]–[4]. In addition, low stored energy levels in small

Manuscript received May 20, 2014; revised October 20, 2014 and February 10, 2015; accepted March 6, 2015. Manuscript received in final form April 5, 2015. This work was supported in part by NSF CAREER ECCS#1151141. Recommended by Associate Editor S. Varigonda. H. Nazaripouya is with the Smart Grid Energy Research Center (SMERC), University of California, Los Angeles, CA 90095 USA (e-mail: [email protected]). S. Mehraeen is with the Division of Electrical and Computer Engineering, Louisiana State University, Patrick F. Taylor Hall, Baton Rouge, LA 70803 USA (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TCST.2015.2421434

grids render their dynamic stability and secure operation a challenging task. Flexible Alternating Current Transmission System (FACTS) devices, on the other hand, have been utilized to improve transient stability in transmission level besides the power flow control tasks [5]–[11]. Similarly, in distribution systems and microgrids, UPQCs can be equipped with proper stabilizing mechanisms to enhance the stability of the grid in addition to their power quality capabilities. For many years, linear system theory has been applied to power grids in order to mitigate oscillations that occur after disturbances and provide stability [11], [12]. In the majority of these works, the nonlinear power grid model is linearized around an operating point. However, these approaches assume that the network variables remain in a small neighborhood of the desired operating point. Power grid model is in general highly nonlinear, and thus nonlinear controller designs are more appropriate for those systems. On the other hand, the closed-loop stability is the sole purpose of most of the available controller designs [13]–[15]. Optimality, however, requires a policy to stabilize the system in an optimal manner according to an overall cost function. The optimal control problem in power system seeks not only to stabilize the system, but also to dampen the speed oscillations effectively with minimum effort that results in lower electrical stress on the equipment while maintaining the performance. The optimal control of nonlinear perturbed dynamical systems leads to solving the Hamilton–Jacobi–Isaacs (HJI) equation, which does not have a closed-form solution [16]. The cost function in HJI can be approximated by employing offline and online approximation methods. Offline methods [17]–[24] require previously collected data sets for training as opposed to online methods [25]–[27] that find the optimal controller in real-time fashion, and thus, no data sets are required prior to training. While online methods relax the need of large data sets, they find the optimal solution with larger error and require the persistently exciting input [25], [27] so that the online training can be conducted. Discrete-time nonlinear optimal control for power grids has been addressed in a limited manner. In this paper, optimal stabilization of weak/islanded grids that comprise renewable energy sources and conventional synchronous generators (SGs) is studied. While in the conventional stability methods, the main target is stabilization with less emphasis on the control effort, the goal in this paper is not only to stabilize the grid, but also, to achieve the stability with minimum control effort (cost) in order to reduce electrical stress on the controller’s power

1063-6536 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 2

electronic devices and increase their lifetime. While the power grid may be equipped with conventional stabilizers such as power system stabilizers (PSSs), the UPQC controlled by the optimal controller proposed in this paper provides additional control to improve the stability. Here, the term weak is utilized to imply medium-sized and islanded power networks with low stored energy level. The entire grid, which includes renewable generator (RG), SG, and UPQC, is modeled as a discrete-time affine nonlinear system. While UPQC has traditionally been introduced to improve the quality of power at the distribution level, the goal in this paper is to show that by controlling the UPQC series voltage, stabilizing capability can be added to the device to improve transient stability with minimum cost, especially in small-sized grids. However, finding the optimal stabilizing controller for the perturbed power grid, leads to solving the nonlinear discrete-time HJI equation with no known closed-form solution. This paper builds upon [17] in the field of discrete-time optimal control and enhances the theoretical results therein to power grid stability studies. First, a proper nonlinear discretetime dynamical model of the power grid equipped with UPQC is developed in this paper. This model is appropriate for grids with renewable energy sources connected through a grid-tie inverter (GTI) as well as grids with SGs. Next, a reduced UPQC structure is proposed in the presence of the RGs (with GTI) where the UPQC’s rectifier (along with the shunt transformer) and dc-link capacitor are removed. This will introduce a significant cost reduction to the controller structure. Subsequently, the grid’s HJI formulation is presented and solved for a cost function by utilizing a recursive algorithm in a two-player zero-sum game approach using dynamic programming where the players are UPQC control (series voltage) and grid’s unknown disturbances. Next, the optimal controller is obtained using the cost function and is applied to the UPQC. One significance of the proposed model is that it does not consider an infinite bus, as infinite bus does not exist in power systems with low stored energy levels such as islanded or weak networks. Rather, the developed model considers variation of the bus voltages as disturbances in the HJI formulation. Then, the discrete-time HJI problem for the developed power system model is solved using two-player policy iteration in dynamic programming to obtain the target cost function. The iterative approach uses the Taylor series expansion of the cost function and approximation properties of neural networks (NNs) [28] to approximate the cost function of RG/SG states, UPQC control effort, and network disturbances. Next, from the obtained cost function, the UPQC optimal series voltage (controller) is found that minimizes the cost function and mitigates the oscillations optimally. Simulation results on practical grids show the effectiveness of the proposed optimal UPQC controller over the conventional approaches. The rest of this paper is organized as follows. First, the discrete-time model of the grid including UPQC is derived in Section II. Next, background information on the nonlinear optimal control systems is presented in Section III. Subsequently, the grid’s HJI formulation is conducted and an iterative approach is utilized to solve the HJI equation for cost function using NN in Section IV. The proposed stabilizing

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY

Fig. 1.

RG connected to weak grid via dc link and inverter.

controller is tested using simulations in Section V, while the conclusion is provided in Section VI. II. M ODELING THE G RID W ITH UPQC The power system with UPQC model is developed in this section through swing and power balance equations. In this model, the variation in the UPQC bus voltage is modeled as an unknown disturbance affecting the grid dynamics. The presented dynamical model applies for both SG and RG, and thus the proposed optimal controller (introduced later) can be applied to both as will be shown in Section V. A. Renewable Generator Model In this paper, it is assumed that the RG is connected to the grid through a GTI, as shown in Fig. 1. Renewable energy sources such as photovoltaic (PV) arrays, fuel cells, variable-frequency wind turbines, or microturbine generators are majorly linked to the main grid through an inverter connected to a dc-link capacitor as shown in Fig. 1. The function of the GTI’s dc-link capacitor is to store energy and provide a constant dc voltage at the inverter’s terminal as the renewable power source may not be capable of supplying enough reserve power in the case of disturbances. When power oscillations occur, the capacitor dynamic has a great influence on the grid stability. The capacitor voltage is governed by the capacitor input–output power balance as C VC V˙C = Pin − Pe

(1)

where C is the capacitance, VC is the capacitor voltage, Pin is the injected power by the renewable energy source, and Pe is the delivered power to the inverter. In addition, Vs represents the output voltage of the inverter, ϕs = ϕˆ s + ϕs0 is the inverter’s voltage phase angle with respect to the grid’s reference node r (ϕr = 0), with ϕs0 and ϕˆ s being the phase steady-state and error values, respectively, while V1 and θ1 denote the voltage and phase angle of the grid bus connected through admittance B. The GTI has two control parameters, gain ks and phase angle ϕs , which can be adjusted to control the GTI’s output voltage and power. The relation between the desired inverter output rms voltage Vs and the dc-link voltage is Vs = ks VC .

(2)

Remark 1: In (2), Vs is the fundamental harmonic of the unfiltered GTI output voltage. Equation (2) is a valid assumption when the carrier frequency is significantly higher than that of the sinusoidal reference [29]. The effect of the filter can then be incorporated in admittance B connected to Vs . Voltage Vs is proportional to the dc-link voltage VC ; however,

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. NAZARIPOUYA AND MEHRAEEN: MODELING AND NONLINEAR OPTIMAL CONTROL OF WEAK/ISLANDED GRIDS USING FACTS DEVICE

3

the inverter duty cycle ks can be dynamically controlled in an inner loop such that desired output voltage results, i.e., ks = Vs−desired /VC . Here, the effect of the inverter filter is modeled through series admittance similar to those of the transmission lines leading to the conventional differentialalgebraic model widely used in power system transient analysis. The RG output power can be altered by controlling GTI’s phase angle ϕs . From (1), the auxiliary parameter υ can be defined such that υ/τ ˙ = VC V˙C = (1/C) × (Pin − Pe )

(3)

where τ is a scaling factor. Because the dc-link capacitor’s stored energy can be altered by the GTI output power (through output voltage phase error ϕˆ s ), one can consider ϕ˙ˆ s = υ

(4)

to control phase ϕˆs . Phase rate of change (4) along with power balance (3) suggests that the inverter phase angle varies with the change in the dc-link stored energy to adjust the GTI’s output power. Here, the stored energy is calculated in terms of capacitor voltage VC in (1). Thus, inverter phase angle error ϕˆ s is defined as a new state variable besides variable υ that represents the change in the capacitor stored energy calculated as 2 /2 (5) υ = τ × VC2 − VC0 where VC0 is the capacitor steady-state voltage. Remark 2: The concept of virtual SG (VSG) and synchronverter have been used in [30]–[34]. Majority of the research in this area do not consider the dynamic of the dc-link capacitor and the stability of the overall system. For example, in [30], storage is considered and the required damping power to an SG-like inverter is supplied by the storage. However, the method adds to the integration costs as the required energy is supplied by an external storage rather than the dc-link capacitor. In [31] and [32], the renewable source is connected to the main grid (infinite bus) and the inverter power is controlled to respond to the frequency variations; however, the dc-link voltage stability and weak grid/islanded mode are not studied. In [32], the inverter is modeled similar to an SG by considering fictitious rotor angle and field current to provide desired active and reactive powers. In this model, the GTI phase angle mimics an SG rotor angle. However, the dc-link dynamic is assumed decoupled from the function of the inverter, an assumption that requires considering a storage device and higher implementation costs. In [33], the dynamic of the dc-link capacitor is assumed negligible. While this paper considers small signal analysis to study RG islandedmode operation, the method highly relies on large dc-link capacitance, which is not always practical. In addition, when applying the concept of rotating mass in the VSG, the method requires the derivative of the speed signal. In all the mentioned work, the stability of the overall system is presumed and the similarity to SG is considered enough to stabilize the entire system, an assumption that may not be fulfilled in power systems in general and in weak grids and islanded networks

Fig. 2.

Vector diagram of the UPQC voltage.

in particular, especially with the occurrence of faults. It is worth mentioning that in many of the commercially available GTIs used with RGs (especially at high power ratings), the dc-link capacitor is the only means of storing electrical energy. Thus, this paper proposes a GTI dynamical model that includes the dc-link capacitor dynamic. Then, the stability of the overall system is studied and improved via a nonlinear optimal stabilizer. The GTI is modeled as an SG where the GTI’s inherent storage, i.e., the dc-link capacitor, is considered to emulate the mechanical energy stored in the mass of an SG. In addition, due to selection of the dc-link energy variation, derivative of the speed is not required in the proposed stabilizing controller. Here, the equivalent classical-model SG [35] has an inertia M = C/τ, a rotor angle δ = ϕs , and a rotor speed error ω = υ (representing capacitor voltage). Thus, the dynamic of the dc-link capacitor is explicitly taken into account (3) similar to the rotor dynamic in the generator classical model. A similar model is developed in [34]; however, the inverter angle in [34] is not equal to the imaginary rotor angle. In the model presented in this paper, no storage device or extra large capacitance is utilized and short-term energy is provided by the conventional dc-link capacitor. B. UPQC Model The UPQC connected in the grid is shown in Fig. 1. The UPQC consists of two parts: 1) a shunt branch and 2) a series branch. The main duty of the shunt branch is to control the voltage magnitude at the sending-end bus by generating or absorbing reactive power, compensate for load imbalance and harmonics, and provide demanded real power at the UPQC dc link for the series converter. The series branch compensates for line current imbalance and harmonics. The shunt branch control and power quality performance of the UPQC are not aimed in this paper; instead, the series branch is employed to enhance grid’s stability as an additional function added to the conventional compensation tasks of the UPQC. UPQC series branch can be modeled as a controlled voltage source, governed by the shunt branch voltage. Fig. 2 shows the UPQC voltage phasor diagram. Series voltage Vu can be decomposed into two components Vup and Vuq , as shown in Fig. 2. The values for the voltage components are produced by an optimal controller that will be discussed in the following section. Once the proposed optimal controller generates the values for Vup and Vuq , inverter gains α(t) and β(t) are adjusted as α(t) = Vup /Vr , β(t) = Vuq /Vr

(6)



to generate voltage Vu at the UPQC series transformer. Here, Vr is the voltage magnitude at the point of connection of the UPQC shunt branch [36], [37]. Thus, in the optimal controller development (introduced later), voltages Vuq and Vup are the optimal control inputs (optimal policies) from which inverter gains α(t) and β(t) can be derived and applied to the UPQC series inverter. C. Dynamical Model Development The inverter losses are considered proportional to the inverter output power [38]. Consequently, referring to Fig. 1, the delivered power to the GTI from the dc link can be represented as Pe = (1 + χ)Vs V1 B sin(ϕs − θ1 ), where χ 1 is a coefficient that models the inverter losses proportional to the inverter output power. In addition, according to Fig. 2, it can be shown that cos θ1 = (Vr + Vuq )/V1 and sin θ1 = Vup /V1 . These along with (3) can be used to obtain (C/τ )υ˙ = Pin − (1 + χ)Vs Vr B sin(ϕˆ s + ϕs0 ) + (1 + χ)Vs B cos(ϕˆs + ϕs0 )u 1 − (1 + χ)Vs B sin(ϕˆ s + ϕs0 )u 2

(7)

where u 1 = Vup and u 2 = Vuq are the design control inputs. Next, shunt terminal voltage Vr can be decomposed into two components: 1) a constant part V¯r (steady-state value) and 2) a varying part V r (disturbance) such that Vr = V¯r + V r . Thus, the second term in (7) can be rewritten as Vs Vr B sin(ϕˆ s + ϕs0 ) = Vs V¯r B sin(ϕˆ s + ϕs0 )

+ Vs V r B sin(ϕˆs + ϕs0 )

(8)

where V r is considered as disturbance and will be shown by w(t) in the system’s representation later. Therefore, here, variations in terminal voltage Vr are taken into account during power system transients as opposed to [36] and [39] that assume a constant Vr (infinite bus). Remark 3: For the nonlinear optimal controller development adopted in this paper, the functions in the system representation [such as term Vs Vr B sin(ϕˆ s + ϕs0 ) in (7)] need to be functions of the dynamical states ϕˆ s and υ. Such a representation requires solving the grid nonlinear algebraic power balance equations for bus voltages and is in general difficult to obtain when infinite bus does not exist (i.e., terminal voltage Vr varies during transients). Thus, despite the availability of its measurement, here Vr is split into a known and a disturbance part such that Vr = V¯r + V r as explained. According to (3), (4), (7), and (8), the system state-space equations can be written as ⎤ ⎡ υ ˙ ϕˆs ⎦ = ⎣ τ υ˙ (Pm − Vs V¯r B sin(ϕˆs + ϕs0 )) C ⎤ ⎡ 0 0 ⎦u(t) + ⎣τ −τ Vs B cos(ϕˆs +ϕs0) Vs B sin(ϕˆs +ϕs0 ) C⎤ ⎡C 0 ⎦ w(t) (9) + ⎣ −τ Vs B sin(ϕˆ s + ϕs0 ) C

where u(t) = [Vup Vuq ]T and w(t) = V r are the control input vector and unknown disturbance, respectively, while τ = (1 + χ)τ and Pm = Pin /(1 + χ). Note that the UPQC dynamic gains α(t) and β(t) can be synthesized as α(t) = u 1 (t)/Vr (t) and β(t) = u 2 (t)/Vr (t), using measured voltage Vr (t), and applied to the UPQC series inverter once the optimal voltages Vup and Vuq are generated by the controller. In addition, |Vu | = (α(t)2 + β(t)2 )1/2 Vr and Vu = tan−1 (α(t)/β(t)) can be used to dynamically adjust the UPQC series inverter gain and phase angle. In the proposed approach, the controller starts after the clearance of the fault. Consequently, there is no excessive voltage variation in the network, and thus, UPQC gains α(t) and β(t) are in tolerable ranges. By assuming the discretizing time step T, the system dynamical equations can be written in discrete time as ϕˆs (k + 1) υ(k + 1) ⎤ ⎡ υT + ϕˆ s (k) ⎦ = ⎣T τ (Pm − Vs V¯r B sin(ϕˆs (k) + ϕs0 )) + υ(k) ⎡ C ⎤ 0 0 ⎦u(k) +⎣T τ Tτ Vs B cos(ϕˆ s (k)+ϕs0) − Vs B sin(ϕˆ s (k)+ϕs0) ⎤C ⎡C 0 ⎦ w(k) (10) +⎣−T τ Vs B sin(ϕˆ s (k) + ϕs0 ) C or equivalently x k+1 = f (x k ) + g(x k )u k + h(x k )wk , where x k = [ϕˆ s (kT ) υ(kT )]T ∈ 2 . Here, for simplicity, the notations x k = x(k) for the state vector, u k = u(k) for the control input, and wk = w(k) for the unknown disturbance have been used in the power system model at step k. Due to the selected disturbance term, all functions are strict functions of states ϕˆs and υ, while the disturbance is a function of time. This step is essential in the development of the control strategy introduced in the following section. It should be noted that when discretizing the continuoustime system, the sampling rate needs to be large enough to capture the dynamics of interest. All dynamics at or greater than the Nyquist frequency are neither controllable nor observable [29]. In order to avoid the adverse effect of the sampling time criterion, f sample > f switching should be satisfied that requires reasonably higher sampling frequency than switching frequency. In [36] and [39], FACTS device (unified power flow controller) is employed to improve SG transient stability where in a generator-infinite-bus system model, the receiving end bus voltage is considered unity as opposed to varying voltage Vr introduced here. The proposed model, which does not require an infinite bus, is more appropriate for weak and islanded grids and applies to both SG and RG through GTI model (10) as opposed to that of [36] and [39]. Remark 4: In model (10), it is assumed that the input power to the dc link is constant during the grid transients. In the wind power generators, the input power can be assumed constant since the wind speed does not undergo significant variation


Fig. 3. UPQC with reduced structure connected with renewable energy source and power system.

with admittance B can be modeled as an ideal current source Iu = − j B Vu in parallel with the admittance B (power injection model [37]). Total admittance B is equivalent to the series combination of UPQC series (filter) admittance and GTI (filter) admittance B. For simplicity, one may ignore UPQC series admittance such that B ≈ B. The series voltage source Vu can be represented in terms of magnitude and phase angle as Vu = μVs e− j θu

in the period of study (i.e., transient analysis). Though this approach is majorly developed for weak grids including wind power, it is helpful to discuss the solar power in the context of the proposed optimal controller. The input power in the case of PV panels can experience more drastic fluctuations in transients than in wind turbine generators. Solar power is a function of the dc-link voltage in the case that no dc–dc converters are utilized to interface the solar cells to the GTI. In this case, input power Pe can be split into constant and variable parts similar to the UPQC terminal voltage where the variable part of the power can be included into the disturbance. An alternative solution is to employ a dc–dc converter with adequately fast response to control the input solar power at a constant rate. Remark 5: It is assumed that the controller starts after the fault is removed. This can be assured by activating the controller after the breakers perform through using the breakers status signals. Moreover, the effect of the UPQC absorbed power on the terminal voltage can be ignored due to low power requirements in the proposed stability method (usually a few percent of the total line power). This way, the proposed optimal controller does not experience excessive variation of the UPQC terminal voltage (i.e., disturbance) and is expected to perform satisfactorily according to the underlying stability assumption on the introduced disturbance [17], which will be explained in the following section. So far, model (10) can represent an RG or an SG (represented in classical model [35]) with rotor angle error ϕˆs and rotor speed error ω = υ. D. Reduced UPQC Structure Model in the Presence of RG As mentioned in Section II-B, the UPQC’s demanded real power for its series converter (at its dc link) is provided by the shunt branch. However, in the presence of an RG, due to the existence of a dc link in the GTI, the UPQC’s shunt branch and dc link can be removed, while the energy for the series compensator can be directly provided by the GTI’s dc link, as shown in Fig. 3. The proposed structure (shown in Fig. 3) results in significant economic savings and a simpler structure. In this topology, the load harmonic/imbalance and reactive power compensations should be accomplished by the GTI (through addition of compensation tasks in the GTI controller), while current compensation is still performed by the reduced UPQC’s series branch. In order to develop the model of the proposed reduced UPQC structure, the injected power by the UPQC to the system is calculated. UPQC series branch voltage Vu along

5

(11)

where μ and −θu are the series branch gain and phase angle. Here, the series voltage angle is defined as negative that in the conventional structure. Current source Iu corresponds to two injected power Ss and Sr into nodes s and r , respectively, calculated as Ss = Vs (− Iu )∗ = BμVs2 sin θu − j BμVs2 cos θu Sr = Vr ( Iu )∗ = BμVr Vs sin(ϕs − ϕr − θu ) + j BμVr Vs cos(ϕs − ϕr − θu )

(12) (13)

where ϕr = 0 is the reference angle. The net power provided by the current source Iu is equal to Ss + Sr . In the conventional UPQC structure (Fig. 1), the shunt branch provides an active power, which is then injected into the system by the series branch. This power is now provided by the GTI’s dc link. As a result ∗ Pseries = real Vu Isr = μBVs Vr sin(ϕs − θu ) − μBVs2 sin θu

(14)

where Isr is the current flowing from node s to node r . Using (12) and (14), the net power from the RG can be derived and the dynamical power equation for the GTI (shown in Fig. 3) can be developed as (C/τ )υ˙ = Pin −(1+χ)(Vs Vr B sin(ϕs )−μVs Vr B sin(ϕs −θu )). (15) . By assuming that inverter losses coefficient (χ) for both inverters are the same, along with relationships μ cos θu = Vuq /Vr , μ sin θu = Vup /Vr , τ = (1 + χ)τ , Pm = Pin /(1 + χ), and ϕs = ϕˆs + ϕs0 , (15) can be rewritten as (C/τ )υ˙ = Pm − Vs Vr B sin(ϕˆs + ϕs0 ) + Vs B cos(ϕˆs + ϕs0 )u 1 − Vs B sin(ϕˆ s + ϕs0 )u 2 (16) where u 1 = Vup and u 2 = Vuq are the control inputs. Dynamics (16) along with (4) now resembles UPQC dynamics (7) and (4). Therefore, discrete-time dynamics (10) serve for both conventional and reduced UPQC structures connected to an RG through GTI. III. BACKGROUND ON N ONLINEAR O PTIMAL D ISCRETE -T IME C ONTROL So far, dynamics of RG/SG connected in a weak grid or microgrid is presented by (10) or its equivalent x k+1 = f (x k ) + g(x k )u k + h(x k )wk .

(17)



In this section, the optimal control problem formulation is provided for power grid representation (17) aimed in this paper. Recall that x k = [ϕˆ s (kT ) υ(kT )]T , u k = [Vup Vuq ]T . First, an appropriate cost function needs to be defined that is a function of the states to be stabilized and the control effort to be minimized. These can be done by selecting quadratic functions of the GTI/SG states errors and UPQC series voltage to form the cost function. Assume that there exists a control input such that system (17) is asymptotically stabilized, where x k = [ϕˆ s (kT ) υ(kT )]T ∈ 2 in the proposed power system. Then, our goal is to find an asymptotically stable control input u k that can minimize the infinite-horizon cost function Jk =

∞

Q(x j ) + u Tj Ru j − γ 2 w Tj Pw j

A. Hamiltonian Function According to the optimization problem (18), a Hamiltonian function can be constructed as

j =k

= Q(x k ) + u kT Ru k − γ 2 wkT Pwk + Jk+1

(18)

where disturbance wk = w(k) has its worst value. In (18), Q is a positive definite function of the system states, R and P are the design positive definite matrices, and γ is a design constant. This optimal control problem is often referred to as a zero-sum two-player differential game [17], [40] where the two players, optimal controller u ∗k and worst case disturbance wk∗ , are the solutions to (18) such that Jk |(u ∗k ,wk ) ≤ Jk |(u ∗k ,wk∗ ) ≤ Jk |(u k ,wk∗ )

∂ Jk /∂wk = 0, for u ∗k and wk∗ . The pair (u ∗k , wk∗ ) is then called the feedback saddle-point solution to the optimization problem [40], [41]. Solving stationarity conditions for nonlinear dynamics (10) leads to a nonconvex differential-difference equation and is in general very difficult to solve if not impossible [17]–[19]. Therefore, an approximate solution is utilized in this paper using the Taylor series expansion to obtain cost function J . Cost function J is then used to obtain the optimal control input u ∗k , i.e., the UPQC series voltage to overcome the fluctuations in the RG/SG’s states, in an optimal fashion with minimal control effort.

(19)

for ∀u and ∀w that make (18) bounded. In the developed power system model (10), the solution to this optimal control problem, which minimizes positive cost function (18), leads to finding an optimal control strategy u ∗k = [u ∗1 (k) u ∗2 (k)]T for the UPQC series voltage (control input) that not only has a minimal value, but also tends to minimize system states errors (i.e., effectively mitigate oscillations) while the system is perturbed by the disturbance (variations in power system bus voltages due to power fluctuations). Since cost function (18) is a positive definite function of states and input, by minimizing (18), both states and control input are minimized. Minimizing the control input is beneficial, for lower control input is equivalent to lower UPQC voltage that in turn reduces the electrical stress on the UPQC series compensator equipment and increases its lifetime. Zero-sum game formulation (18) allows for the control input to grow as the disturbance increases and reduce the input when the disturbance diminishes. In the following section, an appropriate method to obtain cost function (18) along with the optimal controller (UPQC series voltage u = [Vup Vuq ]T ) that minimizes the cost function is discussed. Once the optimal controller is obtained, it can be implemented in the UPQC series compensator to provide the required optimal stabilizing series voltage. This voltage is expected to achieve satisfactory dampening effect along with minimal required UPQC series voltage. IV. O PTIMAL C ONTROL S TRATEGY Optimal solutions must be obtained by solving (18), using the stationarity conditions [16] ∂ Jk /∂u k = 0 and

H (x k , u k , wk ) = J (x k+1 ) − J (x k ) + Q(x k ) + u kT Ru k − γ 2 wkT Pwk . (20) Note that when H (x k , u ∗k , wk∗ ) = 0, we have (18), which is called the discrete-time HJI equation. One can expand Jk = Jk+1 − Jk around the operating point x k by keeping the first two terms in the Taylor series and considering the higher order terms to be negligible. Hence, one obtains Jk = Jk+1 − Jk 1 ≈ ∇ JkT (x k+1 − x k )+ (x k+1 − x k )T ∇ 2 Jk (x k+1 − x k ) 2 (21) where ∇ J and ∇ 2 J are the gradient vector and the Hessian matrices of J , respectively, with respect to state vector x k calculated at time step k. Now, let u k be an admissible [16] control policy [i.e., it renders finite cost function (18)] applied to the nonlinear discrete-time system (17) that yields H (x k , u k , wk ) = ∇ JkT ( f (x k ) + g(x k )u k + h(x k )wk − x k ) 1 + ( f (x k ) + g(x k )u k + h(x k )wk − x k )T 2 ×∇ 2 Jk ( f (x k ) + g(x k )u k + h(x k )wk − x k ) + Q(x k ) + u kT Ru k − γ 2 wkT Pwk .

(22)

Optimal control input u ∗k and worst case disturbance wk∗ can now be found by setting the first partial derivative of (22) with respect to u k and wk , respectively, equal to zero that yields wk∗ = −Yw−1 h(x k )T ∇ J T + ∇ 2 J ( f (x k ) + g(x k )u ∗k − x k ) u ∗k

=

−Yu−1 g(x k )T

T

∇J + ∇

2

J ( f (x k ) + h(x k )wk∗

(23) − xk ) (24)

h kT (∇ 2 Jk )

= · hk − and where Yw Yu = gkT (∇ 2 Jk )gk + 2R. Approximate HJI (22) is now a nonlinear differential equation in cost function Jk and does not depend on future time step k + 1 as opposed to (20). 2γ 2 P


Despite this fact, finding a solution for the approximate HJI equation is still not easy, and thus, an iterative-based scheme along with an NN is introduced [17] in the following in order to approximate cost function Jk and reach optimal controller u ∗k [using (24)]. Convergence to the saddle point can be shown by the following theorem. Theorem [17]: Let the pair (u k , wk∗ ) correspond to an arbitrary admissible controller and the worst case disturbance as provided by (23) for system (17). In addition, let the pair (u ∗k , wk ) correspond to the optimal controller provided by (24) and an arbitrary disturbance for system (17). Then, the Hamiltonian function (22) satisfies H x k , u ∗k , wk ≤ H x k , u ∗k , wk∗ ≤ H (x k , u k , wk∗ . The theorem suggests that applying optimal controller (24) causes minimum cost function under any disturbance (minimal control effort). In addition, it implies that applying disturbance (23) causes maximum cost function under any stabilizing controller (worst case disturbance). In the following, an algorithm is introduced to find the optimal control and worst case disturbance simultaneously. The resulting optimal controller is expected to overcome a variety of disturbances as it is trained to overcome the worst case disturbance (23). The algorithm starts with an initial admissible UPQC controller applied to the series voltage, namely, u (0) , in the outer loop (control loop) while disturbance is set to zero [i.e., w(0,0) = 0] in the inner loop. Here, superscript (0) in u 0 represents the iteration number in the outer loop, while superscript (0, 0) in w(0,0) represents the iteration numbers in the outer and inner loops, respectively. Next, an NN is chosen to approximate cost function J (0,0) by solving (22) in a subset () of the states. Similarly, superscript (0, 0) in J (0,0) represents the iteration numbers in the outer and inner loops, respectively. In this setting, refers to subset = {x = [ϕˆ s υ]T | ϕˆs,min ≤ ϕˆ s ≤ ϕˆs,max , υmin ≤ υ ≤ υmax } that includes the origin where the upper and lower limits on ϕˆs and υ are selected to cover the range of variations of the two states. Once the NN is trained on and cost function is obtained, proper update is performed on disturbance wk using (23), i.e., w(i, j +1) = T (i, j )−1 h(x)T [∇ J (i, j ) + ∇ 2 J (i, j ) × ( f (x) + g(x)u (i) − x)] −Yw with i = 0 and j = 0. The inner loop proceeds with approximating cost function J (i, j +1) using NN with the new disturbance w( j +1) . This procedure continues until the disturbance does not improve in two consecutive steps, that is, |w(i, j +1) − w(i, j ) | ≤ δw for some j with δw being a user-selected convergence error. Next, control input u k is updated using (24) in the outer loop, that is, u (i+1) = T (i, j )−1 −Yu g(x)T [∇ J (i, j ) + ∇ 2 J (i, j ) ( f (x) + h(x)w(i, j ) − x)] with j = 0. Updates on control input u k proceed until no further improvement is observed in u k , i.e., |u (i+1,∞) − u (i,∞) | ≤ δu for some i with δu being a user-selected convergence error. Convergence of the outer and inner loops has been elaborated in [17]. It is shown that in order to achieve

7

convergence of the loops, it is necessary that = g T (∇ 2 J )g + 2R be positive definite and Yu T Yw = h (∇ 2 J )h − 2γ 2 P be negative definite in subset . A positive definite matrix R along with proper selection of the NN activation functions (to achieve a positive definite ∇ 2 J ) renders Yu positive definite. In order to achieve a negative definite Yw , a large design parameter γ may be taken. However, the minimum value of γ can be found by trial and error [41], which then leads to the nonlinear H∞ optimal controller. As verified by the simulation results, the training procedure is not very long for the proposed RG model (usually a few iterations for each loop), and thus, searching for minimum γ can be added to the design. However, as the focus of this paper is not the nonlinear H∞ solution, this step was not conducted in the simulations.

B. Neural Network Approximation of the Cost Function The proposed policy iteration relies on the solution of approximate HJI (22). However, solving (22) is difficult with no known closed-form solution. NNs are proven to approximate smooth nonlinear functions in a compact set [28]. This capability of NNs can be employed in both inner and outer loops of the algorithm in Fig. 4 to approximate the cost function and solve (22). One can approximate nonlinear cost function (18) with an NN as J (i, j ) ≈

L

al σl (x) = W LT σ¯ L (x)

(25)

l=1

where L is the number of hidden layer neurons, vector T is the activation function vector and forms σ¯ L = σ1 . . . σ L a basis [28], and W L = [ a1 . . . a L ]T is the NN weight matrix. The NN weights will be tuned to minimize the residual error in a least-squares sense over a set of points around the origin. The method of weighted residuals [42] helps find the NN weights in shorter time compared with conventional training (i, j ) methods. For the approximate HJI (22), cost function Jk is replaced by NN (25) with residual error e L (x) that yields H (x, u, w) = e L (x). The weights W L are determined by projecting the residual error onto ∂e L (x)/∂ W L [42] and setting the result equal to zero for ∀x = [ϕˆ s υ]T ∈ using inner product, i.e., ∂e L (x)/∂ W L , e L (x) = 0 where

c(x), d(x) = c(x)d(x)d x

≈

p

c(x)d(x)δx

(26)

i=1

p representing the number of points in the selected region around the origin. By introducing a mesh on , with the mesh size x = [ϕˆ s υ], |ϕˆ s | ≤ εϕ , and |υ| ≤ ευ , where εϕ and ευ are the small design mesh sizes, NN weights can be obtained, using p resulted points, as [17] W L = −(X T X)−1 XY

(27)



Fig. 5.

Block diagram of the proposed control scheme.

C. Successive Approximation of the HJI Equation The proposed algorithm, shown in Fig. 4 [17], performs an iterative scheme on system (17) through two loops: 1) the outer control loop and 2) the inner disturbance loop. Once the NN weight matrix W L is calculated, the cost function and optimal controller are obtained using (24) and (25), respectively, in each iteration of the algorithm. D. Controller Implementation

Fig. 4. Successive approximation procedure in finding the cost function [17].

where X and Y are defined as 1 T 2 X = ∇ σ¯ L x + x ∇ σ¯ L x ... 2 x=x 1 T 1 T 2 ∇ σ¯ L x + x ∇ σ¯ L x 2 x=x p ⎡ ⎤ T T (i) (i) 2 (i, j ) Ru − γ w Pw(i, j ) )x=x1 (Q(x) + u ⎢ ⎥ .. ⎥. (28) Y =⎢ . ⎣ ⎦ (Q(x) + u (i) Ru (i) − γ 2 w(i, j ) Pw(i, j ) )x=x p T

T

The algorithm in Fig. 4 starts with an admissible control. Instead, in our results, we choose a stabilizing nonlinear controller since it is difficult to find an initial admissible controller for the selected grids. Design function Q(x) is selected as the quadratic form x T Qx with a positive definite matrix Q. In selecting design gains Q, R, P, and γ , we assign higher weights to R and P to emphasize on the effect of the disturbance and minimal control efforts. However, selection of the optimum design gains depends on the grid specifications and remains an open research topic. We have noticed that improper selection of the gains can prevent the algorithm from convergence indicating there is no optimal solution. In general, the loops converge relatively fast based on our results for small grids (more details are presented in Section V). In order to achieve the best results, some trial and error should be conducted on the gains; however, as the loops converge fast, this does not impose a great challenge in the controller design. Overall, the proposed controller is expected to perform better than its nonoptimal nonlinear and optimal linear counterparts due to its optimality and less dependency on the operating point. Finally, proper NN activation functions should be selected for faster convergence. Our results show that polynomials (discussed in simulations) are effective for weak grid and microgrid as the number of dynamical states are limited. However, for larger systems, other basis functions may be selected, since with polynomials, the number of the weights to be determined will be excessively high. The overall block diagram of the controller is shown in Fig. 5. V. S IMULATION R ESULTS In this section, simulation studies are carried out on weak and islanded grids. The proposed UPQC optimal controller is


9

TABLE II G RID PARAMETERS AT L OW P OWER

Fig. 6.

Wind farm connected to weak grid [43]. TABLE I M ICROGRID PARAMETERS AT H IGH P OWER [43]

P = 1, and γ = 20 are selected. The region −1 ≤ ϕˆs ≤ 1, −10 ≤ υ ≤ 10 is used to provide the mesh required in (27) and training is performed using the NN while the mesh size in the (ϕˆ s , υ) plane is chosen to be (εϕ , εv ) = (0.06, 0.6). The activation functions of the NN are polynomial terms obtained by expanding even polynomial (ϕˆ s + υ)2 + (ϕˆs + υ)4 (to address positive definite cost function and Hessian matrix). The cost function is then approximated by the selected NN whose weights are obtained using (27). The fourth-order polynomial is selected by trial and error and to retain simplicity of the training. For simplicity, instead of an initial admissible controller, a nonlinear stabilizing controller is chosen in the training algorithm as u 1 = 125(−υ × C/(BVs τ ) cos(ϕˆ s + .069) − ϕˆs × C/(BVs τ ) cos(ϕˆs + .069)) u 2 = 125(−υ × C/(BVs τ ) sin(ϕˆs + .069) − ϕˆ s × C/(BVs τ ) sin(ϕˆ s + .069)).

(29)

After the completion of the offline training with four outerloop and maximum nine inner-loop (per outer loop) iterations, the final NN weights are W L = [−13.9114 −0.0719 −0.0151 109.4428 0.0009 0.1693 −0.0000 0.1369]T and the cost function is approximated as J (ϕˆ s , υ) = −13.9114ϕˆ s4 − 0.0719ϕˆ s3υ − 0.0151ϕˆ s2υ 2 + 109.4428ϕˆ s2 + 0.0009ϕˆ s υ 3 obtained through the algorithm in Fig. 4 and applied to GTI and conventional SG in the grids. Case Study 1—GTI Control: The first test grid, shown in Fig. 6, represents a wind farm connected to a small power grid [43] through a GTI. Here, the weak grid is simulated by a small SG. The RG in the network under consideration represents wind turbines (in a wind farm) connected to the network using inverter (GTI). The UPQC is employed to add damping to the oscillations that occur after perturbations in an optimal fashion with minimal control effort besides its power quality tasks. The network is modeled in the MATLAB/Simulink environment using the data given in Table I [43]. The SG model includes high-order stator and rotor dynamics while speed governor and PSS are applied to the SG in all the simulations. The RG is modeled using dynamics (3) and (4). The effect of the UPQC is modeled using (10) that presents both conventional and reduced topologies. In all the simulations, in verifying the effectiveness of the modeling and design, detailed transmission lines’ models including inductances, resistances, and capacitances are considered. In the optimal controller training algorithm, design parameters Q(x k ) = x kT Qx k , Q = 0.01 × I2×2 , R = I2×2 ,

+ 0.1693ϕˆ s υ + 0.1369υ 2.

(30)

Using (30), nonlinear optimal controller (24) is applied to control the UPQC series voltage through a zero-order hold to enhance the stability. In order to verify the performance of the proposed optimal controller in the selected weak grid, a three-phase fault is placed in the network as shown in Fig. 6 at t = 0.2 s and is removed at t = 0.25 s. The controller is bypassed during the fault to avoid excessive series voltage, and activated after fault removal (t = 0.25 s). In addition, the robustness of the proposed controller is investigated under high and low load levels, as shown in Tables I and II, respectively. Figs. 7–10 show the satisfactory damping effect of the UPQC controlled by the proposed nonlinear optimal controller on the considered system with high load levels compared with those of the initial stabilizing controller. In addition, these results are compared with damping performance of a conventional PQ controller (in the absence of UPQC). PQ controller consists of a PID controller for P with the following parameters: k p = 1.163, ki = 0.105, kd = 0, and a PID controller for Q with k p = 0.0001, ki = 0.001, and kd = 0.001. In all the cases, the SG is equipped with governor and PSS. These results show the effectiveness of both the optimal and


Fig. 7. Case study 1—performance comparison of the proposed optimal nonlinear controller, nonoptimal nonlinear controller, and conventional PQ controller—SG speed and RG dc-link voltage; high load levels.

Fig. 8. Case study 1—UPQC series voltage comparison of nonoptimal nonlinear controller and the proposed optimal nonlinear controller—high load levels.


Fig. 11. Case study 1—performance comparison of the proposed optimal nonlinear controller, nonoptimal nonlinear controller, and conventional PQ controller—SG speed and RG dc-link voltage; low load levels.

Fig. 12. Case study 1—UPQC series voltage comparison of nonoptimal nonlinear controller and the proposed optimal nonlinear controller—low load levels.

Fig. 13. Case study 1—cost function comparison of nonoptimal nonlinear controller and the proposed optimal nonlinear controller—low load levels.

Fig. 9. Case study 1—injected power of UPQC comparison of nonoptimal nonlinear controller and the proposed optimal nonlinear controller—high load levels.

Fig. 10. Case study 1—cost function comparison of nonoptimal nonlinear controller and the proposed optimal nonlinear controller—high load levels.

nonoptimal controller over the conventional PQ controller and PSS in damping the oscillations. Next, the control effort of the proposed nonlinear optimal controller is compared with that of the nonlinear nonoptimal controller. The instantaneous UPQC series injected voltage and power into the transmission line using the proposed controller are in acceptable ranges as shown in Figs. 8 and 9, where the

maximum UPQC power is around 1.2 MW that is about 5% of the transmission line power. Note that the optimal nature of the proposed controller causes lower control effort under the same damping performance compared with the initial nonlinear stabilizing controller (29), as shown in Figs. 7–9. Fig. 10 represents the cost function (30) comparison for the two controllers where the optimal controller has around 30% less control cost. Overall, Figs. 7–10 show the effectiveness of the proposed controller over the conventional PQ controller and PSS as well as the nonoptimal nonlinear controller (29). Next, optimal controller (29) is tested under low load level shown in Table II under the same fault scenario. Figs. 11 and 12 show the satisfactory damping performance of the proposed controller verifying the robustness of the optimal controller working at different operating points with the cost comparison shown in Fig. 13. Next, the effectiveness of the reduced UPQC structure presented in Section II-D is investigated when optimal controller (24) is applied in the system of Fig. 6. Figs. 14–16 show the damping performance of the optimal controller with the UPQC reduced structure using the parameters of Tables I and II. The simulation results indicate almost


11

Fig. 14. Case study 1—SG speed and RG dc-link voltage; UPQC reduced structure; proposed optimal controller; high and low load levels.

Fig. 17.

Benchmark low-voltage microgrid. TABLE III G RID PARAMETERS W ITH H IGH L OAD L EVELS

Fig. 15. Case study 1—UPQC series voltage using UPQC reduced structure; proposed optimal controller; high and low load levels.

Fig. 16. Case study 1—injected power of UPQC using reduced structure; proposed optimal controller; high and low load levels.

the same UPQC series voltage and power as those of UPQC conventional structure. Case Study 2: In the second simulation scenario, the performance of the proposed HJI optimal controller is compared with its linear optimal counterpart (Linear quadratic regulator (LQR)) in damping power system oscillations. Here, the effectiveness of the HJI controller is studied in the islanded microgrid of Fig. 17 where oscillation in the SG is controlled by UPQC. The microgrid comprises an SG, loads, and renewable sources. The initial task of the UPQC is to protect the SG against the current asymmetry and harmonics caused by the loads and renewables. It also reduces the voltage asymmetry and harmonics caused by the load at the generator bus. Table III shows the specifications of the test system. According to Remark 2, model (10) can be utilized to model a classical-model SG [35]. Thus, here, the proposed nonlinear optimal controller (24) is applied to the UPQC series transformer to improve the SG’s transient characteristics. Despite the simplified modeling, full generator’s stator and rotor dynamics are considered in the simulation using

MATLAB/Simulink to show the effectiveness of the design in the real system. In addition, SG’s speed governor and PSS are in action in all simulations. Loads are constant impedances with power factors 0.9 and 1. The renewable energy sources are modeled to operate under a constant phase angle difference with the network in order to transmit constant power to the network under steady-state conditions. These sources are connected to the grid through impedances. In addition, unlike the previous scenario, it is assumed that the renewable energy sources are equipped with storage to help reduce sudden transmitted power fluctuations as the proposed control is now only applied to the SG and not the RGs. In the training algorithm, design parameters Q(x k ) = x kT Qx k with gains Q = 0.01 × I2×2 , R = I2×2 , P = 10, and γ = 10 are selected. The initial admissible controller is an LQR with the same design gains. The region −1 ≤ ϕˆs ≤ 1, −10 ≤ υ ≤ 10 is considered to provide the mesh and training is performed using the NN while the mesh size is selected to be (εϕ , εv ) = (0.06, 0.6).



Fig. 18. Case study 2—performance comparison of the proposed optimal nonlinear controller and optimal linear controller (LQR)—speed of SG; high load levels; fault duration 0.2 s.

Fig. 20. Case study 2—injected power of UPQC comparison of the proposed optimal nonlinear controller and optimal linear controller (LQR)—high load levels; fault duration 0.2 s.

Fig. 21. Case study 2—performance comparison of the proposed optimal nonlinear controller and optimal linear controller (LQR)—speed of SG; high load levels; fault duration 0.1 s. Fig. 19. Case study 2—UPQC series voltage comparison of the proposed optimal nonlinear controller and optimal linear controller (LQR)—high load levels; fault duration 0.2 s.

Note that ϕˆ s and υ are the SG’s angle and speed errors, respectively. Using the same-degree polynomial as in the previous scenario, the cost function is obtained using NN and weighted residual method (27). By finishing the offline training (with three outer loop iterations and maximum 14 inner loop iterations), the final NN weights −0.1956 −0.0143 129.6711 are W L = [−55.5171 0.0010 0.2134 0.0000 0.0236]T, that is, the cost function is approximated as J (ϕˆ s , υ) = −55.5171ϕˆ s4 − 0.1956ϕˆ s3υ − 0.0143ϕˆ s2υ 2 + 129.6711ϕˆ s2 + 0.001ϕˆ s υ 3 + 0.2143ϕˆ s υ + 0.0236υ . 2

(31)

A three-phase fault is placed at the UPQC bus at t = 3 s and is removed at t = 3.2 s. The controller acts after fault removal (t = 3.2 s). Fig. 18 shows the damping effect of the UPQC controlled by optimal controller (24) as compared with that of the LQR optimal controller (used in the training algorithm) in the presence of PSS + governor. From Fig. 18, the nonlinear optimal controller can damp the after-fault oscillations with less overshoot and settling time compared with LQR. The instantaneous UPQC series voltages are shown in Fig. 19, while the UPQC injected powers into the line in the two cases are shown in Fig. 20. The results shown in Figs. 18–20 show the effectiveness of the proposed controller where better transient response is achieved by the proposed nonlinear optimal controller, while the voltage and power injected by UPQC series

Fig. 22. Case study 2—UPQC series voltage comparison of the proposed optimal nonlinear controller and optimal linear controller (LQR)—high load levels; fault duration 0.1 s.

transformer are also lower than those of linear optimal controller LQR. Figs. 19 and 20 show that the series injected voltage is less than 0.1 pu and the maximum injected power by UPQC is around 2 kW, which is about 3% of the line power. Next, in the previous test, the fault duration is changed to 0.1 s and the performances of the controllers are compared. Due to the change in the network postfault conditions, the performance of the LQR controller varies as opposed to that of nonlinear controller (24), as shown in Figs. 21–23. This is due to the sensitivity of the linear controller (LQR) to the system operating point when using the network linearized model as opposed to the proposed nonlinear controller. As shown in Figs. 22 and 23, LQR, even with the more control effort, cannot damp the oscillation as effective as controller (24). At the end, the proposed controller is tested under a different load condition given in Table IV in the microgrid


Fig. 23. Case study 2—injected power of UPQC comparison of the proposed optimal nonlinear controller and optimal linear controller (LQR)—high load levels; fault duration 0.1 s. TABLE IV G RID PARAMETERS W ITH L OW L OAD L EVELS

13

Fig. 25. Case study 2—UPQC series voltage comparison of the proposed optimal nonlinear controller and optimal linear controller (LQR); low load levels; fault duration 0.1 s.

VI. C ONCLUSION A nonlinear optimal controller design scheme is utilized to mitigate the oscillations in renewable energy resource and SG connected in weak grid and microgrid. Through a novel modeling approach, the grid equipped with a UPQC is modeled as a nonlinear discrete-time dynamical system that accommodates GTI and SG in weak grid and islanded microgrid, and thus an infinite bus is not assumed. The UPQC performs as a damping controller through injected series voltage besides its power quality tasks. Subsequently, a zero-sum two-player game approach is adopted to design an HJI nonlinear optimal controller through successive approximation of a cost function using NNs. The nonlinear optimal control is then applied to mitigate the oscillations in the weak and islanded grids where it is shown that effective damping performance is obtained in both GTIs and SGs. When applied to GTIs, a reduced structure UPQC is shown effective in improving the dynamic stability of the grid.

Fig. 24. Case study 2—performance comparison of the proposed optimal nonlinear controller and optimal linear controller (LQR); speed of SG; low load levels; fault duration 0.1 s.

and applying the same controllers used with the high load levels. In this scenario, PSS is not able to mitigate the speed oscillations as it operates at a different operating point than it is designed for. Under the same fault conditions (occurring at 3.0 s and ending at 3.1 s), Figs. 24 and 25 show the damping performances of optimal controller (24) and LQR used in the training algorithm. The results verify the robustness of the proposed nonlinear controller and its effectiveness over both the LQR and PSS + governor when the system equilibrium is changed as a result of load change. In fact, since the linear optimal controller (LQR) is designed at a specific operating point (due to linearized system), the performance of the controller drops and makes the system unstable as the load changes in the power network. By contrast, optimal controller (24) can still improve damping the oscillations. However, note that the steady-state UPQC series voltage is not zero as a result of simplified SG model used in the control design, as shown in Fig. 25.

R EFERENCES [1] V. Khadkikar, “Enhancing electric power quality using UPQC: A comprehensive overview,” IEEE Trans. Power Electron., vol. 27, no. 5, pp. 2284–2297, May 2012. [2] K. Karanki, G. Geddada, M. K. Mishra, and B. K. Kumar, “A modified three-phase four-wire UPQC topology with reduced DC-link voltage rating,” IEEE Trans. Ind. Electron., vol. 60, no. 9, pp. 3555–3566, Sep. 2013. [3] G. J. Li, F. Ma, S. S. Choi, and X. P. Zhang, “Control strategy of a cross-phase-connected unified power quality conditioner,” IET Power Electron., vol. 5, no. 5, pp. 600–608, May 2012. [4] P. E. Melin et al., “Analysis, design and control of a unified power-quality conditioner based on a current-source topology,” IEEE Trans. Power Del., vol. 27, no. 4, pp. 1727–1736, Oct. 2012. [5] E. W. Kimbark, “Improvement of system stability by switched series capacitors,” IEEE Trans. Power App. Syst., vol. PAS-85, no. 2, pp. 180–188, Feb. 1966. [6] Y. Tang and A. P. S. Meliopoulos, “Power system small signal stability analysis with FACTS elements,” IEEE Trans. Power Del., vol. 12, no. 3, pp. 1352–1361, Jul. 1997. [7] M. H. Haque, “Improvement of first swing stability limit by utilizing full benefit of shunt FACTS devices,” IEEE Trans. Power Syst., vol. 19, no. 4, pp. 1894–1902, Nov. 2004. [8] L.-J. Cai and I. Erlich, “Simultaneous coordinated tuning of PSS and FACTS damping controllers in large power systems,” IEEE Trans. Power Syst., vol. 20, no. 1, pp. 294–300, Feb. 2005. [9] M. H. Haque, “Evaluation of first swing stability of a large power system with various FACTS devices,” IEEE Trans. Power Syst., vol. 23, no. 3, pp. 1144–1151, Aug. 2008.


[10] N. Yang, Q. Liu, and J. D. McCalley, “TCSC controller design for damping interarea oscillations,” IEEE Trans. Power Syst., vol. 13, no. 4, pp. 1304–1310, Nov. 1998. [11] H. Wang, “A unified model for the analysis of FACTS devices in damping power system oscillations. III. Unified power flow controller,” IEEE Trans. Power Del., vol. 15, no. 3, pp. 978–983, Jul. 2000. [12] C.-T. Chang and Y.-Y. Hsu, “Design of UPFC controllers and supplementary damping controller for power transmission control and stability enhancement of a longitudinal power system,” IEE Proc.-Generat., Transmiss., Distrib., vol. 149, no. 4, pp. 463–471, Jul. 2002. [13] Y. Pipelzadeh, B. Chaudhuri, and T. C. Green, “Control coordination within a VSC HVDC link for power oscillation damping: A robust decentralized approach using homotopy,” IEEE Trans. Control Syst. Technol., vol. 21, no. 4, pp. 1270–1279, Jul. 2013. [14] B. Chaudhuri, R. Majumder, and B. C. Pal, “Application of multiple-model adaptive control strategy for robust damping of interarea oscillations in power system,” IEEE Trans. Control Syst. Technol., vol. 12, no. 5, pp. 727–736, Sep. 2004. [15] C. Meza, D. Biel, D. Jeltsema, and J. M. A. Scherpen, “Lyapunov-based control scheme for single-phase grid-connected PV central inverters,” IEEE Trans. Control Syst. Technol., vol. 20, no. 2, pp. 520–529, Mar. 2012. [16] F. L. Lewis and V. L. Syrmos, Optimal Control. New York, NY, USA: Wiley, 1995. [17] S. Mehraeen, T. Dierks, S. Jagannathan, and M. L. Crow, “Zero-sum two-player game theoretic formulation of affine nonlinear discrete-time systems using neural networks,” IEEE Trans. Cybern., vol. 43, no. 6, pp. 1641–1655, Dec. 2013. [18] R. W. Bea, “Successive Galerkin approximation algorithms for nonlinear optimal and robust control,” Int. J. Control, vol. 71, no. 5, pp. 717–743, 1998. [19] M. Abu-Khalaf, F. L. Lewis, and J. Huang, “Neurodynamic programming and zero-sum games for constrained control systems,” IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1243–1252, Jul. 2008. [20] Z. Chen and S. Jagannathan, “Generalized Hamilton–Jacobi–Bellman formulation-based neural network control of affine nonlinear discretetime systems,” IEEE Trans. Neural Netw., vol. 19, no. 1, pp. 90–106, Jan. 2008. [21] A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, “Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 943–949, Aug. 2008. [22] H. Zhang, Y. Luo, and D. Liu, “Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints,” IEEE Trans. Neural Netw., vol. 20, no. 9, pp. 1490–1503, Sep. 2009. [23] H. Zhang, Q. Wei, and Y. Luo, “A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 937–942, Aug. 2008. [24] J. Huang, “An algorithm to solve the discrete HJI equation arising in the L 2 gain optimization problem,” Int. J. Control, vol. 72, no. 1, pp. 49–57, 1999. [25] S. Mehraeen and S. Jagannathan, “Decentralized optimal control of a class of interconnected nonlinear discrete-time systems by using online Hamilton–Jacobi–Bellman formulation,” IEEE Trans. Neural Netw., vol. 22, no. 11, pp. 1757–1769, Nov. 2011. [26] H.-N. Wu and B. Luo, “Neural network based online simultaneous policy update algorithm for solving the HJI equation in nonlinear H∞ control,” IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 12, pp. 1884–1895, Dec. 2012. [27] T. Dierks and S. Jagannathan, “Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using timebased policy update,” IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1118–1129, Jul. 2012. [28] J. Sarangapani, Neural Network Control of Nonlinear Discrete-Time Systems. Boca Raton, FL, USA: CRC Press, 2006. [29] P. W. Lehn and M. R. Irvani, “Discrete time modeling and control of the voltage source converter for improved disturbance rejection,” IEEE Trans. Power Electron., vol. 14, no. 6, pp. 1028–1036, Nov. 1999. [30] M. P. N. van Wesenbeeck, S. W. H. de Haan, P. Varela, and K. Visscher, “Grid tied converter with virtual kinetic storage,” in Proc. IEEE Bucharest PowerTech, Jun./Jul. 2009, pp. 1–7. [31] J. Ekanayake and N. Jenkins, “Comparison of the response of doubly fed and fixed-speed induction generator wind turbines to changes in network frequency,” IEEE Trans. Energy Convers., vol. 19, no. 4, pp. 800–802, Dec. 2004.


[32] Q.-C. Zhong and G. Weiss, “Synchronverters: Inverters that mimic synchronous generators,” IEEE Trans. Ind. Electron., vol. 58, no. 4, pp. 1259–1267, Apr. 2011. [33] M. F. M. Arani and E. F. El-Saadany, “Implementing virtual inertia in DFIG-based wind power generation,” IEEE Trans. Power Syst., vol. 28, no. 2, pp. 1373–1384, May 2013. [34] S. Kazemlou and S. Mehraeen, “Novel decentralized control of power systems with penetration of renewable energy sources in small-scale power systems,” IEEE Trans. Energy Convers., vol. 29, no. 4, pp. 851–861, Dec. 2014. [35] P. W. Sauer and M. A. Pai, Power System Dynamics and Stability. Champaign, IL, USA: Stipes, 2000. [36] E. Gholipour and S. Saadate, “Improving of transient stability of power systems using UPFC,” IEEE Trans. Power Del., vol. 20, no. 2, pp. 1677–1682, Apr. 2005. [37] M. Noroozian, L. Angquist, M. Ghandhari, and G. Andersson, “Use of UPFC for optimal power flow control,” IEEE Trans. Power Del., vol. 12, no. 4, pp. 1629–1634, Oct. 1997. [38] E. Romero-Cadaval, V. Minambres-Marcos, and M.-I. Milanes-Montero, “Analysis and optimization of sinusoidal voltage source inverter losses for variable output power applications,” in Proc. 7th Int. Conf.-Workshop Compat. Power Electron. (CPE), Jun. 2011, pp. 230–235. [39] H. Nazaripouya and S. Mehraeen, “Control of UPFC using Hamilton–Jacobi–Bellman formulation based neural network,” in Proc. IEEE PES General Meeting, San Diego, CA, USA, Jul. 2012, pp. 1–8. [40] T. Ba¸sar and P. Bernhard, H∞ -Optimal Control and Related Minimax Design Problems. Basel, Switzerland: Birkhäuser, 1995. [41] M. Abu-Khalaf, F. L. Lewis, and J. Huang, “Policy iterations on the Hamilton–Jacobi–Isaacs equation for H∞ state feedback control with input saturation,” IEEE Trans. Autom. Control, vol. 51, no. 12, pp. 1989–1995, Dec. 2006. [42] B. A. Finlayson, The Method of Weighted Residuals and Variational Principles. New York, NY, USA: Academic, 1972. [43] Z. Saad-Saoud, M. L. Lisboa, J. B. Ekanayake, N. Jenkins, and G. Strbac, “Application of STATCOMs to wind farms,” IEE Proc.-Generat., Transmiss., Distrib., vol. 145, no. 5, pp. 511–516, Sep. 1998.

Hamidreza Nazaripouya received the B.S. degree in electrical engineering from the University of Tehran, Tehran, Iran, in 2007, the M.S. degree in power electronics from the Sharif University of Technology, Tehran, in 2010, and the M.S. degree in power systems from Louisiana State University, Baton Rouge, LA, USA, in 2013. He is currently pursuing the Ph.D. degree with the Smart Grid Energy Research Center, University of California at Los Angeles, Los Angeles, CA, USA. He has conducted several projects for utility companies during his education. His current research interests include the application of power electronics in power system, renewable energy integration, power system stability, microgrid technologies, and electric vehicle.

Shahab Mehraeen (S’08–M’10) received the B.S. degree in electrical engineering from the Iran University of Science and Technology, Tehran, Iran, in 1995, the M.S. degree in electrical engineering from the Esfahan University of Technology, Esfahan, Iran, in 2001, and the Ph.D. degree in electrical engineering from the Missouri University of Science and Technology, Rolla, MO, USA, in 2009. He joined Louisiana State University, Baton Rouge, LA, USA, as an Assistant Professor, in 2010. He conducts research on decentralized, adaptive, and optimal control of dynamical systems. He holds a U.S. patent in energy harvesting. His current research interests include microgrids, renewable energies, power systems dynamics, protection, and smart grids. Dr. Mehraeen received the National Science Foundation CAREER Award.