GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS Jesus VelásquezBermúdez, Eng. D. Chief Scientist, DecisionWare  DO Analytics LLC
[email protected]
ABSTRACT This technical paper presents theoretical considerations about the solution of dynamic optimization problems integrating the Benders Theory, the Dynamic Programming approach and the concepts of Control Theory. The so called Generalized Dual Dynamic Programming Theory (GDDP) was initially published by Velasquez (2002); it can be considered as an extension of previous approach known as Dual Dynamic Programming (DDP) (Pereira and Pinto, 1985, 1991) which is a variation of Nested Benders (NB) methodologies (Murphy 2013). From the point of view of Dynamic Programming (DP), the main difference between the traditional Nested Benders (NB) methodologies and GDDP is that the conceptual formulation NB considers all the variables of the problem as state variables, while the GDDP makes a distinction between state variables and control variables. This distinction permits a more detailed algorithm in which the subproblems are smaller than in the NB. The NB has been widely used in the economic dispatch of power systems. The main limitation of NB, including DDP, is that it only solves linear models. The paper of Velasquez (2002) was not followed by the publication of the experimental results of GDDP. This article revisits Velasquez studies extending the theory, analyzing the implementation and presenting experimental results, using models oriented to the electric sector Key words: Benders Decomposition, Nested Benders, Dynamic Programming, Dual Dynamic Programming, Discrete Control Theory, Electric Sector Optimization 1.
Introduction
1.1. Background This paper presents the computational implementation of the Generalized Dual Dynamic Programing (GDDP) whose theory was developed by J. M. Velasquez (2002). The GDDP is based on the chained application of J. F. Benders Theory (BT) (Benders, 1962) applied to the solution of multiperiod dynamic problems using the Dynamic Programming (DP) and the Discrete Control Theory (DCT) approaches. Two previous works must be kept in mind as an initial reference of GDDP, both were called Dual Dynamic Programming (DDP). First, in a series of papers published starting in 1985 M. V. F. Pereira and M. V. G. Pinto (Pereira and Pinto, 1985, 1991) (Pereira, 1989) introduced a methodology that made it possible to apply the DP principles to problems with multiple state variables. Pereira and Pinto used Benders' Partition technique to the multiplestage problem, which allowed the replacement of the discretization of the future cost function in the classic DP approach by a series of hyperplanes generated using BT. DDP limits the relations of the state variables to two consecutive periods; in 1995, Velasquez (1995) studied the application of BT to a more general problem which allows the direct relationship between the state variables of a period t with state variables of any previous period, this case is included en GDDP. The main difference between DDP and GDDP is that the conceptual formulation DDP considers all the variables of the problem as state variables, while the GDDP makes a distinction between state variables and control variables. This distinction permits a more detailed algorithm in which the subproblems are smaller than in the DDP. The DDP has been widely used in the economic dispatch of power systems, its main limitation is that DDP only solves linear models. E. G. Read and others developed the second work (Casseboom and Read, 1987) (Read, 1989) (Read and George, 1990), that they called Dual Dynamic Programming (DDPR). The conceptual formulation of the DDPR makes the distinction between control variables and state variables. The DDPR may be seen as a dual formulation of the classic DP approach in the sense that DP chooses an arbitrary grid of primal state variables and for each one finds the optimal decision and the shadow price of the state variables (dual variables). The DDPR solves parametrically a subproblem in each period to produce points of the "supply function" for that period and, using backward recursion, combines these supply functions with the endofperiod function value to define an optimal strategy.
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
The problem GDDP: considers a more detailed formulation than DDP in which the variables that couple two consecutive periods, related to the stock level and the state of the infrastructure (onoff), correspond to the state variables (xt), while the production and distribution variables in the period t, not involved directly in the dynamic relationship, may be thought as the control variables (ut). The application of GDDP theory solves a largescale optimization problem as a coordinated sum of very simple problems; it entails a significant increase in speed to solve large problems and in the dimensionality of these. Additionally, the structure of GDDP is oriented to asynchronous parallel optimization and distributed realtime optimization. The GDDP is applicable to a wide variety of systems and dynamic problems. This paper presents as an example its utility in the modeling of the electric sector. 1.2. Benders' Partition Theory BT considers the problem P: composed by two types of variables: y, the coordination variables, and x, the coordinates. P: = { Min z = cTx + f(y)  F0(y) = b0 Ax + F(y) = b xR+ , yS } (1) BT restricts the model on x to be a linear problem, while it doesn’t impose conditions on y that may be continuous, or discrete. Additionally, the functions f(y) and F(y) may be nonlinear convex functions. The P: problem is partitioned in two coordinated problems: CY: over y and SP(y): over x which is defined as SP(y): = { Min Q(y) = cTx  Ax = b  F(y), xR+ }
(2)
The coordinator CY: on y can be formulated as CY: = { Min z = f(y) + Q(y)  F0(y) = b0 , yS Q(y) (k)T[b  F(y)] k IT 0 (k)T[b  F(y)] kIN }
(3)
where represents the vector of dual variables of the restrictions Ax = bF(y), IT the set of iterations, an extreme ray on the feasibility region and IN the set of iterations on which no feasibility was obtained. Benders proposed the solution of P: by a hierarchical algorithm that works on two levels: i) the coordination level solves the problem CY: and generates a sequence of yk values; ii) on the second level, yk is used as a parameter of the subproblem SP(y): to generate a sequence of feasible extreme points, k, and extreme rays, k, of the dual feasible zone of SP(y):, this vectors are used to include cutting planes in CY:, that includes two types of cuts. The first type, that we call optimality cutting planes (OCP) restricts the feasible zone of y to obtain the optimal y; it has the following structure Q(y) (k)T[b  F(y)]
k IT
(4)
The second type, that we call feasibility cutting planes (FCP) restricts the feasible zone of y to maintain feasible x in SP(y), it has the following structure 0 (k)T[b  F(y)]
kIN
(5)
For simplicity, in the following sections the FCPs are ignored, but they can be included following the same way used for the OBCs. 1.3. DP: Dynamic Programming Dynamic Programming (DP), as it was formulated by Bellman and Dreyfus (1962), is a numerical methodology for solving optimization problems related to dynamic technoeconomic systems, understood as those problems whose 2
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
planning horizon covers multiple stages/periods; whose results, in the absence of computational limitations, coincide with those obtained using exacts methodologies based on Mathematical Programming. The principle of optimality of DP, which was developed by Richard Bellman (1952, 1957), established that “an optimal path has the property that whatever the initial conditions and control variables (choices) over some initial period, the control (or decision variables) chosen over the remaining period must be optimal for the remaining problem, with the state resulting from the early decisions taken to be the initial condition. To define the dynamic evolution of the system are used state variables, that link two consecutive stages. This principle guarantees the optimality if the separability condition is satisfied. For this study, we are interested in a DP problem in which all the mathematical elements must be separable in the time domain, with mixednonlinearconvex relations for the state variables (xt) and linear relations for all control variables (ut). DP: = { Min z = t [ ct(xt) + dt ut ]  Ft(xt1, xt ) = ft t At(xt1, xt) + Btut = bt t Gt ut = gt t + utR txtS t }
(6)
where t represents the periods, xt the vector of state variables and ut the vector of control variables. Ft() and At() technoeconomical functions, Et , Bt , and Gt are technoeconomical matrices, ft , bt and gt are resources vectors, ct() cost function for the state variables and dt cost vector for the control variables. 1.4. DDP: Dual Dynamic Programming The mathematical formulation considered in the DDP, and in general in the methodologies referred to as Nested Benders, is based on a linear model DDPO: = { Min t=1,T ctT yt  At yt = bt  Et1 yt1 , t=1,T ; ytR+ }
(7)
where yt is a vector that represents, jointly, the state variables and the control variables, or yt = { xt , ut }
(8)
If the vector yt is divided in state and control variables we can see the problem as DP:, equations (6). DDP solves a more restrictive problem than DP because DDP only applies to linear dynamic problems (DDP:); this is because the DDP considers all the variable as state variables. The DDP approach implies the multilevel decomposition (in the time domain) of multiple problems that are decomposed using BT, generating new problems that again are decomposed using BT (Nested Benders, NB). Considering that DDP only has state variables, the reduce form of previous problem is: DDP: = { Min z = t ctTxt  At xt + Et1 xt1 = bt t, xtR+ }
(9)
As a starting point, DDP divided the variables in two groups: i) coordination variables associated with stages/periods between 1 and T1 {x1, x2, ... , xT1}, and ii) coordinated variables associated with stage T {xT}. Based on the direct application of BT the coordinator model for the period {1,T1} is CDT1: = { Min z = t=1,T1 ctTxt + T(xT1)  At xt = bt  Et1 xt1 t=1,T1 , xtR+ t=1,T1) T(xT1) + kTTET1 xT1 kTTbT kIT(T1) }
3
(10)
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
where t(xt1) represents the future cost function considering that the system is on state xt1 at the beginning of period t, IT(t) the amount of cuts included in the coordinator CDt, and kt the vector of dual variables of the restrictions At xt = bt  Et1 xkt1 obtained as a solution of the subproblem SDt(xkt1). We call the cuts as Future Cost Benders Cuts (FCBC). For the stage T the linear subproblem SDT(xkT1) is defined as SDT(xkT1): = { Min T(xkT1) = cTTxT ATxT = bT  ET1xkT1 , xTR+ }
 (11)
Using a recursive approach, the coordinator for each intermediate stage t (less than T) is CDt: = { Min z = cTx + t+1(xt)  Ax = b  E1x1 , xR+ t+1(xt) + jt+1TEtxt t+1jjIT(t) }
(12)
and the subproblem for each intermediate stage t is SDt(xjt1): = { Min t(xjt1) = ctTxt + t+1(xt)  Atxt = bt  Et1xjt1 , t+1(xt) + kt+1TEtxt t+1kkIJ(t,j) }
xtR+ (13)
where IJ(t,j) represents the number of cuts included in the coordinator CDt: when we solve the subproblem SDt(xjtj 1). t is a constant value calculated as tj =
{
jTTbT jtTbt + k=1,IJ(t,j) k,jtt+1k
t=T t = 1,T1 (14)
where k,jt represents the dual variable of the kth Benders cut in the subproblem SDt(xjt1). The index j represents the iteration of the coordinator of SDt(xjt1) and the index k the cuts that have been included in the process of solution of SDt(xjt1). 2.
GDDP: Generalized Dual Dynamic Programming
2.1. GDDP Basic Theory In 2002, Velasquez (2002) considered the solution of the same linear dynamic problem that considers DDP. Due to the partitioning the variables in state variables and control variables, the previous formulation may be relaxed to include: i) discrete variables in the vector of state variables; ii) nonlinear nonseparable convex relations between the state variables; and nonlinear nonseparable convex cost functions; then, the GDDP: problem may be formulated as: GDDP: = { Min z = c(x1 , x2 , … , xT) + t dtTut  F(x1 , x2 , … , xT) = f A(x1 , x2 , … , xT) + Btut = bt t Gt ut = gt t + utR txtSt t }
(15)
where ct(x) is the cost function of the state variables, Ft(x) and At(x) vectorial functions, and St the domain of x that may be continuous or discrete The previous formulation represents many industrial systems in which state variables are associated with: i) the stocks (inventories) and ii) the state of the production units (onoff); and control variables are associated with the production and distribution activities through the supplychain and/or the production factories which are separable between periods. The solution of GDDP: using BT considers a twostage process: i) first, we define the state variable xt as the coordination variables to decouple the problem at temporary level solving static subproblems SUt(xt1,xt): for each t, optimizing the control variables ut and ii) in the second stage, the GDDP solves the coordinator problem, CX:, 4
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
optimizing the state variables xt. CX: is stated as CX: = { Min z = c(x1 , x2 , … , xT) + t t(xt1,xt)  F(x1 , x2 , … , xT) = f { Min t(xt1,xt) = dtTut  Btut = bt  A(x1 , x2 , … , xT) , Gtut = gt , utR+ } xtS t }
t (16)
where t(xt1,xt) represents the optimum operation costs in the period t as consequence of the border condition (x1 , x2 , … , xT) and it corresponds to the objective function of the linear subproblems SUt(xt1,xt): defined as SUt(xt1,xt): = { Min t(xt1,xt) = dtTut  Btut = bt  A(x1 , x2 , … , xT) , Gtut = gt , utR+ }
(17)
The dual problem of SUt(xt1,xt): is DSUt(xt1,xt): = { max t(xt1,xt) = tT[bt  A(x1 , x2 , … , xT) + tTgt  tTBt + tTGt dtT }
(18)
where t represents the dual variables vector of the restrictions Btut = bt  A(x1 , x2 , … , xT) and t is the dual variables vector of Gt ut = gt. Considering the decoupled cuts generated by each subproblem SUt(xt1,xt): the coordinator CX: is CX: = { Min z = c(x1 , x2 , … , xT) + t t(xt1,xt)  F(x1 , x2 , … , xT) = f t(xt1,xt) + (tk)TEt1 xt1 + (tk)TA(x1 , x2 , … , xT) (tk)Tbt + (tk) Tgt t kIU(t) }
(19)
where IU(t) represents the number of cuts generated for each subproblem SUt(xt1,xt). The cuts that integrated the coordinator CX: are called Supply Benders Cuts (SBC), because the represents the cost supply function of the industrial system parametrized on the values of the vector of state variables, xt. Now, we consider the solution of the coordinator problem CX:. If the coordinator CX: is linear it has a dynamic structure similar to the problem DDP:, then we may solve it by using the DDP theory. Velasquez (2002) presents the mathematical methodology for this case. The other two cases, GDDP: and DP: must be solved as an integrated dynamic model. In these cases, CX: may has continuous and/or discrete variables, and the mathematical relation must be convex nonlinear. It isn’t necessary that the functions may be nonseparable on the tdomain. 2.2. GDDP Robustness GDDP is more robust than the Nested Benders Methodologies, like DDP, because it is capable to solve more formats problem, the next table present the cases. Format Problem LP MIP NLP MINLP
Benders Master LP MIP NLP MINLP
Benders Subproblem LP LP LP LP
DDP
GDDP
YES NO NO NO
YES YES YES YES
2.3. GDDP Special Cases 2.3.1.
Unified Benders Cuts
Velasquez (2002, 2018) introduces the concept of Unified Benders Cuts (UBC) which is useful when the subproblems are associated with a dual feasibility region equal for all of them. In this case, when a subproblem is solved it generates cuts for all the remaining subproblems. On large scale problems, the UBC entail a significant acceleration of the solution process. Below the inclusion of UBC in the GDDP is showed; however, for more details the reader is invited 5
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
to review the referenced publication. From an economic point of view, t(xt1,xt) represents the optimum operation costs in the period t as a consequence of the boundary conditions (xt1 , xt). Frequently, t(xt1,xt) is time independent in the short and the medium term planning horizons, due to the fact that in the planning horizon the technology, the topology and the price index don’t change. When the dual feasibility zone of the problems SUt(xt1,xt) is static, which implies that the matrices Bt and Gt and the cost vector dt are time independent; then, the dual problem is DSUt(xt1,xt): = { max t(xt1,xt) = T[bt  Et1xt1  Atxt] + T gt  TB + TG dT }
(20)
In this case, a feasible vector of dual variables { ,} for any DSUt(xt1,xt): is feasible for all DSUt(xt1,xt): independent of the value of t; when we solve SUt(xt1,xt) for a specific value of t we can generate supply Benders cuts (SBC) for all periods and the coordinator CX: can be expressed as CX: = { Min z = c(x1 , x2 , … , xT) + t t(xt1,xt)  F(x1 , x2 , … , xT) = f t(xt1,xt) + (k)TEt1 xt1 + (k)TA(x1 , x2 , … , xT) (k)Tbt + (k) Tgt t kIU }
(21)
where the dual variables vector {, } is time independent; then, for each iteration of the coordinatorsubproblems we need to solve only one problem, or a subgroup of problems, SUt(xt1, xt) to generate cuts for all periods. IU represents the iterations that are realized at subproblems level. In this case we call the Benders cuts as Unified Benders Cuts (UBC). This fact is very important when the number periods is very large, such as in the case of the discrete control problems, that require many short periods to represent adequately continuous movement and constraints of the state variables. This situation is very common for matrices G and B because they are related with the technology and with the topology of the industrial modeled system, which is normally time independent for the medium/short term (including realtime). Velasquez (2018) discusses several methodologies to use when the vector d is time dependent, these methodologies can be incorporated in GDDP. 2.3.2.
Partially Unified Benders Cuts
In some cases, vector dt can be divided two types of elements: the timedependent and timeindependent; then, it is possible to make some reformulation oriented to have an equivalent problem that has for all subproblems a dual feasible zone independent of t. Then, the control variables vector is divided in: i) zt variables with cost dependent of t, and ii) wt variables with cost independent of t. The variables zt will be part of the state GDDPPUBC: = { Min z = c(x1 , x2 , … , xT) + t dwTzt dwTzt  F(x1 , x2 , … , xT) = f Axt = bt  Et1xt1  Bzzt  Bwwt t=1,T Gzzt + Gwwt = gt t=1,T {zt , wt }R+ t=1,T , xtR+ t=1,T }
(22)
where ut = {zt , wt } , dt = {dzt , dw }
(23)
Then, variables zi are defined in the coordinator and wi in the subproblems, and it is possible to use UBC. 2.3.3.
Benders Cuts Based on Inexact Solutions
When the coordinator problem is a MIP (Mixed Integer Programming) we introduce the concept of Benders Cuts based on Inexact Solutions (BCIS). The concept of inexact solution is based on allowing the solution of the coordinator model is not accurate, allowing a gap, k, which is a function of the iteration and that forms a descending series that meets 6
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
≥ ≥ ≥ … k≥ ≥ 0
(24)
[ { f(yk) + Q(yk) }  { f(yk*) + Q(yk*) } ] / [ f(yk*) + Q(yk*) ] ≤ k
(25)
The stop rule for the iteration k is
where yk belongs to the feasible area of the coordinator in the iteration k and yk* the optimal solution at this iteration, Q() represents the dual estimate of the summation of the subproblems objective functions (dual lower bound). This approach is convergent and may be fastest that the standard stop rule. All the experiments with MIP coordinator were made using BCIS. 3.
GDDP Implementation
3.1. General Considerations Benders’s methodology provides an optimal solution of the integrated problem GDDP: solving iteratively the master problem and the set of subproblems. For minimization, BT algorithm close the gap between dual lower bound (LB) and the primal upper bound (UB) by improving the linear representation of each subproblem cost. The termination criteria can be defined as the relative difference between the primal and dual bound is less than a tolerance GDDP  UBk  LBk  /  UBk +  ≤ GDDP
(26)
where is an infinitesimal element. UBk is defined as the sum of the subproblems objective functions the this is: UBk = t dtTukt
(27)
where ukt represents the optimal solution of control variables in the iteration k. The lower bound LBk is defined as the summa of the estimate cost for each subproblem: LBk = t kt(xt1,xt)
(28)
where kt(xt1,xt) represents the optimal estimate of the subproblem cost in the iteration k. Three alternatives for implementation of the GDDP methodology can be considered: 1. SBC (Standard Bender’s Cut): SBC solves an integrated subproblem and generates only one cut that coupled all periods for each iteration; 2. DBC (Decoupled Bender’s Cuts): DBC solves T problems and generates one decoupled cut for each period, the cuts are coupled in the objective function; 3. UBC (Unified Bender’s Cuts): UBC generates N decoupled cuts for each period, the cuts are coupled in the objective function (Velasquez, 2018). Implementation of DBC and UBC cuts implies two levels of loops in the algorithm: i) the outer loop moves iterations counter of BT; and on ii) the inner loop the subproblems associated with each period is solved. The difference between the two alternatives is: ▪ UBC solves N subproblems (1 ≤ N ≤ T) and DBC always solves T subproblems. ▪ UBC a cut for each period it is generated by each subproblem, independently of the period associated to the subproblem (N × T cuts) and DBC always add one cut for each period, it is generated by the associated period subproblem. The implementation of the DBC is direct because it hasn’t alternatives for implementation; for UBC there are various technical aspects that must be analyzed since they can influence the performance of the GDDP (Velasquez, 2018). 3.2. Unified Bender’s Cuts Based in the Unified Bender Cuts theory, when technological matrix Bt and cost vector dt are time independent, a solution of a single subproblem SUt(xt1,xt) can generate appropriate cut coefficients for the piecewise approximation of the operational cost for all periods. In this case GDDP is modified to generate coefficients faster than the traditional 7
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
BT and DBC schemes and thus decrease, significantly, the solution time. Algorithmic improvements considerations are included in Velasquez (2018). Here is presented a briefing: 1.
UBC must solve sets of subproblems SUt(xt1,xt): in which all of them belong to a family of problems identified by the B matrix and by d vector and differentiated by bt vector. Considering that optimal solution x*t is ever dual feasible for any period t, it is possible to use the dualsimplex algorithm to recover the optimality.
2.
Short iterations (SITE) where a little number subproblems (N < T) are solved, which are selected according to a specific criteria and problem rearrangement. These iterations do not allow the evaluation of the primal upper bound (UBk), because it only have a partial sum of the subproblems objective functions.
3.
Full iterations (FITE) where all T subproblems are solved oriented to obtain a primal upper bound (UBk) to check the convergence of GDDP process.
4.
The “big gap” criteria is used to select the problem to solve in a SITE is based on the following principle: a subproblem will be included in the next SITE if the difference between the dual bound, k(xkt1,xkt), and the primal bound, dTukt is greater than a tolerance SU; dTukt should be evaluated during FSITE. Mathematically this is expressed as: kt =  k(xt1,xt)  dTukt  /  k(xt1,xt) + 
(29)
kt ≤ SU
(30) kt is a measure of the gap that must be closed to the each subproblem SUt(xt1,xt):.This indicator can be used to establish a dynamic order for chose the problems to be included in SITE. 5.
The process used to select a problem to solve in a SITE is based on the following rules: ▪ Establishes a sequential order to solve subproblems, ORDDIN(t). ▪ Sets a maximum number (N) of subproblems to be solved in SITE. ▪ Starting with the first subproblem in the list, solve the first N problems that meet the big gap criteria. ▪ The next subproblem to the last solved will be the first to solve in the next SITE, it is the first on the list.
6.
To avoid duplication of the Benders hyperplanes it is necessary the application of a fast comparison criteria to eliminate duplicated hyperplanes. This is aimed to reduce the size of the master problem and speed up the generation of proper cut coefficients. The comparison criteria is based on the square of the norm of the dual vector k , k defined as NUk = ǁ k ǁ + ǁ k ǁ N = Minj=k1(  NUk – NUj  ) Nk / (NUk+) ≤ NU k
(31) (32) (33)
where NUk represents the square of the norm of the dual vector solution in the iteration k and Nk the closest square norm; if Nk is letter that the tolerance NU it is assumed that the vector of dual variables already exists in the previous coefficients and it is ignored. The GDDP algorithm is summarized in the following flowchart:
8
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
Flow Chart  GDDP  UBC ISW = 1 O(t)=t  t=1,T
START
CX: = { min z = t=1,T ( ctTxt + t )  t + (k)T Et1 xt1 + (tk)T At xt tk t=1,T k=1,k } X = { x1, x2, … , xT } QtA = t(xt1,xt) O(t)=f(z) LB = z
tt = 1 UB= kt = Qt,hB – Qt,hA
ISW =  ISW
NC = #t
NC =YES T
t =O(tt)
YES ISW=1 NOT
YES
tt > NC
NOT
UBk – LBk < GDDP
tt = tt+1 k = k+1 UBk= UBk + z
NOT
kt ≥ NU
YES
YES
STOP
{ min z = dT u  B u = bt – E xt1A xt G u = gt uR+ }
ISW=1 Sk ISW=1 Fk
4.
tt = tt+1
NOT
k+1 QtB = dTu* Zt = QtB  QtA
Electric Sector GDDP Modeling
To illustrate the implementation of the GDDP methodology and its computational performance, the methodology has been applied to an electric system selected from the IEEE test cases (Diniz, 2010). which is composed of 12 hydro plants, 8 reservoirs, 6 thermal plants, 1 “deficit: plant and 1 node of electricity demand; electricity transmission network wasn’t considered. Two types of applications were studied: i) Linear Economic Dispatch (LPED) and ii) Mixed Economic Dispatch (MIPED), equivalent to a Unit Commitment (UC) problem. The objective of LPED and UC is to determine the minimum generation cost of the hydro plants and thermal units, subject to several physical, operational and electricity demand constraints. REFERENCE ELECTRIC SYSTEM
CONSUMER
RESERVOIR
Actual Plants
~ HYDRO PLANTS  CONFIGURATION BUS
▪
Units
Capacity (MW)
Total Capacity (MW)
i01
Ui01  Ui12
85
1020
Ui01Ui02 Ui03Ui06 Ui07Ui08 Ui01  Ui03 Ui01 Ui01Ui02 Ui01Ui04 Ui01Ui02 Ui01Ui04 Ui01Ui03 Ui01Ui03 Ui04Ui06 Ui01Ui02 Ui03Ui05 Ui01Ui02 Ui03Ui04 Ui05Ui06
38 51 54 180 40 130 200 15 300 125 380 380 20 52 150 180 280
i02
HYDROELECTRIC
Actual Capacity Reservoirs (hm3) i01 17000 i02 2600 i05 200 i06 10000 i08 12500 THERMOELECTRIC i09 1000 i10 12400 i12 5000
Hydro Plant
i03 i04 i05 i06 i07 i08 i09 i10 i11
12 8 6 1 24
Hydro Plants Reservoirs Thermal Plants Deficit Plant Hour Planning Horizon
i12
Total Capacity (MW)
Mathematical Modeling of Electric System
This section presents the algebraic formulation of the reference electric system.
9
388 540 40 260 800 30 1200 375 2280 196 1220
8349
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
4.1. Notation This section presents the mathematical notation used in the models LPED and UC Indexes i, p d j k t
Hydro plant Deficit plant Thermal unit Hydro unit Period
Sets I J T ITER UI(i) UP(i)
Hydro plants Thermal generation units Time stages Active iterations Hydro generation units installed in plant i Hydro plants immediately upstream of plant i
Parameters Lt k GHmaxk QHmaxk Vmini Vmaxi Qmaxi At,i GTmaxj CTj CARR
Electricity demand Generation efficiency Upper hydro generation bound Upper hydro unit outflow Lower reservoir volume Upper reservoir volume Upper turbined outflow limit Natural water inflow Upper thermal generation limit Thermal plant variable generation cost Thermal plant start cost
MWh MWh/(m3/s) MWh m3/s hm3 hm3 m3/s m3/s MW/h $/MWh $
Variables UHt,k QHt,k GHt Vt,i Qt,i SPt,i GTt,j SRt,j STt,j
hydro unit outflow hydro unit power generation total hydraulic generation reservoir operating volume turbined outflow spillage outflow thermal generation Start thermal plant State thermal plant (off, on)
m3/s MW MW hm3 m3/s m3/s MW 01 01
4.2. Linear Economic Dispatch (LPED) Constraints The constraints of mathematical formulation of the LPED problem can be stated as (between parenthesis the name of the constraint): ▪
Operational bounds Vmini ≤ Vt,i ≤ Vmaxi
ti
(34)
0 ≤ Qt,i ≤ Qmaxi
(35)
0 ≤ QHt,k ≤ QHmaxk
ti tikUI(i)
0 ≤ GUt,k ≤ GUmaxt,k
tikUI(i)
(37)
10
(36)
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
jJt
(38)
ti
(39)
0 ≤ GTt,j ≤ GTmaxj ▪
Turbinated outflow (QQHt,i) Qt,i =kUI(i) QHt,k
▪
Water balance (WATt,i) Vt,i  Vt1,i = At,i + pUP(i) Qt,p + SPt,p) – (Qt,i + SPt,i)
▪
tikUI(i)
(41)
Total hydraulic generation (GUTt,i) GHt =iI kUI(i) UHt,k
▪
(40)
Hydraulic generation (GQTt,i) GHt,k = k × QHt,k
▪
ti
t
(42)
Electricity demand (ELDt,i) tT
GHt +jJ GTt,j = Lt
(43)
The objective function of LPED is Min tjJ CTj × GTt,j
(44)
LPED formulation is always feasible for any value of electricity demand due the inclusion of an additional thermal plant with infinite generation capacity (plant d). For the case of level of the reservoirs it is feasible ever than the total inflows plus the initial water storage is enough to satisfy the minimum level of the reservoirs. For simplicity, in the experiments Vmini was defined as 0.0. This is important because the application of Benders Theory does not require of Feasible Benders Cuts. 4.3. Mixed Economic Dispatch (MIPED) Constraints The MIPED model includes all the restrictions of the LPED model plus the constraints required to represent the start/stop process in thermal plants, which requires binary variables. ▪
▪
Thermal availability (STNt,j y STXt,j) STt,j × GMINt,j ≤ GTt,p ≤ STt,j × GMAXt,j
(45)
STt,j  STt1,j ≤ SRt,j
(46)
Min tjJ ( CTj × GTt,j + CARRj × SRt,j )
(47)
Start/stop process (SSTt,j)
The objective function of UC is
4.4. Dynamic Programing Conceptualization The first step in the implementation of the GDDP to a specific problem is the division of variables between state variables and control variables. In this case the state variables are related with: reservoir level, hydro generation, water releases and state of thermal plants; the control variables are associated with thermal generation.
11
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
DYNAMIC REPRESENTATION OF ELECTRIC SYSTEMS
Control Variables
ut
Thermal Generation ut = { GTt,j }
xt1
xt
State Variables
State Variables Reservoir Level Hydro Generation Spillage Water Releases Start Thermal Plants State Thermal Plants
Cost Function
ctTxt + dtTut
xt = { Vt,i , St,i , Qt,i , GHt , GUt,j, SRt,j STt,j }
LPED and UC models must be decomposed into multiple problems, masters and subproblems, according the principles of GDDP. The problems are: ▪ MLPED: Master of LPED ▪ SPLPED(t): Subproblems of LPED, indexed in t ▪ MMIPED: Master of MIPED ▪ SP MIPED(t): Subproblems of MIPED, indexed in t To ensure that all solution generated, for the state variables, by coordinators problems always generates feasible solutions in the subproblems is required to include in the master problem a restriction on the maximum of hydroelectric generation as a function of the demand of the system. This restriction is formulated as ▪
Maximum hydraulic generation (MGHt) GHt ≤ Lt
(48)
MGHt may be included as an operational bound. The following table present the variables and the constraints of each problem considering in this study. Model
Problem
MLPED
UHt,k QHt,k GHt Vt,I Qt,i SPt,i
Variables Hydro unit outflow Hydro unit power generation Total hydraulic generation Reservoir operating volume Turbined outflow Spillage outflow
SPLPED(t)
GTt,j
MUC
UHt,k QHt,k GHt Vt,I Qt,i
LPED
UC
Constraints QQHt,i WATt,i GQTt,i GUTt,i MGHt
Turbinated outflow Water balance Hydraulic generation Total hydraulic generation Maximum hydraulic generation
Thermal generation
GUTt,i ELDt,i
Total hydraulic generation Electricity demand
Hydro unit outflow Hydro unit power generation Total hydraulic generation Reservoir operating volume Turbined outflow
QQHt,i WATt,i GQTt,i GUTt,i MGHt
Turbinated outflow Water balance Hydraulic generation Total hydraulic generation Maximum hydraulic generation
12
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
Model
Problem
SPUC(t) 5.
SPt,i SRt,j STt,j
Variables Spillage outflow Start thermal plant State thermal plant (off, on)
GTt,j
Thermal generation
JESUS VELASQUEZBERMUDEZ
STNt,j STXt,j SSTt,j
Constraints Maximum thermal availability Maximum thermal availability Start/stop process
GUTt,i ELDt,i
Total hydraulic generation Electricity demand
Results of Realistic Experiments
Below, are presented the results obtained in the analysis of the behavior of the GDDP in the solution of the problems LPED and MIPED. The models were implemented in GAMS, using CPLEX as optimizer. The implementation did not use high level parallel processing for the solution of the problems generated using GDDP, this will be discussed in a next paper. The solution was obtained in a computer HP with a CPU i77500U (4 cores), 2.9 GHz and 12 MBytes of cache memory. 5.1. Linear Economic Dispatch Experiments with the linear economic dispatch (LPED) aim to prove the convergence of GDDP and compare the performance of: ▪ GDDP using Unified Benders Cuts (GDDPUBC) ▪ GDDP versus DDP ▪ An integrate coordinator (GDDPDBC) versus a coordinator using DDP (GDDPDDP). Not were tested using the BT in its original form. Three orders of the subproblems were used for case GDDPUBC ▪ ORDTEM(t): sequential order according to the period ▪ ORDDEM(t): order according to the total demand ▪ ORDALT(t): alternate order according to the period (1, T, 2, T2, 3, T3, … ) The results are presented in the following table, including the FULL case which refers to the integrated problem without using large scale methodologies. The number of periods for all cases was 24.
Methodology
Order
Dual Bound
FULL
Primal Bound
GAP (%)
T. Solution (secs)
Times FULL
1866850.066
0.0000
0.078
1.00
Times GDDPUBC (TEM)
GDDPUBC
TEM
1866850.066
1866963.730
0.0061
1.651
21.17
1.00
GDDPUBC
DEM
1866850.066
1866963.730
0.0061
1.970
25.26
1.19
GDDPUBC
ALT
1866850.066
1866963.730
0.0001
2.006
25.72
1.22
GDDPDBC
1866850.066
1866906.898
0.0030
2.775
35.58
1.68
DDP
1866850.066
1866885.648
0.0019
12.696
162.77
7.69
GDDPDDP
1866850.066
1866982.674
0.0071
42.912
550.15
25.99
The complexity of the case does not justify the largescale methodologies; but it is useable to compare the largescale methodologies and to prove the convergence of all of them. The conclusions are: 1. All the methodologies are convergent. 2. GDDP with Unified Benders Cuts is faster than any other largescale Benders methodology; independently of the order of selection the subproblem 3. The order followed to solve the problems affecting solution time, in this case up to 20%. 4. GDDP with unified cuts is approximately 8 times faster than DDP 5. GDDP with decoupled cuts is approximately 4.5 times faster than DDP 6. GDDP with integrate coordinator is approximately 15.5 times faster than GDDP with DDP coordinator In the remaining experiments using methodology GDDPUBC ordered according to the period. 13
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
JESUS VELASQUEZBERMUDEZ
5.2. Mixed Linear Economic Dispatch Additionally to LPED, the Unit Commitment (MIPED) model includes: ▪ Start/stop cost for thermoelectric plants Experiments with the linear economic dispatch (MIPED) aim to: ▪ Prove the convergence of GDDP for mixed coordinator ▪ Evaluate the performance of GDDP as function of the number of periods. GAP
Solution Time (secs)
0.00000
0.373
0.00000
1.277
GDDPUBC
0.0000
1.947
FULL
0.00000
0.896
0.00000
4.673
GDDPUBC
0.00000
5.424
FULL
0.00000
1.309
0.00002
17.546
GDDPUBC
0.00000
13.126
FULL
0.00000
27.466
0.00002
44.582
GDDPUBC
0.00000
16.540
FULL
0.00060
1001.147
0.00007
75.094
GDDPUBC
0.00000
33.032
FULL

NS
0.00008
185.228
0.00000
130.555
Model
Periods
FULL GDDPDBC
GDDPDBC
GDDPDBC
GDDPDBC
GDDPDBC
GDDPDBC GDDPUBC
24
48
96
192
384
768
Real Variables
Binary Variables
Constraints
Nonzeroes
576
288
769
3391
2214
384
1537
6799
4225
1152
3073
13615
9217
1536
6145
27427
18433
3072
12289
54511
36865
6144
24577
109039
The conclusions are: 1. GDDP is convergent for any type of Benders cuts (decoupled and unified) 2. For large problems, GDDP with Unified Benders Cuts is faster than GDDP with Decoupled Benders Cuts If we studied the relation solution time versus dimensionality (number of periods), we can see that it is lineal, that implies that is possible to manage very large problems; additionally, solution time per period indicated a learning process that it decreases in the first part of the optimization process.
14
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
DETERMINISTIC RESULTS  MIP UNIT COMMITMENT
JESUS VELASQUEZBERMUDEZ
DETERMINISTIC RESULTS  MIP UNIT COMMITMENT
Solution Time x Period vs. Periods
Solution Time vs. Periods
200
0.25
secs
GDDPS1OCI GDDPUBC
150
secs/period
0.20
GDDPSTCI GDDPDBC 0.15
100 0.10
50 GDDPDBC GDDPSTIBC
0.05
GDDPUBCOIBC GDDPUBC
0 0
200
400
600
0.00
800
0
200
400
600
6.
800 Periods
Periods
Conclusions
The conceptual formulation of the GDDP problem enables development of efficient algorithms based on the partition and the decomposition of the original problem using Benders' Theory and following the conceptualization of Dynamic Programing. The solution of the original problem is found by the coordinated solution of multiple problems of smaller dimension than other largescale methodologies. In some cases, it is possible to visualize special matrix structures to generate Benders´ cuts for all periods and eliminates the need to solve a problem for each period, using the UBC approach. This is of special importance when the number of periods is very large, as in the case of optimal control problems. The following steps in the research about GDDP are: ▪
GDDP Extensions Taking into consideration the greater speed in optimization of dynamic systems and the lower memory requirement, that show the GDDP compared with other alternatives of modeling, the next step is the relaxation of the linearity restriction demanded to the subproblems. Using results from other research that extend the Benders Theory to convex nonlinear (Geoffrion 1972, Velasquez 2005) and to mixed linear subproblems (). The GDDP variations may be: • NLGDDP: NonLinear Generalized Dual Dynamic Programming • GDDiP: Generalized Dual Dynamic Programming • MINLPGDDP: Mixed NonLinear Generalized Dual Dynamic Programming Format Problem LP MIP NLP MINLP
▪
Benders Master LP MIP NLP MINLP
Benders Subproblem LP MIP NLP MINLP
GDDP YES NO NO NO
NLGDDP YES NO YES NO
GDDiP YES YES NO NO
MINLPGDDP YES YES YES ?
DiscreteTime Optimal Control Optimal control theory deals with the time continuous problem of finding a control law for a given system such that a certain optimality criterion is achieved. The method is largely due to the work of Lev Pontryagin and Richard Bellman in the 1950s, it is an extension of the calculus of variations and is a mathematical optimization method for deriving control policies. Optimal control solutions are now often implemented digital computers, contemporary control theory is now primarily concerned with discrete time systems and solutions, it is called DiscreteTime Optimal Control. The main feature of these problems is that it requires an infinitesimal time representation, which leads to problems in which the number of periods is “infinite”. This kind of problems could be resolved efficiently with the GDDP using unified cuts.
15
GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS
▪
JESUS VELASQUEZBERMUDEZ
GDDP High Level Asynchronous Parallel Optimization Related to the effective use of multiple processors 9cores) to solve the integrated problem using the GDDP.
▪
GDDP Commercial Implementation The GDDP is part of the largescale methodologies implemented in OPTEX Mathematical Modeling System (Velasquez, DO Analytics, 2017).
7.
Acknowledgments
The author is grateful, in very special way, to Eng. Juan José Torres the work done in the implementation of the GDDP and presented cases. Eng. Torres made valuable contributions in the implementation of the Unified Benders Cuts. This research is financed by DecisionWare Ltda. and DO Analytics LLC. 8.
References
Bellman, R.E. (1952), “On the Theory of Dynamic Programming”, Proceedings of the National Academy of Sciences, 1952 Bellman, R.E. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ. Republished 2003: Dover, ISBN 0486428095. Bellman, R. and Dreyfus, S.E. (1962) “Applied Dynamic Programming”. Princeton University Press. Princeton, New Jersey, 1962 Benders, J. F. (1962) "Partitioning Procedures for Solving Mixed Variables Programming Problems". Numer. Math 4, 238252. Casseboom, P. D. and Read, E. G. (1987) "Dual Dynamic Programming for Coal Stockpiling". Proc. Operations Research Society of New Zealand, 1518, (1987). Diniz, A. L. (2010) “Test Cases for Unit Commitment and Hydrothermal Scheduling Problems”. Published in: Power and Energy Society General Meeting, 2010 IEEE Geoffrion, A. M. (1972) “Generalized Benders Decomposition”. Journal of Optimization Theory and Applications”, Vol. 10, No. 4, 1972 Murphy, J. (2013). “Benders, Nested Benders and Stochastic Programming: An Intuitive Introduction”. Cambridge University Engineering Department Technical Report CUED/FINFENG/TR.675, December 2013. Pereira, M. V. F. (1989) "Stochastic Operation Scheduling of Large Hydroelectric Systems". Electric Power & Energy Systems, Vol 11(3), 161169. Pereira, M. V. F. and Pinto, L. M. G. (1985) "Stochastic Optimization of a MultiReservoir Hydroelectric System: A Decomposition Approach". Water Resources Research, 21, 779792. Pereira, M. V. F. and Pinto, L. M. G. (1991) "Multistage Stochastic Optimization Applied to Energy Planning". Mathematical Programming, 52, 359375. Read, E. G. (1989) "A Dual Approach to Stochastic Dynamic Programming for Reservoir Scheduling". In Dynamic Programming for Optimal Water Resources Systems Analysis (Ed. A. O. Esogbue), 361372. PrenteciHall Englewood Cliffs, N. J. (1989). Read, E. G. and George, J. A. (1990) "Dual Dynamic Programming for Linear Production/Inventory systems ". Computers Math. Aplicc. Vol 19, No. 11, 2942, (1990) Velásquez, J. M. (1995) OEDM: Optimización Estocástica Dinámica Multinivel. Teoría General”. Universidad Nacional de Colombia, Revista Energética No. 13 (1995) (http://www.doanalytics.net/Documents/DWDT03301OptimizacionEstocasticaDinamicaMultinivel.pdf) Velásquez, J. M. et al. (1999) “Dual Dynamic Programming: A note on Implementation”. Water Resources Research Vol 35. No. 7, 22692271, (July 1999). Velásquez, J. M. (2002) “GDDP: Generalized Dual Dynamic Programming Theory”, Annals of Operation Research, November 2002, Volume 117, Issue 1–4, pp 21–31 Velásquez, J. M. (2005) “NonLinear Dual Dynamic Programming (NLDDP)”, White Paper (http://www.doanalytics.net/Documents/DWDT028NLDDP.pdf) Velásquez, J. M. (2017) “OPTEX: The First Automatic Generator of Large Scale Optimization Models”. Conference (http://www.doanalytics.net/Documents/OPTEXPresentation.pdf) Velásquez, J. M. (2018) “Benders Decomposition Using Unified Cuts”, White Paper DWDT0472018, (http://www.doanalytics.net/Documents/BendersDecompositionUsingUnifiedCuts.pdf)
16