revision de la formulacion de la programacion

GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS Jesus Velásquez-Bermúdez, Eng. D. Chief Scientist, DecisionWare - DO Analytics LLC [email protected]

ABSTRACT This technical paper presents theoretical considerations about the solution of dynamic optimization problems integrating the Benders Theory, the Dynamic Programming approach and the concepts of Control Theory. The so called Generalized Dual Dynamic Programming Theory (GDDP) was initially published by Velasquez (2002); it can be considered as an extension of previous approach known as Dual Dynamic Programming (DDP) (Pereira and Pinto, 1985, 1991) which is a variation of Nested Benders (NB) methodologies (Murphy 2013). From the point of view of Dynamic Programming (DP), the main difference between the traditional Nested Benders (NB) methodologies and GDDP is that the conceptual formulation NB considers all the variables of the problem as state variables, while the GDDP makes a distinction between state variables and control variables. This distinction permits a more detailed algorithm in which the sub-problems are smaller than in the NB. The NB has been widely used in the economic dispatch of power systems. The main limitation of NB, including DDP, is that it only solves linear models. The paper of Velasquez (2002) was not followed by the publication of the experimental results of GDDP. This article revisits Velasquez studies extending the theory, analyzing the implementation and presenting experimental results, using models oriented to the electric sector Key words: Benders Decomposition, Nested Benders, Dynamic Programming, Dual Dynamic Programming, Discrete Control Theory, Electric Sector Optimization 1.

Introduction

1.1. Background This paper presents the computational implementation of the Generalized Dual Dynamic Programing (GDDP) whose theory was developed by J. M. Velasquez (2002). The GDDP is based on the chained application of J. F. Benders Theory (BT) (Benders, 1962) applied to the solution of multi-period dynamic problems using the Dynamic Programming (DP) and the Discrete Control Theory (DCT) approaches. Two previous works must be kept in mind as an initial reference of GDDP, both were called Dual Dynamic Programming (DDP). First, in a series of papers published starting in 1985 M. V. F. Pereira and M. V. G. Pinto (Pereira and Pinto, 1985, 1991) (Pereira, 1989) introduced a methodology that made it possible to apply the DP principles to problems with multiple state variables. Pereira and Pinto used Benders' Partition technique to the multiple-stage problem, which allowed the replacement of the discretization of the future cost function in the classic DP approach by a series of hyperplanes generated using BT. DDP limits the relations of the state variables to two consecutive periods; in 1995, Velasquez (1995) studied the application of BT to a more general problem which allows the direct relationship between the state variables of a period t with state variables of any previous period, this case is included en GDDP. The main difference between DDP and GDDP is that the conceptual formulation DDP considers all the variables of the problem as state variables, while the GDDP makes a distinction between state variables and control variables. This distinction permits a more detailed algorithm in which the sub-problems are smaller than in the DDP. The DDP has been widely used in the economic dispatch of power systems, its main limitation is that DDP only solves linear models. E. G. Read and others developed the second work (Casseboom and Read, 1987) (Read, 1989) (Read and George, 1990), that they called Dual Dynamic Programming (DDP-R). The conceptual formulation of the DDP-R makes the distinction between control variables and state variables. The DDP-R may be seen as a dual formulation of the classic DP approach in the sense that DP chooses an arbitrary grid of primal state variables and for each one finds the optimal decision and the shadow price of the state variables (dual variables). The DDP-R solves parametrically a sub-problem in each period to produce points of the "supply function" for that period and, using backward recursion, combines these supply functions with the end-of-period function value to define an optimal strategy.

GDDP: GENERALIZED DUAL DYNAMIC PROGRAMMING. THEORY & ELECTRIC SECTOR APPLICATIONS

JESUS VELASQUEZ-BERMUDEZ

The problem GDDP: considers a more detailed formulation than DDP in which the variables that couple two consecutive periods, related to the stock level and the state of the infrastructure (on-off), correspond to the state variables (xt), while the production and distribution variables in the period t, not involved directly in the dynamic relationship, may be thought as the control variables (ut). The application of GDDP theory solves a large-scale optimization problem as a coordinated sum of very simple problems; it entails a significant increase in speed to solve large problems and in the dimensionality of these. Additionally, the structure of GDDP is oriented to asynchronous parallel optimization and distributed real-time optimization. The GDDP is applicable to a wide variety of systems and dynamic problems. This paper presents as an example its utility in the modeling of the electric sector. 1.2. Benders' Partition Theory BT considers the problem P: composed by two types of variables: y, the coordination variables, and x, the coordinates. P: = { Min z = cTx + f(y) | F0(y) = b0 Ax + F(y) = b xR+ , yS } (1) BT restricts the model on x to be a linear problem, while it doesn’t impose conditions on y that may be continuous, or discrete. Additionally, the functions f(y) and F(y) may be non-linear convex functions. The P: problem is partitioned in two coordinated problems: CY: over y and SP(y): over x which is defined as SP(y): = { Min Q(y) = cTx | Ax = b - F(y), xR+ }

(2)

The coordinator CY: on y can be formulated as CY: = { Min z = f(y) + Q(y) | F0(y) = b0 , yS Q(y)  (k)T[b - F(y)] k IT 0  (k)T[b - F(y)] kIN }

(3)

where represents the vector of dual variables of the restrictions Ax = b-F(y), IT the set of iterations,  an extreme ray on the  feasibility region and IN the set of iterations on which no feasibility was obtained. Benders proposed the solution of P: by a hierarchical algorithm that works on two levels: i) the coordination level solves the problem CY: and generates a sequence of yk values; ii) on the second level, yk is used as a parameter of the sub-problem SP(y): to generate a sequence of feasible extreme points, k, and extreme rays, k, of the dual feasible zone of SP(y):, this vectors are used to include cutting planes in CY:, that includes two types of cuts. The first type, that we call optimality cutting planes (OCP) restricts the feasible zone of y to obtain the optimal y; it has the following structure Q(y)  (k)T[b - F(y)]

k IT

(4)

The second type, that we call feasibility cutting planes (FCP) restricts the feasible zone of y to maintain feasible x in SP(y), it has the following structure 0  (k)T[b - F(y)]

kIN

(5)

For simplicity, in the following sections the FCPs are ignored, but they can be included following the same way used for the OBCs. 1.3. DP: Dynamic Programming Dynamic Programming (DP), as it was formulated by Bellman and Dreyfus (1962), is a numerical methodology for solving optimization problems related to dynamic techno-economic systems, understood as those problems whose 2



planning horizon covers multiple stages/periods; whose results, in the absence of computational limitations, coincide with those obtained using exacts methodologies based on Mathematical Programming. The principle of optimality of DP, which was developed by Richard Bellman (1952, 1957), established that “an optimal path has the property that whatever the initial conditions and control variables (choices) over some initial period, the control (or decision variables) chosen over the remaining period must be optimal for the remaining problem, with the state resulting from the early decisions taken to be the initial condition. To define the dynamic evolution of the system are used state variables, that link two consecutive stages. This principle guarantees the optimality if the separability condition is satisfied. For this study, we are interested in a DP problem in which all the mathematical elements must be separable in the time domain, with mixed-non-linear-convex relations for the state variables (xt) and linear relations for all control variables (ut). DP: = { Min z = t [ ct(xt) + dt ut ] | Ft(xt-1, xt ) = ft t At(xt-1, xt) + Btut = bt t Gt ut = gt t + utR txtS t }

(6)

where t represents the periods, xt the vector of state variables and ut the vector of control variables. Ft() and At() techno-economical functions, Et , Bt , and Gt are techno-economical matrices, ft , bt and gt are resources vectors, ct() cost function for the state variables and dt cost vector for the control variables. 1.4. DDP: Dual Dynamic Programming The mathematical formulation considered in the DDP, and in general in the methodologies referred to as Nested Benders, is based on a linear model DDPO: = { Min t=1,T ctT yt | At yt = bt - Et-1 yt-1 , t=1,T ; ytR+ }

(7)

where yt is a vector that represents, jointly, the state variables and the control variables, or yt = { xt , ut }

(8)

If the vector yt is divided in state and control variables we can see the problem as DP:, equations (6). DDP solves a more restrictive problem than DP because DDP only applies to linear dynamic problems (DDP:); this is because the DDP considers all the variable as state variables. The DDP approach implies the multilevel decomposition (in the time domain) of multiple problems that are decomposed using BT, generating new problems that again are decomposed using BT (Nested Benders, NB). Considering that DDP only has state variables, the reduce form of previous problem is: DDP: = { Min z = t ctTxt | At xt + Et-1 xt-1 = bt t, xtR+ }

(9)

As a starting point, DDP divided the variables in two groups: i) coordination variables associated with stages/periods between 1 and T-1 {x1, x2, ... , xT-1}, and ii) coordinated variables associated with stage T {xT}. Based on the direct application of BT the coordinator model for the period {1,T-1} is CDT-1: = { Min z = t=1,T-1 ctTxt + T(xT-1) | At xt = bt - Et-1 xt-1 t=1,T-1 , xtR+ t=1,T-1) T(xT-1) + kTTET-1 xT-1 kTTbT kIT(T-1) }

3

(10)



where t(xt-1) represents the future cost function considering that the system is on state xt-1 at the beginning of period t, IT(t) the amount of cuts included in the coordinator CDt, and kt the vector of dual variables of the restrictions At xt = bt - Et-1 xkt-1 obtained as a solution of the sub-problem SDt(xkt-1). We call the cuts as Future Cost Benders Cuts (FCBC). For the stage T the linear sub-problem SDT(xkT-1) is defined as SDT(xkT-1): = { Min T(xkT-1) = cTTxT ATxT = bT - ET-1xkT-1 , xTR+ }

| (11)

Using a recursive approach, the coordinator for each intermediate stage t (less than T) is CDt: = { Min z =  cTx + t+1(xt) | Ax = b - E-1x-1  , xR+  t+1(xt) + jt+1TEtxt t+1jjIT(t) }

(12)

and the sub-problem for each intermediate stage t is SDt(xjt-1): = { Min t(xjt-1) = ctTxt + t+1(xt) | Atxt = bt - Et-1xjt-1 , t+1(xt) + kt+1TEtxt t+1kkIJ(t,j) }

xtR+ (13)

where IJ(t,j) represents the number of cuts included in the coordinator CDt: when we solve the sub-problem SDt(xjtj 1). t is a constant value calculated as  tj =

{

jTTbT jtTbt + k=1,IJ(t,j) k,jtt+1k

t=T t = 1,T-1 (14)

where k,jt represents the dual variable of the k-th Benders cut in the sub-problem SDt(xjt-1). The index j represents the iteration of the coordinator of SDt(xjt-1) and the index k the cuts that have been included in the process of solution of SDt(xjt-1). 2.

GDDP: Generalized Dual Dynamic Programming

2.1. GDDP Basic Theory In 2002, Velasquez (2002) considered the solution of the same linear dynamic problem that considers DDP. Due to the partitioning the variables in state variables and control variables, the previous formulation may be relaxed to include: i) discrete variables in the vector of state variables; ii) non-linear non-separable convex relations between the state variables; and non-linear non-separable convex cost functions; then, the GDDP: problem may be formulated as: GDDP: = { Min z = c(x1 , x2 , … , xT) + t dtTut | F(x1 , x2 , … , xT) = f A(x1 , x2 , … , xT) + Btut = bt t Gt ut = gt t + utR txtSt t }

(15)

where ct(x) is the cost function of the state variables, Ft(x) and At(x) vectorial functions, and St the domain of x that may be continuous or discrete The previous formulation represents many industrial systems in which state variables are associated with: i) the stocks (inventories) and ii) the state of the production units (on-off); and control variables are associated with the production and distribution activities through the supply-chain and/or the production factories which are separable between periods. The solution of GDDP: using BT considers a two-stage process: i) first, we define the state variable xt as the coordination variables to decouple the problem at temporary level solving static subproblems SUt(xt-1,xt): for each t, optimizing the control variables ut and ii) in the second stage, the GDDP solves the coordinator problem, CX:, 4



optimizing the state variables xt. CX: is stated as CX: = { Min z = c(x1 , x2 , … , xT) + t t(xt-1,xt) | F(x1 , x2 , … , xT) = f { Min t(xt-1,xt) = dtTut | Btut = bt - A(x1 , x2 , … , xT) , Gtut = gt , utR+ } xtS t }

t (16)

where t(xt-1,xt) represents the optimum operation costs in the period t as consequence of the border condition (x1 , x2 , … , xT) and it corresponds to the objective function of the linear sub-problems SUt(xt-1,xt): defined as SUt(xt-1,xt): = { Min t(xt-1,xt) = dtTut | Btut = bt - A(x1 , x2 , … , xT) , Gtut = gt , utR+ }

(17)

The dual problem of SUt(xt-1,xt): is DSUt(xt-1,xt): = { max t(xt-1,xt) = tT[bt - A(x1 , x2 , … , xT) + tTgt | tTBt + tTGt  dtT }

(18)

where t represents the dual variables vector of the restrictions Btut = bt - A(x1 , x2 , … , xT) and t is the dual variables vector of Gt ut = gt. Considering the decoupled cuts generated by each sub-problem SUt(xt-1,xt): the coordinator CX: is CX: = { Min z = c(x1 , x2 , … , xT) + t t(xt-1,xt) | F(x1 , x2 , … , xT) = f t(xt-1,xt) + (tk)TEt-1 xt-1 + (tk)TA(x1 , x2 , … , xT) (tk)Tbt + (tk) Tgt t kIU(t) }

(19)

where IU(t) represents the number of cuts generated for each sub-problem SUt(xt-1,xt). The cuts that integrated the coordinator CX: are called Supply Benders Cuts (SBC), because the represents the cost supply function of the industrial system parametrized on the values of the vector of state variables, xt. Now, we consider the solution of the coordinator problem CX:. If the coordinator CX: is linear it has a dynamic structure similar to the problem DDP:, then we may solve it by using the DDP theory. Velasquez (2002) presents the mathematical methodology for this case. The other two cases, GDDP: and DP: must be solved as an integrated dynamic model. In these cases, CX: may has continuous and/or discrete variables, and the mathematical relation must be convex non-linear. It isn’t necessary that the functions may be non-separable on the t-domain. 2.2. GDDP Robustness GDDP is more robust than the Nested Benders Methodologies, like DDP, because it is capable to solve more formats problem, the next table present the cases. Format Problem LP MIP NLP MINLP

Benders Master LP MIP NLP MINLP

Benders Subproblem LP LP LP LP

DDP

GDDP

YES NO NO NO

YES YES YES YES

2.3. GDDP Special Cases 2.3.1.

Unified Benders Cuts

Velasquez (2002, 2018) introduces the concept of Unified Benders Cuts (UBC) which is useful when the subproblems are associated with a dual feasibility region equal for all of them. In this case, when a subproblem is solved it generates cuts for all the remaining subproblems. On large scale problems, the UBC entail a significant acceleration of the solution process. Below the inclusion of UBC in the GDDP is showed; however, for more details the reader is invited 5



to review the referenced publication. From an economic point of view, t(xt-1,xt) represents the optimum operation costs in the period t as a consequence of the boundary conditions (xt-1 , xt). Frequently, t(xt-1,xt) is time independent in the short and the medium term planning horizons, due to the fact that in the planning horizon the technology, the topology and the price index don’t change. When the dual feasibility zone of the problems SUt(xt-1,xt) is static, which implies that the matrices Bt and Gt and the cost vector dt are time independent; then, the dual problem is DSUt(xt-1,xt): = { max t(xt-1,xt) = T[bt - Et-1xt-1 - Atxt] + T gt | TB + TG  dT }

(20)

In this case, a feasible vector of dual variables { ,} for any DSUt(xt-1,xt): is feasible for all DSUt(xt-1,xt): independent of the value of t; when we solve SUt(xt-1,xt) for a specific value of t we can generate supply Benders cuts (SBC) for all periods and the coordinator CX: can be expressed as CX: = { Min z = c(x1 , x2 , … , xT) + t t(xt-1,xt) | F(x1 , x2 , … , xT) = f t(xt-1,xt) + (k)TEt-1 xt-1 + (k)TA(x1 , x2 , … , xT) (k)Tbt + (k) Tgt t kIU }

(21)

where the dual variables vector {, } is time independent; then, for each iteration of the coordinator-sub-problems we need to solve only one problem, or a sub-group of problems, SUt(xt-1, xt) to generate cuts for all periods. IU represents the iterations that are realized at sub-problems level. In this case we call the Benders cuts as Unified Benders Cuts (UBC). This fact is very important when the number periods is very large, such as in the case of the discrete control problems, that require many short periods to represent adequately continuous movement and constraints of the state variables. This situation is very common for matrices G and B because they are related with the technology and with the topology of the industrial modeled system, which is normally time independent for the medium/short term (including real-time). Velasquez (2018) discusses several methodologies to use when the vector d is time dependent, these methodologies can be incorporated in GDDP. 2.3.2.

Partially Unified Benders Cuts

In some cases, vector dt can be divided two types of elements: the time-dependent and time-independent; then, it is possible to make some reformulation oriented to have an equivalent problem that has for all subproblems a dual feasible zone independent of t. Then, the control variables vector is divided in: i) zt variables with cost dependent of t, and ii) wt variables with cost independent of t. The variables zt will be part of the state GDDPPUBC: = { Min z = c(x1 , x2 , … , xT) + t dwTzt dwTzt | F(x1 , x2 , … , xT) = f Axt = bt - Et-1xt-1 - Bzzt - Bwwt  t=1,T Gzzt + Gwwt = gt  t=1,T {zt , wt }R+  t=1,T , xtR+  t=1,T }

(22)

where ut = {zt , wt } , dt = {dzt , dw }

(23)

Then, variables zi are defined in the coordinator and wi in the subproblems, and it is possible to use UBC. 2.3.3.

Benders Cuts Based on Inexact Solutions

When the coordinator problem is a MIP (Mixed Integer Programming) we introduce the concept of Benders Cuts based on Inexact Solutions (BC-IS). The concept of inexact solution is based on allowing the solution of the coordinator model is not accurate, allowing a gap, k, which is a function of the iteration and that forms a descending series that meets 6



≥ ≥ ≥ … k≥  ≥ 0

(24)

[ { f(yk) + Q(yk) } - { f(yk*) + Q(yk*) } ] / [ f(yk*) + Q(yk*) ] ≤ k

(25)

The stop rule for the iteration k is

where yk belongs to the feasible area of the coordinator in the iteration k and yk* the optimal solution at this iteration, Q() represents the dual estimate of the summation of the subproblems objective functions (dual lower bound). This approach is convergent and may be fastest that the standard stop rule. All the experiments with MIP coordinator were made using BC-IS. 3.

GDDP Implementation

3.1. General Considerations Benders’s methodology provides an optimal solution of the integrated problem GDDP: solving iteratively the master problem and the set of subproblems. For minimization, BT algorithm close the gap between dual lower bound (LB) and the primal upper bound (UB) by improving the linear representation of each subproblem cost. The termination criteria can be defined as the relative difference between the primal and dual bound is less than a tolerance GDDP | UBk - LBk | / | UBk + | ≤ GDDP

(26)

where  is an infinitesimal element. UBk is defined as the sum of the subproblems objective functions the this is: UBk = t dtTukt

(27)

where ukt represents the optimal solution of control variables in the iteration k. The lower bound LBk is defined as the summa of the estimate cost for each subproblem: LBk = t kt(xt-1,xt)

(28)

where kt(xt-1,xt) represents the optimal estimate of the subproblem cost in the iteration k. Three alternatives for implementation of the GDDP methodology can be considered: 1. SBC (Standard Bender’s Cut): SBC solves an integrated sub-problem and generates only one cut that coupled all periods for each iteration; 2. DBC (Decoupled Bender’s Cuts): DBC solves T problems and generates one decoupled cut for each period, the cuts are coupled in the objective function; 3. UBC (Unified Bender’s Cuts): UBC generates N decoupled cuts for each period, the cuts are coupled in the objective function (Velasquez, 2018). Implementation of DBC and UBC cuts implies two levels of loops in the algorithm: i) the outer loop moves iterations counter of BT; and on ii) the inner loop the subproblems associated with each period is solved. The difference between the two alternatives is: ▪ UBC solves N subproblems (1 ≤ N ≤ T) and DBC always solves T subproblems. ▪ UBC a cut for each period it is generated by each subproblem, independently of the period associated to the subproblem (N × T cuts) and DBC always add one cut for each period, it is generated by the associated period subproblem. The implementation of the DBC is direct because it hasn’t alternatives for implementation; for UBC there are various technical aspects that must be analyzed since they can influence the performance of the GDDP (Velasquez, 2018). 3.2. Unified Bender’s Cuts Based in the Unified Bender Cuts theory, when technological matrix Bt and cost vector dt are time independent, a solution of a single subproblem SUt(xt-1,xt) can generate appropriate cut coefficients for the piecewise approximation of the operational cost for all periods. In this case GDDP is modified to generate coefficients faster than the traditional 7



BT and DBC schemes and thus decrease, significantly, the solution time. Algorithmic improvements considerations are included in Velasquez (2018). Here is presented a briefing: 1.

UBC must solve sets of subproblems SUt(xt-1,xt): in which all of them belong to a family of problems identified by the B matrix and by d vector and differentiated by bt vector. Considering that optimal solution x*t is ever dual feasible for any period t, it is possible to use the dual-simplex algorithm to recover the optimality.

2.

Short iterations (S-ITE) where a little number subproblems (N < T) are solved, which are selected according to a specific criteria and problem rearrangement. These iterations do not allow the evaluation of the primal upper bound (UBk), because it only have a partial sum of the subproblems objective functions.

3.

Full iterations (F-ITE) where all T subproblems are solved oriented to obtain a primal upper bound (UBk) to check the convergence of GDDP process.

4.

The “big gap” criteria is used to select the problem to solve in a S-ITE is based on the following principle: a subproblem will be included in the next S-ITE if the difference between the dual bound, k(xkt-1,xkt), and the primal bound, dTukt is greater than a tolerance SU; dTukt should be evaluated during F-SITE. Mathematically this is expressed as: kt = | k(xt-1,xt) - dTukt | / | k(xt-1,xt) +  |

(29)

kt ≤ SU

(30)  kt is a measure of the gap that must be closed to the each subproblem SUt(xt-1,xt):.This indicator can be used to establish a dynamic order for chose the problems to be included in S-ITE. 5.

The process used to select a problem to solve in a S-ITE is based on the following rules: ▪ Establishes a sequential order to solve subproblems, ORDDIN(t). ▪ Sets a maximum number (N) of subproblems to be solved in S-ITE. ▪ Starting with the first subproblem in the list, solve the first N problems that meet the big gap criteria. ▪ The next subproblem to the last solved will be the first to solve in the next S-ITE, it is the first on the list.

6.

To avoid duplication of the Benders hyperplanes it is necessary the application of a fast comparison criteria to eliminate duplicated hyperplanes. This is aimed to reduce the size of the master problem and speed up the generation of proper cut coefficients. The comparison criteria is based on the square of the norm of the dual vector k , k  defined as NUk = ǁ k ǁ + ǁ k ǁ N = Minj=k-1( | NUk – NUj | ) Nk / (NUk+) ≤ NU k

(31) (32) (33)

where NUk represents the square of the norm of the dual vector solution in the iteration k and Nk the closest square norm; if Nk is letter that the tolerance NU it is assumed that the vector of dual variables already exists in the previous coefficients and it is ignored. The GDDP algorithm is summarized in the following flowchart:

8



Flow Chart - GDDP - UBC ISW = -1 O(t)=t | t=1,T

START

CX: = { min z =  t=1,T ( ctTxt + t ) | t + (k)T Et-1 xt-1 + (tk)T At xt tk t=1,T k=1,k } X = { x1, x2, … , xT } QtA = t(xt-1,xt) O(t)=f(z) LB = z

tt = 1 UB=   kt = Qt,hB – Qt,hA

ISW = - ISW

NC = #t

NC =YES T

t =O(tt)

YES ISW=1 NOT

YES

tt > NC

NOT

UBk – LBk < GDDP

tt = tt+1 k = k+1 UBk= UBk + z

NOT

 kt ≥ NU

YES

YES

STOP

{ min z = dT u | B u = bt – E xt-1-A xt G u = gt uR+ }

ISW=1  S-k ISW=-1  F-k

4.

tt = tt+1

NOT

 k+1 QtB = dTu* Zt = QtB - QtA

Electric Sector GDDP Modeling

To illustrate the implementation of the GDDP methodology and its computational performance, the methodology has been applied to an electric system selected from the IEEE test cases (Diniz, 2010). which is composed of 12 hydro plants, 8 reservoirs, 6 thermal plants, 1 “deficit: plant and 1 node of electricity demand; electricity transmission network wasn’t considered. Two types of applications were studied: i) Linear Economic Dispatch (LP-ED) and ii) Mixed Economic Dispatch (MIP-ED), equivalent to a Unit Commitment (UC) problem. The objective of LP-ED and UC is to determine the minimum generation cost of the hydro plants and thermal units, subject to several physical, operational and electricity demand constraints. REFERENCE ELECTRIC SYSTEM

CONSUMER

RESERVOIR

Actual Plants

~ HYDRO PLANTS - CONFIGURATION BUS

▪

Units

Capacity (MW)

Total Capacity (MW)

i01

Ui01 - Ui12

85

1020

Ui01-Ui02 Ui03-Ui06 Ui07-Ui08 Ui01 - Ui03 Ui01 Ui01-Ui02 Ui01-Ui04 Ui01-Ui02 Ui01-Ui04 Ui01-Ui03 Ui01-Ui03 Ui04-Ui06 Ui01-Ui02 Ui03-Ui05 Ui01-Ui02 Ui03-Ui04 Ui05-Ui06

38 51 54 180 40 130 200 15 300 125 380 380 20 52 150 180 280

i02

HYDRO-ELECTRIC

Actual Capacity Reservoirs (hm3) i01 17000 i02 2600 i05 200 i06 10000 i08 12500 THERMO-ELECTRIC i09 1000 i10 12400 i12 5000

Hydro Plant

i03 i04 i05 i06 i07 i08 i09 i10 i11

12 8 6 1 24

Hydro Plants Reservoirs Thermal Plants Deficit Plant Hour Planning Horizon

i12

Total Capacity (MW)

Mathematical Modeling of Electric System

This section presents the algebraic formulation of the reference electric system.

9

388 540 40 260 800 30 1200 375 2280 196 1220

8349



4.1. Notation This section presents the mathematical notation used in the models LP-ED and UC Indexes i, p d j k t

Hydro plant Deficit plant Thermal unit Hydro unit Period

Sets I J T ITER UI(i) UP(i)

Hydro plants Thermal generation units Time stages Active iterations Hydro generation units installed in plant i Hydro plants immediately upstream of plant i

Parameters Lt k GHmaxk QHmaxk Vmini Vmaxi Qmaxi At,i GTmaxj CTj CARR

Electricity demand Generation efficiency Upper hydro generation bound Upper hydro unit outflow Lower reservoir volume Upper reservoir volume Upper turbined outflow limit Natural water inflow Upper thermal generation limit Thermal plant variable generation cost Thermal plant start cost

MWh MWh/(m3/s) MWh m3/s hm3 hm3 m3/s m3/s MW/h $/MWh $

Variables UHt,k QHt,k GHt Vt,i Qt,i SPt,i GTt,j SRt,j STt,j

hydro unit outflow hydro unit power generation total hydraulic generation reservoir operating volume turbined outflow spillage outflow thermal generation Start thermal plant State thermal plant (off, on)

m3/s MW MW hm3 m3/s m3/s MW 0-1 0-1

4.2. Linear Economic Dispatch (LP-ED) Constraints The constraints of mathematical formulation of the LP-ED problem can be stated as (between parenthesis the name of the constraint): ▪

Operational bounds Vmini ≤ Vt,i ≤ Vmaxi

ti

(34)

0 ≤ Qt,i ≤ Qmaxi

(35)

0 ≤ QHt,k ≤ QHmaxk

ti  tikUI(i)

0 ≤ GUt,k ≤ GUmaxt,k

tikUI(i)

(37)

10

(36)



jJt

(38)

ti

(39)

0 ≤ GTt,j ≤ GTmaxj  ▪

Turbinated outflow (QQHt,i) Qt,i =kUI(i) QHt,k

▪

Water balance (WATt,i) Vt,i - Vt-1,i = At,i + pUP(i) Qt,p + SPt,p) – (Qt,i + SPt,i)

▪

tikUI(i)

(41)

Total hydraulic generation (GUTt,i) GHt =iI kUI(i) UHt,k

▪

(40)

Hydraulic generation (GQTt,i) GHt,k = k × QHt,k

▪

ti

t

(42)

Electricity demand (ELDt,i) tT

GHt +jJ GTt,j = Lt

(43)

The objective function of LP-ED is Min tjJ CTj × GTt,j

(44)

LP-ED formulation is always feasible for any value of electricity demand due the inclusion of an additional thermal plant with infinite generation capacity (plant d). For the case of level of the reservoirs it is feasible ever than the total inflows plus the initial water storage is enough to satisfy the minimum level of the reservoirs. For simplicity, in the experiments Vmini was defined as 0.0. This is important because the application of Benders Theory does not require of Feasible Benders Cuts. 4.3. Mixed Economic Dispatch (MIP-ED) Constraints The MIP-ED model includes all the restrictions of the LP-ED model plus the constraints required to represent the start/stop process in thermal plants, which requires binary variables. ▪

▪

Thermal availability (STNt,j y STXt,j) STt,j × GMINt,j ≤ GTt,p ≤ STt,j × GMAXt,j

(45)

STt,j - STt-1,j ≤ SRt,j

(46)

Min tjJ ( CTj × GTt,j + CARRj × SRt,j )

(47)

Start/stop process (SSTt,j)

The objective function of UC is

4.4. Dynamic Programing Conceptualization The first step in the implementation of the GDDP to a specific problem is the division of variables between state variables and control variables. In this case the state variables are related with: reservoir level, hydro generation, water releases and state of thermal plants; the control variables are associated with thermal generation.

11



DYNAMIC REPRESENTATION OF ELECTRIC SYSTEMS

Control Variables

ut

Thermal Generation ut = { GTt,j }

xt-1

xt

State Variables

State Variables Reservoir Level Hydro Generation Spillage Water Releases Start Thermal Plants State Thermal Plants

Cost Function

ctTxt + dtTut

xt = { Vt,i , St,i , Qt,i , GHt , GUt,j, SRt,j STt,j }

LP-ED and UC models must be decomposed into multiple problems, masters and subproblems, according the principles of GDDP. The problems are: ▪ M-LP-ED: Master of LP-ED ▪ SP-LP-ED(t): Subproblems of LP-ED, indexed in t ▪ M-MIP-ED: Master of MIP-ED ▪ SP- MIP-ED(t): Subproblems of MIP-ED, indexed in t To ensure that all solution generated, for the state variables, by coordinators problems always generates feasible solutions in the subproblems is required to include in the master problem a restriction on the maximum of hydroelectric generation as a function of the demand of the system. This restriction is formulated as ▪

Maximum hydraulic generation (MGHt) GHt ≤ Lt

(48)

MGHt may be included as an operational bound. The following table present the variables and the constraints of each problem considering in this study. Model

Problem

MLP-ED

UHt,k QHt,k GHt Vt,I Qt,i SPt,i

Variables Hydro unit outflow Hydro unit power generation Total hydraulic generation Reservoir operating volume Turbined outflow Spillage outflow

SPLPED(t)

GTt,j

MUC

UHt,k QHt,k GHt Vt,I Qt,i

LP-ED

UC

Constraints QQHt,i WATt,i GQTt,i GUTt,i MGHt

Turbinated outflow Water balance Hydraulic generation Total hydraulic generation Maximum hydraulic generation

Thermal generation

GUTt,i ELDt,i

Total hydraulic generation Electricity demand

Hydro unit outflow Hydro unit power generation Total hydraulic generation Reservoir operating volume Turbined outflow

QQHt,i WATt,i GQTt,i GUTt,i MGHt

Turbinated outflow Water balance Hydraulic generation Total hydraulic generation Maximum hydraulic generation

12


Model

Problem

SPUC(t) 5.

SPt,i SRt,j STt,j

Variables Spillage outflow Start thermal plant State thermal plant (off, on)

GTt,j

Thermal generation


STNt,j STXt,j SSTt,j

Constraints Maximum thermal availability Maximum thermal availability Start/stop process

GUTt,i ELDt,i

Total hydraulic generation Electricity demand

Results of Realistic Experiments

Below, are presented the results obtained in the analysis of the behavior of the GDDP in the solution of the problems LP-ED and MIP-ED. The models were implemented in GAMS, using CPLEX as optimizer. The implementation did not use high level parallel processing for the solution of the problems generated using GDDP, this will be discussed in a next paper. The solution was obtained in a computer HP with a CPU i7-7500U (4 cores), 2.9 GHz and 12 MBytes of cache memory. 5.1. Linear Economic Dispatch Experiments with the linear economic dispatch (LP-ED) aim to prove the convergence of GDDP and compare the performance of: ▪ GDDP using Unified Benders Cuts (GDDP-UBC) ▪ GDDP versus DDP ▪ An integrate coordinator (GDDP-DBC) versus a coordinator using DDP (GDDP-DDP). Not were tested using the BT in its original form. Three orders of the sub-problems were used for case GDDP-UBC ▪ ORDTEM(t): sequential order according to the period ▪ ORDDEM(t): order according to the total demand ▪ ORDALT(t): alternate order according to the period (1, T, 2, T-2, 3, T-3, … ) The results are presented in the following table, including the FULL case which refers to the integrated problem without using large scale methodologies. The number of periods for all cases was 24.

Methodology

Order

Dual Bound

FULL

Primal Bound

GAP (%)

T. Solution (secs)

Times FULL

1866850.066

0.0000

0.078

1.00

Times GDDP-UBC (TEM)

GDDP-UBC

TEM

1866850.066

1866963.730

0.0061

1.651

21.17

1.00

GDDP-UBC

DEM

1866850.066

1866963.730

0.0061

1.970

25.26

1.19

GDDP-UBC

ALT

1866850.066

1866963.730

0.0001

2.006

25.72

1.22

GDDP-DBC

1866850.066

1866906.898

0.0030

2.775

35.58

1.68

DDP

1866850.066

1866885.648

0.0019

12.696

162.77

7.69

GDDP-DDP

1866850.066

1866982.674

0.0071

42.912

550.15

25.99

The complexity of the case does not justify the large-scale methodologies; but it is useable to compare the large-scale methodologies and to prove the convergence of all of them. The conclusions are: 1. All the methodologies are convergent. 2. GDDP with Unified Benders Cuts is faster than any other large-scale Benders methodology; independently of the order of selection the subproblem 3. The order followed to solve the problems affecting solution time, in this case up to 20%. 4. GDDP with unified cuts is approximately 8 times faster than DDP 5. GDDP with decoupled cuts is approximately 4.5 times faster than DDP 6. GDDP with integrate coordinator is approximately 15.5 times faster than GDDP with DDP coordinator In the remaining experiments using methodology GDDP-UBC ordered according to the period. 13



5.2. Mixed Linear Economic Dispatch Additionally to LP-ED, the Unit Commitment (MIP-ED) model includes: ▪ Start/stop cost for thermo-electric plants Experiments with the linear economic dispatch (MIP-ED) aim to: ▪ Prove the convergence of GDDP for mixed coordinator ▪ Evaluate the performance of GDDP as function of the number of periods. GAP

Solution Time (secs)

0.00000

0.373

0.00000

1.277

GDDP-UBC

0.0000

1.947

FULL

0.00000

0.896

0.00000

4.673

GDDP-UBC

0.00000

5.424

FULL

0.00000

1.309

0.00002

17.546

GDDP-UBC

0.00000

13.126

FULL

0.00000

27.466

0.00002

44.582

GDDP-UBC

0.00000

16.540

FULL

0.00060

1001.147

0.00007

75.094

GDDP-UBC

0.00000

33.032

FULL

-

NS

0.00008

185.228

0.00000

130.555

Model

Periods

FULL GDDP-DBC

GDDP-DBC

GDDP-DBC

GDDP-DBC

GDDP-DBC

GDDP-DBC GDDP-UBC

24

48

96

192

384

768

Real Variables

Binary Variables

Constraints

Non-zeroes

576

288

769

3391

2214

384

1537

6799

4225

1152

3073

13615

9217

1536

6145

27427

18433

3072

12289

54511

36865

6144

24577

109039

The conclusions are: 1. GDDP is convergent for any type of Benders cuts (decoupled and unified) 2. For large problems, GDDP with Unified Benders Cuts is faster than GDDP with Decoupled Benders Cuts If we studied the relation solution time versus dimensionality (number of periods), we can see that it is lineal, that implies that is possible to manage very large problems; additionally, solution time per period indicated a learning process that it decreases in the first part of the optimization process.

14


DETERMINISTIC RESULTS - MIP UNIT COMMITMENT


DETERMINISTIC RESULTS - MIP UNIT COMMITMENT

Solution Time x Period vs. Periods

Solution Time vs. Periods

200

0.25

secs

GDDP-S1-O-CI GDDP-UBC

150

secs/period

0.20

GDDP-ST-CI GDDP-DBC 0.15

100 0.10

50 GDDP-DBC GDDP-ST-IBC

0.05

GDDP-UBC-O-IBC GDDP-UBC

0 0

200

400

600

0.00

800

0

200

400

600

6.

800 Periods

Periods

Conclusions

The conceptual formulation of the GDDP problem enables development of efficient algorithms based on the partition and the decomposition of the original problem using Benders' Theory and following the conceptualization of Dynamic Programing. The solution of the original problem is found by the coordinated solution of multiple problems of smaller dimension than other large-scale methodologies. In some cases, it is possible to visualize special matrix structures to generate Benders´ cuts for all periods and eliminates the need to solve a problem for each period, using the UBC approach. This is of special importance when the number of periods is very large, as in the case of optimal control problems. The following steps in the research about GDDP are: ▪

GDDP Extensions Taking into consideration the greater speed in optimization of dynamic systems and the lower memory requirement, that show the GDDP compared with other alternatives of modeling, the next step is the relaxation of the linearity restriction demanded to the subproblems. Using results from other research that extend the Benders Theory to convex nonlinear (Geoffrion 1972, Velasquez 2005) and to mixed linear subproblems (). The GDDP variations may be: • NL-GDDP: Non-Linear Generalized Dual Dynamic Programming • GDDiP: Generalized Dual Dynamic Programming • MINLP-GDDP: Mixed Non-Linear Generalized Dual Dynamic Programming Format Problem LP MIP NLP MINLP

▪

Benders Master LP MIP NLP MINLP

Benders Subproblem LP MIP NLP MINLP

GDDP YES NO NO NO

NLGDDP YES NO YES NO

GDDiP YES YES NO NO

MINLPGDDP YES YES YES ?

Discrete-Time Optimal Control Optimal control theory deals with the time continuous problem of finding a control law for a given system such that a certain optimality criterion is achieved. The method is largely due to the work of Lev Pontryagin and Richard Bellman in the 1950s, it is an extension of the calculus of variations and is a mathematical optimization method for deriving control policies. Optimal control solutions are now often implemented digital computers, contemporary control theory is now primarily concerned with discrete time systems and solutions, it is called Discrete-Time Optimal Control. The main feature of these problems is that it requires an infinitesimal time representation, which leads to problems in which the number of periods is “infinite”. This kind of problems could be resolved efficiently with the GDDP using unified cuts.

15


▪


GDDP High Level Asynchronous Parallel Optimization Related to the effective use of multiple processors 9cores) to solve the integrated problem using the GDDP.

▪

GDDP Commercial Implementation The GDDP is part of the large-scale methodologies implemented in OPTEX Mathematical Modeling System (Velasquez, DO Analytics, 2017).

7.

Acknowledgments

The author is grateful, in very special way, to Eng. Juan José Torres the work done in the implementation of the GDDP and presented cases. Eng. Torres made valuable contributions in the implementation of the Unified Benders Cuts. This research is financed by DecisionWare Ltda. and DO Analytics LLC. 8.

References

Bellman, R.E. (1952), “On the Theory of Dynamic Programming”, Proceedings of the National Academy of Sciences, 1952 Bellman, R.E. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ. Republished 2003: Dover, ISBN 0-486-42809-5. Bellman, R. and Dreyfus, S.E. (1962) “Applied Dynamic Programming”. Princeton University Press. Princeton, New Jersey, 1962 Benders, J. F. (1962) "Partitioning Procedures for Solving Mixed Variables Programming Problems". Numer. Math 4, 238-252. Casseboom, P. D. and Read, E. G. (1987) "Dual Dynamic Programming for Coal Stockpiling". Proc. Operations Research Society of New Zealand, 15-18, (1987). Diniz, A. L. (2010) “Test Cases for Unit Commitment and Hydrothermal Scheduling Problems”. Published in: Power and Energy Society General Meeting, 2010 IEEE Geoffrion, A. M. (1972) “Generalized Benders Decomposition”. Journal of Optimization Theory and Applications”, Vol. 10, No. 4, 1972 Murphy, J. (2013). “Benders, Nested Benders and Stochastic Programming: An Intuitive Introduction”. Cambridge University Engineering Department Technical Report CUED/F-INFENG/TR.675, December 2013. Pereira, M. V. F. (1989) "Stochastic Operation Scheduling of Large Hydroelectric Systems". Electric Power & Energy Systems, Vol 11(3), 161-169. Pereira, M. V. F. and Pinto, L. M. G. (1985) "Stochastic Optimization of a Multi-Reservoir Hydroelectric System: A Decomposition Approach". Water Resources Research, 21, 779-792. Pereira, M. V. F. and Pinto, L. M. G. (1991) "Multi-stage Stochastic Optimization Applied to Energy Planning". Mathematical Programming, 52, 359-375. Read, E. G. (1989) "A Dual Approach to Stochastic Dynamic Programming for Reservoir Scheduling". In Dynamic Programming for Optimal Water Resources Systems Analysis (Ed. A. O. Esogbue), 361-372. Prenteci-Hall Englewood Cliffs, N. J. (1989). Read, E. G. and George, J. A. (1990) "Dual Dynamic Programming for Linear Production/Inventory systems ". Computers Math. Aplicc. Vol 19, No. 11, 29-42, (1990) Velásquez, J. M. (1995) OEDM: Optimización Estocástica Dinámica Multinivel. Teoría General”. Universidad Nacional de Colombia, Revista Energética No. 13 (1995) (http://www.doanalytics.net/Documents/DW-DT033-01-Optimizacion-Estocastica-Dinamica-Multinivel.pdf) Velásquez, J. M. et al. (1999) “Dual Dynamic Programming: A note on Implementation”. Water Resources Research Vol 35. No. 7, 2269-2271, (July 1999). Velásquez, J. M. (2002) “GDDP: Generalized Dual Dynamic Programming Theory”, Annals of Operation Research, November 2002, Volume 117, Issue 1–4, pp 21–31 Velásquez, J. M. (2005) “Non-Linear Dual Dynamic Programming (NL-DDP)”, White Paper (http://www.doanalytics.net/Documents/DW-DT-028-NL-DDP.pdf) Velásquez, J. M. (2017) “OPTEX: The First Automatic Generator of Large Scale Optimization Models”. Conference (http://www.doanalytics.net/Documents/OPTEX-Presentation.pdf) Velásquez, J. M. (2018) “Benders Decomposition Using Unified Cuts”, White Paper DW-DT-047-2018, (http://www.doanalytics.net/Documents/Benders-Decomposition-Using-Unified-Cuts.pdf)

16