daily streamflow forecasting using artificial neural

DAILY STREAMFLOW FORECASTING USING ARTIFICIAL NEURAL NETWORKS Emrah DOĞAN Research Assist., Sakarya University, Civil Engineering Department, [email protected]

Sabahattin IŞIK Assist. Prof. Sakarya University, Civil Engineering Department, [email protected]

Tarık TOLUK MSc. Civil Eng., University, Civil Engineering Department, [email protected]

Mehmet SANDALCI Assist. Prof. Sakarya University Civil Engineering Department, [email protected]

ABSTRACT

Forecasting of streamflows is required for proper water resources planning and management. This study presents the application and comparison of artificial neural network (ANN) approaches and autoregressive (AR) method. ANN and AR(4) methods are employed to predict daily streamflows at Çifteler station in the Sakarya River. Three different ANN methods such as feed-forward backpropagation neural networks (FFNN), radial basis neural networks (RBNN), and recurrent neural networks (RNN) are selected in modeling hydrological time-series and generating synthetic streamflows. Daily streamflows of Çifteler between 1989-1991 (1091 variables) and between 1992-1993 (486 variables) were used for traning and test periods, respectively. Determination coefficients of AR(4), FFNN, RBNN, and RNN models were found as 0.7547, 0.9495, 0.9479, and 0.9991, respectively. Finally, RNN model yields the best result with a determination coefficient of 0.9991. Keywords: Streamflow modelling, Autoregressive model, Artifical neural network INTRODUCTION

Forecasting of streamflows are vital important for flood caution, operation of flood-control-purposed reservoir, determination of river water potential, production of hydroelectric energy, allocation of domestic and irrigation water in drought seasons, and navigation planning in rivers [Bayazıt, 1988].

449

RIVER BASIN FLOOD MANAGEMENT

Stochastic streamflow models are commonly used in hydrology. Recently, artifical neural network (ANN) models are also employed to water resources and hydrology problems [Gavin et. al., 2005]. A number of studies have been reported in literature. Some of them are given the below. Oğuz [1983] developed a mathematical model that simulates movements of yearly flows. Karabörk and Kahya [1998] obtained mathematical expressions of multivariate periodic autoregressive (PAR) and periodic autoregressive moving average (PARMA) models for monthly streamflow observations of 12 stations located in the Sakarya Basin. Jain and Srivastava [1999] used ANN methods to predict reservoir inflows in resevoir operation. They compared ANN and ARIMA models, and concluded that ANN yielded better result. Zealand et. al. [1999] investigated the utility of artificial neural networks (ANNs) for short term forecasting of streamflow. Birikundavyi et. al. (2002) investigated the performance of ANN methods in prediction of daily streamflows. It is shown that ANN method yielded better results than ARMA models. Cigizoglu [2003] incorporated ARMA models into flow forecasting by artificial neural networks to overcome the limitation of the data. Kumar et. al. [2004] employed RNN model in streamflows forecasting. Kişi [2004] investigated the application of artificial neural networks (ANNs) in predicting mean monthly streamflow and compared with AR models. Huang et. al. [2004] compared ANN and ARIMA models in streamflow forecasting. In this study, a stochastic model, autoregressive AR(4), ANN methods, feedforward backpropagation neural networks (FFNN), radial basis neural networks (RBNN), and recurrent neural networks (RNN) were used to forecast streamflows and compared with each other. The models were applied to daily streamflows between 1989-1993 at Çifteler station in the Sakarya River. The models were trained for 1989-1991 daily streamflows (1091 variables) and tested for 1992-1993 daily streamflows (486 variables). STREAMFLOW FORECASTING MODELS

AR(p) Model Time series models are used to forecast streamflows in hydrology. General equation of AR (autoregressive) model can be written as below [Bayazıt,1998]. p

yi = Σφ j y j −1 + ε i j =1

(1)

450

INTERNATIONAL CONGRESS ON RIVER BASIN MANAGEMENT

φ: regression coefficients of model, ε: independent variable. AR(p) model can be given as the following matrix.

 ρ1   1 ρ   ρ  2  1  .   .  =  .   .  .   .     ρ p   ρ p −1

[ρ ] = [

ρ1 1

.

. . 1

.

. ρ p −1   φ1  . ρ p − 2   φ2   . .   x  . .     1 ρ1      . . 1  φ p −1 

. . . 1

P

(2)

] x [φ ]

φ = P −1ρ

(3)

φ values are obtained from eq. (3) and by subsituting the following equation, general equation of model can be obtained [Haan, 2002]. p

yi = Σ φ j yi − j + ε i = φ1 yi −1 + φ2 yi − 2 + ........ + φ p yi − p + ε i

(4)

j =1

Basic Principles of the Neural Networks Artificial Neural Networks (ANNs) consist of large number of processing elements with their interconnections. ANNs are basically parallel computing systems similar to biological neural networks. They can be characterized by three components: ♦ Nodes ♦ weights (connection strength) ♦ An activation (transfer) function ANN modeling is a nonlinear statistical technique; it can be used to solve problems that are not amenable to conventional statistical and mathematical methods. In the past few years there has been constantly increasing interest in neural networks modeling in different fields of hydrology engineering [ASCE, 2000].

451


The basic unit in the artificial neural network is the node. Nodes are connected to each other by links known as synapses, associated with each synapse there is a weight factor. Usually neural networks are trained so that a particular set of inputs produces, as nearly as possible, a specific set of target outputs. Feed-Forward Backpropagation Neural Networks (FFNN) The most commonly used ANN is the three-layer feed-forward ANN. In feedforward neural networks architecture, there are layers and nodes at each layer. Each node at input and inner layers receives input values, processes and passes to the next layer. This process is conducted by weights. Weight is the connection strength between two nodes. The numbers of neurons in the input layer and the output layer are determined by the numbers of input and output parameters, respectively. In the present feed-forward artificial neural networks are used. The model is shown in Figure 1. In the Figure 1, i, j, k denote nodes input layer, hidden layer and output layer, respectively. w is the weight of the nodes. Subscripts specify the connections between the nodes. For example, wij is the weight between nodes i and j. The term ʺfeed-forwardʺ means that a node connection only exists from a node in the input layer to other nodes in the hidden layer or from a node in the hidden layer to nodes in the output layer; and the nodes within a layer are not interconnected to each other.

i wij

j wjk

Input Layer

Hidden Layer

k

Output Layer

Fig. 1 A typical three-layer feed forward ANN

Radial Basis Neural Networks (RBNN) RBNN were introduced into the neural network literature by Broomhead and Lowe [1988]. Radial basis functions (RBF) are powerful techniques for interpolation in multidimensional space. An RBF is a function which has built into it a distance

452


criterion with respect to a center. Such functions can be used very efficiently for interpolation and for smoothing of data. Radial basis functions have been applied in the area of neural networks where they are used as a replacement for the sigmoidal transfer function. Such networks have 3 layers, the input layer, the hidden layer with the RBF non-linearity and a linear output layer. The most popular choice for the nonlinearity is the Gaussian. RBF networks have the advantage of not being locked into local minima as do feed-forward networks. The basis functions in the hidden layer produce a significant non-zero response to input stimulus only when the input falls within a small localized region of the input space. Hence, this paradigm is also known as a localized receptive field network [Lee and Chang, 2003]. The type of input transformation of the RBNN is the local nonlinear projection using a radial fixed shape basis function. After nonlinearly squashing the multi-dimensional inputs without considering the output space, the radial basis functions play a role of regressors. Since the output layer implements a linear regressor the only adjustable parameters are the weights of this regressor. These parameters can therefore be determined using the linear least square method, which gives an important advantage for convergence. In this study, different numbers of iterations and spread constants are examined for the RBNN models with a simple trial-error method adding some loops to the program codes. Recurrent neural networks (RNN) Forecasting of hydrologic time series is based on the previous values of the series depending on the number of persistence components. Recurrent neural networks (RNN) are networks that include feedback connections in addition to the feedforward connections commonly used in artificial neural networks. In general, an RNN includes an input layer, an output layer, and hidden layers. Several types of RNN architectures have been proposed for modelling complex time-dependent phenomena [Williams and Zipser, 1989; Haykin, 1998]. The RNN used in this study is the Elman RNN [Elman, 1990], which has feedback connections from its hidden layer neurons back to its inputs. This is a discrete-time recurrent two-layer network with feedback loops that allow for adaptability and non-linearity. The temporal representation capabilities of the RNN are better than those of purely feed-forward networks, even those with tapped-delay lines [Saad et al., 1998]. An important step in designing models driven by neural networks is the selection of the number of hidden neurons. Because the target function is unknown, it is difficult to predict in advance what the optimal network size should be. The appropriate network should neither overfit nor underfit the data. In order to develop the optimum network model, many networks are trained.

453


APPLICATION OF MODELS

Definition of Study Area Daily streamflows between 1989-1993 at Çifteler river gauging station in the Sakarya River are used in this study. Daily streamflow data are obtained from Electrical Power Resources Survey and Development Administration (EIE) [Toluk, 2006]. Çifteler river gauging station is close to Aktaş village which is locacted on 25 km southwest of Çifteler County in Eskişehir Province. Statistical variables of Çifteler stations are given in Table 1. Table 1. Statistical variables of Çifteler stations (m3/s)

Çifteler

Average

Standart Deviation

Skewness

Maximum

General Data

5.22

2.0008

-0.2914

10.90

1.39

Training Data

5.12

2.0721

-0.1679

10.90

1.39

Test Data

5.44

1.8433

-0.6170

8.92

1.56

Minimum

Average and standart deviation of general data are 5.22 and 2.000; streamflows vary from 10.90 to 1.39. Average and standart deviation of traning data are 5.12 and 2.0721; streamflows vary from 10.90 to 1.39. Average and standart deviation of test data are 5.44 and 1.8433; streamflows vary from 8.92 to 1.56.

Application of AR(p) Model AR(p) model was applied to daily streamflows of Çifteler station. The application of AR model was performed by using Microsoft EXCEL. Since time lag was taken as 4 days in ANN models and data were delayed 4 days, p was also taken as 4 days AR(p) application. Correlation coefficients, ρ1, ρ2, ρ3, ρ4, are given in Table 2.

Table 2. Correlation coefficients

Çifteler

ρ1 0.973129

ρ2 0.934881

ρ3 0.891842

ρ4 0.8500755

If ρ coefficients are subsituted in eq.(2), φ coefficients can be calculated from eq.(3) as in Table 3.

454


Tablo 3. φ Coefficients

Çifteler

φ 0.85027

φ2 0.195

φ3 0.19751

φ4 -0.28206

If obtained φ coefficents are subsituted in eq.(4), AR(4) equation of Çifteler stations can be determined as: yi=0.85027yi-1 + 0.195yi-2 + 0.19751yi-3 - 0.28206yi-4 + εi Synthetic time series are generated by using this equation. Generated values and test values were compared and given in Table 4. Some iterations of trials for AR(4) model are given in Table 4. The best results were obtained in 4. trial with a determination coefficient (R2) of 0.759 and in 5. trial with mean square error (MSE) and average absolute error (AARE) of 1.091 and 17.71. Finally, values obtained in 5. trial were concluded as the best result for AR(4) model. Table 4. AR(4) Results

Iteration No : 1 2 3 4 5

R2 0.700 0.743 0.756 0.759 0.754

MSE 1.28161 1.16942 1.15801 1.10927 1.09096

AARE 18.042 19.037 18.804 18.235 17.715

Application of Feed-Forward Backpropagation Neural Networks In this study, before the training of the network both input and output variables were normalized within the range 0.1 to 0.9 as follows:

xi = 0.8

( x − xmin )

( xmax − xmin )

+ 0.1

(5)

where xi is the normalized value of a certain parameter, x is the measured value for this parameter, xmin and xmax are the minimum and maximum values in the database for this parameter, respectively. Networks are sensitive to the number of nodes in their hidden layers. Too few nodes can lead to underfitting and too many nodes can result in overfitting. In order to reach an optimum amount of hidden layer nodes, 2, 3, 5, 10 nodes are tested. Within this range, an FFNN model, having 4 inputs and two hidden layers with 2 nodes and 5000 iteration number, gives the best choice.

455


Application of Radial Basis Neural Networks In this study, supervised learning algorithm was used. This algorithm has ability to produce processor components. To develop RBNN model nerb function was used via “MATLAB” software [MATLAB, 2004]. Different numbers of hidden layer neurons and spread constants are examined for obtaining an appropriate RBNN model. After trial and error processes, RBNN model, having 4 inputs and 6 spread constants 200 iteration number, gives the best choice. The determination coefficient (R2), average absolute error (AARE) and mean square error (MSE) values of each RBNN in test period are given in Table 5. Table 5. RBNN results for the test period

200

Iteration Number

0.1

200

200

0.2

0.3

200

200

200

200

0.4

1

2

5

R2

0.5662

0.14831 0.02476

0.02135

0.04283

0.4549

0.9324

AARE(%)

13.4358 19.0767 32.4529

44.3932

18.3271

8.6812

5.3026

MSE

1.9277

12.5741 108.422 412.72356 41.8262 3.56819

0.23855

Spread Constant

200

Iteration Number

200

200

200

200

200

10

15

200

6

7

8

9

R2

0.9479*

0.9447

0.9464

0.9472

0.94768 0.94608

AARE(%)

4.97414 4.94873 4.95237

4.93795

4.94347 4.88147* 5.04109

MSE

0.19198* 0.19952 0.19411

0.19212

0.19241 0.194418 0.198954

Spread Constant

20 0.94561

Application of Recurrent Neural Networks After the trial and error processes an RNN model, having 4 inputs and one hidden layers with 1 nodes and 10000 iteration number, gives the best choice. The results are shown in Table 6. Table 6. RNN results for the test period

Trial number

Input Number

1 2 3 4

4 4 4 4

Hidden Layer Nodes 1 1 1 1

Output Number

Iteration Number

1 1 1 1

10000 12000 15000 8000

R2

AARE

MSE

0.9996* 0.003142* 0.03484* 0.9707 0.101512 0.20159 0.9641 0.122542 0.22653 0.9784 0.101807 0.20214

It appears that while assessing the performance of any model for its applicability in forecasting flow discharges, it is not only important to evaluate the average prediction error but also the distribution of prediction errors. The statistical performance evaluation criteria employed so far in this study are global statistics (R2 and MSE)

456


and do not provide any information on the distribution of errors [Dogan et al., 2005]. Therefore, in order to test the robustness of the model developed, it is important to test the model using some other performance evaluation criteria such as average absolute relative error (AARE) and threshold statistics (the error percentage which is less than 10 %) (TS10). The AARE and TS10 not only give the performance index in terms of predicting flow discharges but also show the distribution of the prediction errors. After training the all of the neural network models, test performances were checked. The performance of neural network models for prediction of flow discharge is demonstrated in Figure 2 in the form of hydrograph and scatterplot. Figure 2 also shows an analysis between the network outputs (estimations) and the corresponding targets (observed data) for the test dataset. It is obvious that the predicted values trained by the RNN catch the targets very well. It is seen from the hydrographs that the RBNN and FFNN also estimate closely follow the observed values. The underestimations and overestimations are obviously seen for the AR model. This is also confirmed by the scatterplots. As seen from the fit line equations and R2 values in scatterplots, the estimates of all the neural network models are closer to the exact fit line (y=x line) than those of the AR.


457

Fig.2 Comparison Neural Network Models Results with AR

The comparison of models is shown in Table 7 in terms of the R2, MSE, AARE and TS10 statistics in test period. Table 7 indicates that the RNN model has the lowest MSE and AARE values while has the highest R2 and TS10 values.

458


Table 7. The comparison of models in test period

Models AR(4) FFNN RBNN RNN

R2 0.7547 0.9495 0.9479 0.9996

Performance of the Models MSE AARE 1.09096 17.715 0.1829 4.8088 0.19198 4.97414 0.0033142 0.03484

TS10 17.71 38.06 38.27 43.62

RESULTS

In this study, AR(4), feed-forward backpropagation neural networks (FFNN), radial basis neural networks (RBNN), and recurrent neural networks (RNN) were used to forecast streamflows and compared with each other. Models were applied to daily streamflows at Çifteler river gauging stations in the Sakarya River. It is found that the performances of ANN models are better than AR(4) model. Determination coefficients of AR(4), FFNN, RBNN, and RNN models were found as 0.7547, 0.9495, 0.9479, and 0.9991, respectively. Finally, RNN model yields the best result with a determination coefficient and a mean square error of 0.9991 and 0.0033142. REFERENCES

ASCE Task Committee (2000). Artificial Neural Networks in Hydrology. I: Preliminary concepts.’’ J. of Hydrologic Engineering, ASCE, 5(2), 115–123. Bayazıt M. (1988). Hidrolojik Modeller, İ.T.Ü. rektörlüğü, İstanbul. Birikundavyi S., Labib R., Trung H., and Rousselle J. (2002). Performance of Neural Networks in Daily Streamflow Forecasting, J. of Hydrologic Engineering, 7, 5, 392398. Broomhead, D. and Lowe, D. (1988). Multivariable functional interpolation and adaptive networks. Complex Syst. 2, 321–355. Cigizoğlu H., (2003). Incorporation of ARMA Models Into Flow Forecasting by Artificial Neural Networks, Environmetrics, 14, 4, 417-427. Dogan E, Sasal M, and Isik S. (2005). Suspended Sediment Load Estimation in Lower Sakarya River by Using Soft Computational Methods. Proceeding of the International Conference on Computational and Mathematical Methods in Science and Engineering, CMMSE 2005, Alicante, Spain, 395-406. Elman, J.L. (1990). Finding structure in time. Cognitive Science 14: 179–211. Gavin, B., Graeme, D., Holger, M. (2005). Input Determination For Neural Network Models in Water Resources Applications, Part1-background and methodology, Journal of Hydrology, 301,1, 75-92. Haan T. (2002). Statistical Methods in Hydrology, Lowa State Pres. Haykin, S. (1998). Neural Networks - A Comprehensive Foundation (2nd. ed.). PrenticeHall, Upper Saddle River, NJ.


459

Huang W., Bing Xu B., and Hilton A., (2004). Forecasting Flows in Apalachicola River Using Neural Networks, Hydrological Processes, 18, 2545-2564. Jain S., Das A., and Srivastava D. (1999). Applicationof ANN For Reservoir In Flow Prediction and Operation, Journal of WaterResources Planning and Management, 125, 5, 263-271. Karabörk M., Kahya E. (1998). Sakarya Havzasındaki Aylık Akımların Çok Değişkenli Stokastik Modellenmesi, Tr Journal of Engineering and Environmental Science, 23, 133-147. Kişi Ö. (2004). River Flow Modelling Using Artificial Neural Networks, J. of Hydrologic Engineering, 9, 1, 60-63. Kumar D., Raju K., and Sathish T. (2004). River Flow Forecasting Using Recurrent Neural Networks, Water Resources Management, Kluwer Academic Publishers, 18, 143161. Lee, G.C. and Chang, S.H. (2003). Radial basis function networks applied to DNBR calculation in digital core protection systems. Annals of Nuclear Energy, 30, 15611572. MATLAB, (2004). Documentation Neural Network Toolbox Help, Version 7.0, Release 14, The MathWorks, Inc. Oğuz B., (1983). Yıllık Akımların Gidiş Özelliklerini Benzeştiren Bir Matemetik Modelin Araştırılması, Doktora Tez, İ.T.Ü. İnşaat Fakültesi Matbası, İstanbul. Saad E.W., Prokhorov, D.V., and Wunsch, D.C. (1998). Comparative study of stock trend prediction using time delay, recurrent and probabilistic neural networks, IEEE Transactions on Neural Networks, 9(6): 1456–1470. Toluk, T. (2006). Akarsu Akımlarının Yapay Sinir Ağı Metotları Kullanılarak Modellenmesi, SAÜ Fen Bilimleri Enst., Yüksek Lisans Tezi, Sakarya. Williams R, and Zipser D. (1989). A learning algorithm for continually running fully recurrent neural networks. Neural Computation 1: 270–280. Zealand C., Burn D., Simonovic S. (1999). Shortterm Streamflow Forecasting Using Artificial Neural Networks, Journal of Hydrology, 214, 32-48.