a practical guide to model parameter

0 downloads 0 Views 1MB Size Report
7 made. The first is the objective function, the second is the parameters for the. 8 calibration and their ranges. Even though parameters of a conceptual model. 9.
Acta Geophysica vol. ?, no. ?, pp. ?-? DOI: 10.2478/s11600-012-00??-?

3

On the choice of calibration periods and objective functions: a practical guide to model parameter identification

4

Renata J. Romanowicz, Marzena Osuch, Magdalena Grabowiecka

5 6

Institute of Geophysics Polish Academy of Sciences

7

Abstract

8 9 10 11 12 13 14 15 16 17

Despite the development of new measuring techniques, monitoring systems and advances in computer technology, rainfall-flow modelling is still a challenge. The reasons are multiple and fairly well known. They include the distributed, heterogeneous nature of the environmental variables affecting flow from the catchment. These are precipitation, evapotranspiration and in some seasons and catchments in Poland, snow melt also. This paper presents a review of work done on the calibration and validation of rainfall-runoff modelling, with a focus on the conceptual HBV model. We give a synthesis of the problems and propose a practical guide to the calibration and validation of rainfall-runoff models.

18 19

Key words: rainfall-runoff model, HBV, calibration, objective function, Wieprz catchment

1 2

Ks. Janusza 64, 01-452 Warsaw, Poland

20 21 22 23 24 25 26 27

1.

INTRODUCTION

Rainfall-runoff at a catchment scale is governed by an external (global scale) forcing in the form of rainfall, temperature and radiation and internal (local scale) processes in the form of evapotranspiration, natural or artificial water reservoirs and drainage, including river channels, surface runoff, infiltration, groundwater – surface water interactions and snow melt. Human activity in the form of water management and land use also affects the flow. ________________________________________________ © 2012 Institute of Geophysics, Polish Academy of Sciences

2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

RENATA J. ROMANOWICZ, MARZENA OSUCH, MAGDA GRABOWIECKA

All these factors vary in time and space. Also, external and internal processes influencing catchment behaviour act at different spatial and temporal scales. However, complex interactions between catchment processes make it difficult to isolate their actions. Modelling is always done for a purpose and is conditioned on the available observations (Romanowicz and Macdonald 2005). The river discharge measured at a gauging station at a catchment outlet is therefore a product of all these external and internal forces that we can measure only in a very limited form, mostly at a point scale. Even a discharge itself is usually measured indirectly, through water level measurements followed by a transformation of these measurements into discharges using a rating curve. Due to time- and space-variable interactions between processes influencing water movement in the catchment, the modelling of rainfall – runoff still remains a challenge. In mathematical terms, we search for a relationship between external forcing and the discharge, conditioned on the available information on the internal processes influencing the movement of water in the catchment. The relationship depends on the temporal and spatial scales of the processes. It is important to note that the temporal scale of the description (i.e. the model discretisation time) is related to the spatial scale of the problem. Large catchments require larger discretisation time than small catchments and vice versa. Another factor influencing model choice is the goal of the modelling. Among possible goals are the testing of scientific hypotheses and an understanding of the processes, water resources management and reservoir control, flood protection and forecasting, and water quality modelling. Depending on the modelling concepts used, models can be classified into physically-based, e.g. SHE (Abbott et al. 1986 a, b), SWAT (Arnold et al. 1998, Gassman et al. 2007), conceptual, e.g. HBV (Bergström 1976), HYMOD (Boyle 2000, Boyle et al. 2001), and black-box type, e.g. Nash cascade, Stochastic Transfer Function (Young 2003, Romanowicz et al. 2012), linear ARMA and nonlinear ANN (Bruen and Yang 2006). On the other hand, depending on the model structure, they can be divided into lumped, semi-distributed and distributed. The need for distributed predictions requires the application of distributed modelling tools. However, more complex models have more parameters and require more observations for a model set-up. Yet another division originates from the way in which environmental uncertainty is treated. The deterministic approach neglects its existence, while the stochastic approach attempts to take it into account. Once the model structure has been chosen, it is necessary to set model parameters, and, depending on the model structure, its initial and boundary conditions. The procedure for choosing the parameter values is called model calibration. There are many approaches to calibration, depending on the chosen model

ON THE CHOICE OF CALIBRATION PERIODS AND OBJECTIVE FUNCTIONS

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

3

structure and the experience of the modeller. The most popular is choosing parameters that give a good fit of model simulations to the observed flows. In this paper the goal of the modelling is the prediction of flow at the catchment outlet. The rainfall-runoff model HBV (Bergström 1976) is a popular tool used in studies of the influence of climate change on water resources (Graham et al. 2005, Andréasson et al. 2004, Bergström et. al. 2001). Once the model is chosen, there are two more important choices to be made. The first is the objective function, the second is the parameters for the calibration and their ranges. Even though parameters of a conceptual model have their physical interpretation, not all can be identified based on the available observations. Taking into account the identifiability problem, the choice of parameter set for the identification and the objective function are inter-dependent. The choice of the objective function influences the identifiability of the model parameters and this is one of the issues that we want to stress in this study. However, identifiability is also affected by the length of the observation period used in the calibration. Season-based calibration is less robust but gives a better fit to the observations. Moreover, using the season-based model calibration gives an insight into the information content of the data. The aim of this study is the presentation of approaches to rainfall-runoff modelling, as a part of geophysical research, under the topical theme: Advances in Geophysics ‐ Models and Methods. The presentation focuses on model parameter identification that is of major importance in hydrological sciences and also may be of interest to a broader geophysical community. The problem is illustrated using the rainfall-runoff HBV model applied to the Wieprz catchment, Poland. The objectives of this study are to:  Review different objective functions used for the calibration of rainfall-runoff models.  Discuss the identifiability of hydrological model parameters using analyses of response surfaces created by a range of objective functions.  Analyse the identifiability of parameters using winter seasons for calibration.  Assess the validation performance of a rainfall-runoff model as a test of the information content in the data. In the second section we present a review of different calibration tools used in hydrological predictions. The third section describes the River Wieprz case study used to illustrate the discussion. The fourth section presents problems related to model identifiability and the choice of calibration

4

1 2 3

RENATA J. ROMANOWICZ, MARZENA OSUCH, MAGDA GRABOWIECKA

criteria using the Monte Carlo sampling. The fifth section presents calibration of the HBV using winter seasons to assess the information content of the data. The sixth section introduces the conclusions.

4 5 6 7 8 9 10 11 12 13 14 15

2.

16 17 18 19 20 21 22 23 24 25 26 27 28

2.1 Formulation of the parameter identification problem: forward and inverse modelling Forward modelling is understood to be a description of a physical process, here rainfall-runoff processes on a catchment scale, using mathematical rules. It can also be understood as a transformation of inputs (e.g. rainfall and temperature) into the desired output. Knowing model parameters, we can use forward modelling to simulate flows for a given (observed) rainfall and temperature. In the ideal case, these parameters represent measurable catchment characteristics. We want the simulated flows to be as close as possible to the observed values. Therefore, forward modelling (in a discrete time space) can be understood as predicting error-free data (Tarantola 1987, Romanowicz et al. 1996):

29 30 31 32 33 34 35

METHODS OF CALIBRATION AND VALIDATION USED IN HYDROLOGY

It is important to note that in typical modelling practice both the availability of calibration/validation data and the goal of the modelling play important roles in the model formulation and the form of the model error. The available data dictate the model choice, whilst the goal of the modelling dictates the choice of the model output and the methods of data assimilation. The rainfall-runoff process is stochastic in nature. However, due to the problem complexity, the choice of a deterministic model is often more practical. Moreover, even when some sort of a stochastic approach is taken, it usually consists of, possibly, stochastic pre-processing of inputs, deterministic modelling and a stochastic post-processing of the deterministic model results.

yt  H ( , rt )

(1)

where H () is a nonlinear operator working from parameter space to model space,  is a vector of model parameters and rt denotes meteorological data (rainfall, temperature, evapotranspiration), t, t=1,..,T denotes a time step. Inverse modelling describes the process of model parameter estimation and can be formulated as:

Q( )  | H ( , rt )  zt |

(2)

ON THE CHOICE OF CALIBRATION PERIODS AND OBJECTIVE FUNCTIONS

1 2

5

Where: Q is the modelled flow, zt denotes the observations of flow and

|  | denotes a measure of model performance.

3 4 5 6 7 8 9 10 11 12 13 14 15

The inverse problem depends on the errors in the observations of both input (rainfall and temperature) and output (flow) observations, model structure errors, and the measure of model performance selected for the evaluation of the goodness of fit. There is a vast literature on the subject related to all these issues. A review of the choice of goodness of fit criteria is given in the next subsection 2.2. However, as the aim of this paper is to produce a practical guide to parameter estimation techniques, we shall concentrate on the identifiability problem, and in particular, on the shape of the response surface in the vicinity of the optimal parameter values in a deterministic set-up, i.e., neglecting observation and model structural errors. This will be done in two stages. The first will concentrate on the choice of calibration criteria (section 4). The second stage will concentrate on the choice of the calibration period (section 5).

16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

2.2 Important steps and choices in hydrological model calibration using the HBV model as an example The procedure for choosing the parameter values is called model calibration. There are many approaches to calibration, depending on the model structure chosen and the experience of the modeller. The first step is the choice of the model, followed by the choice of output variables, depending on the aim of the modelling. In the next step the objective functions are determined followed by a choice of the optimisation methods, which depends on the problem statement (including the objective function). The problem can be stated as deterministic, meaning that the modeller neglects observation errors, or stochastic, with observation and/or structural errors included. As an acknowledgement of the stochastic nature of rainfall and flow observations is becoming more common among the hydrological community, the stochastic formulation of rainfall-runoff processes starts to prevail in hydrological research. The last step is the choice of the length and location of a calibration period, determined on the basis of data information content.

33 34 35 36 37 38

2.2.1 Objective functions In the literature reviewed, various objective functions were found. In over two decades of investigation it has not been proven that any particular objective function is better suited for calibration than any other (Gupta and Sorooshian 1998). However, some studies (Fenicia et al. 2007, Deckers et al. 2010) showed that modified functions can deliver better outcomes for low

6

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

22

23 24 25 26 27 28 29 30

RENATA J. ROMANOWICZ, MARZENA OSUCH, MAGDA GRABOWIECKA

and high flows. Recent studies (e.g. Kavetsky et al. 2011) showed that the study of time scale dependencies of hydrological models requires an analysis of parameter and flow distributions (flow duration curves and rainfall-runoff cross-correlation characteristics) rather than a straight-forward (point) comparison of simulated and observed flows. Another example of a purposecontrolled choice of the objective function is given by Pushpalatha et al. (2012) who present a thorough review of the efficiency criteria suitable for evaluating low-flow simulations. A popular function used in the literature is the root mean square error (RMSE). Its use is appropriate when data errors are known not to be correlated, to have constant variance or when the properties of data errors are known (Gupta and Sorooshian 1998). The advantage of this criterion is that it has the same units as the output variable (here, flow). The disadvantage lies in non-additivity for varying time horizons of observations. Mean Square Error (MSE) is also used and presents a more statistically sound measure.

RMSE 

1 n  (Qo,i  Qs,i ) 2 n i 1

(3)

where: n denotes number of time steps, Qo,i - observed runoff at time step i; Qs,i - simulated runoff at time step i. The second commonly used indicator of goodness-of-fit is the coefficient of determination (R2). This measure describes how well a regression line fits a set of data.    R2      

  (Qo, i  Qo )(Qs , i  Qs )  i 1  n n  (Qo, i  Qo ) 2 (Qs , i  Qs ) 2   i 1 i 1 

2

n







(4)

where Qs is mean simulated flow and Qo mean observed flow. In the hydrological literature the most common goodness-of-fit indicator is the Nash Sutcliffe coefficient of efficiency (NS) (Nash and Sutcliffe 1970). It is better suited to evaluate model goodness-of-fit than the coefficient of determination, because R2 is insensitive to additive and proportional differences between model simulations and observations. R2 and NS are very sensitive to extreme values because they square the values of differences.

ON THE CHOICE OF CALIBRATION PERIODS AND OBJECTIVE FUNCTIONS

7

i n

1

NS  1 

 [Q

s ,i

 Qo ,i ] 2

i 1 i n

 [Q

o ,i

 Qo ]

(5)

i 1

2 3 4 5 6 7 8 9 10 11

The Nash-Sutcliffe coefficient of efficiency is a normalised measure of MSE (Gupta et al. 2009), which is the RMSE (Eq. 3) without the square root. The values of MSE are normalized with respect to the variance of the observed hydrograph. This normalization and the fact that RMSE is a square root of MSE leads to a different shape of the response surface for NS and RMSE objective functions. Mean absolute error is the third well-accepted absolute error goodness-offit indicator. It describes differences in observed and predicted values in appropriate units (Legates and McCabe 1999).

MAE 

1 n  Qo,i  Qs.i n i 1

(6)

12 13 14 15 16

Romanowicz and Beven (2006) introduced a measure based on an exponential transformation of the sum of square errors. This function has the form of an exponent to the minus of the sum of square errors between simulated and observed discharges divided by the reduced, scaled mean error variance.

17

 n   (Qo, i  Qs , i ) 2    rr  exp i 1   N  var( Qo, i  Qs , i )     



18 19 20 21 22 23 24 25 26 27 28

(7)

where N describes a scaling factor that takes into account the deviation of the error distribution from the Gaussian due to the correlation between the parameters (so-called inflated variance in the statistics literature). Low values of N result in a smaller variance, thus giving the more “peaked” distribution. This approach does not change the shape of the likelihood function (it is symmetrical), but changes its variance. This criterion was discussed by Blasone et al. (2008), who adjusted N so that the observations fall into the desired confidence bounds. This type of likelihood belongs to the class of informal likelihood measures (Smith et al. 2008). In many publications, a combined objective function is used to optimize model parameters. In this case the volumetric error (difference between the

8

RENATA J. ROMANOWICZ, MARZENA OSUCH, MAGDA GRABOWIECKA

1 2 3 4

volume of simulated and observed discharges during the calibration period) is additionally taken into account. Lindström et al. (1997) introduced the new criterion RV, which is a compromise between NS and absolute relative volume error RD:

5

RV  NS   RD (8)

6 7 8 9 10

13 14 15 16 17 18 19 20 21 22 23

(9)

where ω is a weight with the best result close to 0.1 (Lindström et al. 1997). Van den Tillaart et al. (2013) evaluated HBV model performance with objective function Y, based on Akhtar et al. (2009) findings: Y

11

12

1 n Qs ,i  Qo,i  n i1 Qo,i

RD 

RVE 

NS 1 | RVE |

n

Qs,i  Qo,i

i 1

Qo,i



(10)

 RD  n (11)

where, NS is the Nash-Sutcliffe coefficient and RVE the relative volume error. Several studies have shown that most rainfall-runoff model structures are not capable of reproducing the whole hydrograph with a single parameter set. Following that idea, Fenicia et al. (2007) selected two hydrograph characteristics that the model should correctly simulate: high flows and low flows. To focus on the representation of the low flow and high flows separately the weighting functions depending on the relative flow values are introduced. NHF (eq. 12) gives a stronger weight to the error in the peaks of the hydrograph whilst NLH (eq. 13) enhances the model error within low flow values. n

24

25

N HF 



1 ( (Qs ,i  Qo,i ) 2  WHF ,i ) n i 1

 Q o ,i W HF ,i   Q  o,max

   

(12)

2

(13)

ON THE CHOICE OF CALIBRATION PERIODS AND OBJECTIVE FUNCTIONS

1

2

N LF 

1 n ( (Qs ,i  Qo,i ) 2  WLF ,i ) n i1

W LF ,i

 Qo, max  Qo,i   Q o , max 

   

9

(14)

2

(15)

3 4 5 6 7 8 9 10 11

where n is total number of time steps, Qs,i is simulated flow for the time step i, Qo,i is observed flow for the time step i, Qo,max is maximum observed flow.

12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

2.2.2 Optimization algorithms The HBV model is often calibrated using manual procedures. During manual calibration, the best model parameters are achieved within established min max ranges. The procedure is time-consuming and depends on the modeller’s knowledge and experience. Alternatives to manual calibrations are automatic routines. A very good comparison of manual and automatic methods applied in hydrologic modelling is given by Boyle et al. (2000). The authors argued that both approaches combined together can give the best results. The authors also discuss the advantages of introducing many criteria for parameter calibration (see Efstratiadis and Koutsoyiannis (2010) for a comprehensive review of multi-criteria calibration approaches). The manual approach to calibration requires the deterministic statement of the calibration problem, whilst the automatic routines are used for both deterministic and stochastic problem formulations. The automatic routines depend on the choice of the optimisation routine and their stopping criteria. Among those used for the HBV model calibration are simplex methods (Press et al. 2002), global optimisation methods (Differential Evolution with Global and Local neighbours DEGL, Das et al. 2009, Piotrowski and Napiórkowski 2012)) and SCEM-UA (Shuffled Complex Evolution Metropolis, Vrugt et al. 2003). The DEGL uses a random search of the parameter space, but gives only single best parameter values, i.e. it provides a deterministic solution. The SCEM-UA method provides a multidimensional distribution of parameter values, therefore it belongs to the stochastic approach. The PEST - model-independent parameter optimiser (Doherty 2004) is another approach to model calibration. It was used for

Deckers et al. (2010) proposed two objective functions to focus on a good agreement between observed and modelled low flows or high flows. Model calibration for low flows uses the Nash- Sutcliffe coefficient for 51-day flows below 90% discharge exceedence (NSL). To estimate parameters for high flows part of the hydrograph, a Nash-Sutcliffe coefficient for 7-day flows above 5% flow exceedence (NSH) was applied.

10

RENATA J. ROMANOWICZ, MARZENA OSUCH, MAGDA GRABOWIECKA

1 2

HBV calibration by Lawrence and Haddeland (2011). The PEST software uses a Gauss-Marquardt-Lavenberg (GML) algorithm.

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

2.2.3 A stochastic approach to model calibration and validation The optimal parameter values chosen by the optimisation routine in a deterministic problem formulation give the best model performance (according to the specified criterion), for a single realisation of the observed input and output data. This approach gives a biased solution in the presence of observation errors (Box and Tiao, 1992), but is the most common in hydrological practice. Nevertheless, deterministic optimisation methods may be used as means for exploration of a problem response surface, indicating the feasible parameter ranges (Romanowicz et al. 2010). In the stochastic problem formulation the conditioning of model predictions is performed through the assessment of model errors, i.e. a comparison of the simulated output with observations, with the assumption that errors are additive, independent both in time and space, and stationary. When these assumptions are met, computationally efficient, statistical tools for data assimilation may be used. Unfortunately, those assumptions are violated in rainfall-flow modelling. The errors related to hydrological variables are usually heteroscedastic, i.e. their variance depends on the magnitude of the predicted variable (Sorooshian and Dracup 1980) and correlated in time and space (Sorooshian and Gupta 1983). One method for dealing with these problems is to introduce some appropriate transformation of the error to stabilize the variance and take into account the error correlation/covariance structure (Aronica et al. 1998). These methods require introducing additional parameters to describe the error model structure, which might be unidentifiable when data are scarce. The other option is the use of statistically less efficient, but computationally simpler, methods of uncertainty estimation, e.g. Generalized Likelihood Uncertainty Estimation (GLUE), Beven and Binley (1992), including fuzzy set approaches and other less formal methods of data assimilation (Beven 2006). Informal Bayesian methods are becoming increasingly popular in hydrological modelling (Smith et al. 2008). Assumptions regarding error structure are often made in an implicit way, through the choice of a measure of goodness-of-fit. This leads to misunderstanding in the formulation of the research problem. Namely, the choice of criteria suitable for the uncertainty modelling should not be subjective but should follow the error characteristics. These, in turn, depend on the choice of model output defined by the goal of the modelling. Recent developments in hydrological modelling focus on the timevariability of model parameters, as a recognition of changes undergoing in the catchment response, due to climatic, land-use and catchment manage-

ON THE CHOICE OF CALIBRATION PERIODS AND OBJECTIVE FUNCTIONS

11

1 2 3 4 5 6 7

ment changes. Wagener et al. (2003) give an example of the work done to meet these challenges. The paper introduces the procedure termed dynamic identifiability analysis (DYNIA), in an attempt to analyse the performance of the model in a dynamic fashion resulting in an improved use of available information. The procedure is based on a varying-in-time evaluation of the parameters in a search for the best model performance.

8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

2.2.4 Quantity and quality of calibration data Another important choice to be made is the selection of time periods for the calibration. It is usually assumed that the longer the period the better, as the model will be more robust to varying hydrological conditions. However, the underlying assumption is that the model structure describes well the processes in the catchment. Therefore, it is implicitly assumed that the model parameters are identifiable from the observations and constant in time. Under varying conditions different objective functions should be used to achieve a correct evaluation of model performance. Selected single objective functions should consider different aspects of the hydrograph (Deckers et al. 2010). The latter would require choosing the periods of time corresponding to different parts of a hydrograph. However, when the chosen calibration period contains many hydrographs, an averaging of model performance occurs. This problem is similar to the problem of smoothing in statistics (Box and Tiao 1992). In this paper we shall concentrate on optimisation problems emerging from poor model parameter identifability. In particular, we shall review the choice of objective functions and the choice of calibration period. The solution of the inverse problem (Eq. 2) involves optimisation of the goodness-offit criterion. The most popular method is to use optimisation techniques with the objective function spanning a number of years of observations. The outcome depends on the averaging of the objective function over the entire calibration horizon. As a result, when a single objective function is used, the average performance of the model will be satisfactory but the extremes will be not well matched. Moreover, due to the averaging property of the solution, identifiability issues arise. In brief, the objective function will be flat, showing the same value for a number of sets of different parameter values. This behaviour was called equifinality by Beven and Binley (1992), giving an origin to the Generalised Likelihood Uncertainty Estimation (GLUE) method. In statistics, it can be related to the nonlinear regression problem (Hastie and Tibshirani 1990), where the choice of averaging range should be made. Large neighbourhoods (time periods) would produce low variance, but potentially large bias, small neighbourhoods the converse. This exempli-

12

RENATA J. ROMANOWICZ, MARZENA OSUCH, MAGDA GRABOWIECKA

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

fies the trade-off between bias and variance, controlled by a smoothing parameter, which in the case of scatter-plot smoothing is the width of the bin, and in our problem, is the time horizon of the observations used in the calibration. In the section 5 we shall study the influence of the time horizon of the calibration period on the identifiability of the HBV model parameters by means of a classification of observation periods used in calibration according to hydrological seasons (winter). This classification introduces the division of flows into low and high values, as the floods in the River Wieprz catchment occur mainly in the winter season. The optimal parameter sets obtained by global optimization using DEGL, will be applied during the validation stage. The models are validated on the data from all other periods in a similar way to Luks et al. (2011). Additionally, we explore information on the optimal parameter values obtained from the calibration stage to assess the quality of observations during the validation stage. In order to explore the surface response shape, Monte Carlo simulations are carried out in the feasible space of parameter values (Figures 2, 3 and 4). In these figures the results obtained for short winter periods are compared with the long-term (up to 10 year) period calibration. The application of a season-based calibration and validation procedure was aimed at exploring the quality of the data. There are two main issues in this regard. Firstly, the quality of data can vary in time, depending on the external circumstances (e.g. a change of gauging station characteristics), or secondly, by man-induced changes in flow patterns, such as water abstractions, flow damming by ice or human intervention. In both cases, the natural relationship between rainfall and flow will be obstructed on temporal basis. The season-based calibration can indicate when the changes in the rainfall-flow relationship occur. Moreover, it can indicate the time periods for which the rainfall-flow characteristics of the catchment can be identified. This is the first time when the problem of parameter calibration has been formally addressed from the point of view of the data information content, although intuitively it is very obvious.

33 34 35 36 37 38 39 40 41

3.

RIVER WIEPRZ CASE STUDY

The River Wieprz is a right bank tributary of the River Vistula (Fig. 1). Its source is the periodic Lake Wieprzowe situated at an altitude of 274 m.a.s.l. near to Wieprzów Tarnawacki. Wieprz is 303 km long and joins the River Vistula near Dęblin. The catchment area (approx. 10300 km2) is located entirely within the administrative boundaries of the Lublin province. According to the Corine Landcover data for the year 2000, it is covered by agricultural areas (77%), forest and semi natural areas (19%), artificial sur-

ON THE CHOICE OF CALIBRATION PERIODS AND OBJECTIVE FUNCTIONS

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

13

faces (3%) and water bodies (1%). The catchment elevation ranges from 115 m a.s.l. for the gauging station to about 390 m a.s.l. for the highest hills. We do not take into account elevation correction. Via the Wieprz-Krzna channel the Vistula links with the River Krzna and the Krzna joins the Bug river basin. As it is a drainage channel, with no significant flow transfer, the Wieprz remains a natural river due to relatively low anthropogenic activities in the valley (especially in certain sections). It is a meandering river with a wide valley, oxbow lakes, wet meadows with extensive sedges (Cartex grasslands) and local marshlands, creating numerous habitats for many creatures; among others, there is a number of nesting sites of endangered species of wetland birds. Morphologically, the Wieprz catchment is also very diverse. It crosses chalk and loess hills with deep valleys of Roztocze in its upper part. The River Wieprz main tributaries are Łabuńka, Por, Wolica, Bystrzyca and Tyśmienica. In this study we use flow observations from the Kośmin gauging station. The rainfall and temperature data come from Puławy, Sobieszyn, Lublin, Tarnogród, Zamość and Tomaszów Lubelski meteorological stations.

14

RENATA J. ROMANOWICZ, MARZENA OSUCH, MAGDA GRABOWIECKA Krzna

Św

Wisła KOŚ MIN Wieprz

#

. !

Ty śm ink a

. !

ica

LUBARTÓW

#

PUŁ AWY

LUBLIN ! . . !

ie n

Minin a

t rz Bys

Ś winka

yca

Mogilnica

OPOLE LUBELSKIE

#TRAWNIKI KRASNYSTAW

#

Gieł czew

#

Pó r (l)

Wojsł awka Wolica

KRZAK

Legend . !

#

meteorological stations gauging stations Wieprz catchment rivers Wieprz- Krzna channel

F

Thiessen polygons 0 5 10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

20

30

ZAMOŚĆ Łabuń ka . ! ZWIERZYNIEC

#

TOMASZÓW LUBELSKI . ! . !

TARNOGRÓD

40 Kilometers

Figure 1 River Wieprz catchment 4.

DISCUSSION OF IDENTIFIABILITY ISSUES BASED ON AN ANALYSIS OF THE SHAPE OF THE RESPONSE SURFACE

4.1 Rainfall-runoff modelling: HBV model The rainfall-runoff model applied in this paper is a lumped conceptual HBV model (Bergström 1976, Lindström 1997, Lindström et al. 1997). The model has a physically based structure that includes four conceptual storage reservoirs describing soil moisture, snow melt and runoff components of the rainfall-flow processes in the catchment. In this study it has been run on a daily basis. Catchment precipitation and air temperature were estimated from measurements at meteorological stations at Pulawy, Sobieszyn, Lublin, Tarnogród, Zamość and Tomaszów Lubelski using Thiessen polygons. PET was calculated by the Hamon method based on catchment air temperature. The model is calibrated on discharge observations from the Kośmin gauging station. The lumped parameter HBV model, applied in this study, is a very sim-

15

ON THE CHOICE OF CALIBRATION PERIODS AND OBJECTIVE FUNCTIONS

1 2 3 4 5 6 7 8 9 10 11 12

plified representation of spatially varying rainfall-runoff processes in the catchment. Therefore, model parameters have only a very crude resemblance to their physical counterparts. The model has a large number of parameters of which only eight are calibrated. These are:  FC – maximum soil moisture storage [mm],  β - nonlinear runoff parameter,  LP – limit for potential evaporation,  α – nonlinear response parameter, -1  KF – upper storage coefficient [day ], -1  KS – lower storage coefficient [day ],  PERC – percolation rate [mm/day] and  CFLUX- maximum capillary rate [mm/day].

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

4.2 Evaluation of the HBV model performance using different objective functions

29 30 31 32

As described in section 2, different criteria result in a different shape of the response surface and following it, a different identifiability of model parameters. The aim of this section is to illustrate that issue through an exploration of the response surface. This is done by multiple simulations of the model using different criteria for parameters sampled randomly. 100,000 Monte Carlo simulations were used for the analysis of the response surface. The model parameters were drawn randomly from uniform distributions within the defined upper and lower parameter boundaries (Tab. 1). The parameter ranges were chosen following the literature review as being physically realistic (Bergström 1976, Liden and Harlin 2000, Rientjes et al. 2011, Booij and Krol 2010). Table 1 Prior parameter ranges Parameter Unit

FC [mm]

β []

LP [-]

α [-]

Minimum Maximum

1 1000

1 6

0.1 1

0.1 3

KF [day1] 0.0005 0.3

KS [day1] 0.0005 0.3

PERC [mm/day]

CFLUX [mm]

0.1 10

0 2

To evaluate the performance of the HBV model we used the objective functions presented in section 2.2. These were: the root mean square error (RMSE), coefficient of determination (R2), the Nash-Sutcliffe coefficient of efficiency (NS), the mean absolute error (MAE), a measure based on the ex-

16

RENATA J. ROMANOWICZ, MARZENA OSUCH, MAGDA GRABOWIECKA

1 2 3 4 5 6 7 8

ponential transformation of the sum of the square errors (rr), and two combined objective functions RV and Y (equations 3-15). The objective functions obtained for all the simulations were normalised and used as likelihood weights for the derivation of marginal posterior distributions of each randomly sampled parameter. This procedure is equivalent to applying Generalised Likelihood Uncertainty Estimation (GLUE) with subjective likelihood functions, as described by Beven and Binley (1992), and which follows the well-known Bayes formula:

9

f ( | z)  f0 ( )  L( z |  ) / L( z) (16)

10

where f ( | z ) denotes posterior distribution of parameters  , condi-

11 12 13 14

tioned on the observation vector z, f 0 ( ) denotes the prior distribution of parameters (here, uniform prior was assumed); L( z |  ) denotes the likelihood function in the form of any of the studied objective functions (eq. 315), normalised to the range [0,1]; L(z ) denotes a scaling factor.

15 16

The model was run for a 10 year time horizon at daily time steps from 1.11.1966-31.10.1975 with one year for warm-up 1.11.1965-31.10.1966. b

FC

0.8

0.8

0.6

0.6 cdf

1

cdf

1

0.4

0.4

0.2

0.2

NS

RMSE Y

0

0

200

400

600

800

0

1000

0

1

2

4

5

6

a

LP 1

0.8

0.8

0.6

0.6

MAE NHF NLF R2 RV rr

cdf

1

cdf

17 18 19

3

0.4

0.4

0.2

0.2

0

0.2

0.4

0.6

0.8

1

0

0

0.5

1

1.5

2

2.5

3

Figure 2 The marginal cdfs for the FC and β parameters (upper panel) and the LP and α parameters (lower panel) for the years 1966-1975.

17

ON THE CHOICE OF CALIBRATION PERIODS AND OBJECTIVE FUNCTIONS

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Figures 2 and 3 show the posterior marginal cumulative distribution functions (cdfs) for eight parameters, FC, b, LP and aFig. 2and KF, KS, PERC and CFLUX (Fig. 3), obtained using nine normalised objective criteria (Eq. 3-15) and following the GLUE approach (Eq. 16). The difference between the marginal posterior parameter distribution and a line describing the uniform cdf, can be used as a measure of parameter “identifiability” (Abebe et al. 2010). The identifiability of model parameters depends on the parameter and objective functions. The least identifiable parameters are CFLUX and KF. There are very small differences between a priori (uniform) and a posteriori cdfs in both cases. These two parameters describe the percolation (CFLUX) and surface flow (KF). The KF parameter is related to the surface flow which is better represented in the model, but its ranges are limited to (00.3), which means that its control abilities are low. The range of variability is forced by the physical meaning of this parameter within the model set-up. This parameter is related to α parameter, which is better identifiable, but also in a limited range. Both CFLUX and KF parameters could have been assigned constant values. The best identifiable parameters are: FC, β, and PERC and the objective functions NS, RMSE and Y provide the best identifiability of these parameters for the River Wieprz catchment. KF

KS

0.8

0.8

0.6

0.6 cdf

1

cdf

1

0.4

0.4

0.2

0.2

0

0

NS

RMSE Y

0

0.05

0.1

0.15

0.2

0.25

0.3

0

0.05

PERC

0.15

0.2

0.25

0.3

0.8

0.8

0.6

0.6 cdf

1

MAE NHF NLF R2 RV rr

CFLUX

1

cdf

21 22 23

0.1

0.4

0.4

0.2

0.2

0

0

2

4

6

8

10

0

0

0.5

1

1.5

2

Figure 3 The marginal cdfs for the KF and KS parameters (upper panel) and the PERC and CFLUX parameters (lower panel) for the years 1966-1975.

18

1 2 3 4 5 6 7 8 9

RENATA J. ROMANOWICZ, MARZENA OSUCH, MAGDA GRABOWIECKA

A similar analysis was performed using data from one winter season only, to study the influence of the length of time horizon and the catchment flow magnitude on the parameter identifiability. It should be noted that the winter season lasts from the 1st of November until the end of April. In the Wieprz catchment most high flow incidents occur in winter season; therefore that choice gives a simple division into high and low flow periods. Figure 4 presents the marginal cumulative distribution functions (cdfs) for the model response surface obtained via Monte Carlo sampling of the parameter space using the NS criterion for one winter season 1978/1979. b

FC 1 cdf

cdf

1

0.5

0

0

200

400

LP

600

800

0

1000

0.5

0

0.2

0.4

0.6 KF

0.8

cdf

cdf

2

a

3

4

5

6

0

0.5

1

1.5 KS

2

2.5

3

0

0.05

0.1

0.25

0.3

1

0.5

0

0.05

0.1

0.15 PERC

0.2

0.25

0.5

0

0.3

0.15 0.2 CFLUX

1 cdf

1 cdf

1

0.5

0

1

1

0.5

0

0

1 cdf

cdf

1

0

0.5

0

2

4

6

8

10

0.5

0

0

0.5

1

1.5

2

10 11 12 13 14

Figure 4 Marginal cdfs for eight parameters (FC, β, LP, α, KF, KS, PERC, CFLUX) for winter 1978/1979. Vertical black lines denote parameter optimal value for NS objective function. In the case of KF the optimal value is located close to lower boundary.

15 16 17 18 19 20 21 22 23

The plots show the optimal parameter values obtained using a DEGL optimisation routine for the same winter season. Comparison of the shape of the cdfs with those shown in Figs 2 and 3 shows similarities and differences indicating that the identifiability of model parameter depends on the time horizon and climatic conditions. In our example, parameters FC, b α and PERC are the best identifiable. KF has optimal parameters located at the lower end of the parameter ranges. This suggests that this parameter may require a logarithmic sampling, or more simply to tighten the parameter’s range.

ON THE CHOICE OF CALIBRATION PERIODS AND OBJECTIVE FUNCTIONS

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

19

A comparison of a season-based and long-term calibration (see Figure 2, 3 and 4) indicates that a short-period calibration for the winter 1978/1979 provided more information for the parameter α, making the NS criterion more sensitive to its changes. This parameter describes the power relationship between surface flow and upper reservoir storage in the HBV conceptual representation of a rainfall-flow process (Bergström 1976). Its influence on the NS criterion is more pronounced for large water storage volumes and can be averaged out when many seasons are used for the criterion evaluation. The sensitivity of NS to the other parameters was the same in both cases. 5.

ASSESSMENT OF THE QUALITY OF RAINFALL-RUNOFF OBSERVATIONS USING A SEASON-BASED CALIBRATION AND VALIDATION PROCEDURES

The popular approach to the calibration of rainfall-flow models assumes that the length of the calibration period should be not less than several years. This assumption is based on an attempt to average the model performance to make it robust for all possible catchment responses. Because of the inherent model simplifications of the real rainfall-runoff, this leads to an average quality of model results. In order to explain the problem, we note that rainfall is measured at a point scale (even when, as in this case study, observations from six rain gauging stations are available) whilst the flow comprises an integration of all the water sources existing in the catchment (including the groundwater). Therefore there is a good number of reasons why rainfallrunoff model parameters are not easy to estimate. On the other hand, using an event or season-based calibration usually leads to non-robust solutions. Each event produces different optimal parameter sets indicating differences in catchment responses. Even though the parameter sets are not general enough to describe the average model performance during the validation period, they can provide some information on the quality of the data and the relationships between input (rainfall and temperature) and output (flow) variables. Thus using season-based model calibration we can learn about changes taking place in the catchment (Luks et al. 2011). In the present study we use the division into winter and summer periods as a simple means of flow classification and an assessment of the informative content of data. Every hydrological modeller must have encountered catchments where the calibration results depend strongly on the calibration period used, with some periods superior to the others. This observation led to the idea of looking for time-periods (e.g. winter or summer) that give a good rainfall-runoff model performance and try to learn from that experience about the possible causes. The models are calibrated on short time-periods and verified in a similar season-based manner. This procedure was applied to winter seasons over 29

20

1 2 3 4 5

RENATA J. ROMANOWICZ, MARZENA OSUCH, MAGDA GRABOWIECKA

years of observations for the River Wieprz. The HBV model was optimised using the Nash-Sutcliff criterion and the DEGL optimisation routine. The results of the calibration and validation procedures in the form of the obtained NS criteria for the all analysed seasons are shown in Fig. 5. 1965

CALIBRATION

1970

NS>0.75 0.50