and statistical model predictions were used to restore missing wave measurements in smaller gaps, while data from neighbouring sites were employed to ...
Maritime Transportation and Exploitation of Ocean and Coastal Resources – Guedes Soares, Garbatov & Fonseca (eds) © 2005 Taylor & Francis Group, London, ISBN 0 415 39036 2
Filling gaps in wave records with artificial neural networks O. Makarynskyy Western Australian Centre for Geodesy, Curtin University of Technology, Perth, Australia
D. Makarynska Department of Exploration Geophysics, Curtin University of Technology, Perth, Australia
E. Rusu University “Dunarea de Jos” of Galati, Galati, Romania
A. Gavrilov Centre for Marine Science & Technology, Curtin University of Technology, Perth, Australia
ABSTRACT: This contribution presents a neural data interpolation methodology, which was implemented to restore missing wave measurements. The methodology is based on the ability of artificial neural networks to find and reproduce non-linear dependencies within complex geophysical systems. The data were obtained from a field campaign during July 1985-December 1993 near Tasmania. Wave observations from a “Waverider” buoy were broadcasted as a high frequency radio signal via a quarter-wave antenna to a “Diwar” receiver. These measurements were used to train and to validate the neural nets employed. To restore missing data over time periods from 12 to 36 hours, five feed-forward, three-layered, artificial neural networks of a similar structure were implemented. The artificial neural networks’ performance was estimated in terms of the bias, root mean square error, correlation coefficient, and scatter index. The methodology demonstrated reliable results with a fairly good overall agreement between the restored wave records and actual measurements.
1
INTRODUCTION
The problem of temporal interpolation frequently emerges in geosciences on the stage of time series preprocessing (e.g. Emery & Thomson 1997). This is a particularly important step when filling gaps in wave records because the way of data restoration defines outcomes of any further analysis and may affect the quality of data series extension (e.g. Makarynskyy & Makarynska 2005, in press). The question of data interpolation can be considered from the deterministic or stochastic points of view, for both of which a reasonable number of methods were developed. However, there are really few works dealing with applied interpolation of wave parameters, and the commonly used with this purpose method is linear interpolation (e.g. Cunha & Guedes Soares 1999). There were two essentially different approaches to the wave data interpolation adopted by Hidalgo et al. (1995) in their study. In that paper, linear interpolation and statistical model predictions were used to restore missing wave measurements in smaller gaps, while data from neighbouring sites were employed to complete
large gaps. A direct interpolation of gaps in time series records employing various techniques was also performed by Deo & Kiran Kumar (2000) as well as Stefanakos & Athanassoulis (2001). The former article describes the implementation of several numerical methods based upon polynomial fits, artificial neural networks and statistical disaggregation model to interpolate monthly mean wave heights obtained from satellite into weekly periods. It was noted that the quality of neural interpolation was superior comparatively to other methods’ performance. Stefanakos & Athanassoulis (2001) developed and successfully validated their own flexible nonstationary stochastic model to restore time series of significant wave height. Guedes Soares & Cunha (2000) proposed to use bivariate models to fill the gaps in records at one wave gauge involving the parallel highly correlated measurements at another neighbouring station. Adopting a similar approach, artificial neural networks were also implemented to tackle the tasks of reconstruction of significant wave height (Arena & Puca 2004, Tsai et al. 2002), significant wave height and zero-upcrossing wave period (Makarynskyy & Makarynska
1085
2005, in press) and sea level (Makarynskyy et al. 2005a) time series. Importantly, the presence of two wave measuring tools within limits of high order spatial/temporal correlation is rather an exception than a rule in oceanography. Therefore, there is a strong demand for reliable interpolation methodologies able of filling large gaps of order of days while based on solitary gauge registrations. In the present paper, such a methodology is described. The methodology is based on the capability of artificial neural networks to find and reproduce non-linear dependencies within complex geophysical systems, particularly, when short and long surface waves are concerned (e.g. Lee et al. 2005, submitted, Huang et al. 2003, Makarynskyy et al. 2004). In the present study, a feed-forward, three-layered, artificial neural network (ANN) was implemented to restore the missing 3-hourly wave records over time periods from 12 up to 36 hours. The current contribution is structured as follows. Section 2 describes the data used. Section 3 presents briefly the technique of ANNs. The validation statistics used are presented in Section 4. Section 5 gives more detail on the proposed methodology and specific procedures employed here. The results obtained and following discussion are given in Section 6. Concluding remarks are presented in Section 7.
Figure 1.
Schematic representation of the deployment site.
Figure 2. Measurements from Cape Sorell used in the study.
2
SIGNIFICANT WAVE HEIGHT RECORDS USED
The measurements used in this study were obtained from a “Waverider” buoy moored off Cape Sorell (coordinates 42°8.7S 145°9.4E) in water 50 m deep (Figure 1). The site is located in a coastal environment of Tasmania, the island south of mainland Australia. These data were collected during a field campaign of July 1985-December 1993 and processed by the Division of Oceanography, part of the Australian Commonwealth Scientific and Industrial Research Organisation (Reid & Fandry 1994). Nevertheless, the 3-hourly wave records for a much shorter period (30/04/87–21/12/87, Figure 2) were used here due to the following reasons. Firstly, this was the longest time series without large (more than 3 missing records) gaps and with one of the lowest percentage gap of 0.6% in this field campaign (Makarynskyy & Makarynska 2005, in press). Wave measurements were broadcasted as a high frequency radio signal via a quarter-wave antenna on top of each buoy to coastal “Diwar” receivers. Therefore, the gaps were purportedly caused by the loss of/or poor radio signals due to radio interference, airborne seawater spray and/or loss of line-of-sight due to large
waves. The gaps of few (1–3) missing records were restored using linear interpolation, which on such percentage gap does not influence final results. Secondly, this period covers the southern hemisphere autumn and winter rough seas with significant wave heights up to 9 m (Figure 2), as well as more calm spring and summer wave conditions. These factors are of special importance because ANNs might be expected to reproduce complex system behaviour plausibly only after comprehensive training over the entire range of possible system states. Figure 2 exhibits the completed time series from the site. This time series was divided into two non-overlapping data sets. Data set of 1246 wave registrations (2/3 of the series) was used to train the corresponding neural networks, while an independent data set of 623 measurements (1/3 of the series) served for validation purposes. 3
OUTLINE OF ARTIFICIAL NEURAL NETWORKS
The power of the biological brain comes from the number of neurons in it and from up to 200000 connections between these neurons. Artificial parallel
1086
The scatter index was calculated in two different ways: (4) Figure 3.
Artificial neuron.
neuro-computing constructions, or ANNs, are modelled on the biological brain. This artificial intelligence technique is an attempt to simulate the multiple layers of simple computational nodes, though artificially created neurons are much simpler than the biological ones. The input neurons of an ANN (Figure 3) receive some data from the external surround and communicate it to the hidden layers of the system, where the data are processed. Each neuron is connected to its neighbours with different coefficients of connectivity, also called weights. The pattern of the inter-connections between the neurons is called the architecture. To force the entire network to perform in some expected way, these weights have to be adjusted by presenting a training data set to the ANN in a number of training iterations called epochs. The learning continues until a predefined condition (stopping criterion) is fulfilled, and the obtained results are fired to the external environment. More detailed description of ANNs might be found in Fausett (1994), Bishop (1995), Haykin (1999), and others. 4
VALIDATION STATISTICS
The validation of the results of neural simulations was implemented using the following statistics: (1)
(2)
(3)
where RMSE is the root mean square error; R is the correlation coefficient; N is the number of time steps; xi is the value observed at the i-th time step; –x is the mean value of the observations; yi is the value simulated at the same moment of time; and –y is the mean value of the simulations.
(5) The values of these statistics were very similar due to the closeness of x– and y–. Therefore, only the SIrmse (hereafter mentioned as SI) will be analyzed.
5
METHODOLOGY APPLIED
In contrast to time series extrapolation procedures, where as a rule there are no preliminary data to be considered as first-guess values (e.g. Makarynskyy 2004, Makarynskyy et al. 2005b), newly developed interpolation routines might be based on the results of linear interpolation. This is a well-established technique that could restore missing records over short time periods of order of hours reasonably well though generally unable to reflect synoptic variability of wave parameters. Nevertheless, even rough linear interpolated wave estimates obtained over synoptic time intervals (tens of hours) might be of use as a first approximation to actual wave measurements. Therefore, a pre-processing procedure based on linear interpolation was implemented as described below. Firstly, the data over each 12 h, 18 h, 24 h, 30 h and 36 h (4, 6, 8, 10, and 12 3-hourly measurements, respectively) time windows were linearly interpolated. Secondly, the differences between actual wave records and interpolated values were calculated. It was assumed that wave measurements before and after each “gap”, all over equal time periods, contain the necessary information to recover the “missing data”. Following this logic and considering the case of 12 missing records illustrated in Figure 4, 12 Hs observations before and after a “gap” were used as input values for the employed ANN, while 12 residuals formed by subtracting of linearly interpolated Hs from observed ones were presented to the ANN as target values. Therefore, the 5 three-layer, feed-forward ANNs with a non-linear differentiable log-sigmoid transfer function in the hidden layer and linear transfer function in the output layer employed here had the following structure. The number of input units (IU) was always twice larger than the number of output units (OU), while the number of hidden neurons was computed as IU OU 1, as was successfully done in a number of previous studies (e.g. Makarynskyy et al. 2004, Makarynskyy & Makarynska 2005, in press).
1087
Figure 4. Measured and linear interpolated Hs for a randomly selected 108-hour period (36 3-hourly measurements) during 14–16/09/87.
All these ANNs were trained using one of the simplest but still efficient the resilient backpropagation training algorithm. The number of weights and biases to adjust in the process of training of the largest employed ANN, which was the one with 24 input nodes, 37 hidden neurons, and 12 output neurons (hereafter referred to as 24 37 12, and likewise for other architectures), was equal to 1381. This slightly outnumbers the amount of the records in the first data set. Therefore, the selected number of 200 training epochs served as an early stopping criterion to avoid the ANNs overfitting, which leads to the inability of ANNs to generalize. For all the ANNs considered here, the value of the performance function (averaged squared error between the network outputs and the target outputs) gradient was approaching to 0 in 200 epochs, meaning that the training process was accomplished. 6 RESULTS AND DISCUSSION Some illustrative examples of 24 37 12 ANN outcomes are presented in Figure 5. Notably, linear interpolation fails completely to reproduce the time series variability over these two 36 h data gaps. Importantly, the implemented for the cases 24 37 12 ANN produces the results, which follow closely the trends of the Hs variations, though slightly overestimating (top panel in Figure 5) or underestimating (bottom panel in Figure 5) the observed values. Graphs of the validation statistics calculated for the outcomes of the 8 13 4, 12 19 6, 16 25 8, 20 31 10 and 24 37 12 ANNs are presented in Figure 6. Biases of the Hs reconstructed by the first 3 ANNs are very small with the values ranging from 0 to 0.011 m. The simulations with the 2 largest ANNs demonstrate about twice-higher biases.
Figure 5. Measured and interpolated Hs for randomly selected 36-hour periods during 10–12/06/87 (top panel) and 6–8/12/87 (bottom panel).
RMSEs and SIs of the Hs interpolated by the ANNs over all 5 different time intervals behave in a similar way. The lowest values of these statistics (about 0.15 m and 0.06, respectively) were obtained at the first hours of interpolation. The RMSE and SI rise slowly (up to 0.60 m and 0.22, correspondingly, in simulations with the largest ANN) when approaching the central hours of interpolation, and decrease again to the initial level of accuracy towards the last hours. Analogously, Rs of the ANNs’ outcomes demonstrate the highest quality of the simulations at the first and last hours of data restoration, and the lowest quality in the middle of the interpolation intervals. The values of this statistic parameter stagger within the limits 1.00–0.85 (Figure 6) exhibiting overall high performance of the ANNs employed. Such a general pattern of the accuracy’s variations is attributed to the presence of strong interrelations
1088
Figure 6. Validation statistics of neural interpolation with the ANNs of 5 architectures over corresponding time periods.
Figure 7. Scatterplots for the 3rd, 6th, 9th and 12th hours of interpolation.
1089
between the used measurements and interpolated data at the beginning and the end of the interpolation periods. Meantime, closer to the central parts of the periods, these interrelations weaken though still providing sufficient ANN inputs for producing plausible neural estimates of the missing wave records. To make this clearer, scatter plots of the restored versus measured Hs are presented in Figures 7–8. These plots show that there is almost no dispersion around the exact fit line for the first (3rd hour) as well as the last (24th hour) periods of neural interpolation when the 16 25 8 ANN is used. The dispersion increases gradually up to the hours 12 (Figure 7) and 15 after which it decreases again (Figure 8). 7
CONCLUDING REMARKS
This study presented some site-specific results of application of the ANN technique to the task of significant wave height time series restoration. The observations from a single wave gauge were used. The ANNs employed here were three-layer, feed-forward with a non-linear differentiable log-sigmoid transfer function in the hidden layer and linear transfer function in the output layer. An attempt was made to interpolate missing 3-hourly wave records over 12, 18, 24, 30, and 36-hour gaps involving ANNs with corresponding architectures. The pattern of the simulation accuracy’s variations exhibited existence of strong dependencies between the measurements used as the input to the ANNs and restored wave records at the beginning and the end of each interpolation period. The relations between the input and central data points were weaker. It was concluded that, when compared with actual wave records, overall high quality interpolation outcomes were produced for all the time intervals in consideration. This conclusion was based on the low values of bias (0.00–0.02 m), root mean square error (0.15–0.60 m) and scatter index (0.06–0.22) as well as high correlation coefficients (0.85–0.99). Thus this case study clearly demonstrates the usefulness of the proposed neural approach when solving the problem of wave data interpolation over time intervals from 12 up to 36 hours in the coastal environment of Tasmania. Based on sufficiently long and continuous wave measurements, this neural methodology could be successfully generalized to other locations. ACKNOWLEDGMENTS
Figure 8. Scatterplots for the 15th, 18th, 21st and 24th hours of interpolation.
This study was partially funded by a Curtin Strategic Research Scheme Grant. Appreciation is given to the Division of Marine Research, the Australian Commonwealth Scientific and Industrial Research Organisation,
1090
for making available the wave observations. The authors thank the reviewers for their constructive comments. REFERENCES Arena, F. & Puca, S. 2004. The reconstruction of significant wave height time series by using a neural network approach. ASME Journal of Offshore Mechanics and Arctic Engineering 126 (3): 213–219. Babovic, V. 1999. Sub-symbolic process description and forecasting using neural networks. In Garcia-Navarro, P. & Playan, E. (eds), Numerical Modelling of Hydrodynamic Systems Proc. intern. workshop, Zaragoza, Spain, 21–24 June 1999. Bishop, C.M. 1995. Neural Networks for Pattern Recognition. Oxford: Clarendon Press. Cunha, C. & Guedes Soares, C. 1999. On the choice of data transformation for modelling time series of significant wave height. Ocean Engineering 26: 489–506. Deo, M.C. & Kiran Kumar, N. 2000. Interpolation of wave heights. Ocean Engineering 27: 907–919. Emery, W.J. & Thomson, R.E. 1997. Data analysis methods in physical oceanography. New York: Elsevier Science Inc. Fausett, L. 1994. Fundamentals of neural networks. Architectures, algorithms, and applications. Upper Saddle River: Prentice-Hall. Guedes Soares, C. & Cunha, C. 2000. Bivariate autoregressive models for the time series of significant wave height and mean period. Coastal Engineering 40: 297–311. Hagan, M.T., Demuth, H.B. & Beale, M.H., 1996. Neural Network Design. Boston: PWS Publishing. Haykin, S. 1999. Neural networks: a comprehensive foundation. Upper Saddle River: Prentice-Hall. Hidalgo, O.S., Nieto Borge, J.C., Cunha, C.C. & Guedes Soares, C. 1995. Filing missing observations in time series of significant wave height. In Guedes Soares, C. (ed), Fourteenth International Conference on Offshore Mechanics and Arctic Engineering, Copenhagen, Denmark, 18–22 June 1995.
Hornik, K. 1993. Some new results on neural network approximation. Neural Networks 6: 1069–1072. Huang, W., Murray, C., Kraus, N. & Rosati, J. 2003. Development of a regional neural network for coastal water level predictions. Ocean Engineering 30: 2275–2295. Lee, T.-L., Makarynskyy, O. & Shao, C.-C. 2005 (submitted April 2005). A combined harmonic analysis-artificial neural network methodology for tidal predictions. Journal of Coastal Research. Makarynskyy, O. 2004. Improving wave predictions with artificial neural networks. Ocean Engineering 31 (5–6): 709–724. Makarynskyy, O., Makarynska, D., Kuhn, M. & Featherstone, W.E. 2004. Predicting sea level variations with artificial neural networks at Hillarys Boat Harbour, Western Australia. Estuarine, Coastal and Shelf Science 61 (2): 351–360. Makarynskyy, O. & Makarynska, D. 2005 (accepted May 2005). Wave prediction and data supplement using artificial neural networks. Journal of Coastal Research. Makarynskyy, O., Makarynska, D., Kuhn, M. & Featherstone, W.E. 2005a. Using artificial neural networks to estimate sea level in continental and island coastal environments. In Cheng, L. & Yeow, K. (eds.), Hydrodynamics IV: Theory and Applications. London: Taylor & Francis Group. Makarynskyy, O., Pires-Silva, A.A., Makarynska, D. & Ventura-Soares, C. 2005b. Artificial neural networks in wave predictions at the west coast of Portugal. Computers & Geosciences 31 (4): 415–424. Reid, J.S. & Fandry, C.B. 1994. Wave Climate Measurements in the Southern Ocean. Adelaide: CSIRO Marine Laboratories, Report 223. Stefanakos, Ch.N. & Athanassoulis, G.A. 2001. A unified methodology for the analysis, completion and simulation of nonstationary time series with missing values, with application to wave data. Applied Ocean Research 23 (4): 207– 220. Tsai, C.-P., Lin, C. & Shen, J.-N. 2002. Neural network for wave forecasting among multi-stations. Ocean Engineering 29 (13): 1683–1695.
1091
