A Deep Learning Approach Towards Price

A Deep Learning Approach Towards Price Forecasting Using Enhanced Convolutional Neural Network in Smart Grid Fahad Ahmed1 , Maheen Zahid1 , Nadeem Javaid1(B) , Abdul Basit Majeed Khan2 , Zahoor Ali Khan3 , and Zain Murtaza1 1

3

COMSATS University, Islamabad 44000, Pakistan [email protected] 2 Abasyn University Islamabad Campus, Islamabad 44000, Pakistan Computer Information Science, Higher Colleges of Technology, Fujairah 4114, UAE http://www.njavaid.com

Abstract. In this paper, we attempt to predict short term price forecasting in Smart Grid (SG) deep learning and data mining techniques. We proposed a model for price forecasting, which consists of three steps: feature engineering, tuning classifier and classification. A hybrid feature selector is propose by fusing XG-Boost (XGB) and Decision Tree (DT). To perform feature selection, threshold is defined to control selection. In addition, Recursive Feature Elimination (RFE) is used for to remove redundancy of data. In order, to tune the parameters of classifier dynamically according to dataset we adopt Grid Search (GS). Enhanced Convolutional Neural Network (ECNN) and Support Vector Regression (SVR) are used for classification. Lastly, to investigate the capability of proposed model, we compare proposed model with different benchmark scheme. The following performance metrics: MSE, RMSE, MAE, and MAPE are used to evaluate the performance of models.

1

Introduction

Nowadays, electricity plays an important role in economic and social development. Everything is dependent on electricity. Without electricity, our lives are imagined to be stuck. Electricity usage areas are divided into three categories: industrial, commercial and residential. According to [1], residential area consumes almost 65% of electricity from the whole generation. In the traditional grid, most of the electricity is wasted during generation, transmission and distribution. To solve this issue, SGs are introduced. A traditional grid is converted into SG when information, communication and technology (ICT) are integrated into the traditional grid. SG is an intelligent grid system that manages generation, consumption and distribution of energy more efficiently than traditional grid [2]. SG provides the facility of bidirectional communication between utility and consumer. As we know, energy is the most valuable asset of this world. c Springer Nature Switzerland AG 2019 L. Barolli et al. (Eds.): EIDWT 2019, LNDECT 29, pp. 271–283, 2019. https://doi.org/10.1007/978-3-030-12839-5_25

272

F. Ahmed et al.

It is very necessary to utilize energy in an efficient way to increase productivity and to decrease losses and hazards. Energy crises are present everywhere, so industries are moving toward SG. The primary goal of SG is to keep balance between supply side (utility) and demand side (consumer) [3]. SG fulfills, all the demands from the consumer side and gives response to their requests. Consumers send their demands to the utility through Smart Meter (SM). Hence, a huge amount of data is collected via SM regarding the electricity consumption of consumers. Electricity usage may vary depending upon different factors such as: wind, temperature, humidity, seasons, holidays, working days, appliances usage and number of occupants. Utility must be aware of the usage pattern of consumer. [4], Data Analytics (DA) is a process of examining data. DA is basically used in business intelligence, for decision making. When data analyst wants to do an analysis of electricity load consumption and pricing trends, then they take dataset of any specific electricity company. To maintain the load of electricity consumption, many researchers are working on forecasting of electricity load and price [5]. There are three types of forecasting: Short Term Load Forecasting (STLF), Medium Term Load Forecasting (MTLF) and Long Term Load Forecasting (LTLF). STLF consists of time horizon from a few minutes to hours. Day ahead is considered in STLF. MTLF contains the horizon from one month to one year. LTLF consists of time horizon from one year to several years. Different researchers, used different types of time horizon for forecasting. STLF is mostly used for forecasting, because it gives better accurate prediction results as compared to others. Consumers can also take part in SG operations to reduce the cost of electricity by energy preservation and shifts their consumption load from on-peak hours to off-peak hours. Consumers can utilize energy according to their requirements. To manage supply and demand, both residential customers and industries require electricity price forecasting to cope with upcoming challenges [6]. Robustness, reliability, computational resources, complexity, cost of resources are some issues however, accurate price prediction is also an important issue [7]. When the utilization of electricity is maximum then prices are also high [8]. The price of electricity depends on various factors, such as renewable energy, fuel price and weather conditions etc. [9,10]. 1.1

Motivation

In [11], they performed price forecasting of electricity through Hybrid Structured Deep Neural Network (HSDNN). This model is a combination of CNN and LSTM. In this model, batch normalization is used to increase the efficiency of training data. Authors in [12], proposed a model of Gated Recurrent Unit (GRU) in which LSTM is used as a base model for price forecasting accurately. In paper [13], authors predict load consumption using Back Propagation Neural Networks (BPNNs) model. Authors, used this model to reduce forecasting errors. In [14], authors proposed a combined model of Cuckoo Search, Singular Spectrum Analysis and Support Vector Machine (CS-SSA-SVM) to increase the accuracy of load forecasting. In [15], authors used data mining techniques such as k-mean and KNN algorithm for electricity price forecasting. They used k-mean

A Deep Learning Approach Towards Price Forecasting Using ECNN in SG

273

algorithm to make three clusters for weekdays and also used KNN algorithm to divide the classified data into two patterns for the months of February to March and April to January. After classification, a price forecasting model is developed. The price data of 2014 is used as input and results are verified by 2015 data. 1.2

Problem Statement

We reviewed the related works in electricity price forecasting using deep learning techniques and feature engineering. In [11] this paper proposes an electricity price forecasting system based on the combination of two deep neural networks, the Convolutional Neural Network (CNN) and the Long Short Term Memory (LSTM). However, they neglect the problem of over-fitting. Authors Ziming et al. [16], worked on price forecasting by using the hybrid model of nonlinear regression and SVM. However, the big data is not taken in consideration. Renewable resources, DR, and other factors are influenced on price and load [15,17]. The price of electricity changes frequently, that is why traditional methodologies and approaches are not suitable. We need some enhanced methods for price predictions. 1.3

Contributions

In this paper, main goal is to predict electricity price accurately by using data mining and deep learning techniques. To achieve this, we proposed model for price forecasting. In this work, SVR and CNN both classifiers are used for the prediction of price. Enhanced Convolutional Neural Network (ECNN) is used as a proposed classifier, its results are compared with different benchmark schemes. However, it is very hard to tune the parameters of these models according to dataset. The contributions of this paper are summarized as follows: – Hybrid Feature Selector: Hybrid feature selector is proposed in this paper. – Overfitting: Risk of overfitting is mitigated in this model. – Grid Search and Cross Validation are used to tune the parameters of classifiers, by defining the subset of parameters, – Enhance classifiers is used to increase the forecasting accuracy.

2

Related Work

Authors in [11], discussed price forecasting with the proposed model of Hybrid Structured Deep Neural Network (HSDNN) in which the combination of CNN and LSTM is used. The accuracy of this model is compared by performance evaluators i.e., MAE and RMSE with different models. In [12], authors described the prediction accuracy with the proposed model of LSTM and RNN named as Gated Recurrent Units (GRU) compared its accuracy with benchmark models: SARIMA, Markov chain and Naive Bayes. Rohit et al.

274

F. Ahmed et al.

In [4], authors discussed the data pre-processing steps. They have worked on how to choose a technique for feature selection and feature extraction. These two phases are very important in data pre-processing. Feature selection and extraction techniques play very important role in forecasting. Pre-processing of data is a first step in every forecasting process. Normalized data provides better results for accuracy in forecasting. Data, which is present in raw form gives poor result in prediction. In this work, a meta learning approach is implemented and recommends the pre-processing technique, which shows better results. In [17], authors proposed a model for price forecasting using Deep Learning approaches i.e. DNN as an extension of traditional MLP, hybrid LSTMDNN structure, hybrid GRU-DNN structure and CNN model. Wang et al. [18], authors proposed a hybrid framework of feature selection, feature extraction and dimensionality reduction by GCA, KPCA and also predict the price of electricity through SVM. In [19], authors used Stacked Denoising Autoencoder (SDA) and DNN models. They also compared different models including SVM, classical Neural Network and multivariate regression. Lago et al. [20], worked on DNN to improve the predictive accuracy of a market, for feature selection. They used Bayesian optimization and functional analysis of variance. Also proposed, another model to perform price prediction of two markets simultaneously. Raviv et al. [21], used multivariate models for prediction hourly price instead of univariate, also mitigate the risk of overfitting by using dimensionality reduction techniques and forecast combination. Javaid et al. [22], proposed a deep-learning based model for the prediction of price, using DNN and LSTM. They worked on the prediction of both price and load. In [23], authors considered a probabilistic model for hourly price prediction. Generalize Extreme Learning Machine (GELM) is used for prediction. They used bootstrapping techniques, to increased the speed of model by reducing computational time. Abedinia et al. [24], focused on feature selection to performed better predictions. These proposed models, based on information theoretic criteria i.e. Mutual Information (MI) and Information Gain (IG) for feature select. Another contribution of this paper is a hybrid filter-wrapper approach. In [25,26], proposed a hybrid algorithm for price and load forecasting. Also worked on new conditional feature selection, Least Square Support Vector Machine (LSSVM) and proposed a new modification for Artificial Bee Colony Optimization and Quasi-Oppositional Artificial Bee Colony (QOABC) algorithm. Keles et al. [27], proposed a method based on ANN. They also used different clustering algorithms to find optimal parameters for ANN. Wang et al. [28], proposed Dynamic Choice Artificial Neural Network (DCANN), this model is used for day-ahead price forecasting. This model is a combination of supervised and unsupervised learning, which deactivates the bad samples and search optimal inputs for a model to learn. In [29], developed a hybrid model based on Neural Network. Authors, in [30], used Multilayer Neural Network (MLNN) for electricity price forecasting.


3

275

Proposed Model

In this paper, a novel price prediction model is proposed. Figure 1 shows the proposed model for price prediction. Proposed model is divided into four modules. The modules of proposed models are: 1. 2. 3. 4.

Feature Selection, Feature Extraction, Grid Search and Cross Validation, Price Prediction using SVR and CNN.

The individual module is further explained in the following subsections.

Fig. 1. Proposed model for price prediction

3.1

Model Overview

The accuracy of prediction is key issue in electricity price forecasting. As discussed earlier, the electricity price depends on various factors, which make training of classifiers difficult. To improve accuracy of price prediction, hybrid feature selector (i.e., DTC and XG-boost) is used to select most relevant features. At first, RFE is used to remove dimensionality and redundancy of data. In order to tune parameters of classifier, GS is used along with cross validation to select best subset of parameters. Finally, selected features and best parameters are used in classifiers to predict electricity price.

276

3.2

F. Ahmed et al.

Feature Extraction Using RFE

RFE is used to select specified number of features from dataset. It removes weakest feature recursively, until the specified number of features is reached. RFE requires number of feature to select, however, it is difficult to decide in advance that how many features are most relevant. To address this issue, cross validation is used with RFE. Cross validation calculates accuracy of different subsets and select the subset with highest accuracy. 3.3

Feature Selection Using XG-Boost and DT

Using XG-boost and DT, importance of all features is calculated with respect to target, i.e., electricity price. These techniques calculate the importance of features in vector form. The components of this vector, represents the importance of every feature in sequence. However, we can drop features which have less importance. The fusion of Xg-boost and DT gives more accurate results. Figure 4 shows the importance of features. To control feature selection, threshold is used. Features having importance greater than or equal to threshold are considered and rest of the features are dropped. Feature selection is performed using Eqs. 1 and 2. (1) F s = Reserve if IXG [i] + IDT [i] ≥ Drop if IXG [i] + IDT [i] <

(2)

Where, IXG [i] represents the feature importance calculate by XG-boost, IDT [i] is the feature importance calculated by DT. is the threshold values for the feature selection and i represent feature. 3.4

Tuning Hyper-parameters and Cross Validation

Tuning classifier is very important to do accurate and efficient forecasting. There is a strong relationship between hyper-parameter and results of classifier. GS is used to the tune parameters of classifier for higher accuracy. For this purpose, we define subset of hyper-parameters for SVM shown in Table 1. Table 1. Subset of parameter for Grid Search Parameter name Parameter value(s) kernel

[‘linear’, ‘rbf’]

C

[3, 4, 5, 10, 15, 20, 30, 50]

gamma

[‘scale’, ‘auto’, 5, 10, 20, 30, 50]

epsilon

[0.2, 0.02, 0.002, 0.0002]


3.5

277

Electricity Price Forecasting

After feature selection and parameter tuning, the processed data and the best subset of parameters are used in SVR and CNN to forecast electricity price. Hourly price data of two months (November and December 2016) are used to train classifier and predicts the price for first week of January 2017. We compared the results of first January 2017 and first week of January 2017 with actual price of electricity of NYISO. The results of SVR are shown in Fig. 5(a) and (b) whereas, results of CNN are shown in Fig. 5(c) and (d), respectively.

4

Simulation and Results

In this section, the simulation results are discussed in details. 4.1

Simulation Environment

For simulation purpose, we implement the proposed models by using the following python libraries i.e. Keras, Tensorflow, Sklearn, numpy and pandas. Models are implemented on a system with Intel core i3, 8 GB RAM and 500 GB storage capacity. Two different datasets are selected for simulation. Lastly, Dataset [31] is used as input in price prediction model, which is taken from New York Independent System Operator (NYISO). However, dataset 2 contains hourly data of price and electricity generation from 2016 to 2017.

Fig. 2. Result of cross validation (RFE)

278

F. Ahmed et al.

(a) 2nd January, 2017

(b) First Week of January, 2017

Fig. 3. Price prediction without parameter tuning.

4.2

Results of Price Prediction Model

The proposed model is shown in Fig. 1. NYISO dataset [31] is taken as input, which contains 9,314 real-world records. However, for the sake of demonstration, 75 days dataset are used to train model. This dataset invariably contain approximately 2000 h record, i.e., from 1st November, 2016 to 15th January, 2017. The whole simulation process is organized as: 1. Feature extraction using RFE 2. Feature selection by combining the attributes importance calculated by XGboost and DT 3. Parameter tuning using cross validation and Grid Search 4. Prediction using SVR and CNN 5. Results and Comparison with real data of January 2017. Feature Extraction: To remove redundancy and dimensionality of data, RFE is used. Although, it is difficult to determine in advance how many features set is required. To resolve this issue, cross validation is used with REF to select optimal number of features. Cross validation tests every combination of features and calculates the accuracy of each subset. The subset of features with the highest accuracy is used for prediction. Figure 2 shows the maximum accuracy score on seven number of features. Feature Selection: Importance of selected features are calculated by both DT and XG-boost. By adding both importance, combined importance is calculated. For selection of features, a threshold value is defined. Features are selected with importance greater than or equal to threshold value. Figure 4 shows the importance of every feature. Some features have very high importance, i.e., TWI Zonal LBMP, RTC Zonal LBMP and Load. TWI Zonal Price Version shows very less importance as compared to others features. Most of the features have importance greater than 0.15 and that is why we set the values of threshold to 0.15. Those features whose values are less than threshold value are dropped.


Fig. 4. Feature importance for price



(c) 2nd January, 2017

(d) First Week of January, 2017

Fig. 5. Price prediction using SVR and CNN.

279

280

F. Ahmed et al.

Parameter Tuning and Cross Validation: To find the optimal set of parameters for the classifiers, we use the defined a set of parameters as shown in Table 1. Using GS, every possible combination of parameters are tested by the propose model to find optimal combination of parameters. Price Prediction: Hourly data of November and December 2016 is used to the train classifier. SVR and ECNN are used to predict price of electricity for first week of January. To verify the accuracy of model, predicted price is compared with the actual price of first week of January. The results are shown in Fig. 5(a), (b), (c) and (d). These figures show both actual and predicted price for first day and first week of January, 2017. Figure 5(a) and (b) shows the prediction of classifier SVR and Fig. 5(c) and (d) reports the result of CNN classifier. Discussion of Results: As we know, the main goal of this proposed model is to improve the accuracy of classifier to predict price correctly. The results before parameter tuning of classifier are shown in Fig. 3(a) and (b) are less accurate. The MAE of prediction before parameter tuning is approximately equal to 2.83. After feature selection, extraction and parameter tuning through GS, the results are improved. The results after parameter tuning of SVR is shown in Fig. 5(a) and (b). The results of CNN is shown in Fig. 5(c) and (d). After parameter tuning, the accuracy of classifiers are improved, the MAE is reduced to 1.81 The comparison of actual values with before and after tuning classifiers are shown in Fig. 6(a) and (b). The value of MAE before tuning is 2.83 and after parameter tuning the value is 1.81. MAE value is decreased then it shows that results are improved after parameter tuning. Reducing 1% error values of MAE can save thousands of MW of electricity.



Fig. 6. Comparison of predictions before and after parameter tunning.


5

281

Conclusion

In this research study, a new model is established to predict the price of electricity efficiently and accurately. Proposed model is consist of feature selection, feature extraction, parameter tuning and classification. Hybrid feature selector (hybrid of DT and XG-boost) is used to select important features for prediction. For dimensionality reduction and feature extraction, RFE is used. In order to tune the parameters of classifiers, grid search is used, which boost the classifier’s accuracy. Enhanced classifiers like CNN and SVR are used to predict price and load is proposed models for better accuracy. The results of classifiers is satisfactory and show better accuracy than benchmark scheme.

References 1. Masip-Bruin, X., Marin-Tordera, E., Jukan, A., Ren, G.-J.: Managing resources continuity from the edge to the cloud: architecture and performance. Future Gener. Comput. Syst. 79, 777–785 (2018) 2. Osman, A.M.S.: A novel big data analytics framework for smart cities. Future Gener. Comput. Syst. (2018) 3. Chen, X., Zhou, Y., Wei, D., Tang, J., Guo, Y.: Design of intelligent Demand Side Management system respond to varieties of factors. In: 2010 China International Conference on Electricity Distribution (CICED), pp. 1–5. IEEE (2010) 4. Jindal, A., Singh, M., Kumar, N.: Consumption-aware data analytical demand response scheme for peak load reduction in smart grid. IEEE Trans. Industr. Electron. (2018) 5. Bilalli, B., Abell, A., Aluja-Banet, T., Wrembel, R.: Intelligent assistance for data pre-processing. Comput. Stand. Interfaces 57, 101–109 (2018) 6. Mohsenian-Rad, A.-H., Leon-Garcia, A.: Optimal residential load control with price prediction in real-time electricity pricing environments. IEEE Trans. Smart Grid 1(2), 120–133 (2010) 7. Chandrashekar, G., Sahin, F.: A survey on feature selection methods. Comput. Electr. Eng. 40(1), 16–28 (2017) 8. Erol-Kantarci, M., Mouftah, H.T.: Energy-efficient information and communication infrastructures in the smart grid: a survey on interactions and open issues. IEEE Commun. Surv. Tutorials 17(1), 179–197 (2015) 9. Wang, K., Li, H., Feng, Y., Tian, G.: Big data analytics for system stability evaluation strategy in the energy Internet. IEEE Trans. Industr. Inf. 13(4), 1969–1978 (2017) 10. Wang, K., Ouyang, Z., Krishnan, R., Shu, L., He, L.: A game theory-based energy management system using price elasticity for smart grids. IEEE Trans. Industr. Inf. 11(6), 1607–1616 (2015) 11. Kuo, P.-H., Huang, C.-J.: An electricity price forecasting model by hybrid structured deep neural networks. Sustainability 10(4), 1280 (2018) 12. Ugurlu, U., Oksuz, I., Tas, O.: Electricity price forecasting using recurrent neural networks. Energies 11(5), 1–23 (2018)

282

F. Ahmed et al.

13. Eapen, R.R., Simon, S.P.: Performance analysis of combined similar day and day ahead short term electrical load forecasting using sequential hybrid neural networks. IETE J. Res. 1–11 (2018) 14. Chitsaz, H., Zamani-Dehkordi, P., Zareipour, H., Parikh, P.: Electricity price forecasting for operational scheduling of behind-the-meter storage systems. IEEE Trans. Smart Grid (2017) 15. Patil, M., Deshmukh, S.R., Agrawal, R.: Electric power price forecasting using data mining techniques. In: 2017 International Conference on Data Management, Analytics and Innovation (ICDMAI), pp. 217–223. IEEE (2017) 16. Ziming, M.A., Zhong, H., Xie, L., Xia, Q., Kang, C.: Month ahead average daily electricity price profile forecasting based on a hybrid nonlinear regression and SVM model: an ERCOT case study. J. Mod. Power Syst. Clean Energy 6(2), 281–291 (2018) 17. Lago, J., De Ridder, F., De Schutter, B.: Forecasting spot electricity prices: deep learning approaches and empirical comparison of traditional algorithms. Appl. Energy 221, 386–405 (2018) 18. Wang, K., Xu, C., Zhang, Y., Guo, S., Zomaya, A.: Robust big data analytics for electricity price forecasting in the smart grid. IEEE Trans. Big Data (2017) 19. Wang, L., Zhang, Z., Chen, J.: Short-term electricity price forecasting with stacked denoising autoencoders. IEEE Trans. Power Syst. 32(4), 2673–2681 (2017) 20. Lago, J., De Ridder, F., Vrancx, P., De Schutter, B.: Forecasting day-ahead electricity prices in Europe: the importance of considering market integration. Appl. Energy 211, 890–903 (2018) 21. Raviv, E., Bouwman, K.E., van Dijk, D.: Forecasting day-ahead electricity prices: utilizing hourly prices. Energy Econ. 50, 227–239 (2015) 22. Mujeeb, S., Javaid, N., Akbar, M., Khalid, R., Nazeer, O., Khan, M.: Big data analytics for price and load forecasting in smart grids. In: International Conference on Broadband and Wireless Computing, Communication and Applications, pp. 77– 87. Springer, Cham (2018) 23. Rafiei, M., Niknam, T., Khooban, M.-H.: Probabilistic forecasting of hourly electricity price by generalization of ELM for usage in improved wavelet neural network. IEEE Trans. Industr. Inf. 13(1), 71–79 (2017) 24. Abedinia, O., Amjady, N., Zareipour, H.: A new feature selection technique for load and price forecast of electrical power systems. IEEE Trans. Power Syst. 32(1), 62– 74 (2017) 25. Ghasemi, A., Shayeghi, H., Moradzadeh, M., Nooshyar, M.: A novel hybrid algorithm for electricity price and load forecasting in smart grids with demand-side management. Appl. Energy 177, 40–59 (2016) 26. Shayeghi, H., Ghasemi, A., Moradzadeh, M., Nooshyar, M.: Simultaneous dayahead forecasting of electricity price and load in smart grids. Energy Convers. Manage. 95, 371–384 (2015) 27. Keles, D., Scelle, J., Paraschiv, F., Fichtner, W.: Extended forecast methods for day-ahead electricity spot prices applying artificial neural networks. Appl. Energy 162, 218–230 (2016) 28. Wang, J., Liu, F., Song, Y., Zhao, J.: A novel model: Dynamic Choice Artificial Neural Network (DCANN) for an electricity price forecasting system. Appl. Soft Comput. 48, 281–297 (2016)


283

29. Varshney, H., Sharma, A., Kumar, R.: A hybrid approach to price forecasting incorporating exogenous variables for a day ahead electricity market. In: IEEE International Conference on Power Electronics, Intelligent Control and Energy Systems (ICPEICES), pp. 1–6. IEEE (2016) 30. Mosbah, H., El-Hawary, M.: Hourly electricity price forecasting for the next month using multilayer neural network. Can. J. Electr. Comput. Eng. 39(4), 283–291 (2016) 31. NYISO Hourly SMD from 2011 to 2018. https://drive.google.com/open?id=1XOON8Jtmpsntw7GCeenBNFvw AZCMlj