Development of Improved Artificial Neural Network ...

Pratap Kishore Padhiary et al. / International Journal of Engineering Science and Technology (IJEST)

Development of Improved Artificial Neural Network Model for Stock Market Prediction PRATAP KISHORE PADHIARY1 M.Tech (IT) Scholar, S’O’A University, Bhubaneswar, India.

AMBIKA PRASAD MISHRA2 Asst. Professor, Dept. of CSE, S’O’A University, Bhubaneswar, India.

Abstract: In recent year’s prediction of stock market returns is a hottest field of research in finance. Artificial Neural Network (ANN) is a technique that is heavily researched and widely used in applications for engineering and scientific fields for various purposes ranging from control systems to artificial intelligence. This paper surveys key issues in financial forecasting and propose an ANN methodology which could be better for long term (one month, two month) as well as short term (one day) prediction of stock price of any leading stock market indices. Survey of existing literature reveals that adaptive learning rate will give more accurate result than fixed learning rate parameter for ANN models. Many researchers noted that slight parameter changed causes major variations in the behavior of the network. So there is no theory which could be guideline for finding best network topology. The proposed trigonometric functional link artificial neural network (FLANN) model employs standard least mean square (LMS) algorithm with search-then-converge scheduling which could effectively calculate learning rate parameter that changes with time and may require less experiments to train the model. The objective of this paper is to introduce a functional link single layer artificial neural network (FLANN) for long term as well as short term stock market prediction. Keywords: Neural Network, FLANN, stock prediction, Learning rate, LMS, search-then-converge 1. Introduction Stock price prediction is an important field of research in finance because if the market is successfully predicted then the investors may get maximum returns. The stock market or equity market is a public market where a large amount of capital are invested and traded in everyday all over the world. Many researchers claim that the market is dynamic, non-linear, complicated and chaotic in nature. So it is difficult to deal with normal analytical methods like time series analysis. These chaos systems are sensitive to initial conditions. So the neural networks are effective to deal with such a non-linear system. However financial time-series are difficult to forecast because these are noisiest and non-stationary signals [Oh and Kim, (2002)]. Some common financial time-series are currency exchange rates, interest rates, stock prices etc. A number of researchers have given their view on Efficient Market Hypothesis (EMH) [Lowe and Webb, (1991)]. EMH states that the market is efficient so it cannot be predicted because when new information arises, the market corrects itself and absorbs it [Malkiel, (1999)]. There is no such information to predict the market in such a way that the investors earn greater profits from stock market. In the recent years, many researchers claimed that the EMH must be false. From the last years many researches are on this field, still it remains a big task whether the market can be correctly predicted or not. To predict the stock market accurately, various prediction algorithms and models have been proposed by many researchers in both academics and industry. The organization of our work is as follows. Section 2 reveals the various issues and techniques in the context of stock market prediction by making a survey on existing literatures. Section 3 focuses on basic concepts on Neural Network. Section 4 briefly explains the FLANN Model. Section 5 illustrates FLANN Model training by gradient descent method. Section 6 gives our approach towards improved FLANN Model. Section 7 provides a comparison of learning rate schedules. We conclude this paper in Section 8.

ISSN : 0975-5462

Vol. 3 No. 2 Feb 2011

1576


2. Literature Survey The stock market is volatile because of many unseen factors that can influence the share price. There are two types of factors: Qualitative and Quantitative. Qualitative factors include political events, international ‘events’ [Ng and Fu, (2003)], firms’ policies etc. and Quantitative factors such as open rate, close rate, high rate, and low rate for individual equities. A stock market prediction system was designed by [Kohara et al., (1997)] using the prior knowledge and event-knowledge. They incorporated prior-knowledge in stock prediction such as newspaper information on domestic and foreign events. Event-knowledge is extracted from the news paper headlines in accordance with certain prior-knowledge. Prior knowledge is the information that stems from previous experience. Thus, based on the prior-knowledge, decisions can be made whether a particular event can positively influence the stock market tendencies or not. [Hong and Han, (2004)] introduced an automated system (KBN Miner) that extracts event-knowledge from the Internet for the prediction of interest rates. The KBN Miner which is based on a prior-knowledge collect the event information from the Internet automatically and helps in decision making whether a particular event can positively affect the stock market tendencies or not and then to apply the information to a neural network model for interest rate prediction. Web mining technique that they applied for predicting interest rates can also be applied for stock market prediction. In finance, technical analysis is a method of security analysis which forecasts general price direction by analyzing market activity such as past price and volume. In this process technical analyst use technical indicators and chart patterns from historical data.Yao,Tan and Poh [Yao et al.,(1999)] suggest that technical analysis is not appropriate for market prediction as it is highly volatile and the use of technical indicators would be widely adopted by traders. Most common indicators are such as moving average, relative strength index, stochastic oscillator etc. For such reasons researchers have stressed on developing models for accurate prediction based on various statistical and soft computing techniques. Auto-regressive integrated moving average (ARIMA) based model [Schumann and Lohrbach, (1993)] is one of the most important and widely used statistical technique employed in this regard. The ARIMA model is obtained by differentiating an assumed non-stationary process to obtain a locally wide sense stationary (wss) and locally ergodic process. It applies the Box–Jenkins methodology [Box and Jenkins, (1970)] in the model building process. The ARIMA model is an efficient approach but with short comings like correlation analysis. Recent advances in soft computing led to a new era in the field of financial forecasting. In the most recent times the soft computing tools based on such as multilayer artificial neural networks (ANN) [Kingdon, (1997); Refenes, (1995); Ziurilli, (1997)], Fuzzy logic (FL) [Ju et al., (1997)], Genetic Algorithm (GA) [Bhattacharya and Meheta,(1998)], genetic programming (GP) [Neely,(1997)] and hybrid tools [Hassan,(2009); Hassan et al.,(2007); Kim,(2006);Versace et al.,(2004)] have been applied to financial forecasting. Learning algorithms such as support vector machine has been employed [Huang et al., (2005)] for forecasting stock market movement direction. NNs with GA have been used to predict the Singapore Stock Exchange Index and achieved accuracy rate of 81% [Phua et al., (2000)].Kim and Han [Kim and Han, (2000)] also combined NNs with GA and predicted Korea Composite Stock Price Index 200. He achieved 82% of accuracy in predicting both weekly rising and declining stock market tendencies. In some cases, several economic indicators such as interest rate, price of crude oil, and New York Dow Jones average of the closing price are selected and fed them together with event-knowledge into NNs. Their experimental results showed incorporation of event knowledge improved the prediction ability of NNs by reducing the error rate on the 5% level of significance. A study has been made to investigate appropriate selection and effects of various network parameters in the design of back propagation neural network model [Tan Clarence and Wittig, (1993)].The parameters they studied are learning rate, momentum, input noise, number of hidden layers and activation function. Tools based on ANN have increasingly gained popularity due to their inherent capabilities to approximate any nonlinear function to a high degree of accuracy. Neural networks are less sensitive to error term assumptions and they can tolerate noise and chaotic components [Masters, (1993)]. The most commonly used neural network technique in pattern recognition is MLP and classification problems. Multi layer perceptron (MLP) architecture using back propagation (BP) algorithm has gone into the field of application of ANN to stock price prediction. Probabilistic neural network (PNN) has also been employed to stock prediction [Saad et al., (1998); Tan et al., (1995)]. In this work, the model is used to draw up a conservative thirty day stock price prediction of a specific stock: Apple Computers Inc. Due to their bulky nature owing to the large training data, the PNN is not popular among forecasters. Ornes & Sklansky [Ornes and Sklansky, (1997)] in their paper present a visual neural network (VNN), which combines the ability of multi expert networks to give low prediction error rates with visual explanatory power of nonlinear dimensionality reduction. The multi branch neural network (MBNN) proposed by [Yamashita T et al.,(2005)] which has been applied to the TOPIX (Tokyo Stock Exchange) shows that it has higher accuracy than conventional NNs. The hybrid architectures are also being deployed in recent times. Lee [Lee, (2004)] has proposed a hybrid radial basis function recurrent network (HRBFN) stock prediction system called the iJADE stock advisor. All these models involve large computational complexity as well as more prediction and testing time. Garliauskas

ISSN : 0975-5462

Vol. 3 No. 2 Feb 2011

1577


[Garliauskas, (1999)] concluded that in predicting financial time series, NNs have better performance than classical statistical methods. The present paper of our interest is to develop a low complexity and accurate prediction model which is better suited for long term prediction. The objective is to introduce a functional link single layer artificial neural network (FLANN) for developing efficient stock market prediction model using LMS learning rule. 3. Neural Network A neural network is a massively parallel distributed processor made up of simple processing unit which has a natural propensity for storing experiential knowledge and making it available for use [Haykin]. Each connection between units has an associated real-valued weight which simulates the efficacy of biological synapses in the brain. Given a set of labeled input-output pairs, these models accomplish the learning process by adaptively adjusting their parameters (weights) in such a way as to perform a given task. Neural networks have remarkable ability to derive meaning from complicated or imprecise data. They are used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques. The ability of neural networks to discover nonlinear relationships [Phillip and Nostrand, (1989)] in input data makes them ideal for modeling nonlinear dynamic systems such as the stock market. They have remarkable ability to derive meaning from complicated or imprecise data can be used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques. A neural network method can enhance an investor's forecasting ability [Youngohc and George, (1991)]. Neural networks are also gaining popularity in forecasting market variables [Hamid, (2004)].Neural network have good generalization capability and usually robust against noisy or missing data, all of which are highly desirable properties for time series prediction. The limitation of this approach is that such models are black box in nature that means models do not capture the cause of the movements of stock prices in the market. Another serious problem with NNs is the overfitting problem [Haykin]. It occurs when the network has too many free parameters which allow the network to fit well the training data but typically lead to poor generalization. Overfitting occurs because of two main reason, first is if the network have too many nodes and the second is if the network trained more than necessary. Moreover, NNs have some limitations in learning the patterns when input data have high dimensionality. Dash and Liu [Dash and Liu,(1997)] put the emphasis on the feature selection and suggested that reducing the number of input variables sometimes lead to improved model performance for a given data set. The reduction and transformation of the irrelevant or redundant features may shorten the running time and yield more generalized results. 4. FLANN Model The FLANN architecture is used for predicting the stock prices of leading stock indices. The FLANN [Pao, (1989)] is a single layered neural network with nonlinear input and a single neuron at the output. The need of hidden layers has been removed by incorporating functional expansion of the input pattern. Removal of hidden layer makes these networks very simple and computationally cheap. The functional expansion effectively increases the dimensionality of the input vector, and hence the hyper planes generated by the FLANN provide greater discrimination capability in the input pattern space. The computational complexity is also reduced and the neural net becomes suitable for on-line applications. Further, it reaches its global minima very easily. To bridge the gap between the linearity in the single layer neural network and the highly complex and computation intensive multi layer neural network, the FLANN architecture is suggested. This network is a useful substitute of multilayer artificial neural network (MLANN) [Lippman, (1987)]. However, it is structurally simple and involves less computation compared to those of MLANN. It is also reported that for some applications [Patra et al., (1999)] the FLANN performs better than the MLANN. The nonlinear input is generated by functionally expanding the input vector in a nonlinear manner. Different nonlinear expansions may be employed. These are trigonometric (sine and cosine), Chebyshev and power series. With proper choice of functional expansion, the FLANN can perform as good as in some cases, even better than MLP. In this paper the trigonometric expansion based financial model will be developed for exchange rate prediction as it offers better performance compared to when other expansions are used. The motivation behind using trigonometric function lies in Fourier series expansion of the time series. The trigonometric basis functions which are { cos(PI*X),sin(PI*X),cos(2*PI*X),sin(2*PI*X),….,cos(N*PI*X),sin(N*PI*X)} provide a compact representation of the function in the mean square sense. The degree of freedom, i.e., the number of basis functions needed in case of polynomial basis function is more than that of trigonometric basis function within a specified accuracy. When suitable trigonometric polynomials are used after training, the FLANN weights represent a multidimensional Fourier series decomposition of a periodic version of the desired response function. There have been several applications of FLANN including pattern classification and recognition, system identification and control,

ISSN : 0975-5462

Vol. 3 No. 2 Feb 2011

1578


functional approximation, and digital communications channel equalization. The proposed model consists of three basic processes. 4.1. The functional expansion (FE) process Here input elements are nonlinearly expanded to generate more number of inputs. The Process should be nonlinear and should involve simple computations. This preliminary introduction of nonlinearity reduces the number of layers and computational complexity. 4.2. The estimation process It include two steps (i) computing the output of the adaptive model in response to nonlinear input elements (ii) comparing the estimated financial output of the model with the corresponding desired or target result to generate error signal. 4.3. Adaptive process Here the model updates the connecting weights by means of some adaptive learning rule. The combination of these three processes together constitutes the proposed financial model as shown in Fig. 1.

Fig. 1. Adaptive FLANN Model

5. FLANN Model Training by Gradient Descent Method Steepest descent method is a statistical optimization technique for minimizing a function of several variables. Successive adjustments of the weights are in the direction of steepest descent, that is, in the opposite direction of gradient vector (w). For convenience of presentation we write g=

(w)

By differentiating cost function with respect to weight vector. We get g(k) = –e(k)x(k) Where, g(k) = gradient vector at kth nexperiment e(k) = error at kth experiment x(k) = input vector at kth experiment The steepest descent algorithm is described by w (k+1) = w (k) – µg (k) Updating the weights using steepest descent method as w (k+1) = w(k) + µe(k)x(k) Where, µ=learning rate parameter between 0 and 1. w (k+1) = updated value of weight vector at k+1 experiment. w (k) = old value of weight vector at kth experiment. e (k) = error at kth experiment. x (k) = input vector at kth experiment.

(1)

(2) (3)

6. Proposed Approach towards Improved FLANN Model Learning rate parameter plays a key role in determining the performance of error-correction learning. So selection of learning rate parameter is important to ensure the stability or convergence of the iterative learning process is achieved. The review of the existing literature reveals that learning rate parameter may not be fixed. One major problem with fixed learning rate is accuracy of result. More accurate result may be obtained by

ISSN : 0975-5462

Vol. 3 No. 2 Feb 2011

1579


implementing adaptive learning using search-then-converge rule Darken and Moody [Darken and Moody, (1990b)]. The adaptive learning rate provides better result in terms of accuracy, speed of convergence and stability. Here learning rate parameter change with time. Learning rate parameter is large at the beginning of training and gradually decreases as the network converge. The rule consists of two phases. The first phase is searching phase where learning rate parameter is large and almost constant and second phase is the converging phase where learning rate parameter decrease exponentially to zero. Then the parameters used in the FLANN model can be computed and updated using LMS algorithm [Fredric and Ivica] which comprises of following steps: 1. Set k=1, initialize the synaptic weight vector w (k=1) & select values for µ0 and τ. where 2. Compute Where µ0 and τ are user selected constants and 100≤ τ ≥ 500 3. Compute the error

4. Update the synaptic weights as following wi(k+1) = wi(k) + µ(k) e(k) xi(k),

for i=1,2,……n.

5. If convergence is achieved, stop; else set k←k+1, then go to step 2. 7. Comparison of Learning Rate Schedules Fig 2 shows a comparison of the stochastic approximation and search-then-converge schedule for the LMS Learning rate parameter. In standard LMS algorithm the learning rate parameter is constant for all simulation.So it requires more number of training to converge and in stochastic approximation schedule the learning rate decreases rapidly. In search- then-converge, the learning parameter is large at the beginning of the training and decreases gradually as the network converges. So we go for Search-then-Converge schedule.

Fig.

2. Learning rate schedule comparison

8. Conclusion & Future Work FLANN based stock market prediction model is an effective approach both computationally as well as performance wise to foresee the market levels both in short and medium terms future. Numerous approaches have been proposed for successful stock prediction. But still it is a challenging task for the researchers to mitigate the error. In this paper, we have proposed a FLANN based stock market prediction model employing the LMS algorithm which uses search-then-converge rule for learning rate parameter updation. This FLANN model may predict the stock values more accurately than other models. In our future work we will be coming up with a developed forecasting model which involve less computational load, fast forecasting capability and which requires less experiments to train the model and will focus on long term prediction of stock price of stock market. It is expected that the result which will be obtained by exhaustive simulation study of the proposed method will provide better performance than which have been reported earlier in various literatures.

ISSN : 0975-5462

Vol. 3 No. 2 Feb 2011

1580


References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41]

Bhattacharya, S., & Meheta, K. (1998). Staged learning of trading rules using genetic algorithm. In Proceedings of 3rd INFORMS conference on info. sys. and tech (pp. 62–66) Montreal, April. Box, G.E.P., Jenkins, G.M. 1970. Time Series Analysis, Forecasting and Control, Holden-Day, San Francisco,CA, Darken C. and Moody, J. (1990b) Note on learning rate schedules for stochastic optimization. Advances in Neural Information Processing Systems 3, Morgan Kauffman, San Mateo, California. 832-838 Dash, M. and Liu, H. 1997, ‘Feature selection for classifications’, Intelligent Data Analysis: An International Journal, vol. 1, pp. 131156. Fredric M. Ham, Ivica Kostanic, Principle of Neurocomputing for Science and Engineering, 1st Edition, Tata-McGrawhill Edition Garliauskas, A. 1999, ‘Neural Network Chaos and Compuational Algorithm of Forecast in Finance’, Proceedings of the IEEE SMC Conference on Systems, Man, and Cybernetics 2, pp. 638-643, 12-15 October. Hong, T. and Han, I. 2004, ‘Integrated approach of cognitive maps and neural networks using qualitative information on the World Wide Web: KBN Miner’, Expert Systems, vol. 21 no.5, pp. 243-252. Haykin, Simon, Neural Networks, A Comprehensive Foundation, 2nd Edition, Prentice Hall International. Hassan, Md., (2009). A combination of hidden Markov model and fuzzy model for stock market forecasting. Neurocomputing Elsevier, 72, 3439–3446 Hassan, Md., Nath, B., & Kirley, M. (2007). A fusion model of HMM, ANN and GA for stock market forecasting. Expert System with Applications, Elsevier, 33(1), 171–180, July. Huang, W., Nakamori, Y., & Wang, S. (2005). Forecasting stock market movement direction with support vector machine. Computer and Operation Research, Elsevier, 32, 2513–2522. Ju, Y., Kim, C., & Shim, J. C. (1997). Genetic–based fuzzy models; interest rates Forecasting problem. Computer and Industrial Engineering, 33, 561–564. Kohara, K. Ishikawa, T. Fukuhara, Y. and Nakamura, Y. 1997, ‘Stock Price Prediction Using Prior Knowledge and Neural Networks’, Intelligent System In Accounting, Finance and Management, vol. 6, pp. 11-22 vol.16. Kingdon, J. (1997). Intelligent systems and financial forecasting. Berlin: Springer-Verlag. Kim, K. (2006). Artificial neural networks with evolutionary instance selection for financial forecasting. Expert System with Applications, Elsevier, 30, 519–526. Kim, K. and Han, I. 2000, ‘Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index’, Expert System Appliance, vol. 19. Lowe, D and Webb, A. R. 1991, ‘Time series prediction by adaptive networks: A dynamical systems perspective. IEEE Computer Society Press. Lee, R. S. T. (2004). IJADE stock advisor: An intelligent agent based stock prediction system using hybrid RBF recurrent network. IEEE Transactions on Systems, Man and Cybernetics, Part A, 34(3), 421–428. Lippman, R. P. (1987). An introduction to computing with neural nets.IEEE ASSP Magazine, 4, 4–22. Masters, T. (1993). Practical neural network recipes in C++. New York: Academic Press. Malkiel B.G, “A Random Walk Down Wall Street”, W. W. Norton &Company, New York, London, 1999. Ng, A. and Fu, A.W. 2003, ‘Mining Frequent Episodes for Relating Financial Events and Stock Trends’, Lecture Notes in Computer Science, vol. 2637, pp. 27-39. Neely, C. (1997). Is technical analysis in the foreign exchange market profitable: A genetic programming approach. Journal of Financial and Quantitative Analysis, 32(4). Oh, K. J., & Kim, K-J (2002). Analyzing stock market tick data using piecewise non linear model. Expert System with Applications, 22, 249–255. Ornes C. & Sklansky J. (1997). A neural network that explains as well as predicts financial market behavior. In Proceedings of computational intelligence for financial engineering, the IEEE/IAFE 1997 (pp. 43–49). Masters, T. (1993). Practical neural network recipes in C++. New York: Academic Press. Phillip D. Wasserman, Van Nostrand "Neural Computing: Theory and Practice", Van Nostrand Reinhold, New York, 1989 Pao, Y. H. (1989). Adaptive pattern recognition & neural networks. Reading, MA: Addison-Wesley. Patra, J. C., Pal, R. N., Chatterji, B. N., & Panda, G. (1999). Identification of nonlinear dynamic systems using functional link artificial neural networks. IEEE Transactions on systems, man and cybernetics-Part B: Cybernetics, 29(2), 254–262. Phua, P. K. H. Ming, D. and Lin, W. 2000, ‘Neural Network with Genetic Algorithms for Stocks Prediction’, Fifth Conference of the Association of Asian-Pacific Operations Research Societies, 5th - 7th July, Singapore. Refenes, A. (1995). Neural network in financial engineering. In Proceedings of fourth international conference neural networks in the capital market (NNCM-95) (pp.Singapore). World Scientific: WorldScientific. Saad, E. W., Prokhorov, D. V., & Wunsch, D. C. (1998). Comparative study of stock trend prediction using time delay, recurrent and probabilistic neural networks. IEEE Transactions of Neural Network, 9(6), 1456–1470. Shaikh A. Hamid. Primer on using neural networks for forecasting market variables. In proceedings of the a conference at school of business, southern new hampshire university, 2004. Tan Clarence, N. W., & Wittig Gerhard, E. (1993). A Study of the parameters of a backpropagation stock price prediction model. In Proceedings of first New Zealand international two-stream conference on artificial neural networks and expert systems (pp. 288–291). Tan H., Prokhorov D. V. & Wunsch D. C., II (1995). Conservative thirty calendar day stock prediction using a probabilistic neural network. In Proceedings of computational intelligence for financial engineering, the IEEE/IAFE 1995 (pp.113–117). Versace, M. et al. (2004). Predicting the exchange traded fund DIA with a combination of genetic algorithms and neural networks. Expert Systems with Applications, Elsevier, 27, 417–425. Wang, Y. (2003). Mining stock prices using fuzzy rough set system. Expert System with Applications, 24, 13–23. Yao, J., Tan C.J., Poh H.L., Neural networks for technical analysis: a study on KLCI, International Journal of Theoretical and Applied Finance 2 (2) (1999). Yamashita T., Hirasawa K. & Hu J. (2005). Application of multi-branch neural networks to stock market prediction. In Proceedings of IEEE international joint conference on neural networks (IJCNN ‘05) (Vol. 4, pp. 2544–2548). Youngohc yoon and George swales. “Predicting stock price performance: a neural network approach”. IEEE publishing, 1991. Ziurilli, J. S. (1997). Financial prediction using neural networks. London: International Thompson Computer Press.

ISSN : 0975-5462

Vol. 3 No. 2 Feb 2011

1581