Submitted for Publication in {Applied Artificial Intelligence}. Prepared with the T&F Journal Template.
Application of Neural Networks to Stock Prediction in “Pool” Companies
Oscar Sapena Departamento de Sistemas Informáticos y Computación, Universidad Politécnica de Valencia Vicente Botti Departamento de Sistemas Informáticos y Computación, Universidad Politécnica de Valencia
Estefanía Argente Departamento de Sistemas Informáticos y Computación, Universidad Politécnica de Valencia
Address Correspondence to: Oscar Sapena, Departamento de Sistemas Informáticos y Computación, Universidad Politécnica de Valencia, Camino de Vera s/n 46017, Valencia, España. E-mail:
[email protected]
1
Abstract. Nowadays, carrying out precise stock predictions is essential for many companies in order to reduce material in storage. It allows these companies to decrease their investment in material. In the particular case of “Pool” companies, this need becomes greater since they have to maintain the quality of service as well as to minimize the investment in fixed assets. In order to make an accurate prediction, we have used artificial intelligence techniques, specifically artificial neural networks.* Keywords: prediction, stock, neural networks. Introduction Nowadays, many companies require the use of predictions to adjust material stocks stored in their warehouses. “Pool” companies rent certain kinds of containers to groups of clients. These companies are made up of several distribution warehouses which are located in different locations. Each warehouse serves a known set of local clients, who always rent different containers from this warehouse. There are other clients, who can get the containers from other warehouses depending on their location. The route of the rented containers starts at one of the company’s distribution warehouses. There, clients load their trucks with these containers when needed. It is important for clients to find its containers directly at the warehouse, because their immediate availability adds value to the quality of service. After using the containers, the clients return them to the distribution warehouse where they are prepared for reuse. In our work, the “Pool” company carries out its inventory calculations at the end of every week. The current stock of each article in each warehouse is known. The entries of each type of article during the week are also known. The unknown information is the number of each article the clients take from each warehouse. This knowledge would allow the company to adjust the amount of stock, which it should store in each warehouse. It is extremely important to minimize the amount of stock: having the articles immobile is prejudicial to the company, because its principal source of income is the constant rotation of these articles. The predicted stock remaining may be distributed among non-local clients, requiring them to load their trucks at certain warehouses where there is a surplus. If necessary, it is possible to carry out transfers of containers among warehouses in order to balance their stocks.
*
This work has been partially supported by the grant number CICYT-FEDER-0887 of the Spanish Government.
2
This paper shows how to solve the stock prediction problem by means of neural networks. Artificial neural networks are currently being applied to a wide range of scientific and technological areas. The practicability of using neural networks for economic forecasting has already been demonstrated in a variety of problems such as stock market (Thawornwong 2001), currency exchange rate prediction (Yao 2000), market analysis (Poh 1997) and forecasting time series of political economy (Castiglione 2001). Neural networks offer some advantages against other prediction methods. Classical statistical methods (Weigend 1994) are a good alternative, but it is necessary to choose a statistical model for each problem (which forces to make some assumptions about underlying trends). This paper illustrates the applicability of neural networks in a “Pool” company, and it is structured as follows: in section 2, the “Pool” company concept is described in detail, showing the way these companies manage their stocks and what problems arise. Section 3 defines the goals of this work. Section 4 describes the kinds and structures of neural networks which can be used to solve the problem. Section 5 presents the results obtained. The last sections include conclusions and the bibliography. Problem description Transportation costs have been analysed and studied by many types of companies for a long time. This is due to the repercussion these costs have on the final price of the product. The importance of the handling and transport of goods is the reason for the logistics department in every organization. Improvements in stock processing and transportation have resulted in the “Transportation Unit” (TU) concept. Standardization has progressed thanks to the study and analysis of TUs, and the handling and transportation costs have been reduced. As a result, during the 80’s studies were made in order to optimise the loading area of trucks, by using palettes with measurements divisible by the truck box size. Later, a European Law defined alternative uses for the containers, once they have been utilized. Therefore, containers can be: •
Reusable: containers are designed to execute a minimum number of cycles, rotations or uses during their life cycle.
•
Recycled: transformation of container wastes within the production process, but not to generate energy.
•
Increased in value: allows for the exploitation of the resources contained in the container wastes.
•
Energetically recoverable: the use of combustible container wastes to generate energy. 3
•
Disposable: all procedures aimed at the storage or destruction of the container wastes through incineration or other methods which do not involve energy recovery.
“Pool” companies of plastic containers have arisen from the analysis of the previous concepts. “Pool” Company “Pool” companies provide a rental service of containers (CRSs) or transportation units (TUs) to the different suppliers of supermarket distributors. The following are some of the characteristics of these “Pool” companies: •
“Pool” company is the owner of all the CRSs. It rents them to be used by producers/processing companies, charging a rental on the use and a deposit as a return guarantee.
•
“Pool” company controls the surpluses in all the operational centres (i.e. service and regulating warehouses, producers/processing warehouses and distributors). These surpluses originate from the deliveries and collections of CRSs, and are derived from the seasonal nature of the different products utilized during the course of the year. As a part of its services, the “Pool” company maintains and cleans the containers, guaranteeing the different operators’ needs.
•
The warehouses of the company must be well communicated and managed, since it is essential to provide an immediate response to the demand for CRSs of the different zonal and inter-zonal producers/processing companies.
•
Operating system or physical flow of CRSs.
The operating system or physical flow of the containers is described in Figure 1.
Applicable management models in a “Pool” company A “Pool” company must direct its efforts toward the greatest efficiency in its stock management and transportation logistic models in order to render its services adequately. Improvements in stock management processes will generate greater turnovers and less stock durations in every operational centre of the “Pool”. Improvements in logistics models will help determine the best service level for all the operators that take part in the “Pool”. Then, a lower number of CRSs will be necessary to render the same service level. All these improvements will increase the effectiveness and the competitive level of the company. Information system model The company we are working with has the information scheme shown in Figure 2. Each warehouse sends information to the system about the number of containers that each client has rented. Clients inform the “Pool” 4
company about the number of products they have delivered to the distributor. All this remote information is checked and contrasted between the warehouse and the client’s management modules in order to be correctly introduced into the central database. The administration module is in charge of the automatic invoicing and the company’s accounting. One of the most important modules is the stock management module, which calculates the number of container that each client and distributor holds. From this data, this module studies the container requirements for each warehouse. The prediction module is located inside the stock management module and its function is essential. Objectives The objective of this work is to carry out the most accurate prediction of the number of containers that every client is going to take from each warehouse; the company, therefore, will be able to reduce the stock in the warehouses and increase its profits. The prediction must be adaptable so that it automatically adjusts to the new requirements of each client. In order to solve this problem, we have decided to apply artificial intelligence techniques, specifically neural networks. These are briefly commented on in the next section. From the pioneering work of White (1988), the application of neural networks to finances has undergone an exponential growth in research and in publications (Lu 1999). Since then, these models have been successfully applied to economical predictions with many variants. Neural networks are also well-adapted to system changes through retraining. There are a huge number of exogenous variables that can influence the system (environmental factors, epidemics, demand fluctuation, etc.), which depend on the particular activities of each client. The impossibility of analysing each client individually forces us to make predictions based on only a few known variables. Despite these limitations, we attempt to provide improvements in the stock management of the company. Artificial neural networks Artificial neural networks give general mechanisms to build models of systems from samples of the systems. Neural networks are also called connectionist models and consist in a set of primitive units (neurons) which work in parallel and are connected by links (arcs). In these models, information is distributed across the net and is stored in the topology structure and in the arc weights.
5
There are three different types of neurons: input nodes are activated according to the problem variables; output nodes signal the net response to the input pattern; the rest of neurons are called hidden nodes since they are not visible from the outside. Neurons are usually grouped in layers according to their function. The fundamental parameters of a net are: the number of layers, the number of neurons in each layer, the degree of connectivity and the kind of connection among neurons. According to these parameters there are different models of neural networks, such as self-organizing maps (Kohonen 1997), Hopfield networks (Freeman 1993), etc. However, the most interesting models for prediction applications are multilayer feedforward (Bishop 1995) and recurrent networks (Hammer 2000). These models have been used by many researchers to solve financial problems and other forecasting problems, i.e., (Castiglione 2001) and (Giles 2001). Description of the application The design of neural networks is similar to the traditional procedures for computer applications, although it has its own peculiarities. The following subsections present the phases which have been established. Problem definition The only information available for the stock prediction was a database with the movements that clients had made in the warehouses over the last two years. Extensive research was not carried out because there was a large number of external variables which depend on each client activity. This would suppose an enormous effort and it would be impossible to include new clients to the application without modifications. Besides, variables like product seasons and atmospheric conditions, which could take part in the model, are also unforeseeable, so they could not be used as network inputs. As a result, historical series were the only pieces of information which were used during the learning process. Information codifying. Network inputs and outputs The date, separated into day and week, was included as input since it is a relevant factor: a client does not work in the same way all the year (there are clients who only work at particular seasons, according to their products). Likewise, each client has its own preference about the day of week it wants to collect the products on. The network was also tested without including the date as input, including also the month, and making many more combinations, but the result was worse. The year is not included because the network gets adapted by means of the retraining process when new information about the client arrives to the company.
6
A temporal window was also included, which shows the number of containers that the client collected in the preceding six days (see Table 1). Likewise, different window sizes were tested without being able to reduce the error. Due to the activate function of neurons, it is advisable to codify the inputs between 0 and 1. The Johnson coding was used in order to codify the week of the year; this coding has some interesting characteristics which the network can take advantage of: two consecutive weeks are coded with a difference of only one bit, as the last week of the year and the first week of the next year. A neuron for each day of week is also included. Sunday is not included because it is not a working day in our problem. Finally, temporal window coding has been calculated by dividing the number of containers rented each day by a maximum value. This maximum value can vary if the client increases its demand. Nevertheless, this problem is easily resolved by retraining the neural network with the new maximum value. A statistical normalization was also tested, but the resulting values got off the interval [0..1] and the network output had a high error rate. The output layer includes only one neuron, which represents the number of containers that a specific client collected on a specified day. This value must be multiplied by the above-mentioned maximum in order to return the output value to its original unit measure. Network model As there is no rule for selecting a particular topology for a given problem, it is useful to analyze different topologies and different learning algorithms to find the one which is best adapted to the problem under study (Ripley, 1996; Bishop, 1995). Thus, neural network training consists of an iterative, trial-and-error method. During this process the following points have to be optimised: •
Network topology including number of hidden layers and number of neurons in each layer,
•
Training algorithm (usually, Backpropagation, Backpropagation with Momentum) and training parameters, and
•
Number of training data.
In our work numerous network structures were tried: recurrent networks and networks without feedback, different numbers of hidden layers and different number of neurons per layer. Table 2 shows some results obtained with different network topologies. There is not a better topology for all the clients. Therefore, we have selected the network structure which, in general, gives the best behaviour in most of the cases. The configuration selected was a 7
multilayer perceptron with only one hidden layer composed of 52 neurons. The neurons of each layer were fully connected to the neurons of the previous layer. The results obtained with recurrent networks were slightly worse than those using multilayer perceptron. Figure 3 shows that feedforward networks make a tighter prediction than recurrent networks. The recurrent networks tested were Elman networks (Elman 1990). Elman networks were tested using the BPTT (BackpropTroughTime) training function, with a learning factor between 0,05 and 0,2. These networks were not able to predict the peaks in the demand of the clients with enough accuracy. Neural network training Networks have been trained by the “Back-Propagation” algorithm (Freeman 1993), with a learning factor of 0,01. There are other learning methods like the “Quick-Propagation” algorithm (Freeman 1993), the “Back-Propagation with momentum” algorithm (Freeman 1993), etc. These algorithms theoretically converge faster but the obtained results were not as good as the “Back-Propagation” ones. In order to avoid a long loop, a limit in the number of training cycles was established. A verification on the error was also included as a stop training condition; this error was calculated over a set of validation samples (10% of the whole set of samples, randomly selected) and did not have to grow in order to prevent the network from overtraining (Prechelt 1994). In the implemented application, a neural network was trained for each client, type of container and warehouse (starting from the selected network model). The algorithm in Table 3 carried out the automated training of these networks. Results As the graphs show (Figures 4, 5, 6 and 7), the obtained results were very satisfactory. Each graph belongs to a different client and shows the similarity between predictions and real information. Data employed to generate the graphs are samples which were not used in the training process. The results show that first week predictions are very tight. Results get worse as the foreseen period grows; this is due to the fact that previous prediction values are used as network inputs and, therefore, the error propagates. However, this problem is not important since the company in question works with weekly periods. The regular descents in the graphs are due to weekends, since clients only rent containers 6 days per week (eliminating either Saturday or Sunday). The attached table (Table 4) shows the results quantitatively. It shows the 8
prediction for the next week for several different clients; the company, which uses this tool for its predictions, has provided this information. The first column shows the number of containers actually rented during that week, and the second column shows the number predicted. The error made, which appears in the last column, has been calculated in the way the equation (Eq. 1) shows:
% Error =
PredictedValue × 100 − 100 RealValue
(1)
The error made for each client in a warehouse normally ranges between 1 and 10 %. As can be seen in the last row of Table 4 (which shows totals), the prediction of the total number of containers of one type in a warehouse is really good in most of cases (< 2%). This is due to the fact that individual errors compensate for each other. However, the information that the company is interested in is the total number of containers that are going to be rented in each warehouse. Conclusions As the obtained results show, neural networks constitute a great prediction tool for the behaviour of “Pool” company clients, even despite the limited amount of historical information available for training. They work particularly well when working information is non-linear, with noise or with temporal dependences, or when it needs to be adjusted to new situations quickly. Furthermore, the aim of this work is not to compare traditional statistical methods with neural network methods. We are only interested on showing that neural networks are appropriate to solve the stock prediction problem for those domains with features like the ‘Pool’ company problem. On the other hand, it is difficult to give explanation for the result justification because neural networks adopt a “black box” approach. Besides, training periods may be too long in certain situations (not in this case: training a network for only one client, in one warehouse and for one specific type of article, usually takes about 4 or 5 minutes; this implies only a few hours of calculation for the entire process). This system is currently being used for stock prediction in all seven warehouses of the company. During this utilization period, the system behaviour has been very satisfactory. Figure 4 shows the error rate of the prediction for each week and article during the year 2001. Despite there are weeks where the error for a particular article is almost the 4%, in general the prediction error rate remains less than 2%. Finally, this work is an example of the real usefulness of A.I. in the solution of industrial and business problems. This technique has provided an economic profit for the company, since it has kept the service quality and has increased the number of rented containers without increasing fixed assets. 9
Acknowledgements We would like to acknowledge the Computer Science research group of the Universidad Politécnica de Valencia (Grupo de Tecnología Informática), and the staff of the company Logifruit S.L. which, in close collaboration, have helped us to carry out this work. References Bishop, Ch.M. 1995. Neural Networks for Pattern Recognition. Clarendon Press, Oxford. Castiglione, F. 2001. Forecasting Price Increments Using an Artificial Neural Network. In Complex Dynamics in Economics, a Special Issue of Advances in Complex Systems, volume 3, number 1, pages 45-56. Elman, J.L. 1990. Finding Structure in Time. Cognitive Science, volume 14, number 2, pages 179-211. Freeman, J.A., and Skapura D.M. 1993. Redes Neuronales: Algoritmos, Aplicaciones y Técnicas De Programación. AddisonWesley. Giles, C.L., Lawrence, S., and Tsoi, A.C. 2001. Noisy Time Series Prediction Using a Recurrent Neural Network and Grammatical Inference. Machine Learning. Hammer, B. 2000. Learning with Recurrent Neural Networks. Springer. Kohonen, T. 1997. Self-organizing maps. Springer. Lu, Chun-Feng. 1999. Would Evolutionary Computation Help for Designs of Artificial Neural Nets in Financial Applications?. Masterlink Securities Corporation. Poh, H., Yao, J., and Jasic, T. 1997. Neural Networks for the Analysis and Forecasting of Advertising and Promotion. International Journal of Intelligent Systems in Accounting, Finance and Management, Vol. 6 No. 4. Prechelt, L. 1994. PROBEN1-A Set of Benchmarks and Benchmarking Rules for Neural Network Training Algorithms. Technical Report 21/94. Fakultät für Informatik, Universität Karlsruhe. Ripley, B.D. 1996. Pattern Recognition and Neural Networks. Cambridge Univ. Press. Thawornwong, S, and Enke, D. 2001. The Use of Data Mining, Neural Network Forecast Models and Validation Techniques for Predicting Excess Stock Returns. Second International ICSC Symposium on Advanced Computing in Financial Markets. Weigend, A.S., and Gershenfeld, N.A. 1994. Time Series Prediction: Forecasting the Future and Understanding the Past. Addison-Wesley. Yao, J.T., and Tan, C.L. 2000. A Case Study on Using Neural Networks to Perform Technical Forecasting of Forex. Neurocomputing , volume 34, Number 1-4, pages 79-98. Zell, A., et al. 1995. SNNS Stuttgart Neural Network Simulator. User Manual, Version 4.1. University of Stuttgart.
10
Tables
Parameter
# of neurons
Normalized coding
Week of year
27
Johnson coding
Day of week
6
An input neuron for each day of week, except Sundays
Window
6
Output value for the previous six days, except Sundays
Table 1. Input layer of the neural network.
Training data (680 samples) input 1st 2nd output layer hidden hidden layer layer layer 39 15 1 39 25 1 39 52 1 39 70 1 39 10 5 1 39 20 10 1
Client 1
Client 2
Client 3
Client 4
Client 5
0,038 0,037 0,031 0,037 0,056 0,054
0,009 0,010 0,009 0,010 0,009 0,009
0,054 0,054 0,051 0,058 0,054 0,054
0,051 0,051 0,050 0,047 0,048 0,049
0,024 0,027 0,026 0,027 0,024 0,023
Table 2. Mean square error on the test data for some of the neural network topologies tried.
If ExistsNetwork(Client, Article, Warehouse) then LastTrainingDate = GetLastTrainingDate(Client, Article, Warehouse) /* Retrain the network every 30 days */ If CurrentDate – LastTrainingDate > 30 then LastMovements = GetRentedAmountsFromTheLastTraining(Client, Article, Warehouse, LastTrainingDate) RetrainNetwork(Client, Article, Warehouse, LastMovements) StoreLastTrainingDate(CurrentDate) End If Else /* First network training */ Movements = GetMovements(Client, Article, Warehouse) TrainNetwork(Client, Article, Warehouse, Movements) StoreLastTrainingDate(CurrentDate) End If
Table 3. Network training algorithm.
11
Real value
Predicted value
Error
20.196
21.145
4,69 %
4.176
4.228
1,25 %
6.480
6.871
6,03 %
7.380
6.902
6,47 %
12.168
11.706
3,79 %
4.608
4.580
0,6 %
2.412
2.645
9,66 %
3.456
3.575
3,44 %
4.032
3.990
1,04 %
64.908
65.642
1,13 %
Table 4. Achieved results.
Figure captions Figure 1. Physical flow of the reusable transportation containers. Figure 2. General information scheme. Figure 3. Comparative of the prediction using a feedforward multilayer network and an Elman network. Figure 4. Prediction of rented containers for client 1. Figure 5. Prediction of rented containers for client 2. Figure 6. Prediction of rented containers for client 3. Figure 7. Prediction of rented containers for client 4. Figure 8. Real weekly predictions during the year 2001 for each type of article in the main wharehouse.
12
