Artificial Neural Networks

Artificial Neural Networks

Exercise In this Exercise

ANNs

1. 2. 3. 4. 5. 6.

Introduction ANNs and Matlab Part #1 (Fitting a Function / Regression) Part #2 (Pattern Recognition / Classification) Part #3 (Clustering) References

Duration: 120 min

1. Introduction

(10 min)

An Artificial Neural Network (ANN from now on) is a mathematical structure which consists of interconnected artificial neurons that mimics, in a much smaller scale, the way a biological neural network (or brain) works. An ANN has the ability to learn from data, either in a supervised or an unsupervised fashion and can be used in tasks such as regression, classification, clustering and more. A typical artificial neuron is depicted in Figure 1. The scalar inputs pi are transmitted through connections that multiply their strength by the scalar weight wi to form the product wipi, again a scalar. All the weighted inputs wipi are added and to Σwipi we also add the scalar bias b. The result is the argument of the transfer function f, which produces the output a. The bias is much like a weight, except that it has a constant input of 1. p1 p2 pi

w1 w2

Σ

wi wn

pn

b 1

f

a

a=f(Σwipi+b)

Figure 1: An artificial neuron with one input and bias.

Note that wi and bi are both adjustable scalar parameters of the neuron. The central idea of neural networks is that such parameters can be adjusted so that the network exhibits some desired or interesting behavior. Thus, you can train the network to do a particular job by adjusting the weight or bias parameters, or perhaps the network itself will adjust these parameters to achieve some desired end. The most commonly used transfer functions are the hard-limit (or step), the linear and the sigmoid (or logistic), all depicted in Figure 2.

Figure 2: Commonly used transfer functions: hard-limit (left), linear (middle) and sigmoid (right).

The most common ANN architecture consists of many neurons organized in layers. Each neuron in a layer is connected to all the neurons of the next layer. We can distinguish between input, hidden and output layers. There is only one input and output layer whereas more than one hidden layers are

Kokkoras F. | Paraskevopoulos K.

P a g e |1

International Hellenic University


allowed. Note also that it is common for the number of inputs to a layer to be different from the number of neurons. Some other definitions you should aware of are:  

learning rules: procedures for modifying the weights and biases of a network, that is, methods of deriving the next changes that might be made in an ANN training: a procedure whereby a network is actually adjusted to do a particular job

There are two broad categories of learning rules. In supervised learning the network is provided with a set of examples (the training set) of proper network behavior (pairs of input and known/correct output (target)). As the inputs are applied to the network, the calculated network outputs are compared to the targets. The learning rule is then used to adjust the weights and biases of the network in order to move the network outputs closer to the targets. In unsupervised learning, the weights and biases are modified in response to network inputs only. There are no target outputs available. For the purpose of training, the input data are divided into three sets:  



training: it is used for adjusting the weights and biases validation: it is used to decide when to stop the training process, to avoid overfitting, a situation where the network memorizes the training data, rather than learning the law that governs them testing: it is used to measure the performance of the trained network – it is important that this data do not participate in the training process

The most common forms of ANNs are the multilayered perceptrons, the self-organized maps (SOM or Kohonen Networks) and the associative memories (also known as Hopfield networks).

3. ANNs and Matlab

(5 min)

Matlab (R2008a -v.7.6- was used for the tutorial) provides the Neural Networks Toolbox, a set of tools that include GUIs, wizards and functions that allow any user level (from novice to expert) to use and experiment with neural networks with minimal effort. The easiest way to use the toolbox is through the GUIs that perform certain tasks. In this tutorial we will see the following tasks:   

fit a function / regression recognize patterns / classification cluster data

The second easiest way to use the toolbox is through basic command-line operations. The command-line operations offer more flexibility than the GUIs, but with some added complexity. In this tutorial we will use both GUIs and command line operations. In the following exercises we will see all the steps pertained to the ANN usage to solve regression, classification and clustering problems. These steps include: collecting data, creating the network, configuring the network, initializing the weights and biases, network training, network validation and network use. Note though that you cannot use any type of ANN for any type of problem. The datasets used in the following were taken from the Bren School of Information and Computer Science at the University of California, Irvine (Repository Of Machine Learning Databases) [1]. They are available at: ftp://ftp.ics.uci.edu/pub/machine-learning-databases/ The following exercises are adaptations of Matlab's tutorials to local needs. Kokkoras F. | Paraskevopoulos K.

P a g e |2


4. Part #1 (Fitting a Function / Regression)


(35 min)

Neural networks are good at fitting functions. In fact, there is proof that a fairly simple neural network can fit any practical function. Problem: Using data from a housing application [1] we want to design a network that can predict the value of a house (in $1000s), given 13 pieces of geographical and real estate information. The dataset consists of 506 example homes (records) for which we have 13 items of data and their associated market values. The dataset is available at: ftp://ftp.ics.uci.edu/pub/machine-learning-databases/housing/ The file housing.data contains the data whereas housing.names describes what this data is about. Take your time to familiarize yourself with the dataset. The better you know your data the more insight can give you during ANN modeling.

Defining a Regression Problem Data used for regression problems in Neural Network Toolbox, should be arranged in columns (that is, a data record or training instance is a column). Additionally, input and target values should be located in separate matrixes or files. For example, assuming data for modeling the logical AND problem, one should provide: inputs = [0 1 0 1; 0 0 1 1]; targets = [0 0 0 1]; Alternatively, two delimited text files containing: 0 1 0 1 0 0 1 1 and: 0 0 0 1 respectively, are adequate. This might be a little strange because we usually arrange records in separate lines, that is: 0 0 0 0 1 0 1 0 0 1 1 1

Using command line functions 1. To load the data in Matlab type (in Matlab's command line): load houseInputs.txt load houseTargets.txt These two data files where created from housing.data for your convenience. Since this dataset is also available in Matlab, you can alternatively just type: load house_dataset 2. Create a network. For this example, you use a feed-forward network with the default tan-sigmoid transfer function in the hidden layer and linear transfer function in the output layer. This structure is useful for function approximation (or regression) problems. Use 20 neurons (somewhat ar-


P a g e |3



bitrary) in one hidden layer. The network has one output neuron, because there is only one target value associated with each input vector. net = newfit(houseInputs,houseTargets,20); More neurons require more computation, but they allow the network to solve more complicated problems. More layers require more computation, but their use might result in the network solving complex problems more efficiently. Beware though that adding too many neurons compared to the dataset size might result in overfitting phenomena (more on this later). 3. Train the network. The network uses the default Levenberg-Marquardt algorithm for training (backpropagation-like). The application randomly divides input vectors and target vectors into three sets as follows: 60% are used for training, 20% are used to validate that the network is generalizing and to stop training before overfitting. t 20% are used as a completely independent test of network generalization. To train the network, enter: net=train(net,houseInputs,houseTargets) During training, the training window (right) opens. This window displays training progress and allows you to interrupt training at any point by clicking Stop Training. The train function presents all the input vectors to the network at once in a batch. Alternatively, you can present the input vectors one at a time using the adapt function. As you can see in the screenshot, the training stopped when the validation error increased for six iterations, which occurred at iteration (or epoch) 13.

Note: Given the random initialization of the network, every 'run' produces different results. You will get different results from these depicted here but if the modeling process goes well, you should expect results of the same quality. If you click Performance in the training window, a plot of the training errors, validation errors, and test errors appears, as shown in the figure on the left. In this example, the result is reasonable because of the following considerations:


P a g e |4



 The final mean-square error is small (2.6).  Both the test set error and the validation set error have similar characteristics (green and red lines in the plot).  No significant overfitting has occurred by iteration 7 (where the best validation performance occurs). 4. Perform some analysis of the network response. If you click Regression in the training window, you can perform a linear regression between the network outputs and the corresponding targets. The following figure shows the results. Ideally (with zero error), the points should be placed on the target=output line. In our case, the output tracks the targets very well for training, testing, and validation, and the R-value is a bit over 0.95 for the total response. If you need more accurate results, you can try any of these approaches:  Reset the initial network weights and biases to new values with newfit and train again.  Increase the number of hidden neurons.  Increase the number of training vectors (more data).  Increase the number of input values, if more relevant information is available.  Try a different training algorithm. In this case, the network response is satisfactory, so we can use the ANN to predict the value of a house, given the 13 input parameters. To get a better insight on how the training proceeds, keep the Performance and Regression plots open and initialize and re-train the network while watching the plots animate. Improving Results This example demonstrated some simple commands you can use to solve many types of problems. However, if your first attempt does not meet your needs or expectations do not hesitate to retry. If the network is not sufficiently accurate, you can try initializing the network and the training again. Each time your initialize a feed-forward network, the network parameters are different and might produce different solutions. To re-initialize the network type: net = init(net); where net is your neural network. To re-train the network type:

net = train(net,houseInputs,houseTargets);

As a second approach, you can increase the number of hidden neurons above 20. Larger numbers of neurons in the hidden layer give the network more flexibility because the network has more parameters it can optimize. You should increase the layer size gradually. If you make the hidden layer too large, you might cause the problem to be under-characterized and the network must optimize more parameters than there are data vectors to constrain these parameters. Your network will be overfitted. It will memorize the training examples and although it will demonstrate high accuracy on


P a g e |5



the training data, it will work poorly on never-seen data. A better approach is to try using additional training data. Providing additional data for the network is more likely to produce a network that generalizes well to new data.

Using the Neural Network Toolbox Fitting Tool GUI 1. Open the Neural Network Toolbox Fitting Tool with the command: nftool 2. Click Next to proceed. 3. Click the first button to load houseInputs.txt as input parameters and the second one to load the houseTargets.txt as target parameters. You should browse for your files in the Import Wizard and check that the data are properly recognized (you will have no issue with the provided data files – just hit the 'Next' button twice and then Finish). Alternatively, you can load data already loaded in the workspace. Click the Next button. 4. Click 'Next' to display the Validate and Test Data window, shown in the figure on the right. Here you can select the portion of the original dataset that will be used for validation and testing. Keep the default values of 15%. 5. Click Next. The number of hidden neurons is set to 20. You can change this value in another run if you want. You might want to change this number if the network does not perform as well as you expect. 6. Click 'Next' to see a synopsis of the situation and then click on the the situation is known since you get the same windows as in the command line approach done earlier. As mentioned, the results are not exactly the same but are very similar. The fit is almost perfect for training, testing, and validation data.

button. From now on

7. Click 'Next' in the Neural Network Toolbox Fitting Tool (window) to evaluate the network. At this point, you can test the network against new data (using the controls on top-right).

If you are dissatisfied with the network's performance on the original or new data, you can take any of the following steps (by using the buttons on the left): Kokkoras F. | Paraskevopoulos K.

P a g e |6



 Train it again.  Increase the number of neurons.  Get a larger training data set. 8. If you are satisfied with the network performance, click Next to go to the Save Results window. You can use the buttons on this screen to save your results. You can also click Generate M-File to create an M-file that can be used to reproduce all of the previous steps from the command line. Creating an M-file can be helpful if you want to learn how to use the command-line functionality of the toolbox to customize the training process. 9. When you have saved your results, click Finish.

5. Part #2 (Pattern Recognition / Classification)

(35 min)

Neural networks are also good at recognizing patterns. In this exercise we will classify a tumor as benign or malignant, based on uniformity of cell size, clump thickness, mitosis, etc. [3]. Out dataset consists of 699 example cases for which we have 9 items of data and the correct classification as benign or malignant. This breast cancer databases was originally obtained from the University of Wisconsin Hospitals, Madison from Dr. William H. Wolberg. It is available online at: ftp://ftp.ics.uci.edu/pub/machine-learning-databases/breast-cancer-wisconsin/ Note that this dataset contains some missing values (the numbers are replaced with question mark characters '?'). For the needs of this tutorial, we have replaced the question marks with the number 3.5. The dataset is also available in Matlab but except from the missing values issue, all numbers are divided by 10. Finally, the class designators are 2 and 4 in the original dataset but 0 and 1 in the Matlab's dataset. We will also use these designators (0 and 1). More on this right in the following text.

Defining a Classification Problem As with the function fitting problem discussed in the previous section, records should be organized in columns (vectors) in a matrix (or file). Additionally, the same count of target vectors are required to indicate the classes to which the input vectors are assigned. There are two approaches to creating the target vectors:  If there are only two classes we set each scalar target value to either 1 or 0, indicating which class the corresponding input belongs to. For example, you can define the exclusive-or classification problem as follows: inputs = [0 1 0 1; 0 0 1 1] targets = [0 1 1 0];  If inputs are to be classified into N different classes then we create target vectors (again arranged as columns) that consists of N elements, where for each target vector, one element is 1 and the others are 0. For example, in the exclusive-or problem, we can write the target vectors like: targets = [1 0 0 1; 0 1 1 0]; That is, classification problems involving only two classes can be represented using either format. The targets can consist of either scalar 1/0 elements or two-element vectors, with one element being 1 and the other element being 0.


P a g e |7



For convenience, we write this dataset in the way it will be written in a text file. Inputs will be: 0101 0011 whereas target classes will either be: 0110 (direct indication of the class) or 1001 (1st class – we put 1 if this is our case, 0 otherwise) 0110 (2nd class – we put 1 if this is our case, 0 otherwise) The next section demonstrates how to train a network from the command line, after you have defined the problem.

Using Command-Line Functions 1. Load the tumor classification data as follows: load cancerInput.txt load cancerTarget.txt 2. Create a network. For this example, you use a pattern recognition network, which is a feedforward network with tan-sigmoid transfer functions in both the hidden layer and the output layer. As in the function-fitting example, we will use 20 neurons in one hidden layer. The network has two output neurons, because there are two categories associated with each input vector. Each output neuron represents a category. When an input vector of the appropriate category is applied to the network, the corresponding neuron should produce a 1 and the other neurons should output a 0. To create a network, enter this command: net = newpr(cancerInputs,cancerTargets,20); 3. Train the network. The pattern recognition network uses the default Scaled Conjugate Gradient algorithm for training. The application randomly divides the input vectors and target vectors into three sets:  60% are used for training.  20% are used to validate that the network is generalizing and to stop training before overfitting  The last 20% are used as a completely independent test of network generalization.

To train the network, enter this command: net=train(net,cancerInputs,cancerTargets); During training, as in function fitting, the training window opens (see screenshot on the right). This window displays training progress. To interrupt a long training at any point, click 'Stop Training'. This example uses the train function. It presents all


P a g e |8



the input vectors to the network at once in a batch. Alternatively, you can present the input vectors one at a time using the adapt function. See Matlab's Help System for more on this. This training stopped when the validation error increased for 6 iterations, which occurred at epoch 21. 4. To find the validation error, click 'Performance' in the training window. A plot of the training errors, validation errors and test errors appears, as shown in the figure on the right. The best validation performance occurred at iteration 15 and the network at this iteration (epoch) is returned. 5. To analyze the network response, click 'Confusion' in the training window. A display of the confusion matrix appears (figure on the right) that shows various types of errors that occurred for the final trained network. There are 4 tables: each one displays the network response for the training, validation, testing and all datasets. The diagonal (green) cells in each table show the number of cases that were correctly classified, and the off-diagonal cells (red) show the misclassified cases. The blue cell in the bottom right shows the total percent of correctly classified cases (in green text) and the total percent of misclassified cases (in red text). The results for all three data sets (training, validation and testing) show very good recognition (that is, the network response is satisfactory). If you needed even more accurate results, you could try any of the following approaches:     

Reset the initial network weights and biases to new values with init (re-build) and train again. Increase the number of hidden neurons. Increase the number of training vectors. Increase the number of input values, if more relevant information is available. Try a different training algorithm.

To get more experience in command-line operations you can open a plot window (such as the performance plot) during training and watch it animate.

Using the Neural Network Toolbox Pattern Recognition Tool GUI 1. Open the Neural Network Toolbox Pattern Recognition Tool window with the command: nprtool 2. Click 'Next' to proceed. The Select Data window opens (see figure on the right). If you have the datasets (input and target) already loaded in the workspace (as is the case of the figure), then you can select them from the proper combo boxes. Otherwise you can click the button to start the Import Wizard where you browse for the proper file. There exist additional da-


P a g e |9



tasets accessible from the 'Load Example Dataset' button. 3. Click 'Next' to continue to the Validate and Test Data window, shown in the figure on the right. Validation and test data sets are each set to 15% of the original data. You can keep these values as this is the usual case. 4. Click 'Next'. The number of hidden neurons is set to 20. You can change this in another run if you want. You might want to change this number if the network does not perform as well as you expect. 5. Click 'Next' to go to the Train Network window where a synopsis of the parameters set so far is displayed. 6. Click the 'Train' button. The training continued for 32 iterations and stopped when the validation error increased for 6 iterations, which occurred at epoch 32-6=26. 7. In the NN Training window Under the Plots pane, click Performance to see how the training phase proceeded.

We remind here that due to random initialization of weights and biases in the artificial neurons, you cannot get exactly the same results as depicted here.

8. In the NN Training window Under the Plots pane, click Confusion. The figure displayed (right) shows the confusion matrices for training, testing and validation, as well as the three kinds of data combined. The network's outputs are almost perfect, as you can see by the high numbers of correct responses in the green squares and the low numbers of incorrect responses in the red squares. The


P a g e | 10



lower right blue squares illustrate the overall accuracies. In these squares, green text is used for the correct responses and red for the incorrect. 9. In the Neural Network Toolbox Pattern Recognition Tool under the Plots pane, click 'Receiver Operating Characteristic' (ROC). The colored lines (green and blue) in each axis represent the ROC curves for each of the two categories of this problem. The ROC curve is another visualization of the quality of our network. It is a plot of the true positive rate (sensitivity) versus the false positive rate (1 specificity) as the threshold is varied. A perfect test would show points in the upper-left corner, with 100% sensitivity and 100% specificity. For this simple problem, the network performs almost perfectly. 10.In the Neural Network Toolbox Pattern Recognition Tool, click 'Next' to evaluate the network. At this point, you can test the network against new data. If you are dissatisfied with the network's performance on the original or new data, you can train it again, increase the number of neurons, or perhaps get a larger training data set. 11.When you are satisfied with the network performance, click 'Next'. Use the buttons on this screen to save your results in the workspace. If you click Generate M-File, the tool creates an M-file, with commands that recreate the steps that you have just performed from the command line. Generating an M-file is a good way to learn how to use the command-line operations of the Neural Network Toolbox software.

12.When you have saved your results, click Finish.


P a g e | 11



6. Part #3 (Clustering)

(35 min)

Clustering data is another excellent application for neural networks. This process involves grouping data by similarity. For example, you might perform:   

Market segmentation by grouping people according to their buying patterns Data mining by partitioning data into related subsets Bioinformatic analysis by grouping genes with related expression patterns

In this exercise we will cluster flower types according to petal length, petal width, sepal length, and sepal width. We will use the famous 'IRIS' dataset [4] which consists of 150 example cases for which we have these four measurements. You can get the dataset from the following URL: ftp://ftp.ics.uci.edu/pub/machine-learning-databases/iris/ Take your time and examine the description of the dataset (file iris.names). Getting familiar with your dataset usually gives you insight on how to proceed.

Defining a Clustering Problem To define a clustering problem, simply arrange the input records (vectors) to be clustered, as columns in an input matrix. For instance, you might want to cluster this set of 10 two-element vectors: inputs = [7 0 6 2 6 5 6 1 0 1; 6 2 5 0 7 5 5 1 2 2] The next section demonstrates how to train a network from the command line, after you have defined the problem. You will use the irisInput.txt file which is the initial dataset iris.data dataset with the class attribute removed and the rest of the array transposed1 to meet the Matlab requirement of having the data records in columns (they are arranged as rows in the iris.data file).

Using Command-Line Functions 1. Load the data with the command: load irisInputs.txt Note that the initial dataset (iris.data) consists of input vectors and target vectors. However, you only need the input vectors for clustering. This is because, unlike the previous examples, here we are going to use unsupervised learning. 2. Create a network. For this example, you use a self-organizing map (SOM). This network has one layer, with the neurons organized in a grid. When creating the network, you specify the number of rows and columns in the grid: net = newsom(irisInputs,[6,6]); 3. Train the network. The SOM network uses the default batch SOM algorithm for training. net=train(net,irisInputs); 1

It's easy to transpose an array (turn rows in to columns). Open the dataset in MS Excel, Copy the array and Paste Special selecting Transpose. You can do the transposition in Matlab as well but providing the exact steps is beyond the scope of this section.


P a g e | 12



4. During training, the training window opens (see figure on the right) and displays the training progress. To interrupt training at any point, click the 'Stop Training' button. 5. For SOM training, the weight vector associated with each neuron moves to become the center of a cluster of input vectors. In addition, neurons that are adjacent to each other in the topology should also move close to each other in the input space. The default topology is hexagonal. To view it, click 'SOM Topology' from the network training window. In this figure (left), each of the hexagons represents a neuron. The grid is 6-by-6, so there are a total of 36 neurons in this network. There are four elements in each input vector, so the input space is four-dimensional. The weight vectors (cluster centers) fall within this space. Because this SOM has a two-dimensional topology, you can visualize in two dimensions the relationships among the fourdimensional cluster centers. One visualization tool for the SOM is the weight distance matrix (also called the U-matrix). 6. To view the U-matrix, click 'SOM Neighbor Distances' in the training window. In this figure (right), the blue/purple hexagons represent the neurons. The red lines connect neighboring neurons. The colors in the regions containing the red lines indicate the distances between neurons. The darker colors represent larger distances, and the lighter colors represent smaller distances. A band of dark segments crosses from the lower-center region to the upper-right region. The SOM network appears to have clustered the flowers into two distinct groups, each one located on one side of this dark band. 7. Now click the 'SOM Sample Hits' button to see another visualization (see figure on the right). The plot displayed calculates the classes for each flower and shows the number of flowers in each class. Areas of neurons with large numbers of hits indicate classes representing similar highly populated regions of the feature space. Areas with few hits indicate sparsely populated regions of the feature space. Summing up all numbers results to 150. (Why?) As mentioned in earlier parts of this tutorial, during training, you can have a plot window open (such as the SOM weight position plot) and watching it animate as the training proceeds.


P a g e | 13



Note: If you have past experience with the IRIS dataset, you will probably know that there exist three classes: Iris-Setosa, Iris-Versicolor and Iris-Virginica (you can verify this by looking into the dataset). You probably wonder why our clustering network identified only two. To better understand the situation (and the value of familiarize yourself with the data), check some visualization of them in 2D (figure on the left) using each time 2 of the 4 available dimensions. As you can see, only one of the classes is linearly separable from the others (the one displayed with blue points). The other two are not separable, so our ANN failed to identify them and considered them as a single class (cluster). Unlike clustering (which fails to identify the three clusters in the IRIS data), you can train a highly accurate (>97%) classification ANN if you work with the IRIS data like in Part#2 of this tutorial. You will also need target data since this will be a supervised training. We have them prepared already for your convenience in the file irisTargets_x.dat. Try it yourself as an exercise. The situation discussed here does not, in any case, reduce the usefulness of the capability of ANNs to identify clusters in unknown datasets. For this particular dataset, you can find other techniques in literature (for example, fuzzy clustering) that produce better clustering results.

Using the Neural Network Toolbox Clustering Tool GUI 1. Open the Neural Network Toolbox Clustering Tool window with this command: nctool 2. Click Next. The Select Data window appears. This time we will use a different iris dataset existing in the toolbox. Click in the button, select 'Simple Clusters' and press the button. In this dataset two-element vectors are assigned to four classes. As mentioned at the end of the previous section, samples may be classified using clustering (using only input data) or with pattern recognition (classification) or even fitting (with input and target data). The dataset consists of 1000 records. 'simpleclusterInputs' is a 2x1000 matrix of values. 'simpleclusterTargets' is an 4x1000 matrix, where each ith column indicates which category the ith iris belongs to, with a 1 in one element and zeros in the other elements (see figure below).

This simple example can be solved with the Neural Network Pattern Recognition Tool (nprtool) or Clustering Tool (nctool). Here we follow the later approach. You can try the former as an exercise. Kokkoras F. | Paraskevopoulos K.

P a g e | 14



3. In the Select Data window click 'Next' to continue to the Network Size window. The size of the two-dimensional map is set to 10. This map represents one side of a two-dimensional grid. The total number of neurons is 100. You can change this number in another run if you want. 4. Click Next. The Train Network window appears. 5. Click Train. The training runs for the maximum number of epochs, which is 200. 6. Investigate some of the visualization tools for the SOM. Under the Plots pane, click SOM Sample Hits. This figure (right) shows how many of the training data are associated with each of the neurons (cluster centers). The topology is a 10-by-10 grid, so there are 100 neurons. The maximum number of hits associated with any neuron is 32. Thus, there are 31 input vectors in that cluster. We have marked the four classes with red circles. As you ca see, the two classes on the right are not well-separable. 7. You can also visualize the SOM by displaying weight places (also referred to as component planes). Click SOM Weight Planes in the Neural Network Toolbox Clustering Tool. This figure shows a weight plane for each element of the input vector (two, in this case). They are visualizations of the weights that connect each input to each of the neurons. (Darker colors represent larger weights.) If the connection patterns of two inputs were very similar, you can assume that the inputs are highly correlated. In this case, input 1 has connections that are very different than those of input 2. 8. In the Neural Network Toolbox Clustering Tool, click Next to evaluate the network. At this point you can test the network against new data. If you are dissatisfied with the network's performance on the original or new data, you can increase the number of neurons, or perhaps get a larger training data set. 9. When you are satisfied with the network performance, click Next. Use the buttons on this screen to save your results. You can export them into the Matlab's workspace (Save Results button) or use the Generate M-file button to create an M-file with commands that recreate the steps that you have just performed from the command line. Generating an M-file is a good way to learn how to use the command-line operations of the Neural Network Toolbox software. 10.When you have saved your results, click Finish.


P a g e | 15



References 1. Repository Of Machine Learning Databases: ftp://ftp.ics.uci.edu/pub/machine-learning-databases/ 2. D. Harrison and D.L. Rubinfeld, "Hedonic prices and the demand for clean air", J. Environ. Economics & Management, Vol. 5, 1978, pp.81-102. 3. O. L. Mangasarian and W. H. Wolberg: "Cancer diagnosis via linear programming", SIAM News, Volume 23, Number 5, September 1990, pp 1 & 18. 4. R.A. Fisher, "The use of multiple measurements in taxonomic problems" Annual Eugenics, 7, Part II, pp.179-188 (1936); also in "Contributions to Mathematical Statistics" (John Wiley, NY, 1950). 5. Matlab Help


P a g e | 16