Data Clustering Using Artificial Neural Networks - CiteSeerX

Proceedings of National Conference on Challenges & Opportunities in Information Technology (COIT-2007) RIMT-IET, Mandi Gobindgarh. March 23, 2007.

Data Clustering Using Artificial Neural Networks N. Kumar, R.S.Joshi Mechanical Deptt., DBPC Dhuri, Punjab. { [email protected]} Mechanical Deptt., RIMTIET, Mandi Gobindgarh, Punjab. { [email protected]}

Abstract Cluster analysis is the automatic grouping of objects into groups on the basis of numerical data consisting of measures either of properties of the objects, or of the dissimilarities between them. This paper concerns with the ART1 (Adaptive Resonance Theory 1) in Neural Network. Important features of ART1 are similarity measure (criterion), vigilance parameter, and their function to classify the input patterns. We compare the results of different similarity criteria experimentally. The ART algorithm is programmed in C++.

2.

ART1 ARCHITECTURE

The ART1 architecture [10] consists of two layers of neurons, the comparison layer and the recognition layer as shown in figure 1.1. In comparison layer (i.e. bottom layer) numbers of neurons are equal to the number of total features and neurons in the recognition layer (i.e. top layer) are equal to the maximum expected number of part families. As the inputs

Keywords : Adaptive resonance theory (ART), clustering algorithm, ,neural networks, backpropagation network, ,artificial intelligence .

1.

INTRODUCTION

Artificial intelligence[2,3] is a study of intelligent human behavior which, concerns in development of computer programs, which exhibit intelligent human behavior. The neural network is a new algorithm approach which has been the subject of intensive study by mathematicians, statisticians, physicists, engineers and computer scientists. The proposed ART1 approach [9] is based on Stephen Grossberg’s stability plasticity dilemma. This dilemma can be stated as a series of questions. How can a learning system remain adaptive (plastic) in input associated with the retain previously learned information while continuing to learn new things? A key to solving the stability-plasticity destroying old information, automatic switching between stable and plastic modes, and response to significant input, yet remain stable in response to irrelevant input? How does the system know to switch between its plastic and its stable modes? How can the system stabilization of the encoding of the classes done by the nodes. ART1 network gets its name from the particular way in which learning and recall interplay in the network. dilemma is to add a feedback mechanism between the competitive layer and the input layer of a network. This feedback mechanism facilitates the learning of new information.

Figure 1: ART1 Architecture are presented to the network, the model selects the first input vector C (part vector) as the specimen for the first group (i.e. first part family) .The first neuron in the recognition layer is made to identify this group. The group centre is represented by the associated neuron’s top down weight vector Tj. when the next input is presented, it is compared with the first group specimen. If this input is similar to the specimen, within a specified vigilance threshold limit (p), then it is treated as the member of the first group. The weight connected to this group is also updated in the light of new input vector. If the new input is not similar to the specimen, it becomes the second neuron in the recognition layer. This process is repeated for all inputs (parts). ART learns to cluster the input pattern by

197


making the output neurons compete with each other for the right to react to a particular input pattern. The output neuron which has the weight vector that is most similar to the input vector claims this input pattern by producing an output of ‘1’ and at the same time inhibits other output neurons by forcing them to produce ‘0’s. in ART, only the winning node is permitted to alter its weight vector , which is modified in such a way that is brought near to the representative input pattern in cluster concerned. ART attempts to associate an input pattern to a cluster of patterns. The output of ART is an indication of membership of the input pattern in a group with similar characteristics.

Step 5. Vigilance test (i.e. test of similarity with best matching exemplar)

T . X = ∑ t iθ xi number of perfectly matching i

‘1’s between input vector and best matching exemplar

X = ∑ xi

number of ‘1’ s in input vector

i

represent the new class 3.

ART ALGORITHM [8]

The detail procedure of the implemented algorithm is given below .

If similarity

T. X

Step 1. Initialize top down and bottom up connection weights

X

>ρ

Top-down connection weights : tij (0) = 1

go to step 7; else go to step 6.

Bottom-up connection weights :

Step 6. Disable best matching exemplar temporarily

Bij (0) = 1 / (1+N) For all input nodes i = 0, 1, _ _ _ _ _ ( N-1) and

Output of the best matching node selected in step 4 is temporarily set to zero ; other outputs have a inhibition ; then go to step 3 . in step 3, a new neuron in the output layer gets selected to selected to represent the new class.

output nodes j = 0, 1, _ _ _ _ _ _ _ ( M-1)

Step 7. Update best matching exemplar temporarily

biθ (t + 1) =

Select a value for vigilance threshold between zero and one: 0 ≤ ρ ≤ 1

t iθ (t ) xi 0.5 + ∑ t iθ (t ) xi i

Step 8 : Repeat Go to step 2 , after enabling any nodes disabled in step 6.

Step 2. Apply new input vector X, consisting of zero / one elements xi,

4.

then it is treated as the member of the first group.

WORKING OF DEVELOPED CLUSTERING SYSTEM

The developed system as shown in figure 1.2 is based on ART1 Algorithm to cluster the data Based on the algorithm, a complete package has been developed .This

Step 3 . Compute matching scores

software has been tested successfully with a number of

The output µj , of every output node j equals :

examples. One example is shown in figure 1.1 having

µj = ∑ bij (t ) xi

nine parts with different features. The developed package

i

cluster the parts according to the similarities of features. Step 4. Select best matching exemplar i.e. node with maximum output

µθ = max{µ j } j

Outputs of other neurons are suppressed (lateral inhibition); in case of tie choose the neuron with lower j.

198


001001100 100010010 010100001 000010010 001001100 010100101 010100000 100010010 000000100

2 3 4 5 6 7 8 9

Part-feature matrix

Part Part Part Part Part Part Part Part

: : : : : : : :

1 0 0 0 0 0 1 0

0 1 0 0 1 1 0 0

0 0 0 1 0 0 0 0

0 1 0 0 1 1 0 0

1 0 1 0 0 0 1 0

0 0 0 1 0 0 0 0

0 0 0 1 1 0 0 1

1 0 1 0 0 0 1 0

0 1 0 0 1 0 0 0

Cluster : 2 Cluster : 3 Cluster : 2 Cluster : 1 Cluster : 3 Cluster : 3 Cluster : 4 Cluster: 1

When the first input vector (part 1) is presented, it is coded as belonging to cluster 1( i.e. part family 1) and the first neuron of the recognition layer(i.e. top layer) is made to identify this class. Similarly the parts 2 and 3 as cluster 2 and 3 respectively. Next part 4 is classified as cluster 2 (i.e part family 2) based on the level of similarity with part 2. Part 5 is coded as cluster 1, while parts 6 and 7 get coded as cluster 3vectors; when part 8 is presented, it is not found to be similar to part 2, because of the earlier contraction of this exemplar due to part 4, hence part 8 gets coded under a new cluster 4.Thus after all the parts are processed, there four clusters or part families. In a similar we get alternative solutions for the same problem by talking different value of vigilance parameter.

Figure 1.1 Example

6.

CONCLUSION:

It can be concluded by results of example that the ART1 is very fast and effective approach from computational point of view due to its very fast speed of generating results. The output is highly dependent on the sequence of input. The developed ART1 software generate a number of alternative solutions of a problem by simply adjusting the vigilance parameter .It can be seen from results that as we decrease the vigilance parameter the number of part families reduces as less similarity is required between parts. This feature increases the flexibility of making the groups. The present study has demonstrated a ART1 neural network approach, for clustering the data which has the following advantages [4]. Figure 1.2 Developed Clustering package Enter the input file name in which input is defined in the following order given below. No. of Bottom layer neurons (equal to number of features) = 9 Max. Expected number of Groups (equal to total parts) = 9 No. of top layer neurons (equal to total number of parts ) = 9 Vigilance parameter = .7 5.

· · ·

1 Part : 0 0 1 0 0 1 1 0 0

general

REFERENCES [1] [2]

FINAL RESULTS

with (vigilance parameter = 0.7)

Its very high speed of computation. It can handle large input vectors, which are requirement of GT problems. It supports flexibility.

[3]

Cluster : 1

199

Simon Hay Kin, 'Neural Networks’, Pearson Education Asia, Delhi-2004. S. Raja Sakaran, G.A. Vijayalakshmi Pai, 'Neural Networks, Fuzzy Logic and Genetic algorithms PHI’, New Delhi - 11001 (2003). Kevin Knight, “Artificial Intelligence” Tata McGraw-Hill Publishing New Delhi, (1991).


[4] [5] [6] [7] [8] [9]

[10]

James A. Freeman, David M. S. Kapura, 'Neural Networks International Student Edition (1991). M.J. Zurada, "Introduction to Artificial Neural System Jacbio Publishing Mouse”, New Delhi (1999). Kunwoo Lee,'Principles of CAD/CAM/CAE Systems Addition-Wesley (1993). www.taylorandfrancis.com Ming-Laing Chen and Chung-Min Wu , “Journal of the Chinese Institute of Industrial Engineers, Vol. 19, No. 3, PP 62-74 (2002). M.Solimanpur, P.Vari and R.Shankar 'Feasibility and robustness of transiently chaotic neural networks applied to cell formation problem ‘, Int. J. Prod. Res. Vol. 42 No .6, pp 1065-1082 (2004). S. Jayaswal and G.K .Adil ‘Efficient algorithm for cell formation with sequence data, machine replications and alternative process routings ’, Int. J. Prod. Res., Vol. 33, No. 2, pp 2419-2433 (2004).

200