last section concludes the paper and brings remarks on future work. II. RELATED WORK. ELECTRE TRI [9] is a member of the family of ELECTRE [10] methods.
The Impact of Prototype Selection on a Multicriteria Decision Aid Classification Algorithm Amaury Brasil, Plácido R. Pinheiro, André L. V. Coelho University of Fortaleza (UNIFOR) – Master of Applied Computing Av. Washington Soares, 1321 - Bl J Sl 30 - 60.811-341 - Fortaleza – Brasil {abrasil}@gmail.com, {placido, acoelho}@unifor.br Abstract - This paper presents an experimental analysis conducted over a specific Multicriteria Decision Aid (MCDA) classification technique proposed earlier by Goletsis et al. Different from other studies on MCDA classifiers, which put more emphasis on the calibration of some control parameters related to the expert's preference modeling process, this work investigates the impact that the prototype selection task exerts on the classification performance exhibited by the MCDA model under analysis. We understand that this sort of empirical assessment is interesting as it reveals how robust/sensitive a MCDA classifier could be to the choice of the alternatives (samples) that serve as class representatives for the problem in consideration. In fact, the experiments we have realized so far, involving different datasets from the UCI repository, reveal that the proper choice of the prototypes can be a rather determinant issue to leverage the classifier's performance.
I. INTRODUCTION A classification problem refers to the assignment of a group of alternatives to a set of predefined classes, also known as categories. During the last decades these problems have been tackled using a high variety of statistical and machine learning techniques. Recently, the area of Multicriteria Decision Aid (MCDA) [1] has also brought about new methodologies and techniques to solve these problems. The main difference between the MCDA classification methods and others coming from related disciplines (for instance, artificial, neural networks, Bayesian models, rule-based models, decision trees, etc.) [2] lies in the way that the MCDA methods incorporate the decision maker's preferences into the categorization process. According to Doumpos and Zopounidis [3], in MCDA the classification problems can be divided into two distinct groups: ordinal sorting problems and the nominal sorting problems. The first one is used when the problem has its classes (groups) defined in an ordinal way. A good example of this type of problem is the bankruptcy risk evaluation problem [4]. In this problem, there is an ordinal definition of the groups, since it is obvious that for a decision maker that the healthy firms are in better situation than the bankrupt ones. The second type of problem refers to the assignment of alternatives into classes that do not present a preferential order. When the problem to be solved is an ordinal sorting problem, the MCDA methods usually introduce a fictitious alternative (sample), called a reference profile, in order to delimit the boundary between two consecutive groups. Conversely, in a nominal sorting problem, the boundaries between classes cannot be properly defined beforehand as, usually, the knowledge about lower and upper boundary samples is not readily available. To overcome such difficulty, Belacel [5] has cast a new interpretation to the role of the reference profiles (also known as prototypes) in his PROAFTN method: That of a good representative sample for a specific class.
As pointed out by Zopounidis and Doumpos [6], the great majority of works conducted on the MCDA classification theme has focused on the development of novel MCDA classification methods, not giving much emphasis on characterizing and comparing their distinctive problems. Likewise, the authors also advocate that future research on this field should consider a more deep investigation into some important practical issues, such as the analysis of the interdependencies of the control parameters of the algorithms, the statistical validation of the generated models, the analysis of performance over large data sets, and the establishment of links between MCDA classifier models and those coming from related disciplines, such as Pattern Recognition, Machine Learning, and Data Mining [2]. In this context, this work investigates the impact that the prototype selection task exerts on the classification performance exhibited by a MCDA model while coping with a nominal sorting problem. We understand that this sort of empirical assessment is pertinent as it reveals how robust/sensitive a given MCDA classifier could be to the choice of the alternatives (samples) that serve as class representatives for the problem in consideration. In fact, the experiments we have realized so far, involving the MCDA classification model proposed earlier by Goletsis et al [7] and different datasets taken from the UCI repository [8], reveal that the choice of the prototypes can be a key issue to be properly dealt with in order to leverage the classifier's performance. The rest of the paper is organized as follows. The next section presents an overview of some related work on MCDA classification. The third section outlines the main conceptual ingredients of the MCDA classification algorithm under consideration. The next section provides details of some of the experiments we have conducted so far, discussing the main results achieved. Finally, the last section concludes the paper and brings remarks on future work. II. RELATED WORK ELECTRE TRI [9] is a member of the family of ELECTRE [10] methods. The ELECTRE methods are based on the outranking relation techniques, and the ELECTRE TRI, specifically, was designed to solve classifications problems. The objective of ELECTRE TRI is to assign a discrete set of alternatives into groups that are defined in an ordinal way. This method introduced the concept of reference profile as a fictious alternative that is a boundary between two consecutive groups. The N-TOMIC method [11], was developed for addressing classification problems when the groups are defined in an ordinal way. The method pre-specifies nine different groups or classes to which the alternatives should be assigned. The groups indicate the
aspects related to the performance of the alternatives (high performance, low performance, inadequate performance, etc.) in relation to two reference profiles. The nine classes are basically settled into three different categories: good alternatives, uncertain alternatives, and bad alternatives. The concept of reference profile used in the N-TOMIC outranking relation model assumes a different meaning from what was originally described in ELECTRE-TRI. Instead of a boundary between the classes, in NTOMIC, a reference profile denotes either a "good" or "bad" alternative. Different from ELECTRE-TRI and N-TOMIC approaches, PROAFTN [5] presents itself as an MCDA method suitable to cope specifically with nominal sorting problems. PROAFTN defines a fuzzy indifference relation that measures the strength of the affirmation "alternative a is indifferent to prototype p". To determine this, some computations based on the ELECTRE-TRI are realized. This method has adapted the concept of reference profile, to the prototype one. Instead of representing the upper and the lower bounds of a boundary alternative, the prototype can be considered a good representative of a specific group. In the Machine Learning field, a particular classification algorithm known as k-NN [12] is similar to some MCDA classification algorithms. This algorithm calculates the Euclidian distance that can be weighted, between an alternative to be classified and each training neighborhood alternative. The new alternative will be assigned to the most frequent class among the k neighbors. A problem that have been widely investigated that it is concerned to the k-NN, is the difficulty that it has to deal with large datasets due to the computational costs involved. To solve that issue, some research was developed to apply instance selection to reduce the number of alternatives of a dataset. For the k-NN, methods such as: CNN [13], ENN [14], VSM [15], and Multedit [16] have been successfully applied over the instance selection problem. III. GOLETSIS' MCDA CLASSIFICATION MODEL The methods presented in the last section have been successfully applied to real world problems. The major difficulty in applying these methods, however, is that, in order to produce models that comply with the decision maker's expectations, a set of control parameters, such as threshold variables, weights, coefficients, etc., needs to be properly set in advance, which turns out to be a hard task to be dealt with. Some authors, like Belacel [5] and Jacquet-Lagrèze and J. Siskos [17] have already provided some alternatives to counter this sort of drawback, although their solutions seem to be rather specific to the contexts that were investigated and yet no general recipes are available to be deployed in all methods. The MCDA classification method that we have chosen for investigation was that proposed by Goletsis et al [7]. Like PROAFTN [5], this method makes use of prototypes to serve as references against which the new alternatives are compared (matched) with. One distinctive aspect of this scheme with respect to other MCDA-based ones is that it presents less control parameters to be adjusted (only some thresholds and criteria weights). In what follows, we provide further details of Goletsis' algorithm. Analytically, the model can be defined as in the following way: Let A be the finite set of alternatives, F the set of n features (in the nominal sorting problem it is also known as criteria), with n>=1, w
the weight of a specific criterion, = 1, C is the set of categories of a problem where C = {C 1, C 2,C 3,…, C k} and K > 1, and Bh={b | 1,…,Lh and h=1,…,K} the set of prototypes of the category Ch, where b represents the p prototypes of the category Ch and Lh the number of the prototypes of this category. Each alternative in A and B is characterized by a feature vector containing its feature values for all n criteria in the F set. Each alternative is compared with each prototype b under each criterion j. As described by Goletsis et al. [7], during this comparison the first thing to be computed is the Similarity Index (SIj (a, b )). This index is calculated for each criterion, and its objective is to model the criteria into a five zone similarity index. In order to compute this index, two thresholds must be specified. The first threshold that needs to be specified is the similarity threshold, , that represents the maximum allowed criterion difference between the alternatives and the prototypes. Using this, the alternatives can be judged similar under a specific criterion. The second threshold used by the (SIj (a, b )) computation is the dissimilarity threshold, , representing the minimum allowed criterion difference between an alternative a and prototype b This threshold needs to be defined in order to considerate the criteria totally dissimilar. The similarity index (SIj (a, b )) is computed as described below: SIj (a, b ) = 1, if
(1)
