A fuzzy-based decision-making procedure for data ... - Semantic Scholar

Expert Systems with Applications Expert Systems with Applications 32 (2007) 939–953 www.elsevier.com/locate/eswa

A fuzzy-based decision-making procedure for data warehouse system selection Hua-Yang Lin a

a,*

, Ping-Yu Hsu a, Gwo-Ji Sheen

b

Chung Shan Institute of Science & Technology and Department of Business Administration, National Central University, No. 300, Jungda Road, Jhongli City, Taoyuan 320, Taiwan, ROC b Institute of Industrial Management, National Central University, Taiwan, ROC

Abstract The increase in the number of companies seeking data warehousing solutions, in order to gain significant business advantages, has created the need for a decision-aid approach in choosing appropriate data warehouse (DW) systems. Owing to the vague concepts frequently represented in decision environments, we have proposed a fuzzy multi-criteria decision-making procedure, to facilitate data warehouse system selection, with consideration given to both technical and managerial criteria. The procedure can systematically construct the objectives of DW systems selection to support the business goals and requirements of an organization, and identify the appropriate attributes or criteria for evaluation. In the fuzzy-based method, the weight of each criterion and the rating of each alternative are described using linguistic terms, which can also be expressed as triangular fuzzy numbers. The fuzzy algorithm aggregated the decisionmakers’ preference rating for criteria, and the suitability of data warehouse alternatives versus the selection criteria, to calculate fuzzy appropriateness indices, through which, the most suitable data warehouse system was determined. A case study of a Bar Code Implementation Project for Agricultural Products in Taiwan was conducted to illustrate this method’s effectiveness. Ó 2006 Elsevier Ltd. All rights reserved. Keywords: Multi-criteria decision making; Data warehouse system; Fuzzy sets; Objectives hierarchy; Software selection

1. Introduction As enterprises experience an increase in data, many look towards implementing enterprise-wide data automation software, in order to organize data and assist in making sensible business decisions; thus, the implementation of data warehousing software is expected to grow rapidly (Mukherjee & D’souza, 2003; Shin, 2002; Watson, Goodhue, & Wixom, 2002). When correctly implemented, a data warehouse (DW) system enables companies to enjoy many benefits and obtain timely information for decision making (Gorla, 2003). However, because of the complexity and variety in the functionality of data warehouse systems,

*

Corresponding author. Tel.: +886 3 4227151x66147; fax: +886 3 4254604. E-mail address: [email protected] (H.-Y. Lin). 0957-4174/$ - see front matter Ó 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2006.01.031

the evaluation and ultimate selection of data warehouse products, to fit a company’s needs, can be daunting. Software selection is not a technical procedure, but is rather, a subjective and uncertain decision process (Stamelos, Vlahavas, Refanidis, & Tsoukia`s, 2000). Selecting a suitable software system, among many, depends on the assessment of objective, measurable criteria (e.g., acquisition costs and training costs), as well as subjective criteria (e.g., compatibility, vendor selection and technical factors). Software selection decisions involve the simultaneous consideration of multiple criteria, including tangible and intangible factors; prioritizing these factors can be challenging. When evaluating and selecting data warehouse software, Kimball, Reeves, Ross, and Thornthwaite (1998) suggested that the evaluation should encompass both business andtechnical requirements. A systematic approach to assist companies in making decisions was not proposed, however.

940

H.-Y. Lin et al. / Expert Systems with Applications 32 (2007) 939–953

Selection of the most suitable data warehouse system from a set of alternatives, on the basis of many criteria, creates a multi-criteria decision-making problem. The values given to selection criteria are often qualitatively described or imprecisely measured. The importance of each criterion may also vary, under different requirements and situations. It is easier for a decision maker to describe his/her desired value, and the importance of a criterion, by using common language. The purchaser may state, for example, that vendor support is considered ‘‘very important’’ in the selection of a data warehouse system, or that vendor A can provide ‘‘good’’ technical support during system implementation. Owing to the imprecise nature of software selection, there is a need to develop a multi-criteria decision-making method, based on fuzzy set theory. Fuzzy set theory was developed to address this exact premise, that the key elements in human thinking are not numbers, but linguistic terms or fuzzy set labels (Zadeh, 1965). Decision-making methods, using fuzzy set theory, have gradually gained acceptance, over the last decade. Fuzzy decision-making applications have also become more diverse, including such areas as technology transfer strategy selection in biotechnology (Chang & Chen, 1994), steel tool materials selection for manufacturing applications (Wang & Chang, 1995), material selection decisions in engineering design applications (Liao, 1996), performance evaluation of urban public transport systems (Yeh, Deng, & Chang, 2000), location selection of distribution centers (Chen, 2001), evaluating advanced manufacturing system investments (Karsak & Tolga, 2001), enhancing information delivery in extended enterprise networks (Lau et al., 2003), improving the configuration items selection process for flight simulator software development (Wang & Lin, 2003), supporting financial analysis in the corporate acquisition process (McIvor, McCloskey, Humphreys, & Maguire, 2004), selection of object-oriented simulation software for production system analysis (Cochran & Chen, 2005), and selection of material handling equipments (Kulak, 2005). Fuzzy set theory can play a significant role in this kind of vague decision-making environment. Previous studies focused on the development of some criteria for different decision-making applications. However, it did not explain how to construct a specific objectives structure relating to organizational strategies and how to extract the proper criteria for evaluating the fulfillment of the organizational requirements. Little research has addressed the issue of objectives structure for fuzzy-based decision-making applications. In this study, a fuzzy-based approach to assist decision makers in the systematic selection of the most suitable DW system is proposed. The systematic procedure can help to construct the objectives structure that considering organizational strategies and thus extract the associated attributes for evaluating DW systems, and facilitates the group decision-making process. The paper is organized as follows: in Section 2, a review of software selection methodology and framework is provided; the fuzzy-based decision-making approach is

described in Section 3. While in Section 4, the selection procedure for data warehouse systems is described; the application of the fuzzy method for data warehouse system selection, using a case study in Taiwan, is presented in Section 5. Finally, Section 6 concludes the study. 2. Software selection review Decision making in the field of software selection has become more complex due to a large number of software products in the market, ongoing improvements information technology, and multiple and sometimes conflicting objectives. Decision makers in diverse sectors have made software selection decisions. The proliferation of numerous software systems has created a difficult and complex decision problem for enterprises or users to select and evaluate the best software products. The following five factors make software selection and evaluation difficult and complex (McLoughlin, Rose, & Clark, 1985; Timmreck, 1973; Zahedi, 1985): (1) The tremendous number of software products available in the market. (2) The continual advancements and improvements in information technology. (3) The existence of incompatibilities between various hardware and software systems. (4) The functional dissimilarities are difficult to evaluate among software packages. (5) The users lack the technical knowledge and experience for software selection decision making. The literature reviewed was limited to software selection applications by used different methodologies and frameworks. The coverage is not exhaustive, however it demonstrates the diversity of software selection applications where decision making is used. Table 1 summarized major literature reviewed for software selection. A variety of methodologies and frameworks for software selection and evaluation have been developed. Le Blanc and Jelassi (1989) developed a multi-criteria decision methodology for decision support system (DSS) selection. The proposed methodology consisted of three stages: (1) DSS software screening; (2) DSS generator evaluation; (3) specific DSS design. Stylianou, Madey, and Smith (1992) presented a socio-technical framework and the taxonomy of expert system shells evaluation criteria. A field survey was conducted to identify the importance and critical of evaluation criteria. Boloix and Robillard (1995) proposed a comprehensive framework for software system evaluation. The framework had three dimensions: project, system and environment and each dimension had its own considered factors. Hluoic and Paul (1996) presented a methodology for manufacturing simulation software selection. The evaluation processes consisted of six stages: (1) need for purchasing simulation software; (2) initial software survey; (3) evaluation; (4) software selection; (5) software contract negotiation; (6) software purchase. Morisio


941

Table 1 A summary of the major literature reviewed for software selection Author(s)

Types of software system

Methodology/framework

Beck and Lin (1981) Seidmann and Arbel (1983) Seidmann and Arbel (1984) Zahedi (1985) Le Blanc and Jelassi (1989) Roper-Lowe and Sharp (1990) Zahedi (1990) Subramanian and Gershon (1991) Kim and Yoon (1992) Stylianou et al. (1992) Min (1992) Glassco (1993) Mohanty and Venkataraman (1993) Davis and Williams (1994) Boloix and Robillard (1995) Hluoic and Paul (1996) Kim and Moon (1997) Morisio and Tsoukia`s (1997) Carney and Wallnau (1998) Jones et al. (1999)

Automated office system Office automation software Accounting information system Database management system Decision support system Computer operating system Expert system CASE tool Expert system shell Expert system shell Logistics software Text search-and-retrieval package Automated manufacturing system Manufacturing simulation software CASE tool Manufacturing simulation software Workflow management system CASE tools COTS software Computer assisted learning software

Jung and Choi (1999) Lai et al. (1999) Nikoukaran et al. (1999) Ossadnik and Lange (1999) Vlahavas et al. (1999) Stamelos et al. (2000) Teltumbde (2000) Lai et al. (2002) Mamaghani (2002) Meira and Peres (2004) Phillips-Wren et al. (2004) Sarkis and Talluri (2004) Cochran and Chen (2005) Ngai and Chan (2005) Wei et al. (2005)

COTS software Multimedia authoring system Simulation software AHP software Expert system shell Specific information system ERP system Multimedia authoring system Antivirus and content filtering software Two types of educational software Decision support system Supply-chain software and e-commerce communication systems Object-oriented simulation software Knowledge management tool ERP system

AHP AHP AHP AHP Three stages methodology AHP AHP ELECTRE AHP Socio-technical framework AHP Checklist guidance AHP AHP Three dimensions framework Six stages methodology AHP Two stages methodology with ELECTRE Four basic principles framework Context, interactions, attitudes and outcomes (CIAO) model AHP with optimization model AHP Hierarchical framework AHP Expert system with ELECTRE Expert system with ELECTRE AHP and nominal group technique AHP and Delphi method AHP Dialogue-based approach AHP AHP and goal programming model Fuzzy-based approach AHP AHP

and Tsoukia`s (1997) proposed a methodology based on the multi-criteria decision aid approach and some activities such as comparison, assessment and selection of software products. The evaluation process consisted of two main stages: (1) design an evaluation model; (2) applying the model for software selection. Carney and Wallnau (1998) defined a framework for COTS software evaluation that consists of four basic principles. These four principles are: (1) evaluation is strongly related to decision making; (2) evaluation must accommodate the unavoidable element of uncertainty; (3) evaluation has a basis in design theory, especially in the relationship between form and context; (4) evaluation must be situated: particular candidate products for use in actual systems must be evaluated against specific evaluation criteria. Vlahavas, Stamelos, Refanidis, and Tsoukia`s (1999) developed an expert system based on various aspects of the multi-criteria decision aid approach for software evaluation. The expert system can facilitate the decision maker to choose the appropriate criteria according to the evaluation software and create the evaluation model. As Table 1 reviewed, the analytic hierarchy process (AHP) methodology is an excellent technique as it provides

a structure and hierarchy method for synthesizing software selection problems. The evidence of diversity in software system selection shows that the AHP can be applied to the selection of software products consistent with the maximization of the underlying criteria and sub-criteria expectations of the decision makers. The AHP methodology revealed some merits such as (1) structures a decision problem into a hierarchy helping to understand and simplify the problem; (2) the priority weights of criteria show the concerns and preferences of decision makers. However, the method has some disadvantages such as (1) scoring the relative importance among related criteria can be difficult when there are more than 7 of them; (2) oversimplifying the hierarchy may lose important interdependencies among criteria and overextending the hierarchy may increase time and complexity for creating pairwise comparison matrices; (3) the decision makers need to re-evaluate criteria or alternatives when the numbers of criteria or alternatives are changed; (4) sometimes, the managerial meanings of the global composite weights may be elusive. Although only one recent paper uses fuzzy-based approach to select software as shown in Table 1, the

942


fuzzy-based decision-making method has been successful employed on a great diversity of applications. The use of fuzzy set theory improves decision-making procedure by accommodating the vagueness and ambiguity occurred during human decision makings. The decision makers can use linguistic terms to evaluate criteria and alternatives easily and intuitively. Thus, the objective of this paper is to propose a comprehensive DW system selection procedure, in which the objectives structure is constructed and the appropriate criteria are specified to provide detailed guidance for DW system evaluation based on fuzzy set theory. Fig. 1. Membership function of a triangular fuzzy number.

3. The fuzzy-based decision-making approach In this section, a brief introduction to the basic concepts of fuzzy sets, algebraic operations, triangular fuzzy numbers, linguistic variables and ranking fuzzy numbers is presented. 3.1. Fuzzy sets Fuzzy set theory, involving the fuzziness of data, was introduced by Zadeh in 1965. It was developed to solve problems, in which descriptions of activities and observations were imprecise, vague and uncertain. A fuzzy set is a class of objects, with a continuum of membership grades, where the membership grade can be taken as an intermediate value between 0 and 1. A fuzzy subset A of a universal set X is defined by a membership function fA(x) which maps each element x in X to a real number [0, 1]. When the grade of membership for an element is 1, it means that the element is absolutely in that set. When the grade of membership is 0, it means that the element is absolutely not in that set. Ambiguous cases are assigned values between 0 and 1. The theory also allows mathematical operators such as addition, subtraction, multiplication and division, to be applied to the fuzzy sets (Dubois & Prade, 1979; Kaufmann & Gupta, 1988). 3.2. Triangular fuzzy numbers In this study, triangular fuzzy numbers are used as membership functions, corresponding to the elements in a set, as shown in Fig. 1. The reason for using a triangular fuzzy number is that it is intuitively easy for the decision makers to use and calculate. A fuzzy number is a triangular fuzzy number if its membership function can be denoted as follows (Kaufmann & Gupta, 1988): 8 xc < ac ; c 6 x 6 a bx fA ðxÞ ¼ ba ; a6x6b : 0; otherwise where a, b, and c are real numbers and c = a = b.

Zadeh’s extension principle can be used to calculate membership function after mapping fuzzy sets through a function. In this study, only addition and multiplication are used. Defining two triangular fuzzy numbers A1 and A2 by the triplets as A1 = (c1, a1, b1) and A2 = (c2, a2, b2), the addition and multiplication operations of A1 and A2 can be expressed as follows: Addition: if denotes addition. A1 A2 : ðc1 ; a1 ; b1 Þ ðc2 ; a2 ; b2 Þ ¼ ðc1 þ c2 ; a1 þ a2 ; b1 þ b2 Þ Multiplication: if denotes multiplication. A1 A2 : ðc1 ; a1 ; b1 Þ ðc2 ; a2 ; b2 Þ ¼ ðc1 c2 ; a1 a2 ; b1 b2 Þ;

c1 P 0; c2 P 0

3.3. Linguistic terms in triangular fuzzy numbers Fuzzy set theory is primarily concerned with quantifying the vagueness in human thoughts and perceptions, where linguistic terms can be properly represented by the approximate reasoning of fuzzy set theory. The importance weights of various criteria and the rating values of DW alternatives are considered as linguistic terms throughout this paper. A linguistic term can be defined as a variable whose values are not numbers, but words or sentences in natural language. The importance weight can be evaluated by linguistic terms such as very low, low, medium, high and very high. These linguistic terms can be expressed via triangular fuzzy numbers, as shown in Table 2, while the membership functions of the five linguistic values are shown in Fig. 2. In order to determine the suitability of data warehouse alternatives, versus technical and managerial criteria, the rating values can be accessed by linguistic terms, such as

Table 2 Linguistic terms for the importance weight of each criterion Linguistic term

Very low (VL)

Low (L)

Medium (M)

High (H)

Very high (VH)

Membership function

(0, 0, 0.3)

(0, 0.3, 0.5)

(0.2, 0.5, 0.8)

(0.5, 0.7, 1)

(0.7, 1, 1)


Fig. 2. Membership functions for importance weight of criteria defined in Table 2.

very poor, poor, fair, good and very good. These linguistic terms can be expressed by triangular fuzzy numbers, as shown in Table 3, while the membership functions of the five linguistic values are shown in Fig. 3. These membership functions have been used in different areas of application, such as robot selection (Liang & Wang, 1993), measuring manufacturing competence (Azzone & Rangone, 1996), material selection (Liao, 1996), evaluating advanced manufacturing system investments (Karsak & Tolga, 2001), and selection of objectoriented simulation software (Cochran & Chen, 2005). Triangular membership functions are used in this paper because of their intuitive representation and ease in calculation. 3.4. A fuzzy algorithm for data warehouse system selection A systematic approach to the data warehouse system selection problem, based on fuzzy set theory and multi-criteria decision analysis, is described in this section. Many methods have been proposed to combine the opinions of decision makers such as mean, median, max, min and mixed operators (Buckley, 1984). Since the average operation is the most commonly used aggregation method (Chang & Chen, 1994; Chen, 2001; Cochran & Chen, 2005; Wang & Chang, 1995), in this study, the mean operator was used to aggregate the assessments of decision makers. For a data warehouse system selection decisionmaking problem, there are a group of n decision makers (D1, D2, . . . , Dn), who evaluate the importance weights of k criteria (C1, C2, . . . , Ck) and the appropriateness of m DW alternatives (A1, A2, . . . , Am), under each of these k criteria. Let Wtd (t = 1, 2, . . . , k; d = 1, 2, . . . , n) be the weight given to Ct by decision maker Dn. Let Ritd (i = 1, 2, . . . , m; t = 1, 2, . . . , k; d = 1, 2, . . . , n) be the rating assigned to alternative Ai by decision maker Dn under criterion Ct. Wt and Rit are defined as follows:

943

Fig. 3. Membership functions for linguistic rating values defined in Table 3.

Wt ¼

n 1 1X W td ðW t1 W t2 W tn Þ ¼ n n d¼1

ð1Þ

n 1 1X Ritd ðRit1 Rit2 Ritn Þ ¼ n n d¼1

ð2Þ

and Rit ¼

where Wt is the average importance weight of criterion Ct and Rit is the aggregated rating of alternative Ai under criterion Ct. After the weights and ratings have been aggregated, each aggregated rating of alternative Ai and criterion Ct(Rit) can further be weighed by the aggregated weight (Wt) to obtain the final rating Fi, i.e. the fuzzy appropriateness index of each alternative Ai. The Fi can be obtained by aggregating Rit and Wt, denoted as 1 Fi ¼ ½ðRi1 W 1 Þ ðRi2 W 2 Þ ðRik W k Þ k X 1 k ðRij W j Þ ð3Þ ¼ k j¼1 3.5. Ranking fuzzy numbers The prioritization of the aggregated assessments must be ranked among alternatives. Since the aggregated assessments are represented as triangular fuzzy numbers, a method of ranking these fuzzy triangular numbers is required. There are several methods of ranking fuzzy numbers (Bortolan & Degani, 1985; Buckley & Chanas, 1989; Chen, 2001; Chen, 1985; Kim & Park, 1990; Lau et al., 2003; Liou & Wang, 1992). In this paper, the maximizing set and minimizing set methods (Chen, 1985) were applied, because of ease of use and application in previous studies (Cochran & Chen, 2005; Karsak & Tolga, 2001; Wang & Chang, 1995; Yeh et al., 2000).

Table 3 Linguistic terms for rating Linguistic term

Very poor (VP)

Poor (P)

Fair (F)

Good (G)

Very good (VG)

Membership function

(0, 0, 0.2)

(0, 0.2, 0.4)

(0.3, 0.5, 0.7)

(0.6, 0.8, 1)

(0.8, 1, 1)

944


Let Fi (i = 1, 2, . . . , m) be the fuzzy appropriateness index values of m alternatives. Chen (1985) defined the maximizing set M ¼ fðx; fM ðxÞÞ j x 2 Rg with 8 < ðx xmin Þ ; x 6 x 6 x min max ð4Þ fM ðxÞ ¼ ðxmax xmin Þ : 0; otherwise and minimizing set G ¼ fðx; fG ðxÞÞ j x 2 Rg with 8 < ðx xmax Þ ; x 6 x 6 x min max fG ðxÞ ¼ ðxmin xmax Þ ð5Þ : 0; otherwise Sm where xmin ¼inf S; xmax ¼supS; S ¼ i¼1 F i ; F i ¼fxjfFi ðxÞ>0g; i¼1;2;...;m. Further, the right utility value UM(Fi) and the left utility value UG(Fi) for alternative i are defined as U M ðF i Þ ¼ supðfFi ðxÞ \ fM ðxÞÞ;

i ¼ 1; 2; . . . ; m

ð6Þ

U G ðF i Þ ¼ supðfFi ðxÞ \ fG ðxÞÞ;

i ¼ 1; 2; . . . ; m

ð7Þ

and the total utility or ordering value for alterative i is U M ðF i Þ þ 1 U G ðF i Þ ð8Þ 2 The alternative with maximum UT(Fi) value is the optimal choice in the decision-making problem. U T ðF i Þ ¼

4. Procedure for selecting a data warehouse system In this section, we have developed a fuzzy-based decisionmaking procedure for the selection of a data warehouse system. The approach comprises a nine-step procedure as shown in Fig. 4. The details of the selection procedure are presented in next section alone with the case study. 4.1. Group a committee of decision makers The first step is to form a project team that consists of decision makers, functional experts and senior representatives of user departments. The participation and support of top managers significantly influences the success of DW adoption (Hwang, Ku, Yen, & Cheng, 2004). A wide range of information that including international and local DW products and vendors should be collected from the Internet, professional magazines and marketing research report. 4.2. Identify the DW project characteristics Different enterprises or organizations may adopt a data warehouse system for completely different reasons. The size of company, internal needs and competitive pressure would also influence the adoption of DW systems (Hwang et al., 2004). The initial intention or purpose for adopting a DW system affects problem definition, identifying and structuring objectives, measuring the achievement of objectives, and other subsequent decision-making activities. The decision makers need to analyze the DW system selection problem by identifying decision factors such as stakehold-

ers, project objectives, evaluation criteria, number of alternatives, and other concerns in order to ensure the decisionmaking process effectively and efficiently. 4.3. Construct objectives structure The main purpose of identifying and structuring objectives is to provide insight for better decisions. The initial list of objectives for a decision problem includes both fundamental objectives and means objectives (Keeney, 1994). Structuring the objectives involves organizing them so that the decision makers can describe in detail what an organization wants to achieve, and then incorporate these objectives into the decision model appropriately. For a given decision situation, the overall objective is the same for both the fundamental and means objectives structure. It characterizes the reason for interest and defines the breadth of concern. All relating objectives on different levels of a structure will be derived from the overall objective systematically. It is important to distinguish between fundamental objectives and means objectives in the objectives structuring process. Fundamental objectives are those that are important simply because they reflect what the decision makers are really want to accomplish. However, means objectives are those that help achieve other objectives (Clemen & Reilly, 2001). Fundamental objectives can be organized into hierarchies while means objectives can be organized into networks (Clemen & Reilly, 2001). The upper levels in a hierarchy represent more general objectives, and the lower level objectives explain what is meant by the higher level objectives. A key difference between fundamental objectives hierarchy and means objectives network is that means objectives can be connected to several fundamental objectives, indicating that they help to accomplish these objectives. To separate means objectives from fundamental objectives and to establish their relationships, a guiding question ‘‘Why is that important?’’ can be used to accomplish these tasks (Keeney, 1994). For each identified objective, ask, ‘‘Why is that important?’’ Two types of answers seem possible. If the answer is that the objective is one of the essential reasons for interest in the situation, such an objective is a fundamental objective. If the response is that the objective is important only because of its implications for some other objective, it is a means objective. Clemen and Reilly (2001) provided four techniques to organize fundamental and means objectives further. Table 4 summarizes these techniques. 4.4. Extract the attributes for selecting DW systems The decision makers can derive the attributes or criteria to evaluate DW systems from the created objectives structure. The attributes should involve both quantitative (tangible) and qualitative (intangible) measures that satisfy the objectives of project and requirements of an organization. The decision makers should examine and modify the set


945

Fig. 4. The data warehouse system selection procedure.

Table 4 How to construct fundamental objectives hierarchy and means objectives network Fundamental objectives

Means objectives

To Move: Ask:

Downward in the hierarchy ‘‘What do you mean by that?’’

Away from fundamental objectives ‘‘How could you achieve this?’’

To Move: Ask:

Upward in the hierarchy ‘‘What more general objective is this part of?’’

Toward fundamental objectives ‘‘Why is that important?’’

of selected attributes iteratively, so that they are complete, nonredundant, measurable and minimal (Keeney & Raiffa,

1993). Then, these attributes were verified with external professional experts to ensure all attributes were well

946


formulated and properly understood. The final selected attributes will be used to evaluate the DW systems of the decision model. 4.5. Identify DW alternatives and invite vendors to demonstrate In order to identify the final DW alternatives for evaluation from numerous alternatives collected initially, the decision makers need to shorten the list of DW candidates. The detailed and suitable features of DW systems were transferred to specific requirements to from a questionnaire or checklist of systems specifications. The fundamental objectives hierarchy and means objective network can help to review system specifications and ensure that these requirements match with organizational objectives. The final identified vendors are requested to provide information in response to the specific questions included in the checklist and demonstrate the real systems. The decision makers then assess the information and demonstration to evaluate the DW alternatives. 4.6. Evaluate the DW systems by the fuzzy-based decision-making approach The fuzzy-based approach involves five phases as shown in Fig. 4. In this proposed method, the weight of each criterion and the rating of each alternative are described using linguistic terms, which can also be expressed as triangular fuzzy numbers. The fuzzy algorithm aggregated the decision-makers’ preference rating for criteria, and the suitability of data warehouse alternatives versus the selection criteria, to calculate fuzzy appropriateness indices, through which, the most suitable data warehouse system was determined. In the first phase, the decision makers need to choose the appropriate linguistic terms and membership functions for measuring the importance weights of various criteria and the rating values of DW alternatives. In the second phase, the decision makers then used the importance weighting set W and appropriateness rating set R, i.e. W = {Very low, Low, Medium, High, Very high}, R = {Very poor, Poor, Fair, Good, Very good}, to evaluate the importance weight of the selection criteria and the preference rating of the alternatives versus the criteria. In the third phase, to calculate the aggregate weights for each criterion and the average ratings of alternatives with computer assisted. In the fourth phase, to aggregate the values obtained from the third phase to get the fuzzy appropriateness index values for all alternatives. Finally, to compute the ranking value for all alternatives, the DW alternative with the maximal ranking value is the final choice. 5. An illustrative example The fuzzy decision-making procedure, presented in the previous section, is illustrated by a case study. Owing to the competition involved in joining the World Trade Orga-

nization (WTO), Taiwan has faced challenges in the area of agricultural products. For Taiwan to retain its competitive advantage, it must reduce production costs and increase sales of agricultural products. Hence, the Council of Agriculture delegated one County Farmers’ Association (CFA) located in the middle part of Taiwan to execute a threeyear project: the Bar Code Implementation Project for Agricultural Products. The aims of this project are to computerize transaction processes and integrate the supply chain of agricultural products. These agricultural products presently include vegetables, fruits and flowers, but will cover other items in the future. It is expected that about 80% of Taiwan’s agricultural products transactions will be involved in this project. According to the government contract, the system must be able to store 10 years of transaction data for data analysis, forecasting and products tracking. The data volume is estimated to be more than one terabytes. So, the establishment of a data warehouse system is essential. Before implementing a data warehouse system, the CFA must to evaluate the most suitable DW system for the project. The following steps illustrate the data warehouse system selection process. Step 1: Group a committee of decision makers In order to select the most suitable DW system for the Bar Code Implementation Project for Agricultural Products, six experts with IT and business backgrounds, including one acting as collaborating principal director of the project, formed a committee to make the selection decision. The six decision makers had an average of 9.5 years work experience. Step 2: Identify the DW project characteristics Owing to the complexity and large scope of the Bar Code Implementation Project for Agricultural Products, the committee of decision makers identified the DW project characteristics by spent about one month to visit many organizations involved in this project. There are several types of users belonging to different agricultural organizations widespread in Taiwan. These organizations consist of Council of Agriculture, Agriculture and Food Agency, Fruits & Vegetables Wholesale Markets, Farmers’ Associations, and Agriculture Production and Marketing Groups. The users of different units have dissimilar data analysis requirements. Then, the decision makers discussed the goals of DW system implementation, the project scope, organizational resources, potential alternatives, and other concerns. This process was reviewed iteratively in later meetings to ensure that the project characteristic reacted readily to the changing of business environment. Steps 3 and 4: Construct objectives structure The structure of the objectives was constructed after three formal meetings within two weeks. Fig. 5 illustrated the fundamental objectives hierarchy. The ultimate goal is to select the most suitable DW system for the project. The overall goal is separated into two lower-level (second level) objectives; namely, choose the most appropriate DW system and best DW vendor. The decision makers


947

Fig. 5. Fundamental objectives hierarchy of DW system selection.

then discussed ‘‘What does choose the most appropriate DW system mean?’’ It means that the DW system has to match with existing infrastructure, provides complete functionality, has user-friendly interfaces, supports excellent system flexibility, and with minimal overall cost. For the objective of choosing the best DW vendor, it means that the vendor should have good reputation and stability, and provides good technical capability and ongoing supports. These constitute the third level of the fundamental objectives hierarchy. Further, decision makers asked ‘‘What does match with existing infrastructure mean?’’ To drill down in the objective, the answer lay in integration with software systems and hardware platforms which forms the fourth level of the hierarchy. Similarly, the topdown decomposition method can help to decompose the other objectives in the third level. The fourth level shows the details that can be used to evaluate the performance of DW systems.

It is insightful to relate the means objectives to the fundamental objectives. The means objectives networks for system and vendor factors were formulated simultaneously. The interrelationships are displayed in the network in Figs. 6 and 7, respectively, where an arrow from one objective to another indicates that achieving the former objective has a major influence on (or is a means to) achieving the latter. The decision makers started from the bottom objectives of the fundamental objectives hierarchy by asking ‘‘How could you achieve this?’’ to identify the means objectives and establish links among them. For example, in Fig. 6, the answers to the question, ‘‘How can minimize the initial purchasing cost be achieved?’’ were ‘‘minimizing the hardware cost’’, ‘‘minimizing the software cost’’, and ‘‘minimizing the consultant fee’’. Following a similar approach, the decision makers elaborated the interrelationships among all the means objectives into a complete network. During this process, the decision makers needed to repeatedly

948


Fig. 6. Means objectives network of DW system selection: system factor.

examine all means objectives linkages in order to verify every relationship was reasonable. Step 5: Extract the attributes for selecting DW systems It may be impractical to make assessment among the DW alternatives with respect to every detailed of the fundamental objectives hierarchy in Fig. 5. The difficulty arises because too many attributes or criteria lead to numerous evaluations of the importance weight of the selection criteria and the preference rating of the alternatives versus the

criteria, and cause an inefficient decision-making process. The decision makers discussed to shorten the list of criterion candidates with their subjective opinions. The fundamental objectives hierarchy was modified to generate a list of criteria and the wording was changed to assess its logical consistencies, ease of understanding, sequence of items and DW relevance. The means objectives network can help to establish the evaluation measures and specific requirements. This process can ensure that every decision


949

Fig. 7. Means objectives network of DW system selection: vendor factor.

maker follows the same criteria consistently in the later evaluation process. Then, these criteria were verified with four consultants from four data warehousing vendors by interviews, to ensure all criteria were well formulated and properly understood. These respondents had an average of 5.3 years experience in data warehousing projects. The descriptions of the final selection criteria are given in Table 5. Steps 6 and 7: Identify DW alternatives and invite vendors to demonstrate The committee identified three prospective DW alternatives, according to the marketing research report (Strange, 2004). These alternatives were included among the leading DW products in the global market and all the relevant vendors have branch offices in Taiwan. Only top international DW systems were considered owing to the project scope

and huge data volume. Before evaluating the ratings of alternatives, the committee invited three vendors to present and demonstrate their DW products. The selection criteria and a detailed explanation were sent to the three vendors via e-mail. Then, after two weeks, each vendor was given about three hours to address the selection criteria and formally demonstrate the DW system. Step 8: Evaluate the DW systems by the fuzzy-based decision-making approach (1) The decision makers then used the importance weighting set W and appropriateness rating set R, described in Section 3.3, i.e. W = {Very low, Low, Medium, High, Very high}, R = {Very poor, Poor, Fair, Good, Very good}, to evaluate the importance weight of the selection criteria and the preference

950


Table 5 Data warehouse system selection criteria Criteria

Description

C1: C2: C3: C4: C5:

Display interface determines how user-friendly the software is and how flexible the users can analyze data Analysis tools provide interfaces for end users to analyze data stored in the data warehouse system Query functionality stress the software provides a variety of query design tools and supports different kinds of querying Data warehouse software should be compatible with the existing information systems and infrastructure The integration issue examines if the data warehouse system can be integrated with heterogeneous source systems, other decision support systems and third party tools The proposed data warehouse may be implemented on a standard or extended relational database system, a multidimensional database, or hybrid The data staging process collects operational source data and integrates the data into the data warehouse. The ETL tools can help the extraction, transformation and load processes, automatically Data quality impacts the credibility of a data warehouse. To ensure quality data in the warehouse, the data gathering process and full lifecycle of data warehouse must be well managed Metadata maintenance can influence the entire data warehouse, from the initial model through data extraction and load processes to the users’ exploration and access Data warehouse administration tools help system administrators manage data warehouse operations throughout its operational life cycle Direct costs consisting of the software purchase (licensing fee), the cost of the hardware, as well as the external consulting fee to implement the system Indirect costs include the necessary components to allow the users to become familiar with the system, such as training costs, maintenance costs and system upgrade costs The commitments of vendors to their data warehouse products and to continuous improvements will affect the longevity of such products Data warehouse implementation and operations, which will continue into the foreseeable future, require vendor stability, in order to obtain ongoing service and support In view of ongoing data warehouse system operations, reliable vendor support is crucial. The value of a system is dependent on having long-term technical assistance and sustained service support External support contributes to the data warehouse implementation process. Reliable vendors have in-depth experience in all aspects of system implementation

Display interface Analysis tools Query functionality Compatibility Integration

C6: Database support C7: ETL functionality C8: Data quality checks C9: Metadata management C10: DW administration C11: Direct cost C12: Indirect cost C13: Vendor reputation C14: Vendor stability C15: Vendor support C16: Vendor experience

rating of the alternatives versus the criteria. The linguistic terms and membership functions are listed in Tables 2 and 3. (2) From Steps 1, 5 and 6, we know that the six decision makers (D1, D2, . . . , D6) based their decisions on the 16 selection criteria (C1, C2, . . . , C16), when choosing the most suitable DW system among the three alternatives (A1, A2, A3). The weights assigned to the 16 criteria by the six decision makers are given in Table 6. The suitability of the three alternatives, under each of the criteria given by the decision makers, is presented in Table 7. (3) Through triangular fuzzy number aggregation by Eq. (1), the aggregated weights (Wt), of the 16 criteria, determined by the six decision makers, were obtained; this is shown in Table 8. By using Eq. (2), the average fuzzy appropriateness rating (Rit) of alternatives Ai under each criterion Ct could be obtained, as shown in Table 9. For example, the aggregated weight of criterion C1 (display interface) was obtained as follows: W1 ¼

¼

1 ½ð0:2; 0:5; 0:8Þ ð0:7; 1; 1Þ ð0:5; 0:7; 1Þ 6 ð0:5; 0:7; 1Þ ð0:7; 1; 1Þ ð0:5; 0:7; 1Þ 1 ð3:1; 4:6; 5:8Þ ¼ ð0:52; 0:77; 0:97Þ 6

Table 6 Linguistic assessment results for criteria Criteria

Decision makers D1

D2

D3

D4

D5

D6

C1: Display interface C2: Analysis fools C3: Query functionality C4: Compatibility C5: Integration C6: Database C7: ETL functionality C8: Data quality checks C9: Metadata management C10: DW administration C11: Direct costs C12: Indirect costs C13: Vendor reputation C14: Vendor stability C15: Vendor support C16: Vendor experience

M M M L M VL M H M H VH VH H H H L

VH M VH VL L H H H H H VH VH H H H H

H VH VH H VH H H H VH VH H H H VH H H

H VH H H M L VH H H VH H VH M L VH VH

VH H H H H H H VH H VH VH VH H H VH VH

H VH VH VH VH VH VH VH VH H H H M VH VH H

In addition, the average fuzzy appropriateness rating (R11) of alternatives A1 under criterion C1 can be obtained as follows: 1 R11 ¼ ½ð0:8; 1; 1Þ ð0:8; 1; 1Þ ð0:6; 0:8; 1Þ 6 ð0:6; 0:8; 1Þ ð0:6; 0:8; 1Þ ð0:6; 0:8; 1Þ 1 ¼ ð4; 5:2; 6Þ ¼ ð0:67; 0:87; 1Þ 6


951

Table 7 Linguistic assessment results for alternatives under each criterion A2

Alternatives

A1

Criteria

Decision makers

C1: Display interface C2: Analysis tools C3: Query functionality C4: Compatibility C5: Integration C6: Database C7: ETL functionality C8: Data quality checks C9: Metadata management C10: DW administration C11: Direct costs C12: Indirect costs C13: Vendor reputation C14: Vendor stability C15: Vendor support C16: Vendor experience

A3

D1

D2

D3

D4

D5

D6

D1

D2

D3

D4

D5

D6

D1

D2

D3

D4

D5

D6

VG VG VG G G F VG F VP VP VP VP G G F G

VG G VG G VG VG VG VG VG VG F F VG VG VG VG

G G G G G G G G G G G G VG VG VG G

G VG G VG VG G VG G G F P P VG VG VG G

G G G G F G G G F F F F G G G G

G G VG VG G VG G VG G VG G F VG VG G G

F F F P F G F F F F VG VG F F F F

VG G VG G VG VG VG VG VG VG VG VG F F F F

G G G VG VG G G G G G VG VG G G VG G

VG VG G F F G G G VG G G VG F VG VG VG

F F F F F G F G F G VG VG F F F F

G F VG F F G G F F G VG G G G F F

F F VG VG VG G G G G VG F F G G G G

G F VG VG VG VG VG VG VG VG F G G G G G

G G G G G G G G G G F G G VG VG G

G G VG VG G VG VG VG VG VG P F G VG VG G

G F VG VG G G G VG F G F F G G G G

G G VG VG G VG VG VG G G F F VG VG VG VG

Table 8 The aggregated weights (Wt) of criteria Aggregated weights (Wt)

Fuzzy criteria weights

W1: Display interface W2: Analysis tools W3: Query functionality W4: Compatibility W5: Integration W6: Database W7: ETL functionality W8: Data quality checks W9: Metadata management W10: DW administration W11: Direct costs W12: Indirect costs W13: Vendor reputation W14: Vendor stability W15: Vendor support W16: Vendor experience

(0.52, 0.77, 0.97) (0.5, 0.78, 0.93) (0.55, 0.82, 0.97) (0.37, 0.57, 0.8) (0.38, 0.67, 0.85) (0.37, 0.57, 0.8) (0.52, 0.77, 0.97) (0.57, 0.8, 1) (0.52, 0.77, 0.97) (0.6, 0.85, 1) (0.6, 0.85, 1) (0.63, 0.9, 1) (0.4, 0.63, 0.93) (0.48, 0.73, 0.92) (0.6, 0.85, 1) (0.48, 0.73, 0.92)

(4) By using Eq. (3), the aggregation can be achieved by averaging the alternatives over all the criteria; the results of the fuzzy appropriateness index (Fi) values are shown in Table 10. For example, the fuzzy appropriateness index value of alternative A1 can be obtained as follows: " # ðð0:67; 0:87; 1Þ ð0:52; 0:77; 0:97ÞÞ ðð0:67;0:87;1Þ ð0:5; 0:78; 0:93ÞÞ 1 16 ðð0:7;0:9;1Þ ð0:55; 0:82;0:97ÞÞ ðð0:63; 0:83;1Þ ð0:48; 0:74;0:92ÞÞ " # ð0:3484; 0:6699; 0:97Þ ð0:335; 0:6786; 0:93Þ ð0:385;0:738; 0:97Þ 1 ¼ 16 ð0:3024; 0:6059; 0:92Þ

F1 ¼

¼

1 ð4:7026; 9:3549; 13:7329Þ ¼ ð0:2939; 0:5847; 0:8583Þ 16

(5) By using the ranking method described in Section 3.5, i.e. Eqs. (4)–(8), the ranking values of the three alternatives’ fuzzy appropriateness indices can be obtained as shown in Table 11.

Table 9 The average fuzzy appropriateness rating (Rit) of alternatives Criteria

Rit(A1)

Rit(A2)

Rit(A3)

C1: Display interface C2: Analysis tools C3: Query functionality C4: Compatibility C5: Integration C6: Database C7: ETL functionality C8: Data quality checks C9: Metadata management C10: DW administration C11: Direct costs C12: Indirect costs C13: Vendor reputation C14: Vendor stability C15: Vendor support C16: Vendor experience

(0.67, 0.87, 1) (0.67, 0.87, 1) (0.7, 0.9, 1) (0.67, 0.87, 1) (0.62, 0.82, 0.95) (0.62, 0.82, 0.95) (0.7, 0.9, 1) (0.62, 0.82, 0.95) (0.48, 0.65, 0.82) (0.47, 0.63, 0.77) (0.3, 0.47, 0.67) (0.25, 0.42, 0.62) (0.73, 0.93, 1) (0.73, 0.93, 1) (0.65, 0.85, 0.95) (0.63, 0.83, 1)

(0.57, 0.77, 0.9) (0.48, 0.68, 0.85) (0.57, 0.77, 0.9) (0.38, 0.58, 0.75) (0.47, 0.67, 0.8) (0.63, 0.83, 1) (0.53, 0.73, 0.9) (0.53, 0.73, 0.9) (0.52, 0.72, 0.85) (0.58, 0.78, 0.95) (0.77, 0.97, 1) (0.77, 0.97, 1) (0.4, 0.6, 0.8) (0.48, 0.68, 0.85) (0.47, 0.67, 0.8) (0.43, 0.63, 0.8)

(0.55, 0.75, 0.95) (0.45, 0.65, 0.85) (0.77, 0.97, 1) (0.77, 0.97, 1) (0.67, 0.87, 1) (0.7, 0.9, 1) (0.7, 0.9, 1) (0.73, 0.93, 1) (0.62, 0.82, 0.95) (0.7, 0.9, 1) (0.25, 0.45, 0.65) (0.4, 0.6, 0.8) (0.63, 0.83, 1) (0.7, 0.9, 1) (0.7, 0.9, 1) (0.63, 0.83, 1)

952


Table 10 The fuzzy appropriates index of the three alternatives Alternatives

Fuzzy appropriateness index

A1 A2 A3

F1 (0.2939, 0.5847, 0.8583) F2 (0.2765, 0.5608, 0.8266) F3 (0.3105, 0.6147, 0.8902)

Table 11 The ranking values of the fuzzy appropriateness indices for alternatives Alternatives

Fi

UM(Fi)

UG(Fi)

UT(Fi)

A1 A2 A3

(0.2939, 0.5847, 0.8583) (0.2765, 0.5608, 0.8266) (0.3105, 0.6147, 0.8902)

0.6557 0.6255 0.6902

0.6593 0.6834 0.6316

0.4982 0.4710 0.5293

Step 9: Make the final decision From Table 11, the ranking order of fuzzy appropriateness indices for the three alternatives is UT(F3), UT(F1), and UT(F2). Hence, it is obvious that the most appropriate data warehouse system is A3. Thus, the committee can be comfortable in recommending alternative A3 as the most suitable data warehouse system for the Bar Code Implementation Project for Agricultural Products. 6. Conclusions In this paper, a simple, easy to use and systematic procedure, based on the fuzzy set theory, has been proposed to aid in the selection among alternatives, with several decision criteria. The applicability of this procedure was illustrated through a case study of data warehouse system selection for the Bar Code Implementation Project for Agricultural Products in Taiwan The procedure used objectives structure, fuzzy set theory and fuzzy algebraic operations, to solve the decision-making problem of choosing among DW alternatives, using ranking based on linguistic assessment. Although the case study was related to a specific software system and industry, the same concept can be applied to other software products and industrial sectors. The use of fuzzy set theory improves the decision-making procedure by considering the vagueness and ambiguity prevalent in real-world systems. We also found that using triangular fuzzy numbers made data collection, calculation and interpretation of the results easier for decision makers. Further, the proposed method can be computerized. By implementing fuzzy linguistic assessments on a computer, decision makers can automatically obtain the ranking order of alternatives. References Azzone, G., & Rangone, A. (1996). Measuring manufacturing competence: A fuzzy approach. International Journal of Production Research, 34(9), 2517–2532. Beck, M. P., & Lin, B. W. (1981). Selection of automated office systems: A case study. OMEGA, 9(2), 169–176. Boloix, G., & Robillard, P. N. (1995). A software system evaluation framework. IEEE Computer, 28(12), 17–26.

Bortolan, G., & Degani, R. (1985). A review of some methods for ranking fuzzy subsets. Fuzzy Sets and Systems, 15, 1–19. Buckley, J. J. (1984). The multiple judge, multiple ranking problem: A fuzzy set approach. Fuzzy Sets and Systems, 13, 25–37. Buckley, J. J., & Chanas, S. (1989). A fast method of ranking alternative using fuzzy numbers. Fuzzy Sets and Systems, 30, 337–338. Carney, D. J., & Wallnau, K. C. (1998). A basis for evaluation of commercial software. Information and Software Technology, 40(14), 851–860. Chang, P. L., & Chen, Y. C. (1994). A fuzzy multi-criteria decision making method for technology transfer strategy selection in biotechnology. Fuzzy Sets and Systems, 63, 131–139. Chen, C. T. (2001). A fuzzy approach to select the location of the distribution center. Fuzzy Sets and Systems, 118, 65–73. Chen, S. H. (1985). Ranking fuzzy numbers with maximizing set and minimizing set. Fuzzy Sets and Systems, 17, 113–129. Clemen, T. R., & Reilly, T. (2001). Making hard decisions with decisiontools. Pacific Groce, CA: Duxbury. Cochran, J. K., & Chen, H. (2005). Fuzzy multi-criteria selection of object-oriented simulation software for production system analysis. Computers and Operations Research, 32(1), 153–168. Davis, L., & Williams, G. (1994). Evaluation and selecting simulation software using the analytic hierarchy process. Integrated Manufacturing Systems, 5(1), 23–32. Dubois, D., & Prade, H. (1979). Fuzzy real algebra: Some results. Fuzzy Sets and Systems, 2, 327–348. Glassco, R. A. (1993). Evaluating commercial text search-andretrieval packages. Information Technology and Libraries, 12(4), 413–421. Gorla, N. (2003). Features to consider in a data warehousing system. Communications of the ACM, 46(11), 111–115. Hluoic, V., & Paul, R. J. (1996). Methodological approach to manufacturing simulation software selection. Computer Integrated Manufacturing Systems, 9(1), 49–55. Hwang, H. G., Ku, C. Y., Yen, D. D., & Cheng, C. C. (2004). Critical factors influencing the adoption of data warehouse technology: A case study of the banking industry in Taiwan. Decision Support Systems, 37, 1–21. Jones, A., Scanlon, E., Tosunoglu, C., Morris, E., Ross, S., Butcher, P., et al. (1999). Contexts for evaluating educational software. Interacting with Computers, 11(5), 499–516. Jung, H., & Choi, B. (1999). Optimization models for quality and cost of modular software systems. European Journal of Operational Research, 112(3), 613–619. Karsak, E. E., & Tolga, E. (2001). Fuzzy multi-criteria decision-making procedure for evaluating advanced manufacturing system investments. International Journal of Production Economics, 69, 49–64. Kaufmann, A., & Gupta, M. M. (1988). Fuzzy mathematical models in engineering and management science. Netherlands: Elsevier Science Publisher. Keeney, R. L., & Raiffa, H. (1993). Decisions with multiple objectives: Preferences and value tradeoffs. NY: Cambridge University Press. Keeney, R. L. (1994). Creativity in decision marking with valued-focused thinking. Sloan Management Review, 35(4), 33–41. Kim, C. S., & Yoon, Y. (1992). Selection of a good expert system shell for instructional purposes in business. Information and Management, 23(5), 249–262. Kim, J., & Moon, J. Y. (1997). An AHP and survey for selecting workflow management systems. International Journal of Intelligent Systems in Accounting, Finance,and Management, 6, 141–161. Kim, K., & Park, K. S. (1990). Ranking fuzzy numbers with index of optimism. Fuzzy Sets and Systems, 35, 143–150. Kimball, R., Reeves, L., Ross, M., & Thornthwaite, W. (1998). The data warehouse lifecycle toolkit—expert methods for designing, developing, and deploying data warehouses. NY: John Wiley & Sons. Kulak, O. (2005). A decision support system for fuzzy multi-attribute selection of material handling equipments. Expert Systems with Applications, 29(2), 310–319.

H.-Y. Lin et al. / Expert Systems with Applications 32 (2007) 939–953 Lai, V. S., Trueblood, R. P., & Wong, B. K. (1999). Software selection: A case study of the application of the analytic hierarchical process to the selection of a multimedia authoring system. Information and Management, 36(4), 221–232. Lai, V. S., Wong, B. K., & Cheung, W. (2002). Group decision making in a multiple criteria environment: A case using the AHP in software selection. European Journal of Operational Research, 137(1), 134–144. Lau, H. C. W., Wong, C. W. Y., Lau, P. K. H., Pun, K. F., Chin, K. S., & Jiang, B. (2003). A fuzzy multi-criteria decision support procedure for enhancing information delivery in extended enterprise networks. Engineering Applications of Artificial Intelligence, 16, 1–9. Le Blanc, L. A., & Jelassi, M. T. (1989). DSS software selection: A multiple criteria decision methodology. Information and Management, 17(1), 49–65. Liang, G. S., & Wang, M. J. J. (1993). A fuzzy multi-criteria decisionmaking approach for robot selection. Robotics and Computer-Integrated Manufacturing, 10(4), 267–274. Liao, T. W. (1996). A fuzzy multicriteria decision making method for material selection. Journal of Manufacturing Systems, 15(1), 1–12. Liou, T. S., & Wang, M. J. J. (1992). Ranking fuzzy numbers with integral value. Fuzzy Sets and Systems, 50, 247–255. Mamaghani, F. (2002). Evaluation and selection of an antivirus and content filtering software. Information Management and Computer Security, 10(1), 28–32. McIvor, R. T., McCloskey, A. G., Humphreys, P. K., & Maguire, L. P. (2004). Using a fuzzy approach to support financial analysis in the corporate acquisition process. Expert Systems with Applications, 27(4), 533–547. McLoughlin, I., Rose, H., & Clark, J. (1985). Managing the introduction of new technology. OMEGA, 13(4), 251–262. Meira, L., & Peres, F. (2004). A dialogue-based approach for evaluating educational software. Interacting with Computers, 16(4), 615–633. Min, H. (1992). Selection of software: The analytic hierarchy process. International Journal of Physical Distribution and Logistics Management, 22(1), 42–52. Mohanty, R. P., & Venkataraman, S. (1993). Use of the analytic hierarchy process for selecting automated manufacturing systems. International Journal of Operations and Production Management, 13(8), 45–57. Morisio, M., & Tsoukia`s, A. (1997). IusWare: A methodology for the evaluation and selection of software products. IEEE Proceedings of Software Engineering, 144(3), 162–174. Mukherjee, D., & D’souza, D. (2003). Think phased implementation for successful data warehousing. Information Systems Management, 20(2), 82–90. Ngai, E. W. T., & Chan, E. W. C. (2005). Evaluation of knowledge management tools using AHP. Expert Systems with Applications, 29, 889–899. Nikoukaran, J., Hlupic, V., & Paul, R. J. (1999). A hierarchical framework for evaluating simulation software. Simulation Practice and Theory, 7(3), 219–231. Ossadnik, W., & Lange, O. (1999). AHP-based evaluation of AHP software. European Journal of Operational Research, 118(3), 578– 588.

953

Phillips-Wren, G. E., Hahn, E. D., & Forgionne, G. A. (2004). A multiplecriteria framework for evaluation of decision support systems. OMEGA, 32(4), 323–332. Roper-Lowe, G. C., & Sharp, J. A. (1990). The analytic hierarchy process and its application to an information technology decision. The Journal of the Operational Research Society, 41(1), 49–59. Sarkis, J., & Talluri, S. (2004). Evaluating and selecting commerce software and communication systems for a supply chain. European Journal of Operational Research, 159(2), 318–329. Seidmann, A., & Arbel, A. (1983). An analytic approach for planning computerized office systems. OMEGA, 11(6), 607–617. Seidmann, A., & Arbel, A. (1984). Microcomputer selection process for organizational information management. Information and Management, 7(4), 317–329. Shin, B. (2002). A case of data warehousing project management. Information and Management, 39(7), 581–592. Stamelos, I., Vlahavas, I., Refanidis, I., & Tsoukia`s, A. (2000). Knowledge based evaluation of software systems: A case study. Information and Software Technology, 42(5), 333–345. Strange, K. (2004). Magic Quadrant for Data Warehouse DBMSs, 2004. Gartner, Inc, pp. 1–8. Stylianou, A. C., Madey, G. R., & Smith, R. D. (1992). Selection criteria for expert system shells: A socio-technical framework. Communications of the ACM, 35(10), 30–48. Subramanian, G. H., & Gershon, M. (1991). The selection of computeraided software engineering tools: A multi-criteria decision making approach. Decision Sciences, 22(5), 1109–1123. Teltumbde, A. (2000). A framework for evaluating ERP projects. International Journal of Production Research, 38(17), 4507–4520. Timmreck, E. M. (1973). Computer selection methodology. ACM Computing Surveys, 5(4), 199–222. Vlahavas, I., Stamelos, I., Refanidis, I., & Tsoukia`s, A. (1999). ESSE: An expert system for software evaluation. Knowledge-Based Systems, 12(4), 183–197. Wang, M. J. J., & Chang, T. C. (1995). Tool steel materials selection under fuzzy environment. Fuzzy Sets and Systems, 72, 263–270. Wang, J., & Lin, Y. I. (2003). A fuzzy multicriteria group decision making approach to select configuration items for software development. Fuzzy Sets and Systems, 134, 343–363. Watson, H. J., Goodhue, D. L., & Wixom, B. H. (2002). The benefits of data warehousing: Why some organizations realize exceptional payoffs. Information and Management, 39(6), 491–502. Wei, C., Chien, C., & Wang, M. J. (2005). An AHP-based approach to ERP system selection. International Journal of Production Economics, 96(1), 47–62. Yeh, C. H., Deng, F., & Chang, Y. H. (2000). Fuzzy multicriteria analysis for performance evaluation of bus companies. European Journal of Operational Research, 126, 459–473. Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8, 338–353. Zahedi, F. (1985). Database management system evaluation and selection decisions. Decision Sciences, 16(1), 91–116. Zahedi, F. (1990). A method for quantitative evaluation of expert systems. European Journal of Operational Research, 48(1), 136–147.