A Performance Evaluation of Similarity Metrics for ...

A Performance Evaluation of Similarity Metrics for QoS Ranking of Services in Cloud Service e-Marketplace Azubuike Ezenwoke Covenant University, Nigeria [email protected]

Olawande Daramola Covenant University, Nigeria [email protected]

Matthew Adigun University of Zululand, South Africa [email protected]

Abstract Cloud service e-marketplace (CSEM) is a one-stop shop for trading cloud services. It contains a plethora of functionally equivalent cloud services, and these services can be evaluated by matching user’s QoS requirements with QoS information of services, which are usually heterogeneous in nature (classified into quantitative and qualitative information). Similarity Metrics (SM) can be employed to rank services from the CSEM based on QoS information from users and cloud services. While most SM focuses on accurate matching, not all SMs handle heterogeneous data efficiently, meanwhile some heterogeneous similarity metrics (HSM) has been proposed. In this paper, we considered the heterogeneity of QoS information of cloud services and based on real-life QoS dataset compared the performance of five HSM for ranking cloud services. We performed a rank order correlation among the HSM with those obtained from human similarity judgement and evaluated the accuracy of results using precision measure. The experimental results show significant rank order correlation of Heterogeneous Euclidean-Eskin Metric, Heterogeneous Euclidean-Overlap Metric, and Heterogeneous Value Difference Metric with human similarity judgement, compared to other metrics used in the study; thus confirming the applicability of HSM in QoS ranking of cloud services in CSEM with respect to users’ QoS requirements. Keyword: QoS ranking, Cloud Computing, Similarity metrics

1. Introduction Cloud Computing is a model of internet-based service provisioning in which dynamically scalable and virtualized resources, including infrastructure, platform and software, are delivered and accessed as services [1; 2; 3; 4]. The popularity of cloud computing attracts more players to the domain to offer a wide range of capabilities to users [5; 6], a phenomenon that can contributes to the exponential increase in the number of available cloud services [5; 7; 8]. Apart

from the traditional cloud service models, such as Infrastructure as a service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), other capabilities can be offered as services in the cloud. The collective term for this is XaaS (where „X‟ stands for Anything/Everything that can be abstracted and provisioned as services) [9; 10] e.g. Data-Analytics-as-a-Service (DAaaS), Database-as-a-Service (DBaaS), etc. The ability to truly deliver XaaS is enabled by cloud service ecosystem [11; 8], which is further fast-tracked by advancements in Service Orientation and Virtualization [12]. A cloud ecosystem is the collection of heterogeneous cloud providers and services to aggregate resources to realize new cloud services by combining existing services in a manner not previously pre-planned. Furthermore, the maturity of cloud computing will be fast tracked by being able to commoditize cloud services in a kind of cloud service e-marketplace (CSEM) for trading these services [13; 14]. The increase in the volume of available cloud services poses a challenge for users who desire to access services from the e-marketplace. The user is overwhelmed by the number of the available functionally equivalent cloud services [15], which hinder user‟s ability to make satisfactory selection of desirable service(s) and the consequence is that users may end up selecting a suboptimal option or not make any decision at all [16; 17]. This situation can be referred to as service choice overload. Service Choice overload is analogous to choice overload (also known as overchoice [18] and information overload [19] in ecommerce and information retrieval domains respectively). In such a scenario, the more the number of options and the need for personalization with respect to user requirements, the lesser the motivation to choose or the lesser the satisfaction with the final choice [20; 21]. Noteworthy is that cloud services have measurable non-functional attributes that describes and distinguish each service from another, which also determine user‟s perception of the service(s). These attributes are referred to as Quality of Service (QoS) [22; 23], and based on user‟s perception or preferences, QoS forms the criteria by which service selection is made [22; 23; 24]. Examples of QoS attributes include availability, reliability, latency etc. Some QoS attributes can be described qualitatively as they take on ordinal values, and others are represented using quantifiable numerical values. Classifying QoS attributes based on qualitative and quantitative metric-base values is a holistic representation of the QoS model of cloud services, and sufficiently forms the basis for ranking and selecting cloud services. We refer to this holistic representation as the heterogeneous QoS Model of cloud services and services can be selected by matching user‟s QoS requirement with QoS attributes of services available on the CSEM. In view of the growing number of services available on the e-marketplace, one of the ways to minimize the complexity posed service choice overload is to rank cloud services in accordance to their nearness to user‟s QoS requirements, and users can then make selection from the ranked list. Similarity Metrics, used to determine to what extent two vectors are alike, can be applied to perform this task. Similarity Metrics can be employed to determine the nearness of all services available on CSEM to user-defined QoS requirements [25] and several Similarity Metrics have been proposed in

literature, which could be used for this purpose (e.g. Euclidean Metrics, Chebyshev Distance Metrics etc.). The use of regular similarity metrics will not suffice in handling heterogeneous QoS attributes, as the ability to handle heterogeneous QoS attributes comprising of quantitative and qualitative data, significantly impacts on accurate ranking of services. While most SM focuses on accurate matching, not all SMs handle heterogeneous data efficiently, and heterogeneous similarity metrics (HSM) have been proposed in literature in this regards. In this paper, we consider the heterogeneity of QoS information of cloud services and based on real life QoS dataset, we compared the performance of five HSM. The result of the performance evaluation provided an empirical basis to determine the suitability of HSM for QoS ranking and selection of cloud services in cloud service e-marketplace. The remaining part of this paper is as follows: we describe our concept of a cloud service emarketplace and provided a QoS-driven cloud service selection model. The concept of similarity in the context of cloud service e-marketplace was discussed in section 3 together with detailed descriptions of the metrics considered in this paper. Section 4 contains the empirical results of comparing the performance and output from the heterogeneous similarity metrics with human similarity judgement, and the precision measure of each of the metrics. Section 5 compares our work with related works, while discussions and concluding remarks are presented in Section 6. 2. Cloud Service e-Marketplace The cloud service e-marketplace extends the concept of an electronic marketplace. An electronic marketplace is electronic platform where demand and supply for products or services are fulfilled using information and communication technologies [26; 14; 27]. The e-marketplace is central to creating and adding business and economic value to sellers, buyers, intermediaries, by facilitating the exchange of information, goods, services and payments [26]. In the future, there will be a large number of available vertical or horizontal service offerings from multiple providers and brokers [28; 16], and this proliferation will consummate a kind of „bazaar‟ [29] of cloud services. The popularity of cloud computing and the rise of cloud ecosystem will fasttracked the need and ability to commoditize cloud services and provide an e-marketplace to trade these services [13; 30; 11; 31]. The e-marketplace of cloud services provides an electronic emporium where service providers interconnect their offerings in unprecedented ways [32] and offer users a wide range of services to select from in marketplace environment [33; 29; 30]. Similar to Amazon1 or Alibaba2, the goal of a cloud service e-marketplace is to provide facility for finding and consuming cloud services; with such facilities, user can search for suitable services that match their QoS requirements. The profitability of the marketplace is realized by users‟ ability to find suitable service that meets their specific criteria, confirming one of the laws of ecommerce, which states that, “if users cannot find it, they cannot buy it either” [34]. The implication of this is that users should be able to quickly and easily find and consume cloud 1 2

www.amazon.com www.alibaba.com

services. We define a high-level architecture that encapsulates our concept of the cloud service emarketplace based on elaborations presented in [31; 14; 11]. The key modules of the cloud service e-marketplace (Fig. 1) are described below. a)

Service marketplace Store-front

This is the interaction module where users elicit their QoS requirements and the result of service ranking with respect to QoS properties is presented for the user to make suitable selection. b)

Marketplace Middleware

This module consists of the service ranking mechanisms to determine suitable services that closely match user‟s QoS requirement. The ranking is performed using the similarity metrics to measure nearness of the cloud services to user‟s QoS requirement and returns an ordered top-k list of suitable options. c)

Cloud Service Registry

This module contains the list of available cloud services in the marketplace together with their respective QoS data.

Figure 1: Cloud Marketplace Architecture

2.2.

QoS-driven Cloud Services Selection

Modelling QoS-driven Cloud service selection in a marketplace involves several cloud services, with multiple QoS attributes, the user‟s QoS requirements with QoS priority weights. A set of formal definitions [35] of these concepts and how they are used for personalized ranking of cloud services for selection results are described in subsequent sub-sections.

a)

Cloud Services

Definition 1 (Set of Services) [35]: Let S = {S1, S2, S3 … Sn} be a set of n services available on the cloud marketplace out of which the user is expected to select one. Where n ≥ 2 (i.e. there must be at least more than one service available for the user to select from.) b)

Quality of Service (QoS) Attributes

QoS attributes are classified into qualitative and quantitative metrics. The qualitative QoS attributes are described using an ordinal scales consisting of a set of predefined qualitative values that the attributes can assume. For example, specific tags such as High, Medium and Low, can be used to represent how reliable a cloud service is. Qualitative QoS attribute accept categorical values; while quantitative QoS attributes assume nominal linear, continuous or discrete values input. For example, Response Time is measured in numerical continuous/discreet values. Definition 2 (Set of QoS attributes) [35]: Let matrix) of components describing the QoS attributes of a service c)

be the vector ( .

e-Marketplace Cloud Services Matrix

The heterogeneous nature of QoS attributes necessitates that the values be normalized to obtain dimensionless data [35], which forms the basis of defining a matrix of QoS information of all services available on the marketplace. Definition 3 (Services Matrix) [35]: Let of all service where each element and .

be Matrix that contain the QoS information represents the QoS value of the service,

( From , a row vector would describe a service represents the QoS attribute of service . d)

) with QoS attributes where each element

User QoS requirements

The values of a user‟s QoS requirements are captured in a vector that corresponds to the number of QoS attributes that describes available marketplace services. In addition, users have personalized preferences that describe the relative importance of each QoS attributes which may differ from another user‟s. These preferences are defined using priority weights. Similar to [35], we define the user requirements, as:

Definition 4 (User QoS Values) [35]: Let value for the QoS attribute. Definition 5 (User QoS priority Weights) [35]: Let the importance weight assigned by the user to QoS attribute e)

, where

is the user‟s desired

, where each

is

.

QoS-driven Cloud service Ranking

A user is expected to select the nearest service to his/her QoS requirements, from the list of available services and the Similarity Metrics described in section 3 are applied to determine the most nearest. Definition 6 (Cloud service ranking): The cloud service selection is a quadruple where 1) is the matrix of the QoS attributes of all available cloud services on the marketplace, 2) is the vector of the user‟s priority weights on each QoS attribute, and in this paper, we assumed equal priority for each attribute. 3) R is the user‟s defined QoS values for each QoS attribute, and 4) is the HSM employed. 3. Notion of Similarity in the Context of Cloud Service e-Marketplace Similarity is a measure of proximity between two or more objects or variables [36] and it has been applied in domains that require distance computation. Similarity can be measured on two types of data: quantitative data (also called numerical data) and qualitative (also called categorical/nominal data) [37]. Many metrics have been proposed for computing similarity on either of both types of data. However, few metrics have been proposed to handle dataset containing a mixture of both, and usually combines quantitative and qualitative distance functions. For quantitative data, a generic method for computing distance is Minkowsky [38], with widely used specific instances such as the Manhattan (of order 1) and Euclidean (of order 2). The computation of similarity for quantitative data is more direct, compared to qualitative data, because quantitative data can be completely ordered, while comparing two qualitative values is somewhat complex [37]. For example, the overlap metric [39], assign a similarity value of is when two qualitative values are the same and otherwise. In the context of selecting cloud services from available services, ranking services based on the heterogeneous QoS model necessitates the application of similarity metrics that can handle mixed QoS data. The notion of similarity considered here is between vectors with the same set of QoS properties, which might differ in their QoS values i.e. users‟ QoS requirement and service QoS description. The similarity between the user‟s QoS requirement and QoS description vector of a Cloud service is the sum of similarities between each of the corresponding QoS attributes of the vectors.

Figure 2: Cloud service with QoS attributes 3.1.

Figure 3: Notion of Similarity

Similarity Metrics and QoS Ranking

Apart from the Minkowsky [38] metrics and its derivatives (Manhattan and Euclidean), other examples of quantitative distance metrics are Chebyshev and Camberra metrics. Metrics, such as Overlap [39], Eskin [40] (see Eqn. 11), Lin [41] (see Eqn. 14) and Goodall [42] (see Eqn. 17), have been proposed for qualitative distance computation and their performances on outlier detection were presented in [37]. However, these quantitative or qualitative metrics alone are insufficient for handling heterogeneity, except when combined into a unified metrics that applies different similarity metric to different types of QoS attributes [43]. A study of heterogeneous distance metrics was presented in [43], where authors proposed Heterogeneous EuclideanOverlap Metrics (HEOM) and Heterogeneous Value Difference Metric (HVDM) as metrics for computing similarity operations on heterogeneous datasets. The HOEM metric employs rangenormalized euclidean metric (Eqn. 4) for quantitative QoS attributes, while Overlap metric (Eqn. 3) is employed for qualitative QoS attributes; and the HVDM uses the standard-deviationnormalized euclidean distance (Eqn. 8) and value difference metric (Eqn. 7), for quantitative and qualitative QoS attributes respectively. Apart from HEOM and HVDM, we introduced for this first time in this paper, additional three HSM by adapting and extending existing similarity metrics used for either quantitative or qualitative data alone. The new HSM are as follows: Heterogeneous Euclidean-Eskin Metrics (HEEM), Heterogeneous Euclidean-Lin Metrics (HELM), and Heterogeneous Euclidean-Goodall Metrics (HEGM). HEEM (Eqn. 9-11) combines range-normalized euclidean distance for quantitative dataset, while Eskin metric [40] was employed for categorical dataset. While the range-normalized euclidean distance (Eqn. 4) is employed for computing quantitative dataset in both HELM (Eqn. 12-14) and HEGM (Eqn. 1517), HELM applies the Lin metrics and HEGM used the Goodall metrics to compute on categorical datasets. In all, the five HSM considered in this paper are as follows HEOM, HVDM, HEEM, HELM and HEGM; while the mathematical equations that describe each of them are presented next.

Assuming that and are vectors representing the values of the user QoS requirements and a QS vector of a cloud service belonging to service list , such that and ; and corresponds to the value of the QoS attribute of the users requirement and QoS attribute of the cloud service respectively, then: a)

Heterogeneous Euclidean-Overlap Metric (HEOM) √∑

(1)

Where {

(2)

And overlap

and

are defined as

{

(3) (4)

b)

Heterogeneous Value Difference Metric √∑

(5)

And {

(6)

√∑

|

|

√∑

|

|

(7)

(8)

Where   

is the number of instances (cloud services) available on the marketplace that have value for QoS attribute ; is the number of instances available on the marketplace that have value for QoS attribute and output class c; C is the number of output classes in the problem domain (in this case, C=3, corresponding to the High, Medium and Low successability of the cloud services;



is the conditional probability of output class value , i.e.

, computing as

given that QoS attribute

. However, if

has the

, then

is also regarded as 0. c)

Heterogeneous Euclidean-Eskin Metric √∑

(9)

{

(10)

{

d)

(11)

Heterogeneous Euclidean-Lin Metric √∑

(12)

{

(13)

̂ {

(14) ̂

e)

̂

Heterogeneous Euclidean-Goodall Metric √∑

(15)

{

(16)

̂ { 

Where

(17)

= the number of values that QoS attribute can assume (For , ; corresponding to number of values that availability can assume: High, Medium and Low)



Where ̂ and ̂ are the sample probability of QoS attribute to take the value of in the data set (in this case the available services on the marketplace); computed as ̂



and ̂

.

The total number of service alternatives is denoted as .

4. Experimental Assessment An experimental assessment of the five HSM (section 3) on a subset of real dataset (QWSdataset) adapted from real QoS measures of web services [44] and we used it in the context of cloud service e-marketplace. We evaluated the performance of each of the HSM with respect to results obtained from human similarity judgement. Human subjects were asked to rank a sub-set of cloud services with respect to degree of similarity to a user-defined QoS requirement. First, we determined correlation of the QoS ranking among the HMs including the rakings obtained from human judgement; after which we computed the accuracy of the results obtained from the HMs with respect to those obtained from human judgement. The HSM were implemented in Java according to the equations defined in section 3. The experiments were conducted on an HP Pavilion with Intel Core (TM) i3-3217U CPU at 1.80GHz 1.80 GHz processor and 4.00GB RAM on 64-bit Operating System, x64-based processor running Windows 8.1. 4.1.

QoS Dataset Preparation

The QWS dataset consist of measurements of nine QoS attributes for 2507 web services, which included the following with unit of measurement in bracket: 1) Response Time (ms), 2) Availability (%), 3) Throughput (invokes/sec), 4) Successability (%), 5) Reliability (%), 6) Compliance (%), 7) Best Practices (%), 8) Latency (ms) and 9) Documentation (%). However, we modified the QWS dataset to suit the heterogeneous cloud service QoS model described in this paper, in which we considered only six (attributes 1, 2, 3, 5, 8 and 9) of the QoS attributes, divided into three quantitative and three qualitative attributes (Table 1). We converted the percentage values of attributes 2, 5 and 9 to categorical scale. For simplicity, the values 0-30%, 31-60%, 61-100% was mapped to High, Medium and Low respectively. We selected the first 65 QoS vectors, after sorting list by Response Time in ascending order. To determine the suitability of the dataset, we performed classification task on the dataset according to the successability attribute using J48 decision trees, Multilayer Perceptron, and Naïve Bayes classifiers. The results were measured with a 10-fold cross validation and Table 1 shows the performance on accuracy for all classifiers. Table 1: Classification Accuracy for QoS dataset # 1 2 3

CLASSIFIERS J48 decision trees Multilayer Perceptron Naïve Bayes

ACCURACY (%) 100% 100% 93.75

Table 2: The Six QoS Attributes, unit of measurement and value options QUANTITATIVE ATTRIBUTE Attribute Name Unit Response Time Ms Throughput invokes/sec Latency Ms

# 1 2 3

4.2.

# 4 5 6

QUALITATIVE ATTRIBUTE Attribute Name Value Options/Range Availability {High, Medium, Low} Reliability {High, Medium, Low} Documentation {High, Medium, Low}

Human QoS Similarity Judgement

The assessment of human similarity judgement was performed using questionnaires distributed to willing participants. The participants were shown line charts (Fig 3) of a predefined the user QoS requirements (UserQoS) against the QoS values of each cloud service (ServiceQoS). The services matrix A is populated by the 64 QoS vectors selected for this study and the values for user QoS requirement Vector R used for the experiment is {Low, Medium, Low, 63.83, 5.1, 3.92}. For simplicity and purpose of illustration on the graph, the qualitative values High, Medium and Low were mapped to numerical values of 30, 20 and 10 respectively. The participants were shown each graph containing lines for the UserQoS and ServiceQoS and were instructed to agree or disagree (on a 1 to 7 Likert scale) with the proposition: ‘The two Lines are similar.’ We selected a sample size of 27 (n=27 University undergraduates) on the basis that 27 participants offer an acceptably tight confidence interval [45]. We assumed that each QoS attribute carry equal importance to the user thereby neglecting the effect of QoS priority weights. The responses from the 35 participants were analysed and we determined the Mean of the response to each item which indicates unanimously which ServiceQoS is most similar to the UserQoS. We obtained the mean of the responses across the 65 items presented in the questionnaire and set Mean ≥ 5.00 as the basis for determining similarity of ServiceQoS to UserQoS. Only 15 services met the criteria and they were stored as the „gold standard output‟ [46].

(a) Perfect Match of ServiceQoS and UserQoS values

(b) Difference in ServiceQoS and UserQoS Values

Availability

Reliability

Documentation

Response Time

Throughput

Latency

ServiceQoSA

Low(10)

Medium(20)

Low(10)

63.83

5.1

3.92

UserQoS

Low(10)

Medium(20)

Low(10)

63.83

5.1

3.92

Fig.3a

Fig.3b

ServiceQoSB

High(30)

High(30)

Low(10)

41

43.1

1

UserQoS

Low(10)

Medium(20)

Low(10)

63.83

5.1

3.92

Figure 3: Line Graph showing ServiceQoS Vs. UserQoS Table 3: Mean and Mean Frequencies of Responses MEAN VALUE RANGE 1.00-2.99 3.00-4.99 5.00-7.00

4.3.

FREQUENCY 14 35 15 64

QoS Rank Correlation Coefficient

We applied the Kendall Tau Rank Correlation Coefficient [47] metrics to measures the cloud service rankings obtained from the HM. Kendall Tau Rank Correlation Coefficient is computed based on the number of agreeing versus the contradictory pairs between ranks. Table 4: Results Kendall Tau Rank Correlation Coefficients

HUMAN

Correlation Coefficient Sig. (2-tailed) N

HUMAN 1.000 . 64

HEOM .422(**) .000 64

HVDM .416(**) .000 64

HEEM .449(**) .000 64

HELM -.138 .107 64

HEGM -.209(*) .014 64

HEOM


.422(**) .000 64

1.000 . 64

.617(**) .000 64

.962(**) .000 64

.046 .594 64

.065 .444 64

HVDM


.416(**) .000 64

.617(**) .000 64

1.000 . 64

.653(**) .000 64

-.006 .945 64

-.097 .256 64

HEEM


.449(**) .000 64

.962(**) .000 64

.653(**) .000 64

1.000 . 64

.022 .799 64

.028 .746 64

HELM


-.138 .107 64

.046 .594 64

-.006 .945 64

.022 .799 64

1.000 . 64

.768(**) .000 64

HEGM


-.209(*) .014 64

.065 .444 64

-.097 .256 64

.028 .746 64

.768(**) .000 64

1.000 . 64

Table 4 gives the rank order correlation among all five heterogonous metrics, as well as the ranking obtained from human similarity judgment. The results suggest that 8 of 15 correlations were statistically significant and we can conclude that there is a strong positive correlation among the ranking results from human judgement with HEOM, HVDM and HEEM; and a

negative correlation with HEGM. The strongest correlations occur for HEEM with HEOM (0.962), HVDM with HEOM (0.617), HVDM with HEEM (0.653), and HEGM with HELM (0.786). The weaker correlations occur among the following: HELM with HEOM, HVDM, and HEEM; HEGM with HEOM, HVDM and HEEM. Based on these results, our conclusion is that the results of the ranking obtained by HEOM, HVDM and HEEM rates favourably with those contained in the gold standard. The rank order from HEEM (0.449) correlates highly with human similarity judgment, closely followed by HEOM (0.422) and HVDM (0.416). 4.4.

Accuracy Measure

We applied precision measure to evaluate the accuracy of the metrics. Precision, a measure used in information retrieval domains, was adapted here to evaluate the relevance of the output obtained from each metric with respect to the content of the gold standard. Precision is the fraction of cloud services obtained from the HM that are contained in the gold standard. The gold standard output was used as the benchmark to determine the precision of each metric as we determined how many of the top-k services returned by the metrics include the services contained in the gold standard. We computed the precision each metric as we varied the number of k. We define Precision as: ⋂

(12)

Where TKS=Top-k Cloud Services returned by HM and GS= Number of Services in Gold Standard. Table 5: Precision Results #

HSM

k=5

k=10

k=15

k=20

k=25

1

HEOM HVDM HEEM HELM HEGM

0.8 0.4 0.8 0.4 0.4

0.8 0.5 0.8 0.3 0.2

0.733333 0.6 0.733333 0.333333 0.133333

0.55 0.5 0.55 0.25 0.1

0.48 0.44 0.48 0.2 0.08

2 3 4 5

Figure 4: Precision Graph using Gold standard as benchmark

High precision connotes that the HM ranked and returned more relevant services as contained in the gold standard. Based on the analysis, we observed that HEOM and HEEM consistently gave the highest accuracy across the varied value of k, while HEGM has the least, followed closely by HELM. HVDM had its highest precision at k=15. Based on the results of the rank order correlation and precision analysis, HEEM performed relatively well in comparison to HEOM and HVDM viz a viz human judgement, while the results of HEGM and HELM show their nonsuitability for QoS ranking of services in cloud service e-marketplace. 5. Related Works The success of a cloud service e-marketplace is hinged on adequate support for satisfactory selection based on the multiple QoS requirements of the user. The support provided should effectively enable the user determine which services on the marketplace is closely related to the own QoS requirement. There are several approaches proposed in literature to enable selection of cloud services [48; 16; 23]. CloudRank, a personalized ranking prediction framework, that utilizes a greedy-based algorithm, was proposed by [23] to predict QoS ranking by leveraging on similar cloud service user‟s past service usage experiences of a set of cloud services. The ranking is achieved by finding the similarity between the user-provided QoS requirements and those of other users in the past. Similar users are identified based on these similarity values. Our work focuses on the computation of similarity between cloud services and user-defined QoS requirements. CloudAdvisor, a Recommendation-as-a-Service platform was proposed in [16] for recommending optimal cloud offerings based on a given user preference requirements. Users supply preference values to each properties (energy-level, budget, performance etc.) of the cloud offerings, and the platform recommends available optimal cloud offerings that match user‟s requirements. While our approach applies the notion of similarity, service recommendations in [16] are determined by solving a constraint optimization model and users can compare several offerings automatically derived by benchmarking-based approximations.

Selection of cloud services in the face of many QoS attributes is a type of Multi-criteria Decision Making (MCDM) [48]. Considering the multiple QoS criteria involved in selecting cloud services, [48] propose a ranking mechanism based on Analytical Hierarchical Process (AHP) to assign weights to non-functional attributes to quantitatively realize cloud services ranking. Apart from the complexity in computing the pairwise comparisons of the attributes of the cloud service alternatives, this approach is most suitable with very few number of cloud services, as against a service marketplace of numerous services. Also, users have no input to the desired values of the service properties. Our concept of cloud service selection is similar to [35] in that a ranked list of services that best matches user requirement is returned based on nearness of user‟s QoS requirement to the QoS properties of cloud services in the marketplace. Rehman, et al., [35] proposed an approach to select cloud service based on multiple criteria that selects services that best matches user‟s QoS requirements from a list of services by comparison. Authors introduced two methods, Weighted Difference and Exponential weighted Difference, for computing similarity values. It is however assumed in [35] that all cloud service QoS attributes is quantitative. Our approach considers the heterogeneity of cloud QoS Model that combines quantitative and qualitative QoS data; and to the best of our knowledge, our approach is the first attempt that considers the use of heterogeneous similarity metrics for QoS ranking and selecting services in the context of a cloud service e-marketplace. 6. Conclusion The emergence of cloud service e-marketplaces as a one-stop shop for demand and supply of services further contributes to the popularity of cloud computing, as a preferred means of delivering and consuming services. The success of cloud marketplace is hinged on the ease and how quickly services are found and consumed with respect to how they match user-specific QoS requirements. The main goal of this paper is to demonstrate the plausibility of applying heterogeneous similarity metrics in evaluating cloud services based on nearness of service QoS information to users‟ QoS requirements. We evaluated the performance of five heterogeneous similarity metrics, and compared the results with human similarity judgement. The experimental results show that the QoS rankings obtained from HEOM, HVDM and HEEM correlates closely with human similarity assessments compared to other heterogeneous similarity metrics used in this study. Thus, confirming the suitability of heterogonous similarity metrics for QoS ranking of cloud services with respect to user‟s QoS requirements in the context of a cloud service emarketplace. However, we have used only one user‟s QoS requirement as an example to describe a cloud service selection scenario. Similar studies can be performed using variety of user QoS requirements and QoS datasets to determine if the results obtained in this paper will remain the same.

7. References [1] Rimal, B. P., Jukan, A., Katsaros, D., & Goeleven, Y. (2011). Architectural Requirements for Cloud Computing systems: An Enterprise Cloud Approach. Journal of Grid Computing , 9 (1), 3-26. [2] Tan, X., & Kim, Y. (2011). Cloud Computing for Education: A Case of Using Google Docs in MBA Group Projects. International Conference on Business Computing and Global Informatization (pp. 641-644). IEEE Computer Society. [3] Lewis, G. (2011). Architectural Implications of Cloud Computing. Retrieved March 17, 2012, from SEI-CMU. [4] Qaisar, E. J. (2012). Introduction to Cloud Computing for Developers-Key concepts, the players and their offerings. Information Technology Professional Conference (TCF Pro IT), (pp. 1-6). IEEE. [5] Fortis, T.-F., Munteanu, V. I., & Negru, V. (2012). Towards a service friendly cloud ecosystem. Parallel and Distributed Computing (ISPDC), 2012 11th International Symposium on (pp. 172-179). IEEE. [6] Pallis, G. (2010). Cloud computing: the new frontier of internet computing. IEEE Internet Computing , 70-73. [7] Geambasu, R., Gribble, S. D., & Levy, H. M. (2009). Cloudviews: Communal data sharing in public clouds. Proceedings of the 2009 conference on Hot topics in cloud computing (pp. 14-14). USENIX Association. [8] Papazoglou, M., & van den Heuvel, W.-J. (2011). Blueprinting the cloud. IEEE Internet Computing , 74-79. [9] Bastia, A., Parhi, M., Pattanayak, B., & Patra, M. (2015). Service Composition Using Efficient Multi-agents in Cloud Computing Environment. Intelligent Computing, Communication and Devices , 357-370. [10] Hossain, S. (2012). Cloud Computing Terms, Definitions, and Taxonomy. In A. Bento, & A. Aggarwal, Cloud Computing Service and Deployment Models: Layers and Management: Layers and Management (pp. 1-25). IGI-Global. [11] Gatzioura, A., Menychtas, A., Moulos, V., & Varvarigou, T. (2012). Incorporating Business Intelligence in Cloud Marketplaces. IEEE 10th International Symposium on Parallel and Distributed Processing with Applications (ISPA) (pp. 466-472). IEEE.

[12] Li, H., & Jeng, J.-J. (2010). CCMarketplace: a marketplace model for a hybrid cloud. Proceedings of the 2010 Conference of the Center for Advanced Studies on Collaborative Research (pp. 174-183). IBM Corp. [13] Buyya, R., Yeo, C. S., & Venugopal, S. (2008). Market-oriented cloud computing. Proceedings of the 10th IEEE International Conference on High Performance Computing and Communications (HPCC'08) (pp. 5-13). IEEE. [14] Menychtas, A., Vogel, J., Giessmann, A., Gatzioura, A., Garcia Gomez, S., Moulos, V., et al. (2014). 4CaaSt marketplace: An advanced business environment for trading cloud services. Future Generation Computer Systems , 104–120. [15] Alrifai, M., Skoutas, D., & Risse, T. (2010). Selecting skyline services for QoS-based web service composition. Proceedings of the 19th international conference on World wide web (pp. 11-20). ACM. [16] Jung, G., Mukherjee, T., Kunde, S., Kim, H., Sharma, N., & Goetz, F. (2013). CloudAdvisor: A Recommendation-as-a-Service Platform for Cloud Configuration and Pricing. 203 IEEE Ninth World Congress on Services (SERVICES) (pp. 456-463). IEEE. [17] Townsend, C., & Kahn, B. E. (2014). The “visual preference heuristic”: the influence of visual versus verbal depiction on assortment processing, perceived variety, and choice overload. Journal of Consumer Research , 40 (5), 993-1015. [18] Tofﬂer, A. (1970). Future shock . New York : Amereon Ltd. [19] Lucian, R. (2014). Digital Overload: The Effects of The Large Amounts of Information When Purchasing Online. Journal of Internet Banking and Commerce , 2-18. [20] Chernev, A., Böckenholt, U., & Goodman, J. (2015). Choice Overload: A Conceptual Review and Meta-Analysis. Journal of Consumer Psychology , 333–358. [21] Haynes, G. A. (2009). Testing the boundaries of the choice overload phenomenon: The effect of number of options and time pressure on decision difficulty and satisfaction. Psychology & Marketing , 26 (3), 204-212. [22] Chen, X., Zheng, Z., Liu, X., Huang, Z., & Sun, H. (2013). Personalized QoS-Aware Web Service Recommendation and Visualization. IEEE Transactions on Services Computing , 35-47. [23] Zheng, Z., Wu, X., Zhang, Y., Lyu, M. R., & Wang, J. (2013). QoS ranking prediction for cloud services. IEEE Transactions on Parallel and Distributed Systems , 1213-1222. [24] Abdelmaboud, A., Jawawi, D. N., Ghani, I., Elsafi, A., & Kitchenham, B. (2015). Quality of service approaches in cloud computing: A systematic mapping study. Journal of Systems and Software , 101, 159-179.

[25] Mirmotalebi, R. a.-H. (2012). Modeling User‟s Non-functional Preferences for Personalized Service Ranking. In Service-Oriented Computing (pp. 359-373). Berlin Heidelberg: SpringerVerlag. [26] Bakos, Y. (1998). The emerging role of electronic marketplaces on the Internet. Communications of the ACM , 41 (8), 35-42. [27] Akingbesote, A., Adigun, M., Jembere, E., Othman, M., & Ajayi, I. (2014). Determination of optimal service level in cloud e-marketplaces based on service offering delay. International Conference on Computer, Communications, and Control Technology (I4CT) (pp. 283-288). Langkawi, Kedah, Malaysia : IEEE. [28] Zeng, W., Zhao, Y., & Zeng, J. (2009). Cloud Service and Service Selection Algorithm Research. Proceedings of the first ACM/SIGEVO Summit on Genetic and Evolutionary Computation (pp. 1045-1048). ACM. [29] Vigne, R., Mach, W., & Schikuta, E. (2013 ). Towards a Smart Webservice Marketplace. IEEE 15th Conference on Business Informatics (CBI) (pp. 208-215). IEEE. [30] Menychtas, A., Vogel, J., Giessmann, A., Gatzioura, A., Garcia Gomez, S., Moulos, V., et al. (2014). 4CaaSt marketplace: An advanced business environment for trading cloud services. Future Generation Computer Systems . [31] Akolkar, R., Chefalas, T., Laredo, J., Peng, C.-S., Sailer, A., Schaffa, F., et al. (2012). The Future of Service Marketplaces in the Cloud. IEEE Eighth World Congress on Services (SERVICES) (pp. 262-269). IEEE. [32] Barros, A. P., & Dumas, M. (2006). The rise of Web service ecosystem. IT Professional , 8 (5), 31-37. [33] Khadka, R., Saeidi, A., Jansen, S., Hage, J., & Helms, R. (2011). An Evaluation of Service Frameworks for the Management of Service Ecosystems. PACIS 2011 proceedings, (p. Paper 93). [34] Nielsen, J. (2012, January 4). Usability 101: Introduction to Usability. Retrieved September 15, 2015, from Nielsen Norman Group Website: https://www.nngroup.com/articles/usability101-introduction-to-usability/ [35] Rehman, Z. u., Hussain, F., & Hussainz, O. K. (2011). Towards Multi-Criteria Cloud Service Selection. Fifth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (pp. 44-48). IEEE. [36] Ayeldeen, H., Shaker, O., Hegazy, O., & Hassanien, A. E. (2015). Distance Similarity as a CBR Technique for Early Detection of Breast Cancer: An Egyptian Case Study. Information Systems Design and Intelligent Applications , 449-456.

[37] Boriah, S., Chandola, V., & Kumar, V. (2008). Similarity measures for categorical data: A comparative evaluation. Proceedings of the 8th SIAM International Conference on Data Mining (pp. 243–254). Atlanta: SIAM. [38] Batchelor, B. G. (1978). Pattern Recognition: Ideas in Practice. New York: Plenum Press. [39] Stanfll, C., & Waltz, D. (1986). Toward memory-based reasoning. Communications of the ACM , 1213-1228. [40] Eskin, E., Arnold, A., Prerau, M., Portnoy, L., & Stolfo, S. (2002). A geometric framework for unsupervised anomaly detection. Applications of data mining in computer security , 77-101. [41] Lin, D. (1998). similarity, An information-theoretic definition of similarity. Proceedings of Fifth International Conference on Machine Learning (pp. 296-304). San Francisco: Morgan Kaufmann Publishers. [42] Goodall, D. W. (1966). A new similarity index based on probability. Biometrics , 882-907. [43] Wilson, D. R., & Martinez, T. R. (1997). Improved heterogeneous distance functions. Journal of Artificial Intelligence Research , 1-34. [44] Al-Masri, E. &. (2008). Investigating web services on the world wide web. Proceedings of the 17th international conference on world wide web (pp. 795–804). Beijing, China: ACM. [45] Nielsen, J. (2006, June 26). Quantitative Studies: How Many Users to Test? Retrieved May 2, 2015, from Nielsen Norman Group: Evidence-Based User Experience Research, Training, and Consulting: http://www.nngroup.com/articles/quantitative-studies-how-many-users/ [46] Kim, H. D., Zhai, C., & Han, J. (n.d.). Aggregation of Multiple Judgments for Evaluating Ordered Lists. [47] Alvo, M., & Philip, L. (2014). Statistical Methods for Ranking Data. Springer. [48] Garg, S. K., Versteeg, S., & Buyya, R. (2011). SMICloud: A Framework for Comparing and Ranking Cloud Services. 2011 Fourth IEEE International Conference on Utility and Cloud Computing (UCC) (pp. 210-218). IEEE.