Measuring Bidirectional Effects on Trust and Rating on Online Social Networks Yutaka Matsuo
Hikaru Yamamoto
National Institute of Advanced Industrial Science and Technology 1-18-13 Sotokanda Tokyo, Japan
Seikei University 3-3-1 Kichijoji Kitamachi, Musashino-shi Tokyo, Japan 180-8633
[email protected] ABSTRACT Several attempts have been made to analyze customer behavior that is apparent when using online social media. Some studies particularly emphasize the social network of customers. Users’ reviews and ratings of a product exert effects on other consumers’ purchasing behavior. Whether a user refers to other users’ ratings depends on the trust accorded by a user to the reviewer. A recent study shows that the trust that is felt by a user toward another user correlates to the similarity of the two users. This bidirectional interaction that involves trust and rating is an important factor to understand consumer behavior in online communities because it suggests clustering of similar users and emergence of strong communities. This paper presents analyses of @cosme, a viral marketing site. It is one of the largest community sites in Japan. The notable characteristics of @cosme are that users can bookmark their trusted users; in addition, they can post their ratings of products, which facilitates our analyses of the ratings’ bidirectional effects on trust and ratings. We describe an overview of the data in @cosme, analyses of effects from trust to rating and vice versa, and finally propose a measure of the degree to which a brand shows bidirectional effects. Our study is based on a particular dataset, but it conveys important insights and proposes a potentially important measure for mining online social networks.
Keywords trust, social network, rating, measure of bidirectional effect
1. INTRODUCTION Online social systems and knowledge-sharing sites have received much attention as a medium of viral marketing. People write their experiences and opinions about products and services in their blogs and knowledge sharing sites. Several studies have attempted analysis of opinions in the blo-
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WOODSTOCK ’97 El Paso, Texas USA Copyright 200X ACM X-XXXXX-XX-X/XX/XX ...$5.00.
[email protected] gosphere [10, 1]. With regard to knowledge-sharing sites (or recently called consumer-generated media), relations among customers are analyzed in various ways. M. Richardson and P. Domingos propose a method to calculate a network value of online customers [13]. The network value of a customer is high when the customer is expected to influence other users’ probabilities of purchasing the product both strongly and positively. Kemp et. al follows this problem using several widely studied models in social network analysis. The optimization problem of selecting the most influential customer is NP-hard, and they provide the provable approximation for efficient algorithms [7]. Information about customer experiences flows through social relations. Users share their good and bad experiences with their friends and colleagues. They might exchange that information with their friends online [16]. J. Leskovec et al. analyze the recommendations among users on Amazon.com [8]. They show how the recommendation network grows over time. Moreover, they show its effectiveness from the viewpoint of the sender and receiver of the recommendations. T. Furukawa and Y. Matsuo investigate a social network in a blog hosting service and how the information flows on the network [2]. Recently X. Song et al. analyze the online diffusion process and then propose an algorithm called DiffusionRank to predict who is likely to receive information during a limited time period in a social network [15]. All social relations do not function equivalently: Users might trust some people more than others, and be more influenced by them. Even though a certain user might make many recommendations (like a spammer), such a person’s influence is limited: Users do not trust him, and are rarely influenced. This phenomenon is recognized in [8], which reports that “There are limits to how influential high-degree nodes are in the recommendation network. As a person increasingly sends out recommendations past a certain number for a product, the success per recommendation declines.” Then, the question is how will a user come to put trust in others? The answer is suggested by recent studies by Golbeck et al. [4, 21]. Similarity of profile attributes (such as ratings of movies) induces trust among people. They analyzed data from FilmTrust and found that several profile features beyond overall similarity affect the degree to which subjects trust other users. Another characteristic that Golbeck and colleagues use for various applications [5] is the transitivity of trust: If A trusts B, and B trusts C, then A can be inferred to trust C to some degree. The calculation
is validated through experimental studies. Considering the above studies of the degree to which trust is effective and the manner in which trust is formed, we can see bidirectional effects on users’ trust and ratings. Rating to trust Users puts trust in others because their ratings match the other users’ ratings. Trust to rating The rating of a user is influenced by the opinions of trusted others. This bidirectional interaction of trust and opinion can be considered to be ubiquitous in the real world. People in similar cultures flock together; they are influenced by each other, and make the culture more unique. Examples include Otaku culture in Japan, highly fashionable people, academic researchers, and so on. This phenomenon, by which “similar” people gather, is understood as homophily in the context of social network analysis [18]. Online social media induce people to flock together easily. Consequently, the characteristics of this phenomenon are necessary for mining and analyzing online social communities. In this paper, we analyze a knowledge-sharing site called @cosme (www.cosme.net). It is the largest online community site of “for-women” communities in Japan1 , and provides information and reviews related to cosmetic products. Notable characteristics of @cosme are that a user can register other users who can be trusted; she can also post reviews of products. The trusted users are called Okiniiri by her, which signifies a feeling of both favor and trust. Data of more than 700 thousand users gathered over five years enables us to analyze the bidirectional interaction of trust and opinions: (i) How does a user put trust in others from the similarity of ratings? (ii) What effects does that trust have on users’ purchase behavior and ratings? These data, which include user trust and product ratings, are precious (the few readily available data include the TrustFilm dataset, which was used in [6]). Therefore, the bidirectional effect is newly revealed in this study. We believe that this analysis provides important insights for understanding various online social media. The contributions of the paper are summarized as follows: • The bidirectional effects of trust and opinions are analyzed both theoretically and empirically. Cosmetic products are a typical experience good. For that reason, other users’ opinions are useful for decision-making.
The network characteristics are described to show the effectiveness of the measure. Finally, we present discussion of the results and implications, and then conclude the paper.
2. 2.1
1
The site does not prohibit males. Actually, many male users exist, but in fact, 99% of the users are female.
Viral Marketing Site
Since its opening in December 1999, @cosme has acquired a growing number of users: as of Fall 2005, it had a half million registered users, 1.1 million visitors per month, and 115 million page-views. According to the operator (istyle inc.), it is intended to be a “viral marketing” site related to cosmetics. Although there might be some cultural biases (because it is a Japanese site), the data would be an interesting source of information about users’ trust, ratings, and their interaction. Users of @cosme can post their reviews (called Kuchikomi) on cosmetic products (100.5 thousand items in 11 thousand brands) on the system. A review consists of a text describing the experience and the rating (from 1 (bad) to 7 (good)) of the product. We do not use the text messages for this study, so we use a review and a rating interchangeably in this paper (when not confusing). Figure 1 shows the top page of the site. A visitor can select a product and see reviews about it. She can also browse other products through use of the hierarchical classification of products or clicking reviewers’ other reviews. Once a user registers to the site, she becomes able to log in to the system. She is directed to a personalized page called “MyPage.” News on favorite brands and latest reviews announced by her trusted persons (Okiniiri) are shown. A user can bookmark the reviewer as Okiniiri if she finds someone’s review trustworthy and useful. We use Okiniiri as a (directional) trust relation. The reviewer is notified that she has acquired a new user who registered her as Okiniiri; in other words, that she has acquired a new fan. Because the system ranks users by their respective numbers of fans, some users are extremely motivated to accumulate more fans. Figure 2 shows the newly added reviews, new bookmarks, and new users for each month since the site’s opening. The number of users grows steadily, as do the number of trust relations and reviews. The previous survey of actual users of @cosme [20] introduces an interesting comment from an actual user: Considering my experience in @cosme, there are two periods: in the first period, I used @cosme simply as a product database. Then, I began to pay attention to reviewers and utilize bookmarks, which is in the second period. My bookmarks include persons whom I nicknamed e.g., master of soap, master of fragrance, master of pack and serum, and so on. I frequently check their reviews. I think I can use @cosme more than before when I realized the importance of focusing on people.
• We propose a (potentially) useful measure, called BEM to characterize a product and a brand. We identify a situation in which the online community becomes more clustered; trust and opinion have strong mutual effects. It is important for us determine the potential applicability of analysis for marketing decisions. The paper is organized as follows. In the next section, we provide an overview of the @cosme site. Then, we propose our model of trust and rating in Section 3. Experimental results are shown in Section 4, where two classification problems are addressed. After contrasting the results with the theoretical model, we propose a new measure to calculate the bidirectional effects on each product in Section 5.
OVERVIEW OF @COSME
This comment provides a good insight into how a user has come to pay attention to other reviewers, and how reviewers influence others.
2.2
Data overview
We were provided the official user log data for more than five years, from December 1999 to April 2006. The data
Table 1: Most-reviewed products. #reviews product name 1 18717 eyelash curler 2 16599 nail polish 3 15126 deep cleansing oil 4 12287 hair oil Ohshima Tsubaki 5 10877 cleansing oil 6 10508 eye shadow 7 10238 eyelash curler 8 10086 liquid foundation 9 9808 petroleum jelly 10 9570 skin conditioner
Figure 1: Screenshot of the top page at @cosme.
Table #reviews 2179 2121 1927 1818 1779 1754 1705 1702 1678 1612
1 2 3 4 5 6 7 8 9 10
2: Active reviewers. user id registration date 93218 2002-04 91277 2002-04 230515 2003-07 80600 2002-01 564706 2005-06 5138 2000-04 148458 2002-12 360334 2004-04 68275 2001-10 116062 2002-08
age 26 27 42 28 26 39 33 38 27 27
140000 #reviews #new bookmarks #new users 120000
100000
80000
60000
40000
20000
0 01/01/1999 01/01/2000 01/01/2001 01/01/2002 01/01/2003 01/01/2004 01/01/2005 01/01/2006 01/01/2007
Figure 2: Number of new users, new bookmarks, and new reviews per month. consist of the following three tables. Product review (Kuchikomi) 4,310,346 records with user id, product id, date, and rating (1(bad) – 7(good)). Trust relation (Okiniiri) 530,598 records with user id, (her trusted) user id, and date. User profile 670,040 unique users with user id, date of registration, birth date, types of skin (six choices)2 , and profession (ten choices)3 .
2.2.1
Product Reviews
We introduce some further information about the data: Among 4,310,346 reviews overall, 72,522 products have at least one review. Therefore, one product has, on average, 59.4 reviews, which is quite a large number, showing high
activities among users in the community. About one-third of users have written at least one review. On average, a user posts 6.43 reviews. A user who posts at least once writes 20.47 reviews, on average. Table 1 shows the most-reviewed products. Low-price and commonly used products are listed higher, such as nail polish, cleansing oil, eye shadow, lotion, and so on. Table 2 shows the most active reviewers (i.e., the user with the largest number of reviews). The most active user joined in April 2002. She posted 2179 reviews since then for four years, which is about 1.5 reviews per day. The distribution of the number of reviews seemingly fits a power law, as shown in Fig. 3. We can see a strange gap between x = 9 and x = 10, which might be caused by the fact that a user with ten or more reviews can utilize a personalized recommendation function by the system. This motivates users to post ten or more reviews.
2.2.2
Trust Relation
The entire set of 530,598 trust relations comprises 49,685 users targeting 61,556 users. In other words, 7.4% of users use the bookmark function at least once, and 9.2% of users are trusted by others. On average, 10.7 trust relations are made by a single user who uses the trust function at all. Table 3 shows users with the largest number of fans. They can be considered highly influential. Interestingly, the correlation between the number of fans and the number of reviews 5
10
"res_dist.txt"
104
2
103 Count
The distributions are: mixed skin (40.0%), dry skin (19.2%), normal skin (15.8%), oily skin (12.2%), sensitive skin (10.1%), atopic skin (2.7%). 3 The distributions are: full-time employee (38.3%), full-time homemaker (13.5%), university/graduate student (12.3%), part-time worker (11.4%), high school student (9.0%), junior/career college (3.4%), junior-high/elementary school student (2.2%), employee in cosmetics/beauty business (1.2%), employee in survey/media company (0.3%), other (8.3%).
2
10
101
100 100
101
102 Number of reviews
103
104
Figure 3: Numbers of reviews by users and their count.
1 2 3 4 5 6 7 8 9 10
Table 3: #fans 2056 1638 1623 1599 1362 1335 1334 1325 1267 1167
Popular users. UserID #reviews 29261 905 396118 341 458268 48 93218 2179 243506 330 462931 276 232524 580 214089 804 119999 192 31388 72
5
5
10
10
"res_dist_okini.txt"
103
103
"res_dist_okini_in.txt"
Figure 7: Active reviewer network.
Count
104
Count
104
2
2
10
10
101
101
100 0 10
10
1
2
10 Number of bookmarks
3
10
Figure 4: Outdegree distribution of bookmarks.
10
4
100 0 10
10
1
2
10 Number of (in-)bookmarks
3
10
Figure 5: Indegree distribution of bookmarks.
is not high. Spearman correlations are as follows; between the number of reviews and the number of her trusted users, 0.703; between the number of reviews and the number of users trusting her, 0.658; and between the number of her trusted users and the number of users trusting her, 0.067. Figures 4 and 5 are the degree distributions when considering a trust relation as a directed edge. Both exhibit a linear relation on the log-log plots. Figure 6 and 7 are networks consisting of bookmark relations for the most-popular or most-active 100 reviewers. We can find that active reviewers are not connected densely, whereas the popular reviewers are more densely connected.
3. MODEL OF TRUST AND RATINGS After the brief introduction of the @cosme dataset given in the preceding section, we describe a model of bidirectional interaction of trust and rating in this section. We assume that trust relations are realized by the similarity of users, and that the rating of a product is influenced by her trusted users. First, we model the rating of a product from trust rela-
4
10
tions. The rating of user x on product i, denoted as s(x, i) (0 ≤ s(x, i) ≤ 1), is fundamentally determined by the preference of user x to product i. It is also influenced by the rating of other users that user x trusts. Therefore, we model the rating as the summation of her original evaluation s0 (x, i) and the rating of users she trusts. We denote the users who are trusted by user x as N (x). s(x, i)
= λ0 s0 (x, i) X 1 +λ1 t(x, y)s(y, i) |N (x)|
(1)
y∈N (x)
In the equation above, t(x, y) is the trust value [0,1] of user x to user y and λ0 and λ1 are constants. We divide the rating of trusted users by the number of her trusted users |N (x)| in order to take the average of ratings. (Otherwise, if a user trusts more users, her original evaluation gets less weighted, which contradicts the intuition.) This formula is justified empirically in the next section: The average of ratings of one’s trusted users is a good feature to infer the way she has been rated. The rating of user x to product i depends also on the quality of the product itself and characteristic of the user herself; a good product usually receives many good ratings, and some users tend to make good ratings more easily. However, for our purposes, we assume that the effects are included in the term s0 (x, i). Second, we model the trust relation of a user to another user based on the similarity of ratings. As shown in [6], trust is induced by the similarity of ratings between two users. It can also be induced in a transitive manner using the trust values of more distantly related users. Therefore, trust t(x, y) of user x to user y can be modeled as the summation of the similarity of two users plus the trust values calculated from transitive trust, as follows. t(x, y)
(2) = µ0 sim(s(x, I), s(y, I)) X 1 +µ1 X t(x, z)t(z, y) t(x, z) z∈{N (x)∩N (z)} z∈N (x)
Figure 6: Popular reviewer network.
Therein, I is a set of items, s(x, I) is the vector of rating on I, and sim(·) is the function to calculate the similarity between two vectors (e.g., cosine similarity and inner product). The second term, representing the transitive trust, considers only the effect of distance two, following the formulation in
[6]. The total amount of transitive trusts is normalized so that it does not increase even if a user is newly added. Below we use an inner product as a similarity measure for simplicity. As shown later, cosine similarity works well as a similarity measure. If we assume that the vector s is normalized so that the vector length would be 1, the inner product corresponds to cosine similarity. We first ignore the second term of Formula 2. If we substitute Formula 4.2 to Formula 2, after some transformations, we can obtain t(x, y)
=
1 Kxy
λ0 s0 (x, i)s(y, i) 0
µ0 λ1 @ + |N (x)|
(3) 1!
X
t(x, z)s(y, i)s(z, i)A ,
z∈N (x),z=y
where Kxy = 1 −
µ0 λ1 X s(y, i)2 . |N (x)| i∈I
This formula shows that the trust of user x to user y is determined by the similarity of rating and also the similarity among y to the other users. We can see that Kxy becomes large if s(y, i) gets large, meaning that users with good ratings on many items might be less trusted. Similarly, we can obtain s(x, i)
=
λ0 s0 (x, i) +
X 1 λ1 |N (x)| Kxy
λ0 s0 (x, i)s(y, i)2
y∈N (x)
µ0 λ1 + |N (x)|
X
!
t(x, z)s(y, i) s(z, i) 2
z∈N (x),z=y
This formula is complex, but we can understand it as follows: we assume that user x increases her rating on product i with ∆sxi . Then, t(x, y) increases by µ0 ∆sxi s(y, i); if user y has a high rating on product i, the increase on s(x, i) increases the similarity, resulting in the increase of trust t(x, y). If user y has a low (or zero) rating on product i, it does not bring much of an increase (sometimes even a decrease) of t(x, y)4 . Eventually, s(x, i) is increased by λ1 µ0 ∆sxi s(y, i)2 . |N (x)|
(4)
Therefore, increase of rating s(x, i) again brings the increase of rating s(x, i) itself by order of λ1 µ0 s(y, i)2 /|N (x)| through neighboring user y. The user eventually obtains a higher increase on s(x, i) if user x has many neighbors with a high rating on product i. In this way, the ratings of users become similar if they are connected closely. The bidirectional effect depends on the network structure around user x. Because we ignore the transitive trust in Formula 2, if we take into account the second term, there can be an additional effect caused by users within distances that are greater than 1. Therefore, the entire interaction is more complex, but the general trends hold. The rating of users x on product i increases • if λ1 and µ0 is high, meaning that a user tends to trust others and the trust tends to be determined by similarity, Because we assume s to be normalized, in case that s(y, i) is low, t(x, y) actually decreases. 4
• and if the ego-centric network of a user is clustered. These results show the similar rating on product i among neighbors of user x, thereby producing a cluster of users with similar preferences. Depending on product and user characteristics, λ1 and µ0 differ. In other words, the opinions of users on a particular product tend to be similar locally, which produces dense clusters that are emergent from the network. There is a limitation of theoretical analysis on this nonlinear interaction of users. Therefore, we next show empirical results from the use of @cosme data in the next section.
4.
DATA ANALYSIS OF @COSME
To validate both interactions presented in the previous section, we build two prediction problems: prediction for trust and prediction for rating of the product. Trust prediction is, given two users x and y, to predict whether a trust relation from x to y exists. It can be considered as a link prediction problem [3]. We use the features based on two users’ profiles, product ratings, and other trust relations. Rating prediction is complementary; given a user and a product, we seek to predict the rating. Features are generated using her profile, her ratings of the other product, and her trusted users, Both problem resolutions produce a prediction based on the data before the time point. Models to predict trust levels and ratings are learned by (binary) classification algorithms. Although confusing, rating prediction is not intended to predict the exact rating value of user x on product i. It predicts the existence of a rating of the product by users because it can be considered as a proxy of the user’s purchase on the product: @cosme allows a review only after a user purchases or performs an actual trial of the product. We must clearly distinguish the rating before purchase/trial and rating after the purchase. We also conducted an analysis to predict the exact value of the rating, which will be discussed in Section 6. Below we describe the two classification problems and the results.
4.1 4.1.1
Trust Prediction Features
To predict the trust from user x to user y, we use three kinds of features: each corresponds to a table in @cosme data. The overall features are shown in Table 4. The first type of feature is based on profile table; we use the properties of user x and user y, along with the difference and correspondence of the properties. The second type of feature uses the product-review table. The features are invented to measure the similarities of ratings of user x and user y on various products. For example if user x announces ratings on five products as {(P1 , 6), (P2 , 4), (P3 , 5), (P4 , 2), (P5 , 7)}, then Pi represents product i, as annotated by a rating of an integer [1,7]. Assume that user y announces ratings of four products: {(P1 , 5), (P2 , 5), (P6 , 7), (P6 , 3)}. Then, we can calculate the similarity using various measures. Most simply, we can count the overlap of two sets
Table 4: Features for trust prediction (from user X to user Y ). Group Profile
Rating
type skin profession age history good rating
bad rating
all rating
stats Trust
stats graph similarity
features skin-same (binary), skin-X (category), skin-Y (category) profession-same (binary), profession-X (category), profession-Y (category) age-X, age-Y, age-dif history-X, history-Y, history-dif good-product-matching, good-product-product, good-product-cos, good-productjaccard, good-brand-matching good-brand-product, good-brand-cos, goodbrand-jaccard, good-manufacturer-matching, good-manufacturer-product, goodmanufacturer-cos, good-manufacturer-jaccard bad-product-matching, bad-product-product, bad-product-cos, bad-productjaccard, bad-brand-matching, bad-brand-product, bad-brand-cos, badbrand-jaccard, bad-manufacturer-matching, bad-manufacturer-product, badmanufacturer-cos, bad-manufacturer-jaccard all-product-matching, all-product-product, all-product-cos, all-product-jaccard, all-brand-matching, all-brand-product, all-brand-cos, all-brand-jaccard, allmanufacturer-matching, all-manufacturer-product, all-manufacturer-cos, allmanufacturer-jaccard review-n-X, rating-ave-X, rating-std-X, review-n-year-X, popularity-X, over6-X, under2-X, review-n-Y, rating-ave-Y, rating-std-Y, review-n-year-Y, popularity-Y, over6-Y, under2-Y trusted-n-X, trusting-n-X, trusted-n-Y, trusting-n-Y distance common-neighbors-directional, common-neighbors-reverse, common-neighborsundirectional, Jaccard-directional, Jaccard-reverse, Jaccard-both, AdamicAdardirectional, AdamicAdar-reverse, AdamicAdar-undirectional, preferentialdirectional, preferential-reverse, preferential-undirectional
Note: All features are continuous (except some profile features). We could not explain all the features but explain some; The skin-same takes 1 when the skin types of two users are the same. The age-dif is the difference of ages (days after the birthday) of two users. The good-product-matching means the matching coefficient of reviews with good rating (6 or more score). The review-n-X means the number of reviews by user X. The over6-Y is the number of good ratings (6 or more score) by user Y . The trusted-n-X is the number of users whom user X trusts. The trusting-n-Y is the number of users who trusts user Y . The AdamicAdar-directional means the similarity within the directional trust network measured by Adamic/Adar index.
of reviewed products. In this case, P1 , P2 , P5 is rated by both users, so the matching coefficient is three. If we perform √ √ calculations using cosine similarity, the value is 3/ 4 5 = 0.67. Denoting a set of items rated by user x as I(x). The four measures we use are the following. • Matching coefficient: |I(x) ∩ I(y)| • Cosine similarity: |I(x) ∩ I(y)|/(|I(x)||I(y)|) • Jaccard coefficient: |I(x) ∩ I(y)|/|I(x) ∪ I(y)| • Product: I(x) · I(y)5 Users might refer to reviews with good ratings more often, or reviews with a bad rating more often. Therefore, we define a set of items with good/bad reviews as Igood (x) and Ibad (x) correspondingly. We define a good rating as one with a score of 6 points or more; a bad rating has 2 points or less. In the example, user x puts good rating on P1 and P5 , while user y put good rating on P7 . Then, we can define the overlap of Igood (x) and Igood (y), or Ibad (x) and Ibad (y) as well. Users might not be familiar with a product. However, sometimes they make a decision based on its brand, or its manufacturer. Users often have several favorite brands or manufacturers. Therefore, we can calculate the overlap of rated items as categorized by brands, or as categorized by products. Overall, we have 4 (#measures) × 3 (#sets) × 3 (product/brand/manufacturer) = 36 features. The third type of feature is derived from trust relations (except the very relation from x to y, which we seek to predict). Following the link prediction study [9], we build the following attributes. 5
We consider the set as a vector: it is useful when aggregating to each brand, or to each manufacturer.
• the number of neighbors for user x and y. • distance on the network. • common neighbors of user x and user y. • Jaccard coefficient of neighbors of user x and y. P 1 • Adamic-Adar, which is defined as z∈N (x)∩N (y) log |N . (z| • preferential attachment, defined as |N (x)||N (y)|. The trust network is constructed by trust relations of both directions (where we regard the trust relation from x to y as identical to trust relation from y to x), single relations (where we distinguish the relation from x to y and the relation from y to x), and the reverse direction (where we put a link from x to y if a trust relation from user y to user x exists). Therefore, we have three types of networks on each feature.
4.1.2
Results
We randomly choose 1000 pairs of users with trust relations and another 1000 pairs of unrelated users without relations; they respectively correspond to positive and negative sets. We use support vector machine (SVM) with a linear kernel [17] as a classifier6 . Tables 6 shows the performances of classifying trust relations. Each group of attributes contributes to the classification. Trust features and ratings features contribute greatly to the performance compared to profile features. The F1 value is 82.46% if we use all three groups. 6 We compared with several other classifiers including J4.8 and Naive Bayes. They produce similar results overall; the results worsen by a few points.
Table 5: Features for rating prediction (by user X for product A) Group Profile Rating
type user stat user product brand manufacturer
Trust
product brand manufacturer
features skin-X (category), profession-X (category), age-X, history-X user-total-n, user-total-ave, user-total-std, user-total-over6, user-total-under2, user-oldest-review, user-latest-review user-brand-n, user-brand-ave, user-brand-std, user-brand-over6, user-brandunder2, user-manufacturer-n, user-manufacturer-ave, user-manufacturer-std, user-manufacturer-over6, user-manufacturer-under2 product-review-n, product-review-ave, product-review-std, product-review-over6, product-review-under2, product-oldest-review, product-latest-review brand-review-n, brand-review-ave, brand-review-std, brand-review-over6, brandreview-under2 manufacturer-review-n, manufacturer-review-ave, manufacturer-review-std, manufacturer-review-over6, manufacturer-review-under2, trusted-review-n, trusted-rate-ave, trusted-rate-std, trusted-over6, trustedunder2 trusted-brand-review-n, trusted-brand-rate-ave, trusted-brand-rate-std, trustedbrand-over6, trusted-brand-under2 trusted-manufacturer-review-n, trusted-manufacturer-rate-ave, trustedmanufacturer-rate-std, trusted-manufacturer-over6, trusted-manufacturerunder2
We explain some features; The user-total-n is the number of reviews by user X. The user-total-ave and user-total-std are the average and standard deviation of the ratings by user X. The user-latest-review is the days after the latest review is posted by user x. The user-manufacturer-n is the number of reviews by user X on products made by the same manufacture of product A. The product-review-under2 is the number of ratings with score of 2 or under on product A. The brand-review-ave is the average rating of products with the same brand to procuct A. The trusted-review-n is the number of reviews posted by users trusted by user A. The trusted-brand-ave is the average rating of the brand of product A by users who are trusted by user X.
Table 6: Performance of trust prediction. Precision Recall F1 Attributes Profile 54.89% 53.18% 54.02% 77.38% 65.29% 70.82% Rating 90.04% 71.33% 79.60% Trust 77.55% 67.41% 72.12% Profile + Rating 89.78% 72.30% 80.10% Profile + Trust 88.73% 75.52% 81.60% Rating + Trust 88.10% 77.51% 82.46% All
Table 7: tion. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Highly weighted features in trust predictrusting-n-Y trusted-n-X Jaccard-undirectional Jaccard-directional all-product-cos Jaccard-reverse AdamicAdar-directional common-neighbors-directional all-product-jaccard review-n-Y common-neighbors-undirectional bad-product-cos oveer6-X bad-product-jaccard popularity-X
5.6075 5.3342 3.8225 3.7014 2.8291 2.6943 1.7409 1.5182 1.4858 1.1943 1.1883 -1.1712 -1.0746 -1.0309 -1.0119
Table 7 shows features with the highest weights in the obtained model by SVM classifier. We can see that the number of trusted users for y and the number of trusted users by x are the two highest features, which might be readily apparent: highly trusted users are likely to be trusted by a particular user; a user who trusts many others is likely to trust another. Some features in the table are interesting: Jaccard-directional is the overlap of user x’s trusted users and trusted users by Next, we build the rating prediction problem. The feauser y, which implies the transitivity of trust relations. The tures we used, as shown in Table 5, are categorized into all-product-cos is the similarity of all rated items by user profile, rating, and trust, as well as trust prediction. x and those by user y. This can be understood that the Considering the rating by user x of product i, the profile similarity is measured using the cosine of the rated product. features are simply the properties of user x. The rating By selecting four highly weighted features and applying features are: the number of ratings by user x, average of regression using SVM, we can obtain the following formula. rating by user x, standard deviation of the ratings by user x, the number of good ratings, and the number of bad ratings. t(x, y) = µ0 cos(s(x, I), s(y, I)) We also calculate these values for brands and manufacturers. X 1 Then we make summations of ratings on product i: the +µ1 X t(x, z)t(z, y) t(x, z) z∈{N (x)∩N (z)} number of reviews, the average and standard deviation of z∈N (x) ratings, and so on. As for the trust relation, we aggregate the ratings by users ∼ 0.34 × trusting n Y + 0.31 × trusting n X who are trusted by user x. The summation, average, and +0.09 × all product cos so on are calculated to the product, the brand, and the +0.25 × Jaccard directional manufacturer. The result is shown in Table 8. We can see that ratings 4.2 Rating prediction of features and trust features have comparable performance.
Table 8: Performance of rating prediction. Precision Recall F1 Attributes Profile 53.14% 64.36% 58.21% 89.34% 79.73% 84.26% Rating 83.27% 46.39% 59.59% Trust 89.32% 79.62% 84.19% Profile + Rating 81.77% 46.86% 59.58% Profile + Trust 90.01% 83.76% 86.77% Rating + Trust 89.85% 81.85% 85.66% All
Table 9: Highly weighted features in rating prediction. 1 product-review-n 4.5135 2 product-review-over6 3.8816 3 user-total-n 2.5622 4 product-latest-review -2.075 5 product-review-under2 1.8962 6 user-brand-ave 1.706 7 trusted-rate-ave 1.6909 8 trusted-under2 1.5926 9 trusted-brand-review-n -1.4693 10 user-manufacturer-ave 1.4477 11 user-total-over6 1.2228 12 user-manufacturer-n 1.0543 13 brand-review-std -1.0429 14 trusted-manufacturer-review-n -1.0241 15 trusted-review-n 1.015
Table 10: A list of brands with the highest BEM measure. BEM 0.142 0.131 0.122 0.0997 0.0963 0.0745 0.0741 0.0729 0.0728 0.0713
brand Majolica Majorca Chanel Yves Saint Laurent Anna Sui Cosmetics Kate Esfield Lush Baby Pink Guerlain, Canmake
manufacturer Shiseido Chanel Yves Saint Laurent Parfums Anna Sui Cosmetics Kanebo Sfield Lush Bison Guerlain Ida Laboratories
reasonable to consider a brand as the major medium of the bidirectional effects. It is important to measure the bidirectional effects on each brand from a marketing point of view. We defined a new index, called bidirectional effect measure (BEM) on brand b, which considers the effects from trust to rating, and from rating to trust as follows. BEM (b) = µ0 (b) × λ1 (b)
In that equation, µ0 (b) is the coefficient of the regression by Formula 2 and λ1 (b) is the coefficient by formula 4.2. Because the user’s rating increases in a reflective manner in proportion to µ0 λ1 , as shown in Fig. 4, this measure represents a fundamental value of the brand characteristics. If we use all the features, F 1 is 86.8% using SVM. Table 10 shows the products with high E values. We can The highly weighted features can be found in Table 9. We see that some major brands have high BEM values: For excan see that the number of reviews of products and the numample, Majolica majorca is a Shiseido’s make-up brand for ber of reviews by the user are the important features. Interesting features include user-brand-ave and okiniiri-rate-ave. young consumers with strong personality; The key concept of this brand is metamorphosis and magic in a fairy tale. A user has favorite brands, so the average of ratings of the The users of Lush and Anna Sui cosmetics are known have brand is a good feature. The average of ratings of users extremely high brand loyalty. Its unique brand concept has whom the user trusts is also a good feature, justifying the attracted cult, resulting in high BEM. Thus, high BEM imeffect of trusted users in Formula . plies that the brand is strong because it can create strong By selecting highly weighted features and applying regresuser communities. sion, we can construct the following model. s(x, i)
=
λ0 + s0 (x, i) X 1 +λ1 N (x)
5.2 t(x, y)s(y, i)
y∈Trusted(x)
∼
0.27 × user total n + 0.42 × product review n +0.16 × user brand ave +0.14 × trusted rate ave
5. BIDIRECTIONAL EFFECT 5.1 Measuring bidirectional effect In the above analysis, we can infer that µ0 is 0.25 and λ1 = 0.14 for overall users and products in @cosme. However, depending on the product, this value varies. A product, a brand, or a manufacturer with large µ0 and λ1 shows much bidirectional effects. Especially, when a brand has a large bidirectional effect, it strengthens the trust and rating on the brand in a particular community. Consequently, users become more connected, and a new product prevails easily in the community. Each cosmetics manufacturer strives to establish and strengthen its own brands. Therefore, it is
Product Propagation Network
Although BEM is based on the theoretical model described in Section 3, the effectiveness of the measure is not apparent. We attempt to investigate the difference of user behavior depending on different BEM values. We can build a propagation network that is similar to Leskovec’s recommendation network [8] using product reviews and trust relations. We regard a review by trusted persons as a recommendation. If Alice registers Betty as trusted, and Betty puts a good rating on product i at time t, then we regard it as a recommendation from Betty to Alice on product i that occurred at time t. Because @cosme allows a review only after a user purchase or a trial of a product, we can regard a review as a proof of purchase: we can confidently infer that Alice bought i before time t if Alice has a review on product i at time t. Then we can define the success of propagation as follows: If Alice receives a recommendation on product x from Betty at time t1 , and if Alice writes a (first) review on product i at time t2 where t1 < t2 < t1 + T 7 , we consider that the 7
We set T as 180 days.
6
success rate of recommendation
5
4
3
2
1
0 0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
BEM
Figure 10: Plot of BEM and recommendation success rate. Figure 8: Propagation network on an eyelash expander of the most high-BEM brand. (n = 630, e = 858)
Figure 11: Bidirectional interaction of trust and rating
Figure 9: Propagation network on a lotion with the most popular brand (but with low BEM ). (n = 581, e = 496) recommendation is successful. We can draw a propagation network for various products. Figure 9 shows the (success) propagation network on an eyelash expander of Majolica Majorca, which has the highest BEM . We can see there are a core group of users who are densely connected each other. These users diffuses the product to more peripheral users. On the other had, Figures 8 display the network for a lotion of DHC, which is the most popular brand with numerous product reviews, while the BEM value is low (in the 83th place). Although there are almost same numbers of nodes and edges, the network is more flat than that of Majolica Majorca. Users do not make a big clusters. Success rates of propagations are 1.82%, and 0.47% for these two networks. We show a scatter plot of the recommendation success rate and the BEM value for popular brands in Fig. 10. We can view the correlation of these two values. In summary, the BEM measure is a good index to represent a user’s bidirectional effects on trust and rating. A high BEM value implies the power of the brand to produce strong user communities.
6. DISCUSSION So far, we have addressed the interaction between trust and rating. However, theoretically, the rating of a product
changes over time. We buy a product because we think it is good when we purchase it. Through using the product, our own rating varies. Sometimes, we are very satisfied with the product; sometimes we are disappointed with the product after using it. The rating after use depends on various elements such as price, package, and quality. Considering this, there can be (at least) two loops of bidirectional interaction between trust and relation, as shown in Fig. 11. One is the loop consisting of the trust and the rating before use; another is the loop consisting of the trust and the rating after use. The analyses of this paper employ the former, and use the existence of reviews as a surrogate of rating of the product before use. Although we showed no results on the rating after use, we conducted similar classifications of the ratings after use based on trust. The performance is not high compared to the rating before use, which means that the rating after use is independent from the rating of trusted users to a larger degree. Moreover, it is apparent from Table 9 that the trust value is well predicted by the overlap of reviews on all products by user X and user Y, and the overlap of good/bad reviews is not a highly weighted feature. This also supports the validation of loop A, as described in the paper. We assume that, depending on products, the effective loop might be different, but it is a difficult problem to decide in what timing the rating should be used because the rating changes over time. The cosmetic product can be considered as an experience good [12], where the product quality is ascertained only after consumption. This uncertainty regarding the product quality is apparent in various experience goods such as movies and books, but what is unique about the cosmetic product is the presence of physical risk. A physical risk is risk which jeopardizes physical vigor, health, and vitality [14]. Drugs and medical treatment are most sensitive; cosmetic products are also highly sensitive to this risk. The type of good
and the presence of perceived risk makes the product rating useful and valuable for consumers. Our future works include improvement of the model of trust and rating. We assumed, and validated, that the trust relation is built based on the similarity of users. However, trust might not correspond exactly to similarity. Actually, there is apparently a correlation of similarity and trust, but the feeling of admiration or respect between users might influence the trust to a great degree, which can be captured by more elaborate generation of features. The rating of a product by a user is influenced by the ratings of trusted users. We observed that the low rating by trusted users and a high rating of all users have more weight. Consequently, the aggregate rating can be improved depending on the relation of the reviewer and the user. We conducted a simple theoretical analysis in Section 3 because of the nonlinear nature of the formulae. Actually, we can analyze the formula in the form of a matrix calculation. More advanced analysis is also possible through multi-agent simulations such as an agent interaction model in a social network, which is the subject of ongoing research. The difficulty of the application of our method (especially the BEM measure) lies in the fact that we can not easily obtain the trust data and the rating data related to an online community. However, the algorithm can be potentially applied to online shopping sites (such as Amazon.com and eBay) if users can bookmark other users. The development of social networking service might enable the use of social network data at online shopping sites, providing the opportunity to use our algorithm to understand the bidirectional effects of products. Our algorithm is applicable to several existing data: e.g., published data (such as DBLP, Citeseer and Cora database) have information about papers presented at conferences (which corresponds to purchase of a product), and the co-authorship of a paper or co-affiliation to an institute (which corresponds to trust in a person). Consequently, the brand value of a conference, a journal, or an academic field can be measured. By Web mining of a research domain (such as [11]), we can extract such information.
7. CONCLUSION In this paper, we have described the bidirectional effect of trust and rating theoretically and empirically using data on a viral marketing site @cosme. We modeled and validated that the trust is influenced by the similarity of reviews, and that the rating is influenced by the trusted users’ ratings. Depending on a product, this bidirectional effect is large, resulting in highly clustered user groups. It can be considered as brand strength from a user-interaction point of view. Although our model is evaluated only for a particular dataset because of its necessary characteristics, the bidirectional interaction seems to be an essential model for many other online social communities, and the proposed measure BEM is potentially applicable to them. We hope that our study can provide useful insights and evidence to further elucidate mining and analysis of online communities.
8. REFERENCES [1] E. Adar, L. Zhang, L. Adamic, and R. Lukose. Implicit structure and the dynamics of blogspace. In Workshop on the Weblogging Ecosystem, 2004.
[2] T. Furukawa, Y. Matsuo, I. Ohmukai, K. Uchiyama, and M. Ishizuka. Social networks and reading behavior in the blogosphere. In Proc. ICWSM 2007, 2007. [3] L. Getoor and C. P. Diehl. Link mining: A survey. SIGKDD Explorations, 2(7), 2005. [4] J. Golbeck. Trust and nuanced profile similarity in online social networks. Journal of Artificial Intelligence Research, 2006. [5] J. Golbeck and J. Hendler. Inferring trust relationships in web-based social networks. ACM Transactions on Internet Technology, 7(1), 2005. [6] J. Golbeck and B. Parsia. Trust network-based filtering of aggregated claims. International Journal of Metadata, Semantics and Ontologies, 2006. [7] D. Kempe, J. Kleinberg, and E. Tardos. Maximizing the spread of influence through a social network. In Proc. 2003, 2003. [8] J. Leskovec, L. Adamic, and B. Huberman. The dynamics of viral marketing. In Proc. ACM Conference on Electronic Commerce, 2006. [9] D. Liben-Nowell and J. Kleinberg. The link prediction problem for social networks. In Proc. CIKM, pages 556–559, 2003. [10] C. Marlow. Investment and attention in the weblog community. In Proc. HyperText 2006, 2006. [11] Y. Matsuo, J. Mori, M. Hamasaki, H. Takeda, T. Nishimura, K. Hasida, and M. Ishizuka. POLYPHONET: An advanced social network extraction system. In Proc. WWW 2006, 2006. [12] P. Nelson. Information and consumer behavior. Journal of Political Economy, 78(2):311–329, 1970. [13] M. Richardson and P. Domingos. Mining knowledge-sharing sites for viral marketing. In Proc. SIGKDD’02, 2002. [14] M. Solomon. Consumer Behavior: Buying, Having, and Being. Prentice Hall College Div, New Jersey, 7th edition edition, 2006. [15] X. Song, Y. Chi, K. Hino, and B. Tseng. Information flow modeling based on diffusion rate for prediction and ranking. In Proc. WWW2007, 2007. [16] M. Subramani and B. Rajagopalan. Knowledge sharing and influence in online. social networks via viral marketing. Communications of the ACM, pages 300–307, 2003. [17] V. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, 1995. [18] S. Wasserman and K. Faust. Social network analysis. Methods and Applications. Cambridge University Press, Cambridge, 1994. [19] D. Watts. Six Degrees: The Science of a Connected Age. W. W. Norton & Company, 2003. [20] H. Yamamoto. Understanding the online community member and their purchase behavior: A structural modeling approach. Advances in Consumer Studies, 11(1,2), 2005. [21] C. Ziegler and J. Golbeck. Investigating correlations of trust and interest similarity. Decision Support Services, 2006.
