Assessing the Importance and Relationships of ...

Journal of Quality Assurance in Hospitality & Tourism, 11:73–92, 2010

Copyright © Taylor & Francis Group, LLC ISSN: 1528-008X print/1528-0098 online DOI: 10.1080/1528008X.2010.482000

Assessing the Importance and Relationships of Ratings on User-Generated Traveler Reviews BETSY BENDER STRINGAM School of Hotel, Restaurant & Tourism Management, New Mexico State University, Las Cruces, New Mexico, USA

JOHN GERDES, JR.

Downloaded By: [Gerdes, John] At: 16:57 7 June 2010

School of Hospitality, Retail & Sport Management, University of South Carolina, Columbia, South Carolina, USA

DAWN M. VANLEEUWEN Department of Economics & International Business, New Mexico State University, Las Cruces, New Mexico, USA

The growth of Internet travel sites has increased the importance of the traveler reviews provided on those sites. These reviews play an important role in travelers’ hotel selection. Expedia allows travelers to rate hotels not only on overall satisfaction, but also on four important aspects of their hotel stay (Hotel Service, Hotel Condition, Room Cleanliness, and Room Comfort), and their willingness to recommend the hotel to other travelers. This study looked at over 60,000 traveler ratings. The majority of all four quality ratings and overall satisfaction rating were positive. Additionally, nearly 75% of responses indicated willingness to recommend the hotel. All four quality ratings were found to have similarly strong associations with travelers’ overall rating and willingness to recommend. Partial correlations differed, with hotel service and room comfort being highest, followed by hotel condition then room cleanliness. KEYWORDS hotel, rating, quality, service, cleanliness, Internet

Address correspondence to Betsy Bender Stringam, School of Hotel, Restaurant & Tourism Management, MSC 3 HRTM, 129 Gerald Thomes Hall, New Mexico State University, Las Cruces, NM 88003. E-mail: [email protected] 73

74

B. B. Stringam et al.


INTRODUCTION Hoteliers have long sought to understand the factors influencing the guest room purchase process (Lewis & Pizam, 1981; Lockyer, 2005a; Matilla & O’Neill, 2003). Consumers use different means when evaluating different lodging alternatives. Some travelers rely on word of mouth asking friends, coworkers and other associates for recommendations about where to stay, while others seek the advice of experts in the field relying on travel agents, travel reviews and travel rating systems. Traditionally, the hotel industry has been subject to rating systems that assess the quality and service of the facility (Ingram, 1996; Vallen & Vallen, 2005, Vine, 1981). These rating systems have varied from country to country (Cser & Ohuchi, 2008). In some countries, rating systems have been administered by government organizations or tourism associations. Depending on the country, these organizations and associations exert different levels of control and regulation. In some countries, including the United States, the rating of hotels has been performed by commercial organizations. Most traditional ratings systems use the symbol of the star in their reporting, with one star indicating poor hotel condition, poor service, or very limited amenities, and five stars indicating fine service, excellent hotel conditions, and extensive amenities (Ingram, 1996; Yu, 1992). Many of the traditional hotel rating systems provide explicit criteria which must be achieved to obtain specific ratings (Bevans, 2008). These standards include such things as the square footage of recreational amenities, hours of operation for hotel services, condition of the hotel property, and the levels and consistency of service delivery. While achieving high ratings is never easy, hotels for the most part have been able to discern the requirements of the ratings systems and plan their resources accordingly (Cotter & Snyder, 1996). However, with the emergence of Internet-based travel sites the primary source of hotel rating information for consumers has changed. The distribution of hotel rooms via the Internet and the phenomena of technology based social networking are changing the selection process for hotels and the ratings systems (Cannizzaro et al., 2007; Gazzoli, Kim and Palakurthi, 2008; Steinbrink, 2008). The year 2008 experienced $105 billion of U.S. travel sales booked online, with almost 25% of the hotel reservations distributed through third-party travel companies, such as Expedia, Travelocity, Priceline, Orbitz, etc. (Leggatt, 2008). These sites act as both travel agent and travel advisor, providing consumers access to airline tickets, hotel reservations, rental car reservations, and other travel services, as well as providing advice and recommendations. In many cases these third party travel companies also serve as distributor


Ratings and User-Generated Traveler Reviews

75

when hotels, and franchise and management companies contract with them to distribute, or rent their hotel rooms. Social networks, blogs, video, podcasts and user generated reviews have revolutionized the way travel information is communicated (Cannizzaro et al., 2007). Today, many of the third-party travel sites provide user generated reviews and ratings. While the ratings vary in format, most user generated rating systems resemble the traditional star-rating systems, but may be based on traveler perceptions rather than the clearly defined criteria used in traditional rating systems. For many travelers these consumer generated ratings have replaced the ratings and recommendations of travel experts (Cannizzaro et al., 2007; Steinbrink, 2008). Hotels with extensive amenities and services can find themselves with low ratings as a result of consumers’ opinions (Clausing, 2009). Hoteliers are understandably concerned about the impact of negative reviews and ratings (Enz, 2009). High consumer-generated ratings and praise can be helpful to a hotel, while low consumer generated ratings and criticism can be injurious (Gelb & Sundaram, 2002). Since research has shown that properties with higher ratings are able to achieve higher room revenues, user generated ratings and reviews can have a significant effect on the profitability of a hotel (Cotter & Snyder, 1996, Henley, Cotter, and Herington, 2004). Negative recommendations from consumers are spread further and faster than positive recommendations (Anderson 1998; Oliver & Swan, 1989). This viral propagation of user generated reviews makes consumer recommendations a fearful phenomenon as hoteliers cannot always guarantee 100% customer satisfaction (Chipkin, 2005). Many hoteliers report that reviewing consumer generated reviews and ratings of other hotels provides a valuable insight into current consumer concerns (Clausing, 2007). Savvy hoteliers currently read and analyze their own ratings (Clausing, 2007). Many hoteliers also review the comments and ratings of their direct competitors. Yet, heretofore a comprehensive review of ratings across the industry has not been available. This study seeks to determine the relationships between the consumer’s overall rating of the hotel and their ratings that focus on four specific aspects of the hotel’s operation, namely Hotel Service, Hotel Condition, Room Cleanliness, and Room Comfort. This work also assesses how these four individual factors impact the likelihood that travelers would recommend a hotel to another. This study can assist hoteliers in their resource allocation by identifying those factors which have the most impact on the traveler’s overall ratings. Prior research has already established price (Bojanic, 1996), cleanliness (Lockyer, 2003), hotel maintenance (Hanai, Oguchi, Ando, & Yamaguchi, 2007), and service (Chan & Wong, 2006; Heide & Grønhaug, 2009; Shanka & Taylor, 2004) as important factors influencing the hotel selection process. This study seeks to reexamine these associations within the context of online user generated content.

76


LITERATURE REVIEW


Guest Satisfaction Many factors contribute to guest satisfaction. Guests expect friendly, helpful, polite, and prompt service (Atkinson, 1988; Schall, 2003; Wilkins, Merrilees, & Herington, 2007). Cleanliness of the hotel room has been shown to contribute significantly to guest satisfaction (Atkinson, 1988; Kandampully & Suhartanto, 2000; Prayukvong, Sophon, Hongpukdee & Charupas, 2007; Schall, 2003). Lockyer (2003) found a direct relationship between cleanliness of a hotel and the guest’s intent to return. Some experts argue that the style or “feel” of a hotel contributes to guest satisfaction (Wilkins et al., 2007). Researchers have proposed various models to help determine guest satisfaction (Crotts, Pan, & Raschid, 2008; Deng, 2008; Matzler, Fuller, Renzi, Herting, & Spath, 2008). Comparisons have also been made considering market segments and guest satisfaction (Gunderson, Heide, & Olsson, 1996; Hanai et al., 2007; Matzler et al., 2008). Hoteliers have long sought the opinion of the guest to help improve guest satisfaction (Barsky, 1996). Research has shown that feedback regarding hotel operations and guest attitudes are the two most vital elements of determining guest satisfaction (Schall, 2003). Yet, hotel management may have a different opinion from the guest concerning what factors contribute to overall satisfaction (Lockyer, 2005a). Research has examined the purchase decisions of hotel accommodations. Price and price-value relationships have been shown to be an important factor in hotel selection (Lockyer, 2005b; Matilla & O’Neill, 2003). Researchers have also found that convenient hotel location and good service influence hotel selection (Chan & Wong, 2006).

Purchase Decisions Using Internet Reservation Sites Research has examined different aspects related to the distribution of hotel rooms on the Internet. Many studies have explored consumer behavior and web site characteristics of hotel, travel agency and tourism web sites (Bender & Gerdes, 2007; Brewer, Feinstein, & Bai, 2006; Carroll & Siguaw, 2003; Christodoulidou, Brewer, & Countryman, 2007; Gazzoli, Kim, & Palakurthi, 2008; Law & Bai, 2008; O’Connor & Frew, 2002; Pekar & Ou, 2008). Studies examining web site quality of online travel distribution sites have found quality and placement of pictures, ease of use, and information/content to affect travelers’ likelihood to use a web site for travel reservations (Bender & Gerdes, 2007; Park, Gretzel, & Sirakaya-Turk, 2007).

Rating Systems Hotel rating systems have often been debated and contested (Callan, 1995; Vine, 1981). Consumers and researchers expect rating systems to provide an


77

accurate insight into hotel accommodations (Barth & Walsh, 1997; Callan, 1995; Callan, 1998). Ratings can have a strong influence on pricing, guest perceptions, and hotel operations (Cotter & Snyder, 1996; Henley, Cotter and Herington, 2004; Israeli, 2002). As consumer generated rating systems for hotels replace the traditional rating systems there is a need for further research.


User Generated Content for Travel Consumers consider traveler recommendations on travel sites to be more credible and to have a greater effect on the purchase of online travel than recommendations found on hotel sites or online travel agencies (Cannizzaro et al., 2007; Gretzel & Yoo, 2008; Salerno, 2008). Consumers use online travel review sites to gather information that is more up to date, more detailed, and more relevant than can be found through traditional travel resources (Gretzel & Yoo, 2008). Research has shown that 88% of leisure travelers reported being influenced by online travel reviews (Steinbrink, 2008). Hotel rooms are the most subjective of all travel purchases, and as such consumers rely heavily on user-generated recommendations and ratings when choosing a hotel (Steinbrink, 2008). Furthermore, research has shown a significant relationship between online hotel reviews and hotel financial performance (Ye, Law, & Gu, 2009). Despite this prevailing change in how travelers evaluate hotel options and make their hotel purchase decisions, very little research has been conducted examining online user-generated travel reviews and ratings. Gretzel and Yoo (2008) found that travel review readers seek information from virtual travel communities, travel guidebook sites, and travel distribution sites two to three times more often than from company-owned sites. In their study, they found that more than 90% of travel review readers using TripAdvisor considered other travelers’ reviews to be helpful for learning about travel destinations, products, or services, as well as for avoiding places and services they would not enjoy (Gretzel & Yoo, 2008). Pekar & Ou (2008) developed a model for reviewing free-text comments from review sites. Ye et al. (2009) developed a model to determine if online user reviews affect hotel room sales. Several research studies have examined the word usage in online user generated comments. They found travelers to be concerned with cleanliness, hotel location, hotel guest room size, staff, hotel facilities, and breakfast (O’Connor, 2008; Stringam & Gerdes, 2010).

METHODOLOGY Data used in this study were obtained from publicly available hotel reviews posted on the Expedia website. Expedia, Inc. maintains one of the world’s leading online travel companies, namely Expedia.com (Williams, 2007).


78


Expedia reports $17 billion in annual gross travel bookings (Cannizzaro, et al., 2007; Expedia, 2009). Through this online travel distribution site travelers can access information about hotels, airlines and other travel products. Expedia offers travelers’ reviews and ratings of hotels. To ensure that reviews are based on first-hand experience, Expedia only accepts reviews of a hotel from travelers who have booked and paid for a reservation for that hotel through their website. Travelers are asked to provide an overall satisfaction rating for the hotel along with four subcategory ratings, which evaluate different aspects of their stay, namely Hotel Service, Hotel Condition, Room Cleanliness, and Room Comfort. Each score is based on a 1 (poor) to 5 (excellent) rating system. Expedia also asks travelers if they would recommend the hotel to others. For the purposes of this study, we focus on the travelers’ Overall Satisfaction rating, the travelers’ propensity to recommend the hotel, and the four subcategory ratings of Hotel Service, Hotel Condition, Room Cleanliness, and Room Comfort. During the period from December 18th to 29th, 2007 an automated web spider (Gerdes, Stringam, & Brookshire, 2008) visited Expedia.com and collected traveler reviews for all hotels listed by Expedia for the 100 largest U.S. cities (U.S. Census Bureau, 2007). The spider gathered all available reviews and all available data for each traveler review adhering to the site’s Robot Exclusion Standard restrictions (Koster, 2007). This consisted of the traveler’s overall rating for the hotel, the four subcategory ratings (Hotel Service, Hotel Condition, Room Cleanliness, and Room Comfort), and whether the traveler would recommend the hotel. During this time, the spider collected data on 10,537 hotels as well as 60,648 customer comments. The data were analyzed to determine relationships between the four subcategory ratings and both the traveler’s overall satisfaction rating, and willingness to recommend the hotel. Summary statistics of the subcategory variables were generated and reported, including the frequencies, means, and standard deviations. Relationships among variables, particularly the relationship of the four subcategory variables to overall satisfaction and willingness to recommend, were explored using correlations (Pearson and point-biserial), partial correlations, general linear models (both ANOVA and regression), and cross-tabulations. Data were analyzed using SAS version 9.1.3 software (SAS Institute, Inc.). Associations of categorical rating variables with the recommendation variable were assessed using a chi-square statistic using a significance level of α = .05.

RESULTS Hotel Rating Descriptive Statistics Respondents tended to rate all variables highly. For all five variables, Hotel Service, Hotel Condition, Room Cleanliness, Room Comfort, and Overall

79

Ratings and User-Generated Traveler Reviews TABLE 1 Descriptive Statistics for Subcategories and Overall Satisfaction Rating Category Hotel Service Hotel Condition Room Cleanliness Room Comfort Overall Satisfaction

1

2

3

4

5

Mean

SD

4.61% 6.16% 4.66% 5.20% 5.33%

6.60% 8.33% 6.02% 7.50% 8.10%

14.04% 15.13% 11.66% 13.85% 13.72%

31.59% 30.96% 28.96% 28.77% 32.20%

43.16% 39.42% 48.71% 44.68% 40.65%

4.02 3.89 4.11 4.00 3.95

1.12 1.19 1.12 1.16 1.16

One-way frequencies percentages and means and standard deviations (n = 60,648).


Satisfaction, at least 70% of the responses were either a level 4- or 5rating (Table 1). Respondents also tended to be willing to recommend the hotels they reviewed. Of the 60,648 responses, 74.51 % indicated they would recommend their hotel.

Traveler Overall Satisfaction Each of the subcategory ratings (i.e., Hotel Service, Hotel Condition, Room Cleanliness, Room Comfort) was significantly related to Overall Satisfaction rating, with correlations between the individual subcategories and Overall Satisfaction falling between .83 and .87 (see Table 2). For each subcategory the mean Overall Satisfaction was observed to increase steadily with increasing level of the subcategory rating (see Table 2). For example, 2,797 individuals assigned a 1-star rating to Hotel Service. For these individuals, the mean Overall Satisfaction rating was 1.37 with a standard deviation of 0.66. Looking at the subset of individuals that assigned ratings of 2, 3, 4, and 5 to Hotel Service, the corresponding mean Overall Satisfactions were 2.16, 3.03, 3.97, and 4.78, respectively. A similar trend was observed for Overall Satisfaction when summarized by the other subcategories. Subcategory ratings were highly correlated with one another, with correlations between subcategory pairs falling between .71 and .83 (see Table 3). There is a clear lack of independence among the subcategory ratings. To extract the portion of the association between a subcategory rating and Overall Satisfaction that was unique to that subcategory, partial correlations were also summarized in Table 2. These partial correlations were adjusted for all three of the other subcategories. The partial correlations for Hotel Service and Room Comfort were the highest at .49 and .45 followed by the partial correlation for Hotel Condition (.35) and Room Cleanliness (.18). This suggested that, after accounting for the influence of the other three subcategories, Hotel Service and Room Comfort accounted for the most variability remaining in Overall Satisfaction. After adjusting for the other variables, Room Cleanliness accounted for the least variability remaining in Overall Satisfaction. A main-effects analysis of variance (ANOVA) with Overall

80

n 1.37 2.16 3.03 3.97 4.78 .83 .49 .31

.66 .80 .82 .68 .50

OS Mean OS SD 3, 737 5, 049 9, 176 18, 778 23, 908

n 1.39 2.35 3.25 4.09 4.84 .87 .35 .26

.60 .73 .74 .62 .42

OS Mean OS SD

Hotel condition

2, 824 3, 649 7, 070 17, 565 29, 540

n 1.36 2.11 2.92 3.83 4.74 .84 .18 .13

.61 .75 .80 .69 .53

OS Mean OS SD

Room cleanliness

3, 156 4, 549 8, 400 17, 446 27, 097

n

1.35 2.19 3.10 3.96 4.80 .86 .45 .33

OS Mean

Room comfort

.59 .74 .76 .61 .46

OS SD

OS = Overall Satisfaction. ∗ = Correlation between the Overall Satisfaction and subcategory ratings. ∗∗ = Partial correlations between Overall Satisfaction and a subcategory rating are based on adjusting both Overall Satisfaction and the subcategory rating for all three other Subcategory ratings. + = Standardized betas from the multiple regression with overall satisfaction as the response variable.

1 2, 797 2 4, 005 3 8, 512 4 19, 157 5 26, 177 Correlation∗ Partial Correlation∗∗ Standardized Betas+

Rating

Hotel service

TABLE 2 Descriptive Statistics (Means and Standard Deviations, Correlations and Partial Correlations) for Overall Satisfaction by Level of Each Subcategory


81

Ratings and User-Generated Traveler Reviews TABLE 3 Correlations Among Subcategories Subcategory


Hotel Service Hotel Condition Room Cleanliness Room Comfort

Hotel service

Hotel condition

Room cleanliness

Room comfort

1.00 – – –

.74 1.00 – –

.72 .83 1.00 –

.71 .80 .81 1.00

Satisfaction as the response variable and all four subcategories as categorical explanatory variables produced an R-squared value of .8797. That is, the 16 degrees of freedom (df) model fitting all four subcategory variables explained 88% of the variability in Overall Satisfaction. A simpler, 4 df regression model that treated subcategory ratings as quantitative variables produced nearly the same explanatory power with an R2 value of .8793. This model produced the following standardized estimates—for Hotel Service, .31; for Hotel Condition, .26; for Room Cleanliness, .13; and for Room Comfort, .33 (see Table 2). These standardized betas can be interpreted as follows. For Hotel Service, the standardized beta of .31 indicates that, controlling for Hotel Condition, Room Comfort and Room Cleanliness the model predicts a .31 standard deviation increase in the mean of Overall Satisfaction for every one standard deviation increase in Hotel Service. The standardized betas provide a similar picture of the relationships between the subcategory and Overall Satisfaction ratings as do the partial correlations. Based on both approaches to analyzing the data, after adjusting for other subcategory variables, Hotel Service and Room Comfort have similar explanatory power which is greater than the explanatory power of Hotel Condition, which in turn has greater explanatory power than Room Cleanliness. The relationships observed in the partial correlations may be explained in part by considering correlations among subcategory ratings (Table 3). While all subcategories were highly correlated with one another, correlations of Hotel Condition, Room Cleanliness, and Room Comfort with Hotel Service fell between .71 and .74, while correlations among Hotel Condition, Room Cleanliness, and Room Comfort were somewhat higher with observed values between .80 and .83 (Table 3). That all correlations were relatively high suggests either the existence of some factor (possibly related to the individual’s overall experience) that impacts all subcategory ratings, or it may reflect the inherent quality of the hotel that they are rating. However, the high partial correlation of Hotel Service with Overall Satisfaction may be due, in part, to the relatively lower correlation that this variable has with the other three subcategory ratings. Further exploring relationships among the variables through frequencies and ratings patterns provided insight into the relationships among


82


the subcategories and the relationship of the subcategories to the Overall Satisfaction rating. It can also provide perspective on the information contained in the correlations and partial correlations. Of the 60,648 responses, 25,514 (42.1%) gave the same rating to all four subcategories, with 1,122 rating all four subcategories a 1; 464 rating all a 2; 1,364 rating all a 3; 5,863 rating all a 4; and 16,701 rating all subcategories a 5. Even when ratings were consistent across all subcategories, the Overall Satisfaction rating was not completely predictable. For example, of those respondents rating all subcategories a 2, only 90.5% also rated the Overall Satisfaction a 2, while 6.0% rated it a 1, and 3.5% rated it either a 3 or 4. Among respondents who rated all subcategories a 5, 97.8% also rated Overall Satisfaction a 5, leaving 2.2% of respondents rating Overall Satisfaction below 5. Three respondents even rated the Overall Satisfaction a 1 despite having given all subcategories a level 5 rating. Across all the data, 42.1% of responses rated all subcategories the same, and 40.8% had nearly uniform ratings, with the maximum and minimum assigned subcategory rating differing by only 1 rating point. From Table 2 we see that Hotel Service had the highest partial correlation with Overall Satisfaction, while Room Cleanliness had the lowest. This is explained in part by the fact that for those individuals whose ratings were not uniform or nearly uniform, the Hotel Service rating was more likely than the Room Cleanliness rating to be inconsistent with the ratings of the other subcategories. Additionally, an inconsistent Hotel Service rating was more likely to be associated with a shift in the Overall Satisfaction rating than was an inconsistent Room Cleanliness rating. This tendency of Hotel Service ratings to deviate from the other three subcategory ratings was most pronounced when ratings were high overall, because there were far fewer instances of overall low ratings than of overall high ratings. This issue can be illustrated by investigating the subcategory total, along with the one- and two-way frequencies by total. For example, summing the four subcategory ratings gives an aggregate metric of the respondent’s review. Since each individual rating can take values ranging from 1 to 5, the subcategory total can only have a value of 18 when half the subcategory ratings equal 4 and the other half equal 5, or when three of the four ratings are equal to 5, and the remaining rating equals 3. For those respondents with a subcategory total equal to 18 (n = 5,168), 326 rated Hotel Service a 3 but only 51 rated Room Cleanliness a 3. Of those with a total = 18 and Hotel Service = 3, only about 28% rated Overall Satisfaction a 5 but of those with a total = 18 and Room Cleanliness = 3, about 59% rated Overall Satisfaction a 5—more in line with their ratings of the other subcategories. Thus when the total was 18, the Hotel Service rating was more likely than the Room Cleanliness rating to be inconsistent with the other subcategory ratings. Furthermore when Hotel Service was the inconsistent rating, the Overall Satisfaction rating was less likely to be a 5 than when Room Cleanliness was the inconsistent rating.

83



Willingness to Recommend Of the 60,648 responses, 74.51 % indicated they would recommend their hotel. However, when summarized by subcategory rating, the percentages that would recommend varied greatly. Simple associations between the Recommendation (yes–no) and each of the subcategories were assessed with a chi-square test. In all cases, the association was significant. For example, for those rating Hotel Service a 1, the percentage that would recommend the hotel was 7.6% (Table 4). As one might expect, the percentage of respondents willing to recommend increased as the Hotel Service rating increased, with 90.3% Willingness to Recommend the hotel when they rated Hotel Service a 5. Similar trends were observed for each subcategory. To some degree, while the point-biserial correlations were lower than for Overall Satisfaction (i.e., 0.53 to 0.55) a similar picture emerges. The partial correlations suggested that once you adjust for other subcategories, Hotel Service and Room Comfort might be the areas the hotels should target to improve their recommendation rate. While measures of simple associations suggested that Room Cleanliness has as much explanatory power as any other subcategory, once you adjust for other subcategories it has the weakest association. Also relating Willingness to Recommend to Overall Satisfaction, reveals a similar increasing trend with level 1 and level 2 Overall Satisfaction ratings associated with very low recommendation proportions, and level 4 and level 5 ratings associated with very high recommendation proportions (see Table 4). For Overall Satisfaction the partial correlation was taken after adjusting for all four subcategories and the Overall Satisfaction partial correlation is the highest partial correlation. This suggested that, in terms of the traveler’s Willingness to Recommend the hotel, the Overall Satisfaction TABLE 4 Descriptive Statistics for Willingness to Recommend: Percentages of Respondents Willing to Recommend by Level of Subcategory and Overall Satisfaction

Subcategory rating 1 2 3 4 5 Correlation ∗∗ Partial Correlation∗∗∗ ∗

Recommend % by hotel service level

Recommend % by hotel condition level

Recommend % by room cleanliness level

Recommend % by room comfort level

Recommend % by overall satisfaction level∗

7.58% 20.30% 55.55% 82.44% 90.32% .5315 .1537

5.41% 25.85% 65.26% 85.36% 90.62% .5510 .0920

7.19% 19.18% 51.74% 79.86% 90.05% .5392 .0621

6.62% 21.13% 58.40% 83.30% 90.71% .5537 .1390

3.49% 8.28% 58.80% 88.13% 91.54% .6167 .1960

Overall Satisfaction partial correlation with recommend adjusts for all four subcategories. Correlation between the Willingness to Recommend, and Subcategory or Overall ratings. ∗∗∗ Partial correlations between Recommendation and a Subcategory rating are based on adjusting both Recommendation and the Subcategory rating for all three other Subcategory ratings. ∗∗

84


rating carried additional information above and beyond that conveyed by the subcategory ratings. None of these variables was found to be a perfect predictor of the others. While the propensity to recommend the hotel had lower correlations with subcategories than with overall satisfaction, this was primarily due to the binary nature of the Willingness to Recommend variable. It is interesting that even when assigning a subcategory a rating of 1, some individuals will still recommend the hotel. The Willingness to Recommend a hotel is a complicated consumer behavior. Given that hotel choice and recommendation appear to have many factors, further study is recommended to explore these variables.


DISCUSSION This study sought to determine patterns and interdependence in traveler’s overall satisfaction rating for hotels, willingness to recommend the hotel, and ratings in each of the four subcategories: Hotel Service, Hotel Condition, Room Cleanliness, and Room Comfort. Hoteliers have expressed the concern that the majority of online ratings and reviews will be completed by disgruntled or dissatisfied travelers. Previous research had also voiced this same concern when dealing with word of mouse recommendations in ecommerce (Chen, Fay, & Wang, 2003; Chevalier & Mayzlin, 2006). The results of this study indicate that this was overwhelmingly not the case when the travelers rated the hotels. In this broad based study involving 10,537 hotels drawn from the 100 largest U.S. cities, and involving 60,648 reviews, almost three-fourths (74.51%) of the travelers writing reviews would recommend the hotel to others. Similarly, 72.85% of them gave the hotels a level 4 or 5 Overall Satisfaction rating (see Table 1). An additional 13.72% of the travelers gave the hotel a level 3 rating, resulting in 13.43% of travelers assigning a level 1 or 2 overall satisfaction rating to a hotel. The same trend was observed for each of the subcategories, with an overwhelming majority of travelers writing reviews giving hotels level 4 and 5 ratings for each of the subcategories. Travelers overall were pleased with the hotel’s service, condition of the hotel, as well as the condition and cleanliness of the guest room. Travelers’ evaluation of the hotels tended to be internally consistent. All of the subcategory variables were highly related to the Overall Satisfaction rating and the traveler’s Willingness to Recommend the hotel (see Tables 2 and 4). This high degree of agreement across subcategories is reflected in the high correlations and confirms an expected degree of inter-item consistency among subcategory ratings. This may be because a hotel’s performance in each category tends to reflect the overall management of that hotel, whether that management is



85

performing at a high level or a low level. On the other hand, it may also be due to a sort of response bias on the part of the traveler. When a hotel provides outstanding product or service in one subcategory, travelers may be more likely to provide a high overall rating, as well as high ratings on the other subcategories. Similarly, poor service in one category may cloud the traveler’s view of the property and result in low ratings for all categories. The high correlations among subcategories and Overall Satisfaction support previous research in service quality management which posits that the perceptions of the processes or parts of the service delivery are closely related to the overall perception (Gronroos, 2000, p. 51). A conclusion that can be drawn is that hoteliers cannot ignore one subcategory and still receive strong ratings, and recommendations. There is a great deal of internal consistency across the respondent ratings. Of the 60,648 responses, 42.1% of the hotel reviews had the same ratings for each subcategory, and an additional 40.8% had maximum and minimum ratings that differed by one (e.g., ratings of 4,3,3,4 or 1, 2, 2, 2). Only 17.1% of responses had subcategory ratings that differed more than 1 point (e.g., 3,5,5,5 or 1, 1, 3, 4). To some degree, the variation in this smaller group accounts for the differences observed in the partial correlations. Note, that if every review had all five ratings in perfect agreement (i.e., the four subcategories and Overall Satisfaction ratings were either all 1s, or all 2s, or . . . ), then all correlations would have been equal to 1.0, and all partial correlations would have been equal to 0. So the relatively high partial correlations for Hotel Service and Room Comfort suggest that ratings for these categories did sometimes differ from the overall tendency in an individual’s subcategory ratings, and when this occurred, the rating of Overall Satisfaction tended to follow this discrepant rating. Room Cleanliness had the highest percentage of 4’s and 5’s at nearly 78%, and was rated a 5 by nearly 49% of respondents. Also, Room Cleanliness did not have as great a tendency to be given a lower rating when other categories were given high ratings. These factors help to explain its lower partial correlation. It is likely that well managed hotels have effective quality control systems in place to assure that rooms are consistently clean. However, it is more difficult to maintain consistent quality assurance over the many, varied, and often face-to-face interactions that fall under ‘Hotel Service.’ Exploration of the data revealed 5,921 cases (9.76% of the total responses) where Hotel Service was rated lower than the other subcategories. In these cases the Overall Satisfaction rating tended to reflect this lower Hotel Service rating. In part because ratings tended to be high, these cases had a greater influence on the partial correlation than cases where the Hotel Service rating was higher than the other subcategories. With ratings tending to be high, there were simply more opportunities for Hotel Service to be rated uncharacteristically low than there were opportunities for Hotel Service to be rated uncharacteristically high. Based on these data,


86


Hotel Service failures may be costing hotels in terms of their online ratings. The same is likely true of Room Comfort. Despite having a lower partial correlation, it is imperative that hotels continue with effective quality assurance in areas such as room cleanliness. Our findings were consistent with earlier research which determined that the cleanliness of the hotel room significantly affected guest satisfaction and the guest’s intent to return (Lockyer, 2003; Schall, 2003). When a hotel experience a slump in occupancy and/or average rate, the first step is often to cut housekeeping hours (Murray, 2008). If such a cutback impacts the cleanliness of the facility, our results suggest that overall satisfaction ratings would likely suffer. Hotel Service involves simultaneous production and consumption, which makes quality control more difficult than with housekeeping guest room cleanliness standards. With Hotel Service, a single instance of rude or otherwise poor service cannot be easily undone, because it happens face-toface in real-time. Training people to both know how to act in every possible service situation as well as assuring that they never have a bad day is a lot more difficult than having consistent housekeeping inspection systems. Similarly, guest behaviors and actions cannot be controlled easily, further complicating the Hotel Service variable. At the same time, for that same reason, Hotel Service offers opportunities for ‘above and beyond’ positive experiences that positively flavor a customer’s perceptions. Similar experiences are not as likely to be achieved with room cleanliness. Hotels may benefit by training people to know how to conduct themselves in the many service situations that may arise, emphasizing the need to deliver uniformly high-quality service, and empowering employees to provide service. Room Comfort may have a high partial correlation because some hotels have upgraded and some have not. However, many factors contribute to room comfort, with upgrades to product only one factor. As with Hotel Service, the partial correlation for room comfort suggests that this is an area where hotels may be able to improve their ratings. However, based on the available information, it is difficult to provide concrete recommendations for improving room comfort ratings. Factors that matter to one individual may not matter to another. Future research might attempt to assess to what degree personal taste and preference influences room comfort ratings as well as what room features consistently contribute to, or detract from high room comfort ratings. To summarize, the following results emerge from this study: An individual traveler’s Overall Satisfaction rating is generally consistent with each of his or her subcategory ratings. When travelers are pleased or displeased with one aspect of the hotel, it influences their evaluation of the entire hotel stay. Cleanliness is important to travelers. Overall, travelers were pleased with the cleanliness of the hotel room with 78% of travelers assigned a



87

rating of 4 or 5 to Room Cleanliness. It is important to note that when a hotel received the lowest rating for cleanliness, it was nearly impossible to receive a high Overall Satisfaction rating. After adjusting for other categories, Hotel Service (i.e., the quality of the service the traveler receives), and Room Comfort appear to have the highest influence on the Overall Satisfaction rating. A traveler may overlook some discrepancies in the hotel product if the service compensates. However when the service does not compensate for the discrepancy, the traveler is likely to give the hotel a lower rating. A relatively low Hotel Service rating may bring down the Overall Satisfaction rating when a traveler has assigned high ratings to the other subcategories. However, hotels cannot concentrate on one facet of the hotel condition or service at the expense of other factors. A traveler’s perception of hotel service, the condition, comfort and cleanliness of the hotel and the guest room are highly related to one another, and to their overall evaluation of the hotel. If a hotel was to target certain aspects of their services or products in an attempt to leverage higher ratings, the hotel should concentrate on maintaining areas of strength and quality control systems that are in place. For example, they may want to focus on strengthening their systems that target guest room cleanliness, and on consistently delivering high quality hotel service. Apart from that, determining which features their clientele values as contributing to room comfort as well as any features that detract from room comfort might provide added direction for improving online ratings.

LIMITATIONS AND RECOMMENDATIONS FOR FUTURE STUDY This study has some limitations. The study was observational in nature. As such, the interpretations of the results do not imply cause and effect. While the study design does use common subcategories used in assessing the hotel experience, they are not independent measures. The subcategories selection was driven by the available data. The observed high correlation between these factors suggests that a clearer insight into the traveler experience might be obtained using different, more independent measures, incorporating more facets of a hotel stay, such as the arrival and departure processes, amenities, pricing, reservation processes, food and beverage service, recreational facilities, etc. The data, while representing a broad cross-sectional sample of the U.S. market, does not contain any reviews of international sites, or specific segments of the hotel industry. The data were only drawn from a single travel site, and the travelers that use that site. While data from the largest travel site were used, there still may be some selection bias based on the travelers that use this site in contrast to other travel sites. Also, there is a self reporting bias: reviews and ratings were from travelers that opted to complete the


88


optional feedback on their travel. Further analysis using other web sites and other countries would help to validate the findings. While traveler rating sites try to limit their ratings to only those guests making reservations through their web sites, it is known that false recommendations both in favor of and against hotels exist (O’Connor, 2008). Despite Expedia’s reservation requirement for reviewing hotels, it is possible that some comments from travelers could be biased comments from the hotel personnel itself, or from competing hotels. No attempt was made to filter out these ratings, nor would it have been possible to do so. Further work is needed to identify the impact of these biased ratings on the online guest feedback systems. The analysis of hotel ratings provides another limitation. There is a tendency to view level 1, or 1-star ratings as poor, level 5 or 5-star ratings as excellent, and level 3 or 3-star ratings as neutral. However this is not always the case. For instance a hotel with a long history of 5 star ratings would not consider a level 3 or 3-star rating as neutral. It would consider such a rating as poor. And a hotel with limited services may not expect to achieve 5-star or level 5 ratings. With the growing importance of traveler ratings on third party travel sites, it is increasingly important for hoteliers to monitor and address concerns raised in these online forums. These sites provide a great deal of information about the hotel operations, at least from the traveler’s perspective. Expedia, for example provides both the traveler’s and overall average rating of the Hotel Service, Hotel Condition, Room Comfort, and Room Cleanliness, as well as an Overall Satisfaction rating and the Willingness to Recommend the hotel to others. Given the importance of the overall ratings, it is important to determine the impact of the four subcategory ratings on the Overall Satisfaction rating and the Willingness to Recommend. In this study we looked at over 60,000 traveler ratings from a cross-section of hotels across the United States. What emerged is a complex picture, suggesting as one might expect that all four factors are highly related to each other, and also to the traveler’s overall evaluation of the hotel. However, the greatest gains may be from those areas where asserting consistent levels of quality control are most difficult, especially Hotel Service, and where personal preference may make it difficult to attain high marks across a diverse clientele, such as room comfort.

REFERENCES Anderson, E. (1998). Customer satisfaction and word of mouth. Journal of Service Research, 1(1), 5–7. Atkinson, A. (1988). Answering the eternal question: What does the customer want? Cornell Hotel and Restaurant Administration Quarterly, 29 (2), 12–14.



89

Barsky, J. (1996). Building a program for world class service. Cornell Hotel and Restaurant Administration Quarterly, 37(1), 17–28. Barth, J. & Walsh, J. (1997). An empirical approach to developing classification and rating schemes. Journal of Hospitality & Leisure Marketing, 5(1), 15–29. Bender, B. & Gerdes, J. (2007). To book or not to book: The role of web site heuristics. Proceedings of the Hospitality Information Technology Association Conference, Orlando, Florida: 1–22. Bevans, M. (2008, 18 September). Stars losing their luster. Caterer & Hotelkeeper, 198(4545), 18. Bojanic, D. (1996). Consumer perceptions of price, value, and satisfaction in the hotel industry. Journal of Hospitality & Leisure Marketing, 4(1), 5–22. Brewer, P., Feinstein, A. & Bai, B. (2006) Electronic channels of distribution: challenges and solutions for hotel operators. FIU Hospitality Review, 24(2), 68–77. Callan, R. (1998). Attributional analysis of customers’ hotel selection criteria by U.K. grading scheme categories. Journal of Travel Research, 26(3), 20–34. Callan, R. (1995). Hotel classification and grading schemes, a paradigm of utilisation and user characteristics. International Journal of Hospitality Management, 14(3-4), 271–283. Cannizzaro, M., Carroll, W., Offutt, B., Quinby, D., Schetzina, C. & Sileo, L. (2007). PhoCusWright’s U.S. Online Travel Overview (7th ed.). Sherman, CT: PhoCusWright. Carroll, B. & Siguaw, J. (2003). The future of electronic distribution: Effects on hotels and intermediaries. Cornell Hotel and Restaurant Administration Quarterly, 44(6), 38–50. Chan, E. & Wong, S. (2006). Hotel selection: When price is not the issue. Journal of Vacation Marketing, 12(2), 142–159. Chen, Y., Fay, S. & Wang, Q. (2003). Marketing implications of online consumer product reviews. Working paper, Department of marketing (University of Florida, 2003) Retrieved May 6, 2009, from http://plaza. ufl.edu/faysa/review.pdf. Chevalier, J. & Mayzlin, D. (2006). The effect of word of mouth on sales: Online book reviews. Journal of Marketing Research, 43 (3), 345–354. Retrieved August 28, 2006, from http://www.atypon-link.com/AMA/doi/abs/10.1509/ jmkr.43.3.345. Chipkin, H. (2005). Contesting unflattering Web comment proves difficult. Hotel & Motel Management, 220(16), 58. Christodoulidou, N., Brewer, P. & Countryman, C. (2007). Travel intermediaries and metasites: A comparison of customer perception. Information Technology & Tourism, 9(3/4), 227–243. Clausing, J. (2009). Hotel star ratings a standard for inconsistency, confusion. Travel Weekly. Retrieved March 4, 2009, from http://www.travelweekly.com/ article3_ektid190466.aspx. Clausing, J. (2007). Playing fair on the web. Travel Weekly, 66(29), 43. Cornell Roundtable Focuses on Marketing Challenge of Social Media (2008). Cornell Center for Hospitality Research, Ithaca, New York. Retrieved September 24, 2008, from http://www.4hotliers.com/4shots_fshw.php?mwi=3087.


90


Cotter, M. & Snyder, W. (1996). How Mobil stars affect restaurant pricing behavior. Cornell Hotel & Restaurant Administration Quarterly, 37(2), 34–41. Crotts, J., Pan, B. & Raschid, A. (2008). A survey for identifying key drivers of guest delight. International Journal of Contemporary Hospitality Management, 20(4), 462–470. Cser, K. & Ohuchi, A. (2008). World practices of hotel classification systems. Asia Pacific Journal of Tourism Research, 13(4), 379–398. Deng, W. (2008). Fuzzy importance-performing analysis for determining critical service attributes. International Journal of Service Industry Management, 19(2), 252–270. Enz, C. (2009). Key Issues of Concern in the Lodging Industry: What Worries Managers. Cornell Hospitality Report, 9(4). The Center for Hospitality Research, Cornell University School of Hotel Management. Retrieved March 14, 2009, from http://www.hotelschool.cornell.edu/chr/pdf/showpdf/ chr/research/enzmanagerstopost.pdf?my_path_info=chr/research/enzmanagers topost.pdf. Expedia (2009). Expedia, Inc.’s mission is to get the world going by building the world’s largest and most intelligent travel marketplace. Retrieved August 18, 2009, from http://overview.expediainc.com/phoenix.zhtml?c=190013&p= overview. Gazzoli, G., Kim, W. & Palakurthi, R. (2008). Online distribution strategies and competition: Are the global hotel companies getting it right? International Journal of Contemporary Hospitality Management, 20(4), 375–387. Gelb, B. & Sundaram, S. (2002). Adapting to “word of mouse.” Business Horizons, 45(4), 21–25. Gerdes, J, Stringam, B. & Brookshire, B. (2008). Addressing researchers’ quest for hospitality data: Mechanism for collecting data from web resources. Tourism Analysis, 13(3), 309–315. Gronroos, C. (2000). Service Management and Marketing. West Sussex, England: John Wiley & Sons. Gretzel, U. & Yoo, K. (2008). Use and impact of online travel reviews. In Information and communication technologies in tourism: Proceedings of the international information and communication technologies in tourism conference Austria, 2008 (pp. 35–46). New York: SpringerWein. Gunderson, M., Heide, M. & Olsson U. (1996). Hotel guest satisfaction among business travelers. Cornell Hotel and Restaurant Administration Quarterly, 37(2), 72–81. Hanai, T., Oguchi, T., Ando, K. & Yamaguchi, K. (2007). Important attributes of lodgings to gain repeat business: A comparison between individual travels and group travels. International Journal of Hospitality Management, 27(2), 268–275. Heide, M. & Grønhaug, K. (2009). Key factors in guests’ perception of hotel atmosphere. Cornell Hospitality Quarterly, 50(1), 29–43. Henley, J., Cotter, M. & Herington, D. (2004). Quality and pricing in the hotel industry: The Mobil “star” and hotel pricing behavior. International Journal of Hospitality &Tourism Administration, 5(4), 53–65.



91

Ingram, H. (1996). Classification and grading of smaller hotels, guesthouses and bed and breakfast accommodation. International Journal of Contemporary Hospitality Management, 8(5), 30–34. Israeli, A. (2002). Star rating and corporate affiliation: Their influence on room price and performance of hotels in Israel. International Journal of Hospitality Management, 21(4), 405–424. Kandampully, J. & Suhartanto, D. (2000). Customer loyalty in the hotel industry: The role of customer satisfaction and image. International Journal of Contemporary Hospitality Management, 12(6), 346–351. Koster, M. (2007). A standard for robot exclusion. Retrieved February 25, 2009, from http://www.robotstxt.org/wc/robots.html. Law, R. & Bai, B. (2008). How do the preferences of online buyers and browsers differ on the design and content of travel websites? International Journal of Contemporary Hospitality Management, 20(4), 388–400. Leggatt, H. (2008). Online leisure travel market grows, bookings drop. Biz Report: News and Insight for Online Marketers. Retrieved May 5, 2009, from http:// www.bizreport.com/2008/08/online_leisure_travel_market_grows_bookings_ drop.html. Lewis, R. & Pizam, A., (1981). Guest surveys: A missed opportunity. Cornell Hotel and Restaurant Administration Quarterly, 2(3), 37–44. Lockyer, T. (2003). Hotel cleanliness—how do guests view it? A New Zealand study. International Journal of Hospitality Management, 22(3), 297–305. Lockyer, T. (2005a). Understanding the dynamics of the hotel accommodation purchase decision. International Journal of Contemporary Hospitality Management, 17(6), 481–492. Lockyer, T. (2005b). The perceived importance of prices as one hotel selection dimension. Tourism Management, 26(4), 529–537. Matilla, A. & O’Neill, J. (2003). Relationships between hotel room pricing, occupancy, and guest satisfaction: A longitudinal case of a midscale hotel in the United States. Journal of Hospitality & Tourism Research, 27(3), 328–341. Matzler, K., Fuller, J., Renzi., B., Herting, S. & Spath, S. (2008). Customer satisfaction with ski areas: The moderating effects of personal, situational, and product factors. Journal of Travel Research 46(4), 403–413. Murray, C. (2008). Housekeepers feeling decline pinch: Operators are trimming this department to save money, but will it result in lower customer service? Hotel Interactive. Retrieved December 16, 2008, from http://www.hotel interactive.com/article.aspx?articleid=12371. O’Connor, P. & Frew, A. (2002). The future of hotel electronic distribution. Cornell Hotel and Restaurant Administration Quarterly, 43(3), 33–45. O’Connor, P. (2008). User-generated content and travel: A case study on trip advisor.com. In Information and communication technologies in tourism: Proceedings of the international information and communication technologies in Tourism Conference Austria, 2008 (pp. 47–58). New York: SpringerWein. Oliver, R. & Swan, J. (1989). Consumer perceptions of interpersonal equity and satisfaction in transactions: A field survey approach. The Journal of Marketing, 53(2), 21–35.


92


Park, Y., Gretzel, U. & Sirakaya-Turk, E. (2007). Measuring web site quality for online travel agencies. Journal of Travel & Tourism Marketing, 22(1), 15–30. Pekar, V. & Ou, S. (2008). Discovery of subjective evaluations of product features in hotel reviews. Journal of Vacation Marketing. 14(2), 145–155. Prayukvong, W., Sophon, J., Hongpukdee, S. & Charupas, T. (2007). Customers’ satisfaction with hotel guestroom: A case study in Ubon Rachathani provence, Thailand. Asia Pacific Journal of Tourism Research, 12(2), 119–126. Salerno, N. (2008, June 12). Using travel social media to advance website sales. 4Hoteliers, Hospitality, Hotel & Travel News. Retrieved September 24, 2008, from http://www.4hoteliers.com/4shots_fshw.php?mwi=3088. Schall, M. (2003). Best practices in the assessment of hotel-guest attitudes: For guest surveys to provide useful information, those surveys must be properly designed. Cornell Hotel & Restaurant Administration Quarterly, 44(2), 51–65. Shanka, T. & Taylor, R. (2004). An investigation into the perceived importance of service and facility attributes to hotel satisfaction. Journal of Quality Assurance in Hospitality & Tourism, 4(3), 119–134. Steinbrink, S. (2008). The PhoCusWright Business Travel Trends Survey (3rd ed.). Sherman, CT: PhoCusWright. Stringam B. & Gerdes, J. (2010). An analysis of word-of-mouth ratings and guest comments on an online hotel distribution site. Journal of Hospitality Marketing and Management, 19(7), in press. United States Census Bureau, Population Division (2007-06-28). Table 1: annual estimates of the population for incorporated places over 100,000, Ranked by July 1, 2006 population: April 1, 2000 to July 1, 2006 (CSV). 2005 Population Estimates. Retrieved August 1, 2008, from http://www.census.gov/popest/ cities/tables/SUB-EST2006-01.csv. Vallen, G. & Vallen, J. (2005). Check-in, check-out: Managing hotel operations. Upper Saddle River, NJ: Pearson Education. Vine, P. (1981). Hotel Classification—art or science? International Journal of Tourism Management, 2(1), 18–29. Wilkins, H., Merrilees, B. & Herington, C. (2007). Towards an understanding of total service quality in hotels. International Journal of Hospitality Management, 26(4), 840–853. Williams, J. (2007). Top historical facts about Expedia. Associated Content. Retrieved February 3, 2009, from http://www.associatedcontent.com/article/260871/top_ historical_facts_about_expedia.html?cat=3. Ye, Q., Law, R. & Gu, B. (2009). The impact of online user reviews on hotel room sales. International Journal of Hospitality Management, 28(1), 180–182. Yu, L. (1992). Seeing stars: China’s hotel-rating system. Cornell Hotel and Restaurant Administration Quarterly, 33(5), 24–27.