The Effect of Rich Web Portal Design and Floating

International Journal of Human-Computer Interaction

ISSN: 1044-7318 (Print) 1532-7590 (Online) Journal homepage: http://www.tandfonline.com/loi/hihc20

The Effect of Rich Web Portal Design and Floating Animations on Visual Search Pei-Luen Patrick Rau , Qin Gao & Jie Liu To cite this article: Pei-Luen Patrick Rau , Qin Gao & Jie Liu (2007) The Effect of Rich Web Portal Design and Floating Animations on Visual Search, International Journal of Human-Computer Interaction, 22:3, 195-216, DOI: 10.1080/10447310709336962 To link to this article: http://dx.doi.org/10.1080/10447310709336962

Published online: 05 Dec 2007.

Submit your article to this journal

Article views: 152

View related articles

Citing articles: 6 View citing articles

Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=hihc20 Download by: [Tsinghua University]

Date: 18 February 2016, At: 00:18

INTERNATIONAL JOURNAL OF HUMAN–COMPUTER INTERACTION, 22(3), 195–216 Copyright © 2007, Lawrence Erlbaum Associates, Inc.

Downloaded by [Tsinghua University] at 00:18 18 February 2016

The Effect of Rich Web Portal Design and Floating Animations on Visual Search Pei-Luen Patrick Rau Qin Gao Jie Liu Department of Industrial Engineering, Tsinghua University, China

Unlike most Web portals in the world, Chinese Web portals are characterized by a huge amount of information, excessive visual stimuli, and very long Web pages. The objective of this study is to investigate the effect of such rich Web portal designs and floating animations on visual search, emphasizing a comparison between Chinese users and German users. Two experiments were conducted to test 2 proposed hypotheses. Experiment 1 studied the effect of Web portal design (rich and simple) on visual search performance (performance time, errors, and satisfaction) with both Chinese and German participants. Experiment 2 studied the effects of static animations (leaderboards, couplets, and large squares) and floating animations (moving down, moving up/down, and random movement) on visual search performance on Web portals. The dependent variables were the performance time, error, satisfaction, and animation recognition. The results indicated that participants using simple Web portals searched faster, made fewer errors, and were more satisfied than participants using rich Web portals. No significant differences were found between the performance time of Chinese participants and German participants. However, satisfaction of Chinese participants was found to be less influenced by the differences between simple and rich Web portal designs, compared with German participants. No significant differences were found in performance time and animation recognition between static animations and floating animations, which indicated that users are able to detect the pattern of animation movements and were able to avoid floating animations as well as static animations. People searching pages with randomly floating animations were found to use significantly more time compared with those searching pages with no animations. Furthermore, users’ satisfaction for pages with randomly floating animations, moving down animations, and moving up/down animations was significantly lower than for pages with no animations. Implications for designers and for future research are discussed.

Correspondence should be sent to Pie-Luen Patrick Rau, Department of Industrial Engineering, Tsinghua University, Beijing 100084, China. E-mail: [email protected]

196

Rau, Gao, Liu


1. INTRODUCTION The World Wide Web has become an extensive information and media source in China. The latest survey by the China Internet Network Information Center (CNNIC; 2005) reported that 94 million Chinese had gone online. Among the large amount of Internet use, getting information (39.1%) is the primary goal. Three major Chinese Web portals (Sina.com.cn, Sohu.com, and 163.com) are currently the best source for consolidated information. These Web portals present huge amounts of information using strong visual stimuli such as animation and colorful texts, which add to the difficulties for the user to perceive the elements in the environment and results in cognitive burdens (Neerincx, Lindenberg, & Pemberton, 2001). Chinese Web portals, though aware of the cognitive overload on users, try to offer all the available information on the home page and very long Web pages. For example, the page length for news on Sina.com (http://news.sina.com.cn/) is almost four times the length of news pages on Yahoo.com (http://news.yahoo.com/). Cognitive overload, particularly for visual searches on Web portals, is not considered advantageous by many usability design guidelines. However, many Chinese developers and users are used to cognitive overload. They consider providing very rich information on very long Web pages a criterion of a good Web portal site design. The problem with these rich Web pages is the difficulty users have in finding what they need among the large amount of information. Being interrupted or having attention deviated involuntarily by pop-up animation banners on Web portals is typical for Chinese Web users. Many Chinese Web sites try to present strong visual stimuli (e.g., videos, animation, blinking text links) in rapid succession, the location, size, and movement of which are varied from time to time. Specifically, animations floating around the page are prevalent forms of advertising on Chinese Web portals. These features, though considered serious usability problems by human–computer interaction (HCI) experts and usability engineers, have prevailed on Chinese Web portals in mainland China for years. However, such features do not characterize Web portals in Taiwan, Hong Kong, and Japan. Korean Web portals also provide rich information and floating banners, but they are not comparable with Chinese Web portals. Thus, Chinese language or pictorial characters may not be the only reason accounting for such phenomenon. There are several possible reasons why Chinese users can accept these design features. It is possible that the visual search capability of experienced Chinese is enhanced so that these features do not bother them as much as they bother other people. In this case, usability concerns and Web design principles for Chinese should be modified to take into account the discrepant user capacity. However, it is also possible that Chinese accept such designs just because they have no other choice, but they also suffer from usability problems and negative impacts on visual search and browsing experiences. Are Chinese superior in visual searching on rich Web portals? Do they really prefer rich portal designs or do they just accept them as the only alternative? These questions are of interest to both HCI researchers and Web designers, especially for e-commerce Web site developers who are going to explore the enormous Chinese market. The objective of this study is to investigate how rich information and floating animations affect visual searches in terms of standard visual search metrics, such as mean reaction time, error rate, and satisfaction, emphasizing the differences be-

Effect of Design and Animations on Search

197

tween Chinese users and German users. The first research question is whether visual search performance will be influenced by the richness of information on a Web page and how Chinese differ from Germans in visual search tasks. Two languages and two cultural groups were used in the experiment to account for the potential confounding effects of language and familiarity. The second research question is whether visual search performance will be influenced by floating animation. Six types of floating animations were tested on Chinese Web portals, in comparison to a control group using no banners.


2. LITERATURE REVIEW Studies on link arrangement in categorical menus indicate that link arrangements can have a marked effect on search time and user satisfaction (Maldonado & Resnick, 2002; Nygren & Allard, 1996; Parkinson, Sisson, & Snowberry, 1985). Many studies tried to examine the various effects of grouping items in a screen layout to identify a grouped visual links hierarchy for efficient visual searches (Card, 1982; McDonald, Stone, & Liebelt, 1983, Triesman, 1988; Tullis, 1997), and others focused on users’ preferences toward different links or textual information presentation (Bernard, Baker, & Fernandez, 2002; Chaparro & Bernard, 2001; McDonald et al., 1983). Bertera and Rayner (2000) studied the density between a fixed number of randomly placed characters using trials in a search task and found that the search time decreased and the estimated number of letters processed per fixation increased as the density increased. On the other hand, Ojanpää, Näsänen, and Kojo (2002) found that after the distance between the centers of words increased beyond 1.5° of the visual angle, search time increased as the density decreased. The density may be measured as overall density or local density. Overall density is simply the number of items divided by the area occupied by all items. Local density is the number of items in a given region divided by the area of that region. Hornof and Halverson (2003) investigated the effect of local density on visual search tasks in structured, two-dimensional menus. The results showed that sparse groups of words are searched faster and, when presented with dense groups, are searched first. Weller (2004) evaluated the effects of white space measured by the level of text density on visual search time. Screens were presented with each of four graphical layouts—no graphics, contrast, borders, and contrast with borders—under two levels of overall density and three levels of local density. The results showed that the search times were longer with increased overall density, but significant differences were not found between the levels of local density. For the color effect on a visual search task, it has been found that locating targets distinguished by color is significantly faster than locating targets distinguished by a conjunction of two or more features (Christ, 1975). A phenomenon that helps explain this faster search time with color is the “pop-out” effect. Further research has found that people can selectively attend to items of a target color among distracters of different colors, but this ability is task dependent (Brawn & Snowden, 1999). A real-world example might be a user searching a Web page for a particular blue link while ignoring the magenta (visited) links. It is this sort of scenario that has motivated Web design guidelines for link coloring (National Cancer Institute, 2003). If several elements have a similar size, color can be used to discriminate which element is most important.


198

Rau, Gao, Liu

Triesman (1988) concluded that differences in surface characteristics such as color and brightness can provide visual search cues. Hornof and Halverson (2003) investigated the effect of link color effects on user visual search performances. The result offered empirical support for the design recommendation to differentiate visited and unvisited links by color. In addition, this indicates that omitting elements not immediately relevant to the user’s goals allows for a more efficient search. Some studies found that image over text could be attended to first (Brandt, 1954; Stone & Glock, 1981). However, other researchers noticed that users often instinctively ignore the graphics that are presented on a Web site, the so-called banner blindness, in which viewers have learned to ignore certain graphics (Benway, 1998). This effect has been found to have the greatest impact on the effectiveness of banners. Other influencing factors include the type of browsing (Pagendarm & Schaumburg, 2001), image size (Faraday, 2001), and advertisement content. Other studies also revealed that animation may be less effective than nonanimated graphics to some extent (Bayles, 2002; Bayles & Chaparro, 2001; Benway & Lane, 1998; Wright, Milroy, & Lickorish, 1999). Presenting animated banners was found to increase the frustration and mental load of visual search tasks, and no significant search performance deterioration was found with animation presented (Burke & Hornof, 2001). Zhang and Massad (2003) suggested that animation as nonprimary information significantly reduces information-seeking performance in a Web-based environment. As the difficulty of the task increases, viewer performance is less affected by animation. Animation that is similar but irrelevant to a task has a more negative impact on viewer performance than animation that is dissimilar to the task. Animation that is brightly colored has a stronger negative effect on viewer performance than does dull-colored animation. Furthermore, in a different study, Zhang (2001) found that animation on the left side of a screen has a higher negative impact on task performance than animation on the right side; animation that appears during the middle or toward the end of an information-seeking task degrades performance more than animation that appears at the beginning. Over the years, animation retains the significant main effect on onset timing, but the differences between various onset timing effects (at the beginning, in the middle, or near the end of the tasks) become less significant (Zhang & Massad, 2003).

3. HYPOTHESES AND METHODOLOGY Two hypotheses are proposed, and two experiments were designed to test the two proposed hypotheses.

3.1. Hypotheses Hypothesis 1: Users’ visual search performance of simple Web portal would be better (in terms of shorter performance time, less error, and higher satisfaction) than users’ visual search performance of rich Web portal. Compared to a simple Web portal design, a rich Web portal design contains more hyperlinks, graphics, and animations that are interactive in nature. Too much visual


199


stimuli (blinking text links, colorful background, graphics, flash, animations) presented simultaneously can create an attention overload, which has been shown to promote misinterpretation (Weir & Heeps, 2003). It is widely believed that human attention is limited and allocated selectively to stimuli in the visual field (Pashler, 1998). Textual or graphical distracters will increase the demand for cognitive resources and thus decrease the availability of cognitive resources for searching the target stimulus. Previous empirical studies have provided evidence that high overall text density (Weller, 2004); images superimposed over text (Brandt, 1954; Stone & Glock, 1981); and animations, especially brightly colored animations (Burke & Hornof, 2001; Zhang & Massad, 2003), have a negative effect on information-seeking performance on the Web. Therefore, it is predicted that users’ visual search performance of a simple Web portal design would be better than users’ visual search performance of a rich Web portal design. In addition, the differences between Chinese users and Western users were also of interest. Hypothesis 2: a. Users’ visual search performance on Web portals with static animation banners would be better (in terms of shorter performance time, less error, and higher satisfaction) than users’ visual search performance on Web portals with floating animations. b. The recognition of animations (animation recognition) of users on Web portals with floating animations would be better than the animation recognition of users on Web portals with static animations. One potential negative effect of animation is that it disturbs concentration (Nielsen, 1995; Spool, Scanlon, Schroeder, Snyder, & DeAngelo, 1999; Stewart, 1976). Animation will make it difficult for users to focus on the information in their search task, because it constitutes a constant change in the visual field. As human attention is highly sensitive to movement in the environment, the continuous blinking will prevent concentration. Users need to spend attention resources to suppress the interference of animation, and limited resources are left for the central task of visual search. Therefore, presenting animation on a Web page will interfere with information processing on other nonanimated items. On the other hand, traditional studies on capturing a user’s attention assert that humans involuntarily orient their attention toward moving objects regardless of importance (Hillstrom & Yantis, 1994; Jonides, 1981) Therefore, floating animations automatically capture a user’s attention, making any concurrent visual task more difficult assuming limited attention capacity. It is predicted that users’ visual search performances on Web portals with static banners would be better than Web portals with floating animation and the animation recognition of floating animations would be better than static animations.

3.2. Experiment 1 Participants. A two-way factorial design was used. Two types of Web portal designs were tested, simple and rich. Each was developed into two versions, Chinese and English. A total of 33 Chinese participants took part in this experiment.


200

Rau, Gao, Liu

They were divided into two groups: 14 participants used the Chinese version, and the other 19 used the English version. Another 19 German participants as the comparative group participated in the experiment using the English version of the design. Each participant used both the simple design for 10 tasks and the rich Web design for the other 10 tasks. To reduce undesired variations among participants from sources other than cultural difference such as education levels, field of study, gender, age, and so forth, the participants selected from both cultural groups were graduate students majoring in engineering and had little or no prior knowledge about the tasks performed during experimentation. The Chinese participants were graduate engineering students at Tsinghua University who were between 21 and 30 years old (M = 24.6, SD = 2.33). The German participants were students who came to China for less than 6 months. All of the German participants were graduate engineering exchange students from the Aachen University of Technology. The age of the German participants ranged from 22 to 26 (M = 24.3, SD = 1.16). All Chinese participants frequently visit one of the three major Chinese Web portals (http://www.sina.com.cn), whereas all the German participants had no prior experience with any Chinese Web portals.

Tasks. Participants were required to search for and click a series of target words on both rich Web portals and simple Web portals. There were 20 tasks (10 tasks on rich Web portals and 10 tasks on simple Web portals) for each participant. The tasks to be performed were exactly the same for each participant but with targets listed in random order. In the two groups using the English version, 10 participants conducted tasks on rich Web portals first, and the remaining 9 participants conducted tasks on simple Web portals first. In the group using the Chinese version, 7 participants conducted tasks on rich Web portals first, and the remaining 7 participants conducted tasks on simple Web portals first. Each participant’s session, from introduction to final comments, was video- and audio-recorded.

Independent variables. There were two independent variables in Experiment 1: the character effect (between-subjects) and the Web portal design (within-subjects). The character effect had three levels: pictorial for pictorial natives (Chinese for Chinese participants), alphanumeric for pictorial natives (English for Chinese participants), and alphanumeric for alphanumeric natives (English for German participants). The character effect was considered due to concern for its possible confounding effect on the Web portal design. Chinese participants were familiar with the rich Web portal design in Chinese; thus, an English version was also tested. Furthermore, English was familiar to but not the native language for both the Chinese and the German participants. Moreover, pictorial tests for alphanumeric natives were not studied, because alphanumeric natives would recognize pictorial characters as symbols more than meaningful words. The difference between the three levels of character effects was expected, and it was not addressed in this study. Web portal design has two levels: rich and simple. The rich Web portal design contained much more visual stimuli (e.g., graphics, animation, background


201


and link color), larger amounts of information (relatively denser page layout), and clutter links arrangement than the simple Web portal design.

Dependent variables. There were three dependent variables: performance time, error, and satisfaction. Performance time was defined as the time taken to locate the target items. It was recorded to the nearest one-thousandth of a second by the testing system. System response time for displaying objects on the screen is excluded by the computer automatically. Error was defined as the extra hits utilized to perform the tasks and was also recorded by the testing system. Satisfaction of the visual search was the score obtained through a general satisfaction questionnaire completed after participants performed the tasks of one level of Web portal design and was based on the Likert scale of 1 (lowest satisfaction) to 5 (highest satisfaction). The questionnaire was a modified version of the satisfaction questionnaire by Cook (1991).

Testing system. The testing system was developed by HTML/JavaScript and displayed in the Microsoft Internet Explorer Web browser (version 6.0). Web pages were presented on a 17’ CRT display set to 1024 × 768 pixels resolution. The target words were designed to appear randomly in different areas of each Web page. The rich Web portal was designed based on two major Web portal sites in China (sina.com.cn and sohu.com), and the simple Web portal was designed based on Yahoo!. The layout of testing pages was adopted from the existing Web portals, but all the images and texts were replaced with equally sized images and texts. All the hyperlinks in the testing system were replaced randomly by simple English words with 4 to 10 characters. This resulted in testing pages the content of which could not be read or comprehended. Participants could search for the targets words only visually. For the Chinese version, each English word was translated into the corresponding Chinese word. Using short English words made the length of English words and corresponding Chinese words similar and made it possible to use the same page layout for the two versions. Each word appeared only once on each page. The basic elements of the two Web portal designs included textual hyperlinks, animated and moving graphics, logos, and background (Figures 1 and 2). The percentage of each element for the two Web portal designs present on the screen (1024 × 768) without scrolling was compared (Table 1). The rich Web portal design had 47.06% of textual hyperlink, and the simple Web portal design had 33.13% of textual hyperlinks. Moving and animated graphics occupied 13.51% of the rich Web portal design but only 6.03% of the simple Web portal design. Colorful backgrounds occupied 17.14% of the rich Web portal design but only 6.67% of the simple Web portal design. White spaces accounted for 54.17% of the simple Web portal page and 22.24% of the rich Web portal.

Procedure. The experiments were conducted in the HCI and Usability Lab. Each participant was required to perform the tasks alone with no other participants

FIGURE 1 Rich Web portal design.

202




203

FIGURE 2 Simple Web portal design.

Table 1:

The Percentage of Basic Elements for the Two Web Portal Designs Graphics

Web Portal Design

Textual Hyperlinks

Rich Simple

47.06 33.13

Floating and Animated 11.44 1.77

Background Static

Logos

White

Yellow

Blue

Others

1.88 3.60

0.19 0.66

22.24 54.17

6.30 0.00

6.51 6.67

4.38 0.00

except for the moderator. The computer program automatically traced and timed (to the nearest one-thousandth of a second) the participants’ movements. Each participant began the experiment by filling out the informed consent form and a general personal information questionnaire concerning her or his demographical information and Internet experience. Before starting the real tasks, each participant was given on-screen instruction. A brief practice session with eight trials was conducted to help the participants understand the operation of the system and the tasks to be performed. Following the practice session, each participant performed the visual search tasks using the testing system. A task page presented a target word and a start button one at a time. The user was asked to remember the word before starting the task, and the Web portal page showed when the user clicked the button to start. The next target word was presented if the participant completed the previous task. Participants completed 10 tasks for one Web portal design and then were given a satisfaction ques-

204

Rau, Gao, Liu

tionnaire. Another 10 tasks were given for another Web portal design. During both the practice and the testing sessions, participants were instructed to perform the visual search tasks as quickly as possible without sacrificing accuracy.


3.3. Experiment 2 Participants. A single factor design was used. Seventy undergraduate students majoring in the Department of Industrial Engineering at Tsinghua University with no prior knowledge about the tasks to be performed during the experiment participated as part of a course requirement. The participants were randomly assigned to one of the seven experimental conditions. The participants ranged in age from 19 to 23 years old (M = 21.2, SD = 0.77).

Tasks. Participants were required to search for and click a series of target words on a Web portal. There were 20 tasks for each participant, and the 20 target words were given to the participants in random order.

Dependent variables. There were four dependent variables in Experiment 2: performance time, error, satisfaction, and animation recognition. Performance time, error, and satisfaction were defined and measured as in Experiment 1. Animation recognition was measured by asking participants to recognize the animations upon completion of the visual search tasks. The participants were asked to identify the 20 animations from 30 animations, with 10 animations not in the experiment. Participants responded “yes,” “no,” or “no impression” to each animation in the animation recognition test.

Independent variable. Floating animation was the independent variable with seven levels: no animations, static animation: leaderboards (728 × 90 × 3), static animation: couplets (100 × 300 × 2), static animation: large square banner (250 × 250), floating animation: floating squares (100 × 100 × 2), floating animation: moving up/down floating squares (100 × 100 pixels × 2), and floating animation: randomly floating squares (100 × 100 pixels × 2). The designs are introduced in the next section.

Testing system. The testing system was developed by HTML/JavaScript and displayed in the Microsoft Internet Explorer Web browser (version 6.0). Web pages were presented on a 17-in. CRT display set to a 1024 × 768 pixels resolution. The target words were designed to appear randomly in different areas of each Web page. Each Web page contained 32 lines of text links in three columns (see Figure 3). Hyperlinks were 12-point blue underlined Chinese text in Song font, which is a common font on most Chinese Web portals. Only one of the hyperlinks contained



205

FIGURE 3 Task page example for Experiment 2.

the target item each time. The hyperlinks were randomly selected from news headlines on Chinese Web portals. Figures 4 through 9 illustrate the six types of floating animations. For each floating animation combination group, 20 animations were created in this study with Macromedia Flash MX 2004. Each animation had one or two large English words with no relation to the tasks at the same speed. Leaderboards were embedded in the Web pages and did not move. Couplets were fixed and arranged on both sides of the Web page. A large square was on the left center part of the Web page. Floating

FIGURE 4 Static animation: leaderboards (728 × 90 pixels × 3).

FIGURE 5 Static animation: couplets (100 × 300 pixels × 2).


FIGURE 6 Static animation: large square (250 × 250 pixels).

FIGURE 7 Floating animation: moving down floating squares (100 × 100 pixels × 2).

FIGURE 8 Floating animation: moving up/down floating squares (100 × 100 pixels × 2).

FIGURE 9 Floating animation: randomly floating squares (100 × 100 pixels × 2). 206


207


squares were on both sides of the Web page and moved down with scrolling. Moving up/down floating squares were on both sides of the Web page and moved up and down automatically. Randomly floating squares moved along with the scrolling direction horizontally and vertically on the Web page.

Procedure. The experiments were conducted in the HCI and Usability Lab. Each participant began the experiment by filling out the informed consent form and a general personal information questionnaire concerning her or his demographical information and Internet experience. Before beginning the real tasks, each participant was given on-screen instructions. A brief practice session with eight trials was conducted to help the participants understand the operation of the system and the tasks to be performed. Following the practice session, each participant performed the visual search tasks using the testing system. One target word was presented at a time, and the testing Web page showed when the user clicked to start the visual search task. The next target word was presented if the participant completed the previous task. Upon the completion of the 20 tasks, each participant was asked to conduct an animation recognition task and then a complete satisfaction questionnaire. 4. RESULTS AND DISCUSSION 4.1. Testing Hypothesis 1 The intention of the hypothesis was to examine how the Web portal design might affect the visual search performances. Because each participant used both simple and rich Web portals, paired t tests were used for testing the hypothesis. To satisfy the normal distribution assumption of paired t tests, some variables were transformed into logarithms. For the data whose logarithm distribution still differed significantly from the normal distribution, a nonparametric testing method (Wilcoxon Matched Pairs Test) for dependent samples was adopted. The result is presented in Tables 2 and 3 for Chinese participants and Table 4 for the German participants, and the corresponding transformation function is documented below the table.

Table 2:

Data for Chinese Participants—English Version

Richa Variable Time (sec) Error Satisfaction

Simpleb

M

SD

M

SD

t

2,770.90 1.15 2.95

1295.50 1.34 0.51

598.20 0.32 3.09

222.88 0.58 0.56

54.95c Z = –2.11d 4.15e

p < .001* .03* .006*

Difference (%) 78.40 72.1 4.7

an = 19. bn = 19. cTransformation: f(x) = ln(x). dWilcoxon Matched Pairs Test. eTransformation: f(x)=ln(1+x). *Significant differences at p < 0.05.

208

Rau, Gao, Liu Table 3:

Data for Chinese Participants–Chinese Version

Richa Variable Time (sec) Error Satisfaction

Simpleb

M

SD

M

SD

t

p

2,267.80 0.29 2.8

904.20 0.61 0.60

461.00 0.36 2.9

190.30 0.74 0.62

7.820 Z = 0.31c Z = 0.88c

< .001* .75

Difference (%) 79.70 —

= 14. bn = 14. cWilcoxon Matched Pairs Test. *Significant differences at p < .05.


an

As shown in Table 2 for the Chinese participants using English versions, significant differences were found in the performance time (t = 54.95, p < .001), error (Wilcoxon test: Z = –2.11, p = .03 < .05), and satisfaction (t = 4.16, p = .006 < .05). Participants using simple Web portals were 78.4% faster than participants using rich Web portals and made 72.1% fewer errors, and their satisfaction ratings for the simple design were 4.7% higher than for the rich design. As shown in Table 3 for the Chinese participants using Chinese versions, a significant difference was found in performance time, t = 7.82, p < .001. Participants using simple Web portals in Chinese were 79.7% faster than participants using rich Web portals in Chinese. No significant differences were found in error and satisfaction. As shown in Table 4 for the German participants, significant differences were found in performance time (t = 10.1, p < .001), errors (Wilcoxon test: Z = –1.94, p = .05), and satisfaction ratings (t = 7.59, p < .001). Participants using simple Web portals were 68.4% faster than participants using rich Web portals and made 77.5% fewer errors, and their satisfaction ratings for the simple design were 15% higher than for the rich design. An examination of visual search performances between the Chinese and English version for the Chinese participants indicated that no significant differences exist in performance time, error, and satisfaction. The differences between Chinese participants and German participants were tested with t tests, and it was found that the visual search performance, in terms of time, error, or satisfaction ratings of Chinese participants (simple or rich), did not differ significantly from German participants, with rich Web portals or with simple Web portals, as shown in Table 5. To assess the effects of Web portal design for the Chinese participants in contrast to those for the German participants, the interaction between participants’ nationality and types of portal design was examined using repeated measures analysis of Table 4:

Data for German Participants–English Version

Richa Variable Time (sec) Error Satisfaction an

Simpleb

M

SD

M

SD

t

2,131.30 0.89 2.80

674.54 1.59 0.59

604.40 0.20 3.30

208.70 0.54 0.57

10.01 Z = 1.94c 7.59d

p < .001* .05* < .001*

= 19. bn = 19. cWilcoxon Matched Pairs Test. dTransformation: f(x) = ln(x +1). *Significant differences at p < 0.05.

Difference (%) 68.4 77.5 15.2

Effect of Design and Animations on Search Table 5:

209

Comparison Between Chinese Participants (Using English Web Pages) and German Participants Chinesea

Variable


Rich design Time (sec) Error Satisfaction Simple design Time Error Satisfaction an

Germanb

M

SD

M

SD

t

p

2,770.90 1.15 2.95

1,295.50 1.34 0.51

2,131.30 0.89 2.80

674.54 1.59 0.59

1.67c Z = 406d 0.8

.10 .26 .43

598.20 0.32 3.09

222.88 0.58 0.56

604.40 0.20 3.30

208.70 0.54 0.57

0.3c Z = 388.5d 0.95

.76 .47 .34

= 19. bn = 19. cTransformation: f(x) = ln(x). dWilcoxon Two Sample Test.

variance (ANOVA). The results indicated that the interaction effect had no significant influence on performance time (F (1, 72) = 2.78, p = .10), but influenced satisfaction significantly (F (1, 72) = 4.521, p = .04; see Figures 10 and 11). The results indicated that although both Chinese and German participants were more satisfied with the simple design and less satisfied with the rich design, the strength of the effect of the Web portal design (rich/simple) differed. Compared with German participants, the difference between Chinese users’ satisfaction ratings for the simple design and their ratings for the rich design was smaller. The results showed that all three groups of participants performed visual search tasks faster on simple Web portals design than on rich Web portals. The results are consistent with the past studies in visual attention and perception that in general, objects in our peripheral vision can capture our attention (Driver & Baylis, 1989). The meaning of a non-attended stimulus is processed to a certain extent (Allport, 1989; Duncan & Humphreys, 1989; Treisman, 1991). The resources for processing the target stimulus may be affected by the resources used to attend to nontarget stimuli. Users may dominate more visual attention and work load toward each item presented when more stimuli appeared on the screen and thus decrease the

FIGURE 10 Interaction effect between language effect and Web portal design on performance time.


210

Rau, Gao, Liu

FIGURE 11 Interaction effect between language effect and Web portal design on satisfaction.

available resources for processing the target stimulus and therefore the visual search performances. However, no significant differences were found between Chinese users and German users regarding performance time, with either the simple portal design or the rich portal design. For the two groups using the English version, significant differences were found in error and satisfaction with both Chinese and German participants. Rich portals resulted in more errors and lower satisfaction. However, the extent to which participants’ satisfaction was influenced by the design differed between Chinese and German. Chinese participants’ satisfaction was less influenced by the differences between rich and simple Web portals. As all Chinese participants were extremely experienced Web users (averaging 4.3 hr on the Internet per day), the finding can be interpreted as such: Chinese users are as annoyed by those overwhelming attention distracters (e.g., graphs, animations, brightly colored text) on overly rich Web portals when performing visual search tasks as the Germans; on the other hand, the influence of the design (rich/simple) on their subjective evaluation becomes smaller as they are used to rich Web portals.

4.2. Testing Hypothesis 2 The intention of Experiment 2 was to investigate whether a visual search performance is influenced by the combination of floating animation banners. Table 6 shows the descriptive statistics. The difference between static animations (banner, large square, couplets) and floating animations (moving down, moving up/down, randomly floating) was tested with an ANOVA, as shown in Table 7. Where the assumptions of the ANOVA were violated (even after possible transformation of variables), a Wilcoxon two-sample test was used. No significant difference was found in performance time (Z = –1.3823, p = .1669), error (Z = 0.5923, p = .5516), satisfaction (F (1, 58) = 2.01, p = .1618), and animation recognition (Z = –1.3878, p = .1652) between floating animations and static animations.

Effect of Design and Animations on Search Table 6:

211

Descriptive Statistics of Experiment 2 Static Animations

Dependent Variables Time (sec) Error rate


Animation recognition Satisfaction

No Animation X(s)

Leader Boards X(s)

Couplets X(s)

Large Square X(s)

Moving Down X(s)

Moving Up and Down X(s)

Randomly Floating X(s)

43434.91 (11038.68) 0.10 (0.32) —

58253.98 (21845.35) 0.40 (0.52) 5.1 (5.26) 4.41 (0.75)

47774.56 (11406.31) 0.50 (1.27) 7.2 (6.96) 4.47 (0.98)

49259.10 (8286.16) 0.10 (0.32) 3.9 (3.67) 4.27 (0.87)

55358.35 (19584.79) 0.10 (0.32) 7.2 (5.43) 4.13 (0.96)

55072.34 (12438.01) 0.60 (1.58) 6.4 (4.53) 4.09 (1.07)

67426.40 (31187.93) 0.20 (0.42) 6.5 (4.12) 3.89 (1.26)

4.9 (0.57)

Table 7:

Data for Testing Hypothesis 2

Statica Variable Time (sec) Error Satisfaction Recognition an

Floating Animations

M 51,762.55 0.333 4.38 5.400

Floatingb SD

15,230.36 0.8000 0.8465 5.4494

M

SD

F

p

522,85.69 0.300 4.033 6.700

22,432.33 0.9500 1.0669 4.5724

Z = –1.3823c Z = 0.5923c F = 2.01 Z = –1.3878c

.1669 .5516 .1618 .1652

= 30. bn = 30. cWilcoxon Two Sample Test.

Then each animation treatment (banner, large square, couplets, moving down, moving up/down, randomly floating) was compared with the control group (no animations). It was found that participants searching with randomly floating animations (M = 67426.40, SD = 31187.93) were significantly slower (Wilcoxon two-sample test: Z = –2.4568, p = .0140) than those searching with no animation (M = 43434.91, SD = 11038.68). Significant differences in satisfaction were also found between no animation and randomly floating animations (t = 2.327, p = .032), no animation and moving down animations (t = 2.194, p = .042), and no animation and moving up/down animations (t = 2.130, p = .047). The satisfaction with no animation was 26.0% higher than with randomly floating squares, 18.6% higher than with moving down squares, and 19.8% higher with moving up/down squares. Usability experts have found that people searching for specific items on the Web are very likely to ignore salient (large, colorful, animated) items, especially banners, on Web pages (Benway, 1998; Nielsen, 1997). Further, it was found that design and positioning of banners did not have a significant influence on banner recognition (Benway, 1998). On the basis of previous research, this study investigated the influence of the movement (static, moving down, moving up/down, and random floating) of the animated banners. The result showed that only randomly floating animation influenced visual search performance time. Further, the movement of animations was found to have no influence on animation recognition. The reason may be the extended “animation avoidance.” Participants in our study were highly


212

Rau, Gao, Liu

experienced Internet users (4.3 hr/day), and they already have a schema of the form and function of the animated contents on a Web portal. Most often these are advertisements and are irrelevant with their searching tasks on the Web. Users become so used to animation avoiding that even when the animation is floating, they can easily detect the pattern of the movement and neglect the whole area within which the animation is moving. For example, it is possible that users notice that the movement of up/down squares is restricted to the left and the right margin of the page, and then they keep these areas intentionally out of their sight. However, the randomly floating squares move with no restrictions, and users can hardly exclude them from their sight and mind. Attention was diverted and visual search tasks were interfered with in this way, as indicated by the fact that users searching on the pages with randomly floating animations used 55.24% longer to perform the tasks than users searching on pages with no animation. However, the longer exposure time did not lead to better animation recognition. This result indicated that users were reluctant to process any animated contents that they considered advertisement-like and task irrelevant, no matter how attention grabbing the animation might be. It was also found that floating animations came at a cost of user satisfaction, as demonstrated by the fact that users searching pages with floating animations (moving down, moving up/down, and randomly floating) were significantly less satisfied than users searching pages with no animation. Users may perceive the page as advertisement cluttered and of low informational value. They may also perceive the page as obtrusive, because they must ’protect’ themselves from those unwanted contents distracting them from their goal-driven navigation.

5. CONCLUSIONS In Experiment 1, participants using simple Web portals searched faster than participants using rich Web portals, made fewer errors, and were more satisfied. Chinese participants did not differ significantly from German participants in visual search performance on Web pages, either on rich or simple Web pages. But the difference between simple and rich portals imposed less influence on Chinese participants than it did on German participants. The results imply that experience with rich Web portals neither helps Chinese users perform better than German users nor makes them prefer rich Web portals to simple Web portals when performing visual search tasks. But such experience blurs the difference between rich and simple Web portals in their satisfaction. In other words, it makes Chinese users less sensitive to the difference between rich and simple Web portals. The results of Experiment 2 showed that users became so good at animation avoiding that even moving down and moving up/down animation could not grab their attention when they searched on the page. But they could not avoid the randomly floating animation and suffered greatly during the visual search tasks. The moving features (moving down, moving up/down, and randomly floating) of animations did not lead to significant differences in animation recognition, as ex-



213

pected. However, users’ satisfaction with these features was significantly lower than with no animation. The task preciseness was not influenced in any case. These results are limited in several ways. First, we did not control individual attention performance in both experiments. As a result, the difference in attention might distort the results. Second, the duration and frequency of watching a banner was not measured explicitly in this study. Eye-tracking devices should be used in a future study to measure these variables accurately. The allocation of attention can be better investigated and its influence on the performance time of visual search tasks and animation recognition can be better studied in this way. Third, according to the result, Web users seem to have the ability to adapt to problems. However, to prove this, comparative studies between performance of inexperience users and experienced users are needed. In this study, only experienced users were tested. In future studies, it would be interesting to explore such an adaptation further. Despite the limitations, the study did provide some useful implications for Web portal designers and for usability experts. Though Chinese users are experienced with very crowded and rich Web portals, their visual search capacity does not differ significantly from German users. Their satisfaction with performing visual search tasks on rich Web portals is also lower than on simple Web portals. But their satisfaction is less sensitive to the rich/simple design than the Germans’. This result indicates that Chinese users also have usability problems and negative attitudes toward overrich Web portals when searching information. Chinese users are more tolerant of rich Web portals, but they still prefer simple Web portals to rich portals when performing visual search tasks. For Web site developers, simple design should be preferred when designing for Chinese. The higher tolerance of Chinese users may be good news for designers, but attention should be paid to control excessive content or unnecessarily rich representations, which would add difficulties to visual search tasks on the Web page. Animation floating, especially randomly floating elements, on the page should be used with great care. They impair visual search tasks and reduce users’ satisfaction without yielding better animation recognition. New online advertising techniques that are less obtrusive, more customer friendly, and more engaging are needed. Advertisers should provide tailored content rather than trying to grab attention with rich presentations. Furthermore, the higher tolerance of Chinese users could change over time. As Evers (1997) discovered, Chinese found “usefulness” a more discernible variable than ease of use in their satisfaction rating. Therefore, what influences Chinese users’ satisfaction with a Web portal the most might be their belief in the usefulness of the abundant information. Whether this is another underlying reason for the result that Chinese user satisfaction ratings are less influenced by rich/simple design remains an interesting topic for future study. Another direction for future research is to study whether Chinese preference for rich Web portal design can be applied to enterprise portal design. Public Web portals work to satisfy the informational needs of a vast and public audience, but enterprise portals are designed to facilitate communication and collaboration within

214

Rau, Gao, Liu

the enterprise as well as improve efficiency of the enterprise. The influences of these differences on users’ preferences for information design and visual design for the portal should be studied in detail in the future.


REFERENCES Allport, A. (1989). Visual attention. In M. I. Posner (Ed.), Foundations of cognitive science (pp. 631–682). Cambridge, MA: MIT Press. Bayles, M. E. (2002). Web site analysis: Designing online banner advertisements: Should we animate? Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Changing Our World, Changing Ourselves, pp. 363–366. Bayles, M. E., & Chaparro, B. (2001). Recall and recognition of static vs. animated banner advertisements. Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting, pp. 1202–1204. Benway, J. P. (1998). Banner blindness: The irony of attention grabbing on the World Wide Web. Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting (pp. 463–467). Chicago: HFES Press. Benway, J. P., & Lane, D. M. (1998). Banner blindness: Web searchers often miss “obvious” links. Available from http://www.internettg.org/newsletter/dec98/banner_blindness.html Bertera, J. H., & Rayner, K. (2000). Eye movements and the span of effective stimulus in visual search. Perception & Psychophysics, 62, 576–585. Bernard, M., Baker, R., & Fernandez, M. (2002). Paging vs. scrolling: Looking for the best way to present search results. Available from http://psychology.wichita.edu/surl/ usabilitynews/41/paging.htm Brandt, H. (1954). The psychology of seeing. New York: Philosophical Library. Brawn, P., & Snowden, R. J. (1999). Can one pay attention to a particular color? Perception & Psychophysics, 61, 860–873. Burke, M. K., & Hornof, A. J. (2001). The effect of animated banner advertisements on a visual search task (Tech. Rep. No. 02–03). Department of CIS, University of Oregon. Card, S. K. (1982). User perceptual mechanisms in the search of computer command menus. Proceedings of CHI ’82 (pp. 190–196). New York: ACM. Chaparro, B., & Bernard, M. (2001). Finding information on the Web: Does the amount of white space really matter? Proceedings of the Tenth Annual Usability Professionals’ Association Conference. Available from http://psychology.wichita.edu/surl/usabilitynews/ 2w/whitespace.htm Christ, R. E. (1975). Review and analysis of color coding research for visual displays. Human Factors, 17, 542–570. China Internet Network Information Center. (2005). CNNIC’s 15th survey on China Internet development. Available from http://www.cnnic.net.cn/download/2005/2005011801.pdf Cook, J. R. (1991). Cognitive and social factors in the design of computerized jobs. Unpublished doctoral dissertation, Purdue University. Driver, J., & Baylis, G. C. (1989). Movement and visual attention: The spotlight metaphor breaks down. Journal of Experimental Psychology: Human Perception and Performance, 15, 448–456. Duncan, J. G., & Humphreys, W. (1989). Visual search and stimulus similarity. Psychological Review, 96, 433–458. Evers, V. (1997). Human-computer interfaces: Designing for culture. Unpublished doctoral thesis, Faculty of Mathematics, Informatics, Physics, and Astronomy, University of Amsterdam, Amsterdam, the Netherlands.



215

Faraday, P. (2001). Attending to web pages. CHI ’01 Extended Abstracts on Human Factors in Computing Systems (pp. 159–160). Seattle, WA: ACM Press. Hillstrom, A. P., & Yantis, S. (1994). Visual motion and attentional capture. Perception Psychophysics, 55, 399–411. Hornof, A. J., & Halverson, T. (2003). Cognitive strategies and eye movements for searching hierarchical computer displays. Proceedings of the Conference on Human Factors in Computing Systems, Ft. Lauderdale, FL, 5,(1), 249–256. Jonides, J. (1981). Voluntary versus automatic control over the eye’s movements. In J. Long & A. Baddeley (Eds.), Attention and performance IX (pp. 187–203). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Maldonado, C. A., & Resnick, M. L. (2002). Do common user interface design patterns improve navigation? Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting, 1315–1319. McDonald, J. E., Stone, J. D., & Liebelt, L. S. (1983). Searching for items in menus: The effects of organization and type of target. Proceedings of the Human Factors Society 27th Annual Meeting, 834–837. National Cancer Institute. (2003, June 3). Research-based Web design & usability guidelines. Available from http://www.usability.gov/guidelines/index.html Neerincx, M. A., Lindenberg, J., & Pemberton, S. (2001). Support concepts for Web navigation: A cognitive engineering approach. Proceedings of the Tenth World Wide Web Conference (pp. 119–128). New York: ACM Press. Nielsen, J. (1995, December). Guidelines for multimedia on the Web. Alertbox. Available from http://www.useit.com/alertbox/9512.html Nielsen, J. (1997, September 1). Why advertising doesn’t work on the Web. Alertbox. Available from http://www.useit.com/alertbox/9709a.html Nygren, E., & Allard, A. (1996). Between the clicks skilled users scanning of pages. Designing for the Web: Empirical studies. Human Factors and the Web/HTML Conference. Ojanpää, H., Näsänen, R., & Kojo, I. (2002). Eye movements in the visual search of word lists. Vision Research, 42, 1499–1512. Pagendarm, M., & Schaumburg, H. (2001, August 24). Why are users banner-blind? The impact of navigation style on the perception of web banners. Available from http:// jodi.ecs.soton.ac.uk/Articles/v02/i01/Pagendarm Parkinson, S. R., Sisson, N., & Snowberry, K. (1985): Organization of broad computer menu displays. International Journal of Man-Machine Studies, 23(6), 689-697. Pashler, H. E. (1998). The psychology of attention. Cambridge, MA: MIT Press. Spool, J. M., Scanlon, T., Schroeder, W., Snyder, C., & DeAngelo, T. (1999). Web site usability: A designer’s guide (pp. 89–91). San Francisco: Morgan Kaufmann. Stewart, T. F. (1976). Displays and the software interface. Applied Ergonomics, 7, 137–146. Stone, D., & Glock, M. D. (1981). How do young adults read directions with and without pictures? Journal of Educational Psychology, 73, 419–426. Treisman, A. (1988). Features and objects: Fourteenth Bartlett memorial lecture. Quarterly Journal of Experimental Psychology, 40, 201–237. Treisman, A. (1991). Search, similarity, and integration of features between and within dimensions. Journal of Experimental Psychology: Human Perception and Performance, 17, 652–676. Tullis, T. S. (1997). Screen design. In M. Helander, T. K. Landauer, & P. Prabhu (Eds.), Handbook of human–computer interaction (2nd ed., pp. 503–531). Amsterdam: North-Holland. Weir, G. R. S., & Heeps, S. (2003). Getting the message across: Ten principles for Web animation. Paper presented at the IASTED International Conference on Internet and Multimedia Systems and Applications, Honolulu, HI.

216

Rau, Gao, Liu


Weller, D. (2004, July 16). The effects of contrast and density on visual Web search. Available from http://psychology.wichita.edu/surl/usabilitynews/62/density.htm Wright, P., Milroy, R., & Lickorish, A. (1999). Static and animated graphics in learning from interactive texts. European Journal of Psychology of Education, 14, 203–224. Zhang, P. (2001). The impact of animation timing and location on visual search task performance in the Web environment. Paper presented at the Seventh Americas Conference on Information Systems, Boston, MA. Zhang, P., & Massad, N. (2003). The impact of animation on visual search tasks in a Web environment: A multi-year study. Proceedings of the Americas Conference on Information Systems (2265–2272). Tampa, FL: AIS Press.