Innovations online - Wiley Online Library

Innovations online Blackwell Science, Ltd

Betsy Anagnostelis, Librarian, Medical Library, Royal Free and University College Medical School of UCL and Royal Free Hampstead NHS Trust, Rowland Hill Street, London NW3 2PF, UK

In this second part of the article ‘Surfing for health’, the authors present the results of a usercentred evaluation of a commercial health website (Surgerydoor). Twenty non-expert users contributed to the study, which consisted of participant observation and interview methods, gathering data about both open and directed use. Although the findings of the study are not entirely unexpected, it is interesting that they reinforce a number of key messages for both health care website developers and library and information professionals. Regarding website design, the study confirmed that users were not happy with large amounts of text on-screen, or large numbers of contents lists,

Correspondence: Betsy Anagnostelis, Librarian, Medical Library, Royal Free and University College Medical School of UCL and Royal Free Hampstead NHS Trust, Rowland Hill Street, London NW3 2PF, UK. E-mail: [email protected]

site maps or indexes, and disliked commercial features. Use of search facilities demonstrated that direct searching was more popular than site navigation. However, even seemingly adept users were found to experience difficulties in exploiting the search engine functionality, especially where it differed from expected behaviour, e.g. by comparison with established generic web search engines. The authors conclude that there is a significant role for library and information professionals in supporting users in their utilization of health care websites, both in promoting evaluation tools, such as DISCERN,1 and also in enhancing users’ awareness of the existence of gateways to preevaluated quality resources, such as OMNI.2

References 1 DISCERN. Available online: http://www.discern.org.uk /. 2 OMNI. Available online: http://omni.ac.uk /.

Innovations online

Surfing for health: user evaluation of a health information website. Part two: fieldwork Peter Williams*, David Nicholas*, Paul Huntington* and Fiona McLean†, *Internet Studies Research Group, City University, †Health Care Information Service, British Library, London, UK

Abstract This is the second part of a two-part paper examining the question of the evaluation of health

Correspondence: Peter Williams, Internet Studies Research Group, Department of Information Science, City University, London EC1V 0HB. E-mail: [email protected]

214

information websites. It follows an extensive background and literature review that formed part one, and describes a user-centred evaluation of a commercial health site—that of Surgerydoor (http:// www.surgerydoor.co.uk /). This was undertaken as part of an ongoing research programme, Digital Health, looking at the impact of electronic health information systems for the general public.1 Twenty subjects undertook the evaluation, conducted through participant observational and interview methods. In the first part of the evaluation, subjects were invited to discuss such usability issues as navigation, contents labelling, readability and attractiveness of the information, site design and general layout. Following this, a number of retrieval tasks were given, which ranged from searching for factual information to an in-depth

© Blackwell Science Ltd 2002 Health Information and Libraries Journal, 19, pp.214– 225

Innovations online

investigation requiring perusal of a number of pages. Results showed that users were unhappy with the large amount of text on each page, and found it difficult to read lengthy contents lists or activate individual entries. The large number of contents lists, site maps and indexes did not aid navigation or information retrieval. Regarding content, users found the information itself well written and interesting. Commercial features were not well received, being considered inappropriate for a health information site and impinging on the authority of the information. Other aspects related to authority, however, such as attribution, currency, etc., were not mentioned. Content providers are recommended to reduce the number and location of content entries and quantity of material per page, and information professionals to emphasize the importance of the quality of the content of health information to their users.

Introduction: the Surgerydoor website This paper focuses on the medium of the Internet, and primarily on issues of usability. It describes a user-centred evaluation of the commercial Surgerydoor website. This site was chosen as the researchers are working with InTouch With Health, the company which produced the site (in addition to electronic touch-screen kiosks), and is responsible for its design and content. The research team are also undertaking an analysis of the site’s computer log files, and examining use and user reaction to its equivalent touch-screen kiosks as part of the research programme.2,3 The website has described itself4 as offering ‘the UK’s first electronic versions of official NHS information and the country’s biggest online health multistore’. It comprises over 5000 pages of content with access to an additional 40 000 pages of local health service maps and listings via web links to approved (and approving) external organizations. Apart from information provision, the site sells products and services ranging from pharmacy items to health foods and services. Revenue is generated from sponsorship, advertising and content syndication. Site content is divided into the following sections, each of which appear as main content entries on a side bar and contain a series of subheadings:

• Health Daily: this contains a news and health alert, weather, and tips (such as ‘what to look for when considering joining a gym’). • Medical: comprises ‘Emergencies’, a Symptoms Index, ‘Diseases in Depth’, a Medical Dictionary and a Prescription Drug Guide. • Healthy Living: discusses Preventing Accidents, Dental Health Drugs and advice on alcohol consumption. • NHS & Benefits: provides listings on ‘Health in your area’ (searchable by postcode), including surgeries, dentists, hospitals, etc. • Complementary Medicine: includes an ‘A to Z’ of complementary medicine. • Travel Health: includes a section on vaccinations, giving a list of what vaccinations are required for each country, and a ‘Traveller’s Health Kit’—a list of items that may be useful during a holiday. • Community & Fun: including a ‘Surgerydoor Health Magazine’, feedback and suggestions, health surveys, quizzes and competitions. • Shopping: an area where users can buy online. • Today’s Selection of Features and Topics: includes ‘This week’s Radio Times article from Dr Mark Porter’, who is one of the website team.

Aims and objectives The broad aim of the study was to obtain an understanding of how people used the website so that we would be better placed to interpret the emerging transaction logs. Web log analysis is a major strand of the research project.5 The more specific aims were to: • obtain users’ personal opinions about various aspects of the site, particularly regarding usability issues, such as menu options, navigational facilities, site structure, etc., although their views on content were also considered important; • discover how users went about searching it— how they navigated and retrieved information; • test evaluation methodologies and observation techniques as part of an effort by the Internet Studies Research Group to formulate a broad evaluatory framework for medical websites. Although interesting, and of value to the overall research programme, this particular study did not

© Blackwell Science Ltd 2002 Health Information and Libraries Journal, 19, pp.214–225

215

216

Innovations online

explore in any depth issues related to the subjects’ perceived needs for health information, nor the interplay between different sources and the circumstances in which, for example, professional medical help, informal advice or written (printed or electronic) information would be sought. However, comments and opinion about the readability or perceived relevance of information found on the site were examined, despite the latter arguably not informing the ‘usability’ aspect of the research.

A user-centred approach was adopted. A group of end users, i.e. neither web nor domain (in this case health) experts, were observed using the website and interviewed about their experiences afterwards. Use was firstly ‘open’. Here, participants examined the site in their own way, looking at pages of personal choice and navigating through the system as they desired. Secondly, directed use was undertaken, where information retrieval tasks were set. For present purposes, users were required to find information by any routes possible, but the exercise was undertaken with a view to developing a ‘cognitive walkthrough’ type approach appropriate for end users in the future, which might take the form of asking them to determine what possible actions they may take, and to evaluate themselves the efficacy of following various paths. Data gathering was accomplished in three ways: • participant observation (for Part I), in which users were encouraged to ‘chat’ freely about the site and their views of it; • protocol analysis (for Part II), in which users described their actions more formally, and were instructed to describe each individual step; • post test interviews, which focused both on the subjects’ general opinions and on their thoughts on the information retrieval tasks.

literature on evaluating information technology applications, even single figure evaluator numbers are considered adequate. Nielsen and Mack6 recommend the use of between three and five evaluators. They argue that only a small number of evaluators is required to elicit the major issues that arise in, for example, navigation or text size. In keeping with this, the commercial IT evaluation company Serco, who are looking at digital TV health information initiatives in tandem with the present writers, typically invite between 10 and 15 viewers to their evaluation sessions. It was expected that having specifically information science students undertake the evaluation would add value to the research, as they might have specialist knowledge or skills about information organization and presentation that had general applicability (although ‘web experts’ were not sought, as explained below). Potential participants (all those on the MSc course) were assigned random numbers. These were then ordered, and students invited to participate one by one down the list until 20 recruits had been obtained. During the recruitment process, students were asked to state the extent of their web use. Those who indicated they had used the Internet to search for information for more than one hour daily for a period of more than four months, or two hours daily for two months, were deemed too expert. Twenty-three out of 75 students were eliminated using this simple measure of expertise. Students were also asked to withdraw from consideration if they had studied any aspect of medicine or health at an undergraduate or higher level. This exclusion criterion was imposed because, as mentioned above, the object of the exercise was an evaluation by lay and not expert users. There was no quota system by age, gender or other characteristic. The breakdown of 14 females and 6 males reflected almost exactly the population from which the sample was taken.

Sample

Results

Methods

Twenty MSc Information Science students from City University participated in the evaluation. For quantitative data, where the object is to infer generalities from samples, this would constitute a low number of respondents. However, in the

Part I: General opinions/comments about the site The home page One of the most important design features of an Internet site is, of course, the home page. This acts


Innovations online

as a gateway into the site and also provides a statement about what it contains, whom the site belongs to and what the site is all about. One aspect of the home page commented upon was that the Surgerydoor icon is very small, and situated on the right of the screen. Considering that we read from right to left this might, as one respondent pointed out, be a poor position. It means that the nature and source of the site is not immediately obvious. Fifty percent of users found the icon ‘too small’ or ‘not prominent enough’, meaning that there was no clearly defined identity for the site, and this was disconcerting for users. When they were shown the icon and told the name, however, only one person enquired further, to ask about the legitimacy of the company, etc. No fewer than 14 (70%) respondents said the text was too small, and 13 (65%) thought that too much information was provided for a home page. This problem was mentioned with regard to other pages, too, and echoes findings from other research. Nielsen, in his early study of website design, said that ‘users were often overwhelmed by the amount of information on the pages … one user commented that the Sun home page “contains so much stuff that I will just ignore anything that is not clear” ’.7 The link to television programmes was considered confusing by many (6, or 30%), who asked if all programmes or just those on health matters were listed. Three people also thought this feature was inappropriate for a health site, although there was one voice of approval, who liked the feature as it made the site a ‘one stop shop’. A final criticism was that the Multistore entries were considered ‘too commercial’ by 5 (25%), with one user saying this made him suspicious of the motives of the site and the information carried by it. Again, one user was more positive, saying that the information on the site led people to realizing they needed to eat more healthily, etc., and the site helped them find the products. There were a few other positive comments. Three users (15%) thought the page attractive, with two of those citing the colour scheme. Two users also liked the currency element of the page manifested in ‘rolling’ ticker-tape news headlines and the current date predominantly displayed.

Navigation and finding information Contents lists. As with The Times site evaluated by the Internet Studies Research Group,8 Surgerydoor provides a number of content lists. These appear at the right and left-hand side of the page, and at the bottom of the page. Although the lists appear to be (almost) mutually exclusive, there is the possibility that users may be a little confused about all the different options available. Indeed, the general opinion of users regarding the menu entries was that there were too many lists and too many entries in each. Nearly half (45%) of the users mentioned the latter, and 30% complained about having menu options at each side and in the middle column. No other general themes emerged, although there were other criticisms and suggestions. The former included the fact that the entries were not in alphabetical order, that some items were duplicated, and that there was an element of randomness in the choices. It was suggested that some items would be of such universal interest that they merited main contents entries—cancer was the topic that was most mentioned here. The fact that the left contents list was (almost) constant throughout the pages on the site provided grounds for satisfaction, although there was some consternation when it was discovered that it disappeared when certain pages were accessed. (It does so because some of the pages are from external sites.) Although many of these, such as the home page of the Benefits and Entitlements section, remain inside the bordering frame of Surgerydoor, others leave it. Two users (10%) thought that the left and right frame menu lists could have been merged, with a corresponding reduction in listed entries. Those who suggested this, and other users (7, or 35% in total), considered that the main entry item was sufficient for the home page, with more detailed entry lists appearing on activating the main header. The idea of using drop down menus was also mooted. The one on the right side was generally approved of, and two users suggested the same for the left hand one. The home page of the website is pictured below, showing the menu lists and the position of the Surgerydoor title logo (Fig. 1).


217

218

Innovations online

Figure 1 Home page of the Surgerydoor website

Positive remarks about the menu system generally included the fact that the abundance of menu items did at least allow the user to gain an insight into the contents of the site. Browsing was encouraged, someone suggested, by this arrangement. In terms of presentation and aesthetics, a number of observations were made. Firstly, as with pages generally on the site, the font size was criticised. It was so small that anyone with poor eyesight would struggle to read the entries. Exacerbating the problem was the colour scheme. For the left hand menu bar this was white lettering on a lilac/purple background. On the right hand frame the combination was lilac/purple text on a light green background. Both combinations made it difficult to distinguish the text. A different problem relating to the menu bars is that each menu bar obviously takes up valuable space. The text size and amount of information on the page proved to have a big impact on users’ opinions of the site.

Search engine. Despite the comprehensiveness of the menu options, the search engine scored more highly in arriving at the information required. Counting the first three retrieval tasks only (see below), 36 (63%) correct responses were arrived at through a search, as opposed to 21 (37%) from menu entries. The search engine might have scored even more highly, had it not required phrases to be enclosed in parentheses. In spite of instructions on the web page to ‘use a word, or phrase with parentheses’, only 4 (20%) users noticed these requirements before their first use of the facility. On several occasions (see individual tasks below for details) users made one or another of the following errors: • used phrases without parentheses; • enclosed single words in parentheses; • included Boolean operators. The choice of terms was interesting, too. In addressing retrieval task 4 (see below), the users all chose the concrete terms ‘cancer’ and ‘diet’, rather


Innovations online

than the abstract one ‘prevention’. This group of users, at least, also expected high precision. A dim view was taken of hit list entries that did not contain the search terms, and of complete pages that, despite containing the terms input, were only marginally relevant. Users were perhaps more critical of the search engine because they were information science students and had had some experience of using and evaluating search systems. Indeed, the main point made—that there was no information on how the engine worked—was, perhaps, a reflection of this. A second point was that the search terms are not highlighted in the hit list entries, making it somewhat difficult to determine why something was displayed. There were also complaints that some pages did not seem to relate to the topics suggested by the terms, and there was much doubt as to whether there was a weighting system to filter out entries where the search terms only occurred once or twice in the document. An interesting finding from the information locating exercises undertaken was that users did not follow the search box instructions. This is detailed below, in the section on task completion. General comments on navigation. The overall impression seemed to be that the site was simply too complicated and that navigation around it was not easy. The main reason for this seemed to be the changes in menu structure that occur in different parts of the site. One user gave the following example: ‘When I click on Symptoms Index in the home page in the purple column, I was brought to the page that list the various kinds of symptoms. But together with this page is a change in the purple column which now has many different options. It is confusing and I am not sure what I am looking at if I do click on any of these new items listed in the purple column’. Four users originally stated that it was good having a standard format for each page, i.e. with the left and right frames, but changed their minds on discovering that this did, in fact, change according to the source or producers of the particular information. For example, NHS Healthy Living pages within the site have their own structure. Two of these users then mentioned that this was confusing or disorientating.

The overlapping of contents lists also caused problems. One user accessed a page on multiple sclerosis from the Diseases in Depth section, and then found another one from the Medical Conditions links. They were both very similar. Indeed, at first it appeared the same page had been retrieved. Information content Surprisingly, comments about the information content itself were limited. This may have been because of the test situation, where users, although encouraged to read the information as they were making their evaluations, may not have felt that it was appropriate to spend lengthy periods in silent reading. People were, of course, encouraged to continue their perusal of the site in their own time and e-mail comments, but this was only undertaken in one case. Although the various participants naturally chose a variety of pages, one common comment from their brief readings was that the information presented was well written, informative and interesting. The prevailing view was that good advice was being offered on all the pages accessed, with the constant reminder to visit a doctor if symptoms persisted being particularly sensible. Only one reader complained of ‘too much text’, and felt that bullet points would have been more appropriate. For this person, the web was simply there to provide basic information that could be followed up by using more traditional sources. Another said that the text was too technical, and pointed to expressions such as ‘hypertrophy’ to justify their view. Finally, following on from the generally negative attitude towards the advertising, one sceptical student thought that information might be included because of the commercial nature of the site (i.e. promoting healthy eating in order to sell health foods). This issue is discussed later. Evidence of currency was rarely sought. Two users were reassured that the date included on the home page was giving the correct reading, and one said that from this there was an implication that information offered in the health news section, would be current. No one, however, looked for posting dates of any documents, or asked about the issue. Interactivity The interactive element of the site is the part where users can interact with each other by posting


219

220

Innovations online

messages, asking questions and generally becoming part of an online community. The introductory page explains that the idea is ‘to help you communicate with each other through our specially created Message Boards, where you can share your experiences and help others in similar circumstances’. Only one participant referred to this interactive feature of the site, despite its visibility as a main entry item on the home page. She said, in slightly disparaging terms, ‘hearing about the discomfort of other sufferers may just put you off’. In fact, she did not activate the page to look at its features. None of the other participants chose that page to view either. General comments about site content appeared to suggest that, for these participants anyway, the site was very much an information rather than a communication conduit. As indicated in the discussion below, health is not the only area where there has been a lack of interest in ‘interactive’ facilities. Hyperlinks Five (25%) users commented that there should be more external links and that these (and internal ones) should be made in the body of the text. The former point was made most frequently whilst undertaking a set task requiring students to find the address of the British Red Cross. It was felt that a link to the website of that organization, or at least the web address, should have been included. Two users said the link facility was at the heart of the Internet and users therefore expected sites to be ‘peppered with them’. Finally, four (20%) subjects found that the internal site map link was inactive, and one user discovered another internal ‘dead link’; a ‘tips for getting started’ page in the Sport and Fitness section. Frames Site construction, in the form of the ‘frame’ system, was also commented upon extensively. Several problems with this approach were highlighted in addition to that of their changing to accommodate different menus, as mentioned above. One user pointed out that it is not possible to bookmark framed pages—the URL stubbornly remains that of the home or framing page. Thus, pages that one might like to have at hand easily, e.g. those giving details about the symptoms of

meningitis, could not be easily bookmarked. Another remarked that frames are difficult to read and manipulate by visually impaired users. No one, surprisingly, mentioned the problem that frames often confuse printers. For example, often only one frame is printed. This may be acceptable sometimes, but on other occasions either the whole page or one different to that printed is required. At worst, a ‘no pages to print’ message appears on screen creating much frustration. The lack of comment on this was probably because there was no requirement to reproduce pages during the exercise. Advertisements A health information site carrying advertisements for health products may seem quite reasonable— good health may sometimes be better maintained with appropriate medicines or equipment. Women in early pregnancy are advised to take folic acid supplements, for example. However, the advertisements on Surgerydoor are displayed as banners across the top of each page, where they occupy a larger area than the logo of the company itself, and contain flashing, eye-catching and distracting images. In the image of the home page reproduced above, for example, the ‘For Home Bodies’ is a banner ad whose text changes every few seconds. This was not thought to be conducive to the desired impression of an authoritative and serious information provider. Instead, it gives the impression—as seven (35%) users noted—that the key function of the site is for profit rather than to promote healthy living or provide health information services; ‘health shouldn’t be like that—an excuse to make money’. The extent of advertising led some to wonder how the organization was funded, and what the links were with the Department of Health or National Health Service. One respondent excused the advertisements on the grounds that the company had to obtain a revenue ‘otherwise, how are they going to maintain the site? It’s better to have the information—even if surrounded by adverts—than to not have it at all’. Others said, noticing that content was provided by health related bodies, that the company should be subsidized by these, to lower the extent of advertising.


Innovations online Table 1 Retrieval strategies—task one.

Action

No.

%

Followed links to: diseases in depth Followed links to: accident and emergencies Followed links to: map of Britain (bottom, home page) Look for links/menu items first Search: ‘red cross’ Search: ‘red’ ‘cross’ Search: red and cross Search: red cross Search: Red Cross Tries to access site map (but no link) Used search facility at some point Used search facility first Completion of task

4 5 1 8 9 1 6 7 4 3 15 12 19

20 25 5 40 45 5 30 35 20 15 75 60 95

Part II: set tasks Four retrieval tasks were set. Task one was a simple fact finder—to find the telephone number of the British Red Cross. The second was similar, but required users to input data and retrieve information relevant to their own circumstances (names and addresses of the GP surgeries in their own area). The third task was somewhat broader, where the answer could be found from different pages on the site. It was to find the causes and treatments for shock. The final task was much more investigative. It was to answer the following question: To what extent does diet play a part in the prevention of cancer? What types of cancer are most affected by diet, and what types have little or no relationship to it? Task 1: What is the telephone number of the British Red Cross? This task was not as straightforward as envisaged. A large majority of users (75%) took the search engine route at some point in the conduct of the task, 12 (60%) of them doing so as a first step in the exercise. Problems arose when the term ‘Red Cross’ without speech marks, lower or title case, did not yield any page containing the required information. Using the correct procedure, however (i.e. with phrases in parentheses), generated a hit list that did. Other users accessed the required page by browsing techniques—such as following links—and eventually only one subject failed to

complete the task. Table 1 summarizes the actions followed to complete task 1. Task 2: Find the names and addresses of all the GP surgeries in your own area This proved to be a surprisingly simple task. Eleven (55%) users accessed the information easily, using the map of Britain at the bottom of the home page. This leads to an interactive page where users can input the name of their town, obtaining from this a list of surgeries and other facilities. Of note is that one person who accessed the correct page almost failed to spot the data input box, as it appears below screen level, and another user failed to see the link on the home page, also because it was below screen level. Only those users who used the search function for all tasks did so for this one, and no one actually retrieved the necessary page by this method. The successful completion of this task was attributable to two ‘signposts’ on the home page. These were the map of Britain and a menu link to ‘your home area’, both of which obviated the need either for browsing menus and links, or for undertaking a direct search. Table 2 indicates the actions taken to complete task 2. Task 3: What are the causes of and treatments for shock? It was hoped that this question would yield a variety of answers and information retrieval


221

222

Innovations online Table 2 Retrieval strategies—task two.

Table 3 Retrieval strategies—task three.

Action

No.

%

Action

No.

%

Clicked on map at bottom of centre frame Followed links to: Health in your area (NHS) Search: ‘Surgery Croydon’ Search: ‘Surgery list’ Search: NHS Search: NHS and GP Used search facility at some point Used search facility first Completion of task

11 9 1 1 1 1 4 3 18

55 45 5 5 5 5 20 15 90

Clicks on first aid menu item Used search facility at some point Looks in symptoms index menu item Used search facility first Tries to access site map (but no link) Search: shock Search: ‘shock treatment’ Search: ‘shock’ Search: shock and medical Completion of task

6 4 3 3 2 2 2 1 1 20

30 20 15 15 10 10 10 5 5 100

methods. There were, indeed, several search avenues explored in carrying out this task, each one leading to the same page and the same information. The most common tactic (undertaken by 30% of users) was to look at the First Aid menu first. Unlike the first task, more people adopted this browsing behaviour than used the search facility during the search, because there was a main menu item of clear relevance available as a starting point. The symptoms index menu option was also used by four (20%) users. The first aid option led straight to a page on shock, and the symptoms index also led to that page. In fact, the search engine also retrieved the relevant page, but again, there were doubts about its functionality. One user enclosed the single term ‘shock’ in parentheses, whilst another used two terms with a Boolean operator. All subjects were able, by one means or another, to find the information required. Users took an average of 2.4 min to do so, with one subject managing to find the information in under 30 s. Table 3 summarizes the retrieval strategies used to complete task 3. Task 4: To what extent does diet play a part in the prevention of cancer? What types of cancer are most affected by diet, and what types have little or no relationship to it? This was more of an investigative question, for which more time was required, and one to which perhaps the site is not best suited. Many participants spent up to the allotted 20 min looking into this issue. No one found a satisfactory page, although the required information was to

be found within the pages devoted to individual types of cancer. Interestingly, although the search facility was used by all but one of the students, not one person included the word ‘prevention’ as a search term. As can be seen from the table below, several combinations of terms were used, including a one word term in parentheses. One problem searchers had with this task was that, as they accessed several pages in order to retrieve the information, they entered sections of the site where the menu bars changed. Five users (25%), for example, searched on the simple term ‘Diet’. This gave a link to a breast cancer page (amongst others), accessed by four people. The left hand frame changed for this page, which was slightly confusing. Even on returning to the home page, it did not change back to the ‘default’ menu list. Another searcher also accessed the breast cancer page, and went on to look for ‘A Plan for Healthy Lives’ on this alternative menu, but this did not provide the required information. By the end of the session, all users had managed to find pages on various different types of cancer, and, given more time, could have researched the topic from these individual pages. No one, however, was able to find a page presenting information about the disease in general terms and including a section on diet. This prompted two users to say that, as cancer was such an important topic, about which a great many Surgerydoor users probably seek information, it would be a good idea to include the topic as a main menu item in its own right. Table 4 presents the retrieval strategies adopted to complete task 4.


Innovations online Table 4 Retrieval strategies—task four. Action

No.

%

Used search facility at some point Used search facility first Search: Cancer Search ‘Cancer’ Search: Cancer and diet Search: ‘Cancer and diet’ Search: Diet Search: disease cancer Search: ‘disease cancer’ Search; ‘Diet’ Followed links to: Diseases in Depth Followed links to: Medical conditions Followed links to: A to Z of the NHS Followed links to: Healthy Eating Completion of task

19 17 12 7 15 13 5 2 3 2 4 8 4 3 20

95 85 60 35 75 65 25 10 15 10 20 40 20 15 100

Discussion Implications for information professionals A number of issues of relevance to library and information professionals emerged from this study. It was noted, for example, that only one participant enquired about the legitimacy of the company. Considering the concern over the quality of health and medical websites9,10 and how users should evaluate them,11 the lack of attention paid to this aspect, particularly by postgraduate students, is interesting. It is possible that any health site is considered, by definition, as authoritative. If so, there are clear implications for library and information professionals and their role in assisting others to use websites as an information resource. Indeed, the potential use of medical sites by individual users may be helped by a greater consideration of the skills required to evaluate information. The need for such skills is now well recognized, and a large body of literature is building up on the subject of information literacy12 and, more recently, digital literacy.13 Teaching of information seeking and knowledge management strategies throughout the educational system, and pro-active involvement by librarians, are to be recommended. This is supported by Sieving,14 who cites a paper by Geffert and Christensen15 showing that students who used

more sources in their papers, and believed material on the Internet should be viewed with scepticism, had higher grade point averages when compared to their peers. Whilst DISCERN16 would not have applied to all the tasks in this project (as it is primarily of relevance to treatment information), awareness of evaluation tools such as that would constitute useful training, in addition to knowledge of the existence of reputable subject gateways to quality checked sites. Commercial aspects Users were very sensitive to the commercial aspects of the site, and an important message emerged here specifically for health site developers, in the form of the distrust the commercial part of the site generated. This may be due to the fact that in Britain there is a widespread perception that health care (and, by implication, health information) should be free at the point of delivery. It may be, however, that other factors related to the nature of the online environment come into play here. The issue of branding, advertising and brand recognition has taken on a new dimension on the Web. Simon17 writes that the surfeit of choice online produces ‘a concomitant change in consumer attitudes’ moving them from what he describes as ‘receptive space’ to ‘sceptical space’. It may be that with such a glut of information— including that concerned with health—users feel they do not have to tolerate the interference of advertising. They can move effortlessly from one brand to another. Drawing from a number of usability studies undertaken by Forrester Research, Travis18 suggests that fewer than 20% of website visitors look for a favourite brand—which is in keeping with the finding here that the attribution of information and therefore authority was not important for the users studied. One option for users is to limit the sites they use to those that are funded by government or notfor-profit organizations, which do not depend on revenue earned by the site (e.g. sites such as NHS Direct Online19 or Contact a Family,20). Web users expect to have a wide choice of locations to meet their information needs, and even though a restriction such as this would reduce the choice of sites substantially, there would still be a range of options.


223

224

Innovations online

The confusion about how Surgerydoor was funded showed a lack of easily available information about this on the website, and also the confusion caused by having links to external websites within the Surgerydoor frame, including pages produced by the NHS. Issues of the perception of health information and of whether advertising and sponsorship lead to actual or perceived bias of content are clearly important. The NHS is keen to explore this issue as a current digital TV initiative21 includes health information appearing under the auspices of commercial service providers. Site development and navigation Many implications regarding site creation and development, construction and layout could be drawn from the study. For example, a larger and better-placed logo and other measures are needed to provide a clear identity for a site. A number of salutary points emerged regarding site navigation. As the present writers found with a study of newspaper sites,8 contents lists and indexes liberally scattered over various pages—in an attempt to provide a sense of organization—do not always aid the navigation process. Nielsen,22 too, has found from his usability studies that users ignore navigational aids where these are perceived as being excessive. In the study reported here, users simply became confused as to which list to consult. Some even spent time trying to work out the criteria by which some entries appear on each particular list. Equally problematic was the way the left menu bar changed appearance and content when certain sections of the site were accessed. Users feel comfortable with consistency, and a constant point of reference would have been a positive feature. Retrieving information by direct searching rather than site navigation proved popular. As outlined above, many users either did not use the required parentheses or did so inappropriately (i.e. by enclosing individual words of a search term rather than the phrase itself), and so information that was on the site was not retrieved. As described, operators were used, but generally after no relevant articles had been retrieved from an initial search.

Here a clear message emerged that users were by now so familiar with inputting terms without operators, parentheses or any other additional element, that to do otherwise was confusing. It may be that there was an assumption that the site search engine worked in the same way as Yahoo! or AltaVista—by the simple input of a number of words. The concern about retrieving items that contained the search terms but were only marginally relevant implies that if a site is to contain a search facility, a best match term weighting system would be the most appropriate. What the students did not comment on was also revealing. The lack of attention to issues of authority, attribution and currency has already been discussed. The lack of interest in interactivity is also noteworthy. This aspect of the Web is often considered one of its main attractions, although previous work by the present writers23 and others has also shown a surprising lack of take up and interest in this facility. Chapman,24 for example, notes in his account of use of a regional newspaper online, that take up of the interactive element ‘has been noticeably (and interestingly) lacking’. Our participants may be different from some other users, however, in that they were not looking for emotional support or the reassurance of shared experience, which might be relevant to people concerned about their health, or that of a close relative.

Conclusions Health information on the Internet is expanding enormously. However, this small-scale exercise has shown that this information is not necessarily well presented or easily retrievable. A number of issues are highlighted about website construction and organization, and about how people use search facilities. Regarding the latter, the main finding was that even apparently adept users have problems where there are particular aspects of search engine functionality, with many subjects not inputting phrases in the correct manner. The main messages regarding the website design and content are that pages should not be too crowded with text, commercial features are not well received (at least not in the field of health information), and a large number of contents lists,


Innovations online

site maps and indexes do not always aid navigation or information retrieval. It is worth noting that these were almost exactly the points that emerged from the writers’ evaluation of The Times newspaper site.8 Regarding the Surgerydoor site specifically, users found the information itself well written and interesting, although there could have been more links to external sites and more diagrams. Of course, there were a number of differences in opinion, and each individual had their own idea of what made an attractive, well-constructed site. There was, nevertheless, remarkable conformity regarding these main points, which website creators and information providers would do well to consider when producing further webdisseminated information. Participants’ lack of comment about content quality shows a lack of awareness about this crucial aspect of health websites as an information resource. Information professionals have an important role in making their clients aware of the variability of content quality, and of how this can be assessed. Systematic methods for end-user assessment, such as the DISCERN tool, could be introduced, in addition to ensuring clients know about the subject gateways to resources already checked for quality, such as OMNI.25

References 1 Digital Health. The Web, the kiosk, digital TV and the changing face of consumer health information provision: a national impact study. April 2000–December 2002. Available online: http://www.digitalhealth.soi.city.ac.uk/ isrg /doh.htm 2 Nicholas, D., Huntington, P. & Williams, P. Establishing metrics for the evaluation of touch screen kiosks. Journal of Information Science, 2001, 27, 61– 72. 3 Nicholas, D., Huntington, P. & Williams, P. Health kiosk use: a national comparative study. Aslib Proceedings, 2001, 53, 130–40. 4 M2 Presswire UK-specific health website aims to capture a fifth of online health market. M2 Presswire news release (Lexix-Nexis Universe UK News database) 2000. Available online, password access only: https://web.lexis-nexis.com/ universe 5 Nicholas, D., Huntington. P. & Williams, P. Digital consumer information. Health Information on the Internet, 2000, 17, 9 –11.

6 Nielsen, J. & Mack, R. L. Usability Inspection Methods. New York: John Wiley, 1994. 7 Nielsen, J. Report from a 1994 Web Usability Study 1994. Available online: http://www.useit.com /papers / 1994_web_usability_report.html (accessed 23 April 1994). 8 Williams, P. & Nicholas, D. Navigating the news net: a study of how news consumers read the electronic version of a daily newspaper. Libri, 2001, 51(1), 8 –16. 9 Impicciatore, P., Pandolfini, C., Casella, N. & Bonati, M. Reliability of health information for the public on the world wide web: systematic survey of advice on managing fever in children at home. British Medical Journa., 1997, 314, 1875 –81. 10 Culver, J. D., Gerr, F. & Frumkin, H. Medical information on the Internet: a study of an electronic bulletin board. Journal of General Internal Medicine, 1997, 12, 466 –70. 11 HONF (Health On The Net Foundation). HON Code of conduct for medical and health web sites. Health on the Net Foundation web site 1997. Available online: www. hon.ch / HONcode/Conduct.html (accessed 2 August 2001). 12 Doyle, C. S. Information Literacy in an Information Society: a Concept for the Information Age Syracuse, NY: ERIC Clearinghouse, 1994. 13 Gilster, P. Digital Literacy. New York, NY: Wiley, 1997. 14 Sieving, P. C. Factors Driving the Increase in Medical Information on the Web—One American Perspective 1999. Available online: http://www.symposion.com /jmir/1999/1/ e3/index.htm. 15 Geffert, B. & Christensen, B. Things they carry: attitudes toward, opinions about, and knowledge of libraries and research among incoming college students. Reference and User Services Quarterly, 1998, 37, 279– 85. 16 DISCERN. Available online: http://www.discern.org.uk /. 17 Simon, P. The strange online death and possible rebirth of brand theory and practice. Aslib Proceedings, 2001, 53, 245– 9. 18 Travis, D. What Drives Repeat Visitors to Your Website? Available online: http://www.system-concepts.com / articles /forrester.html (accessed 27 January 2002). 19 NHS Direct Online. Available online: http:// www.nhsdirect.nhs.uk/. 20 Contact a Family. Available online: http:// www.cafamily.org.uk /. 21 Gunter, B., Nicholas, D., Williams, P. & Huntington, P. Is TV good for you? Library Association Record, 2001, 103, 558– 9. 22 Nielsen, J. Is navigation useful? Jakob Nielsen’s Alertbox. 9 January 2000. Available online: http://www.useit.com / alertbox /20000109.html. 23 Williams, P. & Nicholas, D. The Internet, a regional newspaper and an attempt to provide ‘value added’ information. Aslib Proceedings 1998, 50, 255–63. 24 Chapman, P. Developing ‘This is the north east’. Aslib Proceedings, 1998, 50, 255– 63. 25 OMNI. Available online: http://omni.ac.uk/.


225