A Case Study of a Multimedia System Using an ...

1 downloads 0 Views 549KB Size Report
900 Dandenong Road, Caulfield East, Victoria 3145, Australia. {natalie.pang, don.schauder}@infotech.monash.edu.au. Abstract. Many usability studies have ...
A Case Study of a Multimedia System Using an Integrated Approach of Usability Evaluation Sanxing Cao1, Natalie L.S Pang2, Dan Li 1, Don Schauder2 1

School of Computer Science and Software, Communication University of China 57#, No.1 East Street Dingfuzhuang, Beijing, China {c3x, dennis_myli}@cuc.edu.cn 2 Faculty of Information Technology, School of Information Systems and Management, Monash University, 900 Dandenong Road, Caulfield East, Victoria 3145, Australia {natalie.pang, don.schauder}@infotech.monash.edu.au Abstract. Many usability studies have evolved around the use of scenarios and heuristics. Here we aim to conduct usability evaluation using scenario-based design, combined with extended heuristic evaluation for hypermedia objects. This integrated approach is then applied on a digital portal developed for a video production project for Olympics 2008, evaluating it for its usability.

1

Modified Heuristics and Scenarios

Several schools of thought govern studies of usability evaluation, namely, the use of scenario, and general heuristic principles. We use these two schools of research to develop a specific process to evaluate an integrated platform of multimedia content management for an advertainment movie production project for Olympics 2008[1].

Fig. 1. Modified Usability Heuristics (1~14), and Characteristic elements of UI Scenarios

Heuristic evaluation [2] is widely employed in usability evaluation of UI design whereby experts trained in design heuristics review interfaces to determine their conformances with these guidelines. The platform studied is supported by a backend content management application to facilitate the syndication of multimedia objects. We are also faced with additional usability criteria caused by users from diversified cultures, together with unique characteristics of multimedia objects. Based on these and Nielsen’s usability heuristics [2], we developed a new set of heuristics evaluation guidelines, as shown in the left part of figure 1, for usability evaluation.

Design scenarios are vital in describing how and why a user interacts with a system [3]. In addition, using heuristics guidelines is not enough, as we would like to take into context the scenarios of use of the system as well, to identify additional requirements for the system if necessary. In this early stage of portal development, it is also necessary that developers and designers gain useful information to help them enhance the designs and usability of the interfaces. See figure 1 for scenario elements.

2

Experiment and Results

6 expert reviewers, trained in both heuristic evaluation and scenario based analysis, were given the same tasks to evaluate the web-based UIs of the system. 2 main tasks were defined, one from each potential group of users: the content producers, and end users. (T1: upload a video object using the file upload module. T2: viewing the object using the media browser.) Reviewers were divided into 2 groups of 3. Group 1 focused on the use of heuristic evaluation while Group 2 used scenario based analysis. All subjects finished the evaluations in 1~3 hours. Results are indicated in Figure 2.

Fig. 2. Evaluation Results

3

Conclusion and Future Work

We employed the help of expert reviewers in this pilot study using 2 usability evaluation methods: heuristics and scenario based design with claims analysis. The findings would be used to refine system prototypes. In the near future we would further evaluate the usability of newer versions of the portal, through participatory design by other stakeholders such as designers, users, and developers. This would be tightly linked with the optimization of the software process for multimedia systems. This work is supported by the CUC Research Fund (YNG0414).

References 1. Cao, S., Klein, R.: Managing Uncertainty in Media Content Platforms, In Xu, Q.R. et al (eds): Proc. ISMOT04. Zhejiang University Press, Hangzhou, China (2004) 171-175. 2. Carroll, J.M.: Making use: Scenario based design of HCI, MIT Press, USA (2000) 3. Nielsen, J. et al. Usability Inspection Methods, John Wiley & Sons, Inc, USA (1994)