CHI’95 MOSAIC OF CREATIVITY ■ May 7-11 1995 ~ Usability Inspection Methods Jakob Nielsen Stmsoft 2550 Garcia Avenue Mountain View, CA 94043-1100 Email: jakob.nielsen@sun.com (primary), nielsen.chi@xerox.com (backup) ABSTRACT Usability inspection is the generic name for a set of cost- effective ways of evaluating user interfaces to find usability problems. They are fairly informal methods and easy to use. Keywords: Usability engineering, heuristic evaluation, cognitive walkthroughs, pluralistic walkthroughs, feature inspection, consistency inspection, standards inspection. INTRODUCTION Software inspection [1] [7] has long been used as a method for debugging and improving code. Similarly, usability inspection [25] has seen increasing use since about 1990 as a way to evaluate user interfaces. The four basic ways of evaluating user interfaces are automatically (usability mea- sures computed by running a user interface specification through some program), empirically (usability assessedby testing the interface with real users), formally (using exact models and formulas to calculate usability measures), and informally (based on rules of thumb and the general skill and experience of the evaluators). Under the current state of the art, automatic methods do not work and formal methods are very difficult to apply and do not scale up well to handle larger user interfaces. Empirical methods are the main way of evaluating user interfaces, with user testing probably being the most com- monly used method. Often, real users can be difficult or expensive to recruit in sufficient numbers to test all aspects of all the versions of an evolving design, leading to the use of inspection as a way to “save users .“ Furthermore, project schedules or budgets sometimes impose restrictions that make informal methods like inspection desirable as a “dis- count usability engineering” solution [16] [21] since they are highly cost-effective [8]. Several studies have shown that usability inspection methods are able to find many usability problems that are overlooked by user testing but that user testing also finds some problems that are overlooked by inspection, meaning that the best results can often be achieved by combining several methods [5] [6] [1 1]. Permission to copy without fee all or part of this material is granted provided that the copies are not made or !istr’ibuted for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of ACM. To copy otherwise, or to republish, requires a fee and/or specific permission. CHI Companion 95, Denver, Colorado, USA @ 1995 ACM 0-89791 -755 -3/95/0005 ...$3.50 INSPECTION METHODS Usability inspection is the generic name for a set of methods that are all based on having evaluators inspect the interface. Typically, usability inspection is aimed at finding usability problems in a design [13], though some methods also address issues like the severity of the usability problems and the overall usability of an entire design [28]. Many inspec- tion methods lend themselves to the inspection of user inter- face specifications [17] that have not necessarily been implemented yet, meaning that inspection can be performed early in the usability engineering lifecycle [19]. ● ● ● ● ● ● ● Heuristic evaluation [23][27] is the most informal method and involves having usability specialists judge whether each dialogue element follows established usability principles (the “heuristics” [15] [18] [21] [22]). Cognitive walkthroughs [12] [29][30] use a more explic- itly detailed procedure to simulate a user’s problem solv- ing process at each step through the dialogue, checking if the simulated user’s goals and memory content can be assumed to lead to the next correct action. Formal usability inspections [9] use a six-step procedure with strictly defined roles to combine heuristic evalua- tion and a simplified form of cognitive walkthroughs. Pluralistic walkthroughs [3] [4] are meetings where users, developers, and human factors people step through a scenario, discussing each dialogue element. Feature inspection [2] lists sequence of features used to accomplish typical tasks, checks for long sequences, cumbersome steps, steps that would not be natural for users to try, and steps that require extensive knowledge/ experience in order to assess a proposed feature set. Consistency inspection [31] has designers representing multiple projects inspect an interface to see whether it does things in the same way as their own designs. Standards inspection [31] has an expert on some inter- face standard inspect the interface for compliance. Heuristic evaluation, cognitive walkthroughs, feature inspection, and standards inspection normally have the interface inspected by a single evaluator at a time (though heuristic evaluation is based on combining inspection reports from a set of independent evaluators to form the list of usability problems). In contrast, pluralistic walkthroughs 377