4.4 Chapter Summary and Discussion

We've looked at three methods for evaluating an interface without users. If you scan through the "Summary" sections at the ends of the examples, you'll see that each method uncovered different problems. The cognitive walkthrough identified problems with finding the controls and suggested that they be moved to the Print dialog box. A one-person heuristic analysis noted the same problem and some others, and suggested that the controls should be relabeled. The formal action analysis suggested a vague question about the printer-type selection and made it clear that there were a lot of actions needed to do this simple task. A back-of-the- envelope action analysis might have given similar results, without details of how long the task took.

Even the combined result of the three techniques didn't catch all the problems. That's partly because there are kinds of problems that none of these methods will catch, and partly because no individual analysis is perfect. One example of something none of the methods caught, although both the cognitive walkthrough and the heuristic analysis noted related problems, is that "Chooser" and "Control Panel" have similar functions (setting options about the system) and also similar names (starting with C, two o's in the middle). It's likely that users will sometimes slip and pick the wrong one even when they know which one they're after, especially since items are alphabetized in the Apple menu. This problem could surface in user testing, or at least in beta testing. Another problem, which another heuristic analyst might have caught, is that the dialog box doesn't let users know which printer type is selected when it first opens. A user who isn't sure what kind of printer is being used may change to the wrong type, after which printing may not work at all.

Here's a recommendation on how to use the various methods in the design process. In general, the cognitive walkthrough will give you the best understanding of problems it uncovers, and we recommend thinking through the interface in walkthrough terms as you develop it. When a substantial part of the interface is complete, heuristic analysis is a good way to catch additional problems -- but you need several evaluators to do it well. Back-of-the-envelope action analysis makes a good "sanity check" as the interface becomes more complex, and it's also useful early in system design to help decide whether a system's features will pay back the effort of learning and using them. Formal action analysis is probably only appropriate for very special cases, as we described in that section. All the methods need to be combined with user testing, which we discuss in the next chapter.