An Example of Formal Usability Inspections in Practice at Hewlett-Packard Company

Cathy Gunn

Usability Engineer
Hewlett-Packard Company
Information Networks Division
19420 Homestead Road, MS 43LE
Cupertino, 95014
cgunn@cup.hp.com
408-447-2949

Abstract

Can usability engineers, not formally educated as human factors engineers, help facilitate improving the ease of use of software products? Can design engineers learn to detect usability defects? The answer to both questions is yes. This is a success story of a partnership between human factors engineers and usability engineers in providing a Formal Usability Methodology that has been accepted and is -continuing to be used by product developers since it was introduced two years ago. The usability engineers have added usability practices and facilitation skills to their -traditional roles as technical writers and support engineers. Design engineers enjoy using the methodology, and learn how to evaluate their products from the user's viewpoint.

Keywords

usability inspections, human computer interaction, user- centered design, process redesign

Introduction

Thirty usability engineers at Hewlett-Packard's Information Networks Division (IND) were trained in a Formal -Usability Inspection Methodology[1] developed by human factors engineers (HFEs) and corporate trainers. Six -usability engineers received additional training as -moderators. The moderators manage the inspection process and have become a resource pool available to product teams. This paper describes the following:

Overview of the inspection process.
Value of usability inspections.
How do we keep the process going?
What we've accomplished.
What we learned.
Usability inspections outside IND.

WHY USABILITY INSPECTIONS?

With a small number of human factors engineers available to IND, the organization wanted to find a way of improving the ease of use of a larger number of products. Roxanne Bradley, the IND HFE, wanted to use a design review methodology that non-HFEs could implement early in the product life cycle. She recommended usability inspections as the appropriate methodology.

HOW: THE TRAINING

Based in part on formal code inspections, Rose -Marchetti, a corporate trainer, and HFEs Michael Kahn and Amanda Prail developed a Formal Usability Inspection -Methodology class. All usability engineers in IND were trained in how to participate in a usability inspection. In addition, six engineers were also trained to moderate inspections.

FORMAL USABILITY INSPECTIONS

In brief, a usability inspection is a formal review of tasks a user completes while using the product. Typically, the IND inspection team includes design engineers, -usability -engineers (technical writers/support engineers), and -customer support engineers, but customers could also -participate. The process is facilitated by one of the trained moderators.

The purpose of the inspection is to evaluate a product from the user's perspective, and find and fix usability concerns, and in so doing, improve the ease of use of the product.

The process consists of six phases[1]:

1. Planning. The moderator helps the product team choose an inspection team, prepare the inspection packet and schedule the meetings. The inspection packet contains user profiles, a set of task scenarios, and information on the product design or a prototype description.
2. Kickoff Meeting. The moderator distributes the packets and reviews what is expected of the inspectors. The moderator is also available if there are any questions about the process.
3. Review. Each of the inspectors reviews the inspection packet materials. On their own, inspectors take the role of users as described in the user profile(s) and try to -perform the task steps in the task scenarios (a walk through). During the review, inspectors note any -usability concerns found as they try to complete the tasks.
4. Logging Meeting. As a team, the inspectors walk through the task scenarios and log usability concerns they found, as well as any new ones discovered during this meeting. The moderator manages the meeting. A scribe logs the concerns. Overhead slides are used to show the concerns to the entire team as they are logged.
5. Rework. Solutions meetings are scheduled to -brainstorm and propose solutions to the concerns found. The product owner (or owners) reviews and fixes the -usability concerns.
6. Follow-up. The moderator requests feedback from the inspection team on the process. The moderator writes and distributes the inspection report.

To help facilitate acceptance of the process, the term -concern rather than defect is used. Inspectors are instructed to express usability concerns in a constructive manner. For example:

CONSTRUCTIVE: "As an experienced user, I know I have to configure the node type, but I can't see how to do it on this screen." NOT CONSTRUCTIVE: "Why didn't you put every- thing on the same screen?"

VALUE OF FORMAL USABILITY INSPECTIONS

Our organization has validated the value of usability inspections as compared to traditional user tests:

Design engineers learn how to evaluate their designs from the perspective of the user.
Inspections can be held very early in the life cycle. More concerns can be fixed, since the cost to fix is less. The code does not have to be complete. Paper prototypes and design documents can be inspected.
The method itself is economical. It requires less plan- ning and set up time than user tests. It is easier to schedule into the product life cycle.
The number of usability concerns found is high.

KEEPING THE PROCESS GOING

A challenge in organizations is how to get teams to -implement new processes and to continue using them. Management support is a key factor required for success. In addition, someone needs to champion the process. The trained moderators have done this in part by forming a group chartered to:

Share best practices.
Manage moderator availability to project teams.
Provide feedback on the process and training to method developers.
Designate a chief moderator to help facilitate group activities and work with human factors -engineers.

WHAT THE MODERATORS HAVE ACCOMPLISHED

As of this writing (December 94), the moderators have:

Inspected 14 products (about 9 per year).
Collected on average 76 usability concerns per p-roduct (range 25-149). Fix rate: 74% (range: 50-95%).
Been recognized for expertise by managers and engineers.
Tailored the inspection method for use at IND.

As the moderators used the process at IND, we:

Revised the inspection data collection forms (using the usability inspection method.)
Provided feedback to corporate trainers.
Refined the role of scribe. We have found that using one of the trained moderators as a scribe is effective in having a successful inspection. The scribe can help assure that concerns are logged accurately and in a user-centered way. It helps to have moderator expertise in the scribe role, especially if teams have inspectors unfamiliar with the process.

WHAT WE HAVE LEARNED

We have learned the following:

Engineers can detect usability concerns.
Engineers enjoy using the method and are eager to participate in additional inspections.
Engineers work very hard to provide constructive feedback during the inspection, which improves the process.
The inspection process increases awareness of user needs.
The process is more successful if most of the team has attended inspector training or has participated in an inspection. Using a scribe who is a moderator is also a key to success at IND.
For continued success, engineers' managers must be supportive of the method.

USABILITY INSPECTIONS OUTSIDE IND

The Formal Usability -Inspection Methodology participant and moderator training is -available to -divisions and to product teams. The method is becoming more prevalent across Hewlett-Packard -divisions. IND's success and implementation model is used as an example for other divisions -considering the methodology.

References

1. Kahn, M.K., and Prail, A., (1994). Formal Usability Inspections in J.Nielsen and R. L. Mack, (Eds.). Us- ability Inspection Methods. John Wiley & Sons, Inc. New York. 141-171.