User Research

There are a variety of approaches to usability evaluation that you may choose to take. The methodologies can be divided into two broad categories: those that gather data from actual users and those that can be applied without actual users present. Of course the approach you choose to take must factor in variables such as the cost of evaluation, the appropriateness of the technique to your project, time constraints of the project, and the cost associated with the implementation and training of new users.

Usability evaluations can be conducted at many stages during and after the design and development process. In choosing a usability evaluation method, cost is invariably a consideration. But it is important to keep in mind that the value of a usability evaluation needs to be calculated not only in time and materials involved, but also in terms of the impact on the end-users, especially considering the cost of losing return visitors to your website due to unusable design.

1. Cognitive Walkthrough is an approach to evaluating an interface based on breaking down and analyzing actions that a user must perform in order to use the system or perform a task.

2. Focus Groups gather groups of users to get their feedback, initial reactions to a design, and discuss their preferences. Focus groups can be useful for raising issues that may not come out during interviews.

3. GOMS is a family of techniques for modeling and describing human task performance. GOMS is an acronym that stands for Goals, Operators, Methods, and Selection Rules.

4. Prototyping involves developing representations of a system for testing purposes and can range from simple sketches to almost fully functional systems.

5. Task Analysis evaluates how the end-user actually uses software or websites. An analyst determines the user goals and tasks, then makes recommendations aimed at increasing efficiency and user-friendliness.

6. Usability Inspection reviews a system based on a set of usability guidelines. Experts familiar with issues of usability in design perform the usability inspection.

7. User Testing observes actual users interacting with software or websites. Users are asked to perform tasks while usability experts observe and take note of their actions.

8. Contextual Inquiry

·An interpretive field research technique used to gather and understand data on user needs.

·The data can be used in system requirements, functional specifications and evaluations.

·Data is gathered in conversations with users (current and potential) in the context of their work.

Cognitive Walkthrough

Cognitive walkthroughs are performed in the early stages of design using a prototype or a conceptual design document.

Based on a user’s goals, a group of evaluators steps through tasks, evaluating at each step how difficult it is for the user to identify and operate the interface element most relevant to their current subgoal and how clearly the system provides feedback to that action. Cognitive walkthroughs take into consideration the user’s thought processes that contribute to decision making.

For example, finding the Usability First website can be broken down to several levels of tasks. At a general level, it requires opening up a browser, remembering the URL and typing it in the text box at the top of your browser. Or, if you do not remember the URL, you must choose a search engine, think of a search term, view the results, scroll through the results, and then click on the link. Each of these actions can be further decomposed.

This approach is intended especially to help understand the usability of a system for first-time or infrequent users, that is, for users in an exploratory learning mode.

Focus Group

Using focus groups to evaluate a system is a very efficient way to get user feedback and gauge initial reactions to a design. Focus groups are also good at discovering how the system being tested differs from the user’s current expectations. As we see it, focus groups provide two major benefits. First, they are less expensive than conducting interviews with the same number of people. Second, they rely on group interaction to trigger memories that may not come up during interviews.

Where task analysis often discovers the standard way people interact with information systems, focus groups can bring out exceptions to the rules. These exceptions are often very important interactions that users simply do not think of in one-on-one sessions.

Conducting only a single focus group can be misleading, however, as some groups are affected by “group-think” or may simply have irregular views. For this reason, at least two groups should be evaluated for any one project.

The focus group leader writes up the impressions and comments of the groups and recommends areas for improvement.


GOMS is a family of techniques for modeling and representing the knowledge necessary for a person to perform a task.

The purpose of this site is to provide information for students, researchers, and practitioners interested in GOMS.

GOMS is a family of techniques proposed by Card, Moran, and Newell (1983), for modeling and describing human task performance. GOMS is an acronym that stands for Goals, Operators, Methods, and Selection Rules, the components of which are used as the building blocks for a GOMS model. Goals represent the goals that a user is trying to accomplish, usually specified in a hierarchical manner. Operators are the set of atomic-level operations with which a user composes a solution to a goal. Methods represent sequences of operators, grouped together to accomplish a single goal. Selection Rules are used to decide which method to use for solving a goal when several are applicable.

Psychological Basis

Most GOMS techniques are, at least partially, based on a simple cognitive architecture known as the Model Human Processor (MHP). This representation of human cognition consists of separate components for cognitive, motor, and perceptual processors (and associated buffers), as well as for long and short-term memory. The components of GOMS map onto this model in one form or another. For instance, control in the MHP is central to the cognitive processor, where execution of methods and selection rules is assumed to take place. Likewise, the execution of operators can be seen as the issuance of commands by the cognitive processor to the other components.

Uses of GOMS

From a research standpoint, GOMS provides a framework for modeling aspects of human performance and cognition. From an applied perspective, GOMS provides a rich set of techniques for evaluating human performance on any system where people interact with machines. GOMS analysis can provide much insight into an system’s usability, such as, task execution time, task learning time, operator sequencing, functional coverage, functional consistency, and aspects of error tolerance. Some type of GOMS analysis can be conducted at almost any stage of system development, from design and allocation of function to prototype design, detailed design, and training and documentation for operation and maintenance. Such analysis is possible for both new designs and redesign of existing systems.


Prototyping techniques involve developing representations of a target system for evaluation and testing purposes.

Prototyping is an essential element of an iterative design approach, where designs are created, evaluated, and refined with the results of testing at each cycle feeding into the design focus of the next cycle.

Prototypes can range from extremely simple sketches (low-fidelity prototypes) to full systems that contain nearly all the functionality of the final system (high-fidelity prototypes).

Here is a list of some prototyping terms and techniques:

o thumbnail sketch

o rough

o comp / mockup

o paper prototype

o video prototype

o wizard of oz prototype

o functional prototype

o rapid-prototyping

o RAD tool (rapid application development)

Rapid Prototyping

Quickly generating mockups of what a system will look like. Rapid prototyping may be done with paper prototyping methods, such as with quick sketches, or with techniques like video prototyping or wizard of oz prototyping. Rapid prototyping systems are software systems, such as HyperCard, Director, or RAD tools, that let you quickly display interface elements and define some behavior for them to obtain a sense of the interaction and the capabilities of the fully-implemented system. These prototypes are useful for determining the target of the development project, doing many types of user testing, and spotting unanticipated complexities in the design

Task Analysis

Task analysis is a method that evaluates how people actually accomplish things with software. Through observation and interviews with users, an analyst determines a set of goals belonging to the target user. Then, a set of tasks that support these goals is determined. These are prioritized based on criteria such as the importance of the goal to the organization and the frequency of task performance.

The highest priority tasks are decomposed into their individual steps. The level of decomposition varies with the budget and type of system evaluated. The analyst then suggests ways to make the task more efficient or suggests new tasks which more effectively support the goals. It is important to recognize that the analysis is done from the perspective of the end-user-not from the point of view of managers or executives who do not necessrily use the system.

Usability Inspection

A usability inspection is a review of a system based on a set of guidelines. The review is conducted by a group of experts who are deeply familiar with the concepts of usability in design. The experts focus on a list of areas in design that have been shown to be troublesome for users.

Usability guidelines are usually derived from studies in human-computer interaction, ergonomics, graphic design, information design, and cognitive psychology. Some areas that get evaluated are the language used in the system, the amount of recall required of the user at each step in a process, and how the system provides feedback to the user. In particular, issues such as clarity, consistency, navigation, and error minimization are analyzed. Once the problems are discovered, the experts make recommendations for resolving these issues.

User Testing

User testing is the mainstay method when it comes to finding usability problems. Nothing is more convincing than watching person after person encounter difficulties with the same part of a software or information system. The difficult areas that repeat themselves between multiple test participants reveal areas that should be studied and changed by the developers. User testing can often uncover very specific areas needing improvement, where focus groups and task analysis often find more general areas needing improvement.

A trained observer conducts user testing often with the assistance of software developers. People who are representative of the target audience are asked to perform representative tasks with the software. The observer writes a user testing report listing the problems and offering recommendations based on their findings.

Key Concepts in Contextual Inquiry

· Context: understand user needs as discovered in the course of their, as they work.

· Partnership: work with users as partners in inquiry, a co-discovery process.

· Focus: listen and explore from a clear perspective (while staying open to discovery).

What is Context?

· Work process is influenced by the environment: arrange to interview where and while users are working.

· Observe and capture on-going experience while it is happening: as opposed to “talking about it after the fact.”

· Concrete data grounded in the work: as distinct from abstract and generalized “task intentions.”

Examples of Data Gathering in Context

At stages throughout the design cycle:

· Work walkthrough to get an overview

· “What if” futures scenario

· Paper prototyping

· Test drive

What is Partnership

· Work and usability are accessed through dialogue

o Use open-ended questions

o Listen

o Be aware of “self-talk”

· The user is the expert

o Let the user lead the conversation Move beyond “roles” to get valid better data for design

· User knowledge of work

· integrated with

· Designer knowledge of technology

What is Focus?

· Focus is a perspective: a way to reveal (and conceal)

· Design of overall system

o Work observed

o User’s model

o Overall workflow

· Design of HCI

o The part of the work addressed

· Design of user-system workflow