Victorian
Government
Contact Centre
1300 366 356

Main Menu

Categories, RSS and More

Please enter the URL that you'd like to shorten or click here to shorten current:

Support Resources: Usability - Testing the Information Architecture

Replaced by Developing and managing an information architecture - Whole of Victorian Government ICT Guideline issued 29 November 2010.

Supporting: Information Architecture/Classification standard (WEB/STD/08)

Version: 2.0

Release Date: 30 November 2007

Overview

This resource supports the Information Architecture /Classification Standard WEB/STD/08. A significant component of effective information architecture design is refinement and review of the information architecture using usability testing. This may be incorporated into a Usability Testing Plan.

This resource provides advice for developing a plan to test a website's information architecture, focusing specifically on supporting users to find information easily. Specific information architecture issues are addressed including:

Alignment of categories, titling and labelling with user language and need
User centred design implications for the information architecture
Methods for obtaining user input to the information architecture
Stakeholder acceptance of user input to the design.

What is a usability testing plan?

Usability testing is a technique for ensuring that the users of a system can carry out the intended tasks efficiently, effectively and satisfactorily. It is an iterative process that is repeated at several stages in a website development or re-development process.

There are six key components in the development of a usability testing plan. They are:

Defining the goals and scope of usability testing
Designing the testing program
Obtaining user input to the information architecture
Conducting the testing program
Analysing test results and determining implications
Gaining stakeholder acceptance of user input to the design.

Some organisations have one formal usability testing plan and others have more frequent, often informal, testing plans. The approach adopted will depend on the number of people conducting the test, the time available, resources and costs.

Some usability experts argue that the development of a usability culture characterised by frequent, small, inexpensive in-house testing can deliver very effective results comparable to or exceeding more formal usability testing. In this culture-of-usability model, the tests are run on a regular, fixed schedule (once a month or more) with a small number of users. Any aspect of the website – e.g. design, perform, task completion, information architecture – can be tested. (1)

Why test the usability of a website's information architecture?

Ease of navigation and efficient discoverability are fundamental to providing a better web experience and service to citizens.

Usability testing of the information architecture reveals any flaws in the architecture, navigation or labelling of the site.

There is a very strong correlation between user satisfaction and loyalty. Fundamentally good usability increases usage and return visits. Usability testing the information architecture ensures users:

find their way around websites
understand the language and groupings used on the website and that they reflect the way users are likely to think about the information
locate the information and services they need
know where they are, where they have been and where they can go in a website
understand what the navigational elements do
recognise the elements placed on the page
are presented with clear, relevant and valuable content.

Testing the validity of the information architecture components of a website can and perhaps should be undertaken regularly throughout the development and subsequent operation of a website. The most successful organisations conduct usability tests frequently, results are quickly integrated into the website, and the total cost of the testing program is smaller. (2)

How to test the usability of an information architecture?

Usability testing the information architecture of a website examines how a user interacts with a webpage or website. It establishes whether there is a match between how the web manager and developer expected or anticipated a user to act with how a user actually acted.

The development of an IA Usability Testing Plan describes the approach that your organisation takes to testing the information architecture. In many instances this plan will be part of a broader approach to testing the usability of the whole website.

Defining the goals and scope of the testing

A usability testing goal is a measurable design objective for how usable the architecture needs to be. It should be aligned with specific user interactions. Usability goals underpin the overall design and interactions available on a website and should be established early in the design process.

Usability goals have three attributes:

Performance – What should the user be able to do?
Conditions – Under what conditions should the user be able to do it?
Criteria – How well must it be done?

IA usability testing references focuses on testing, improving and refining the information architecture. Specific information architecture features that require testing include:

Design – page design, readability, layout, graphics, scrolling
Finding information – navigation, category names, links
Understanding information – content quality and presentation
Search – quality of search results

Designing the testing program

Selecting appropriate user testing methods

There is a rich collection of user testing methods that can be used to test the usability of the information architecture.

1. Cognitive Walkthrough is an approach to evaluating an interface based on breaking down and analyzing actions that a user must perform in order to use the system or perform a task.

2. Focus Groups gather groups of users to get their feedback, initial reactions to a design, and discuss their preferences. Focus groups can be useful for raising issues that may not come out during interviews and are excellent when cost and time are issues.

3. GOMS is a family of techniques for modeling and describing human task performance. GOMS is an acronym that stands for Goals, Operators, Methods, and Selection Rules.

4. One-on-one interviews with users exploring specific aspects of the website presentation and functionality.

5. Prototyping involves developing representations of a system for testing purposes and can range from simple sketches to almost fully functional systems.

6. Scenario-based tasks where a range of scenarios are developed to test the information architecture. Scenario-based testing is appropriate for testing all aspects of the website information architecture.

7. Task Analysis evaluates how the end-user actually uses software or websites. An analyst determines the user goals and tasks, then makes recommendations aimed at increasing efficiency and user-friendliness.

8. Usability Inspection reviews a system based on a set of usability guidelines. Experts familiar with issues of usability in design perform the usability inspection.

9. User Testing observes actual users interacting with software or websites. Users are asked to perform tasks while usability experts observe and take note of their actions. (3)

Developing test methods

The preparation of one or more test methods for inclusion in a testing program is reliant on research and evidence related to the website audience/users and stakeholders. It is vital that the following information sources are used to develop the tests. These include:

Search terms
Site statistics – path analysis and usage patterns
Consumer/user research
Stakeholder and/or business owner research
Benchmark industry websites
Audience interaction models
Call centre queries
Shop-front queries

Determining what standards are acceptable and how results will be collected

Consider and document the measures of acceptable performance and how they will be recorded so that recording is accurate and objective.

Data can be collected through:

observation
measuring task completion
interview
discussion assessments
written assessments.

Information architecture usability testing tools include:

index cards
paper proto-types
wireframes
story-boards
high-fidelity prototypes/ mock-ups
computers.

Data collection tools include:

logging sheets
questionnaires
observation checklists
audio-tapes
tracking software
videotaping
observation rooms/laboratories.

For more detailed advice on testing methods and tools see:

Usability Testing Methods Toolbox available online

Obtaining user input to the information architecture

The selection and recruitment of users to be involved in usability testing is an important factor in achieving effective test results. Returning to the audience profiles documented in the Information Architecture Plan will help to identify the types of users that should be engaged to perform the user testing.

Types of users to test

Users should be selected carefully and should represent the intended audience groups. Informal testing can be less rigorous in terms of user selection however care must be taken to ensure impartiality and limited familiarity with the government or organisational perspective. Users who have no knowledge of the subject domain being tested are generally more likely to identify flaws in the information architecture than those who are familiar with the area.

Number of users to test

Usability testing can involve quite large groups of users particularly if there are several distinct audience/user groups. However leaders in the usability field such as Jakob Nielsen have set a benchmark of 5 carefully selected and representative users being tested over the course of 3 tests. It is argued that this will uncover the bulk of usability problems. The decision regarding the number of users and number of usability tests will be determined by the complexity of the information architecture, the range of user groups, and available resources, funding and business risk associated with the site.

Conducting the testing program

The two main aspects to consider when conducting the testing program are:

clarifying the roles of those involved in the usability testing program
the actual conduct of the testing program.

Identify roles and responsibilities

Typical roles are:

Facilitator – organises the testing program, conducts a dry-run, in some instances confirms the participants, location, equipment and the participant incentive, introduces and contextualises the test session, establishes the test parameters, introduces the observers( if relevant) and closes the session.
Participant – addresses the activities, questions, procedures in the test tool.
Observer/scribe – focuses on documenting the user experience noting responses and summarising user comments.
Observer – focuses on observing the activities of the participants during testing.

Conduct of the testing program

Consideration should be given to ensuring that:

the number of users and the resources requirements align with the appropriateness and scale of the testing program
the venue selected is ‘fit for the purpose”
the equipment e.g. computers, video and audio recorders, has been tested and is ready for use
the tests have been trialled in a ‘dry run' to deliver the most useful results from the testing activity
debrief opportunities are available for users and observers
privacy requirements are observed.

Analysing test results and determining implications

A report on the IA usability testing outcomes should be prepared and used to guide ongoing change and development.

Testing the information architecture will identify a list of issues, problems and areas of user frustration. These items should be prioritised according to a sliding scale reflecting the impact on the overall usability of the site and specifically on the way users find information on the website. Typically the scale would assess issues as:

High - Fundamental usability problem preventing the completion of tasks e.g discoverability of information. These issues or problems should be addressed immediately and may require a substantial review of the information architecture. The modified information architecture must be tested in a second testing program prior to ongoing development.
Medium - Usability problem creating frustration or confusion e.g. term confusion, terminal pathways. These issues should be addressed immediately and tested specifically in a second testing program.
Low - Low impact issue that should be addressed prior to launch of a new or re-developed site but they will not delay other development activities.

Stakeholder acceptance of user input to the design

An effective information architecture must reflect the way users think and wish to interact with the website. User information needs as well as business goals should be well understood and aligned in the information architecture design of the site. Therefore communicating these needs and goals to stakeholders is essential to ensuring that the information architecture is understood, endorsed and supported by stakeholders.

Consideration needs to be given to the strategy for communicating the results of usability testing to stakeholders. These could include:

Making stakeholders aware of the goals of usability testing of the information architecture prior to the testing program
Inviting stakeholders to observe testing sessions
Presenting the results of the testing program and the resultant changes to stakeholders in report and/or presentation