• Home
• About this Site
• Contact Us
• RSS Feeds
• eGovRC Toolbar

Help & Print

Help Print

Resize Text

Categories, RSS and More

Best Practice Benchmarking Study - Benchmark Testing Findings and Recommendations (RS-02) - Victoria Online

Project Version 1.1 Final - 6 December 2001

Prepared by The Hiser Group for Multimedia Victoria

1. Executive summary

1.1 Overview

This document presents the results and recommendations of the research study carried out by The Hiser Group (Hiser) on behalf of Multimedia Victoria for Best Practice Benchmarking (RS-02) in support of the Victorian Online Gateway (VOG) redevelopment project. The purpose of this study is to benchmark features of sites from throughout the world which embody world's best practice for a set of identified features:

• Search
• Browse
• Directories
• Online payment
• Aggregation
• Integration.

The results and recommendations contained in this report will help drive the subsequent design and testing of the Victorian Online Gateway.

1.2 High Level Findings

• A large portion of users find interactive functionality challenging to use. Many users have difficulty using interactive features on web sites. A portion of users have difficulty searching, and many have difficulty using online applications. Almost all users in this study had difficulty using online payment systems.
• Users require simple designs. Users preferred sites that reduce their cognitive load by presenting information in a simple, easy to read manner. Clear, consistent layout was key to making users comfortable on each of the sites.
• Browsing is the primary method of information discovery. Users generally prefer to browse rather than search. Searching is often associated with overwhelming, low quality results. Users are even more likely to browse if they are unsure how to specify their desired target. Well designed link labels and taxonomies are vital to ensuring that users can browse effectively.
• Orientation is important for large sites. Users are very attentive to cues that help orient them as they move through large web sites. Most users make use of multiple cues, such as imagery, headings and breadcrumb trails to help decide whether they are in the right part of a site.
• Customisation is difficult for most people. User-controlled customisation (e.g. ‘My Page') is a feature that many users find challenging. Most would only make use of customisation to gain access to frequently accessed information or if the effort associated with customisation is minimal and the benefits are very clear.
• Users are not very competent at searching and refining searches. Many people have difficulty searching for information, and find refining searches difficult.
• World's best practice sites are quite conservative. World's best practice sites employ well proven techniques to ensure users can carry out tasks with ease.
• No one site is the best. None of the sites benchmarked combine all the key features that users require. Each benchmarked site showcased particular features that are valuable for users.
• There is no significant difference in user competency based on audience segmentation. The audience segmentation carried out in the course of this work has not revealed a significant difference in people's competency using the internet.

1.3 Key Recommendations

1.3.1 Search

• Ensure results are clearly presented and easily scannable.
• Reveal category structure in search results.
• Provide coaching tools to help users search.
• Integrate search and browse.
• Ensure that users can easily refine an existing search by entering additional search terms.
• Promote and encourage standardisation in the tagging of content.
• Promote frequently accessed content to the top of search results.

1.3.2 Browse

• Use imagery to help orient users.
• Ensure category labelling is meaningful.
• Use familiar browsing metaphors such as tabs and folder / tree style navigation.
• Orient users with clear textual cues.
• Reserve areas on a page for particular content and navigational elements.
• Provide role-based navigation as secondary means of accessing content.

1.3.3 Directories

• Provide textual cues to orient users.
• Ensure that search and browse results are easily scannable.
• Provide collateral information in directory listings.
• Ensure browse categories are relevant to users.
• Show categories with search results to encourage discovery.
• Ensure that structured query forms are very clear.
• Combine search and browse.
• Provide coaching for search.
• Ensure visual design does not distract users from task.

1.3.4 Payment

• Use familiar terminology and leverage real-world analogies where possible.
• Ensure process is as simple as possible.
• Provide clear confirmation of actions and outcomes.
• Allow users to abort process at any time.
• Remove all extraneous page elements through purchase process.
• Embed registration in purchase process.
• Set expectations throughout process.

1.3.5 Aggregation

• Assume that most users will be challenged by advanced functionality.
• Combine information into useful groupings.
• Ensure browse design is easy to use.
• Ensure that pages are not cluttered with information.
• Ensure that sign-in and registration systems are clear and simple.
• Target user-customisation to very frequent portal visitors.
• Guide users step-by-step through process of user-customisation.

1.3.6 Integration

• Use persistent navigational elements.
• Provide consistent browsing experience.
• Allow quick access to content.
• Structure the presentation of information so that it is not overwhelming.
• Present information from different sources in consistent manner.
• Integrate browse and search.

1.4 Next Steps

We recommend that the findings and recommendations contained in this report are used in conjunction with organisational and user goals to drive subsequent design and testing activities for the Victorian Online Gateway. In line with international experience, the success of the Victorian Online Gateway can be supported by adopting a user-centred design approach which accounts for organisational goals. The activities described below are a valuable means of ensuring that the Victorian Online Gateway is both usable and useful for its users. It is highly recommended that the Victorian Online Gateway team carries out the following activities:

• Elaborate organisational goals. Interviews with key stakeholders are a means of both identifying and clarifying the organisational goals for a web site. Organisational goals are used in conjunction with user goals to drive the subsequent design and development of a web site.
• Identify important content. Focus groups are a means of rapidly identifying user needs through a series of group activities with potential site users. For example, the ‘Design a Web site' activity allows potential site users identify content and services that are most important to them.
• Define high level information categories and content groupings. Relevant content groupings and category names is vital to the success of the Victorian Online Gateway portal. Through the ‘Card Sorting' exercise, users identify the way they would group content and the names they would give each group.
• Involve users through the design and testing process. Once organisational goals, users goals and content groupings have been defined, it is important to involve users through the design and testing process. This can be achieved through the following activities:
  • Collaborative design sessions bring together a group of stakeholders such as users, developers and analysts to design the user interface of a web site. Before beginning collaborative design, several high level design concepts are developed as ‘straw man' designs. Using paper mock-ups developed through the conceptual design activity, both users and stakeholders review and revise a design to ensure that is meets both user and organisational needs. Collaborative design helps identify potential user interface issues early on in the design process, ensuring that changes are not costly to fix.
  • A usability walkthrough is a quick, flexible evaluation method that aims to collect user (and sometimes other) feedback about the design of a user interface. The method can be used with either mock-ups, or with electronic prototypes. Usability walkthroughs can be used on an entire new interface, or for just those screens that cover new functionality being added to an existing site.
  • Visual design applies detailed graphical treatment to designs developed in collaborative design sessions and usability walkthroughs.
  • Usability testing is a means of evaluating how well a particular system meets its usability criteria. With real users and real tasks, it is the most accurate way to collect data on users' perceptions of a web site and how well a design performs against usability goals. One of the most useful aspects of usability testing is that it looks not only at the tasks that users perform but the context in which they are performed. This helps to uncover usability issues based on the way in which users work together within the environment in which they work.

---

Reports

It is the intention of the Victorian Online Gateway project team to make available individual report findings and recommendations for appropriate use by other government organisations. Materials may be downloaded and printed with applicable copyright and other notices included. Any other use including copying, modifying, displaying or transmitting of the content of these reports requires the prior written permission of the Executive Director of Multimedia Victoria.

The Best Practice Benchmarking Study is available in pdf format (3.13mb) (This document requires the use of Adobe Acrobat Reader.)

Best Practice Benchmarking presentation to the Expert Group by Hiser, 5 December 2001 in Powerpoint 97 (5.1mb)

Last updated: 28 February 2006

Search, Login and Options