When developing online research resource portals, databases and community groups it is often difficult to assess how people will be using a platform as it grows. As online resources for research in international development are often put together without a commercial business framework in mind, there is a potential that the functioning fluidity of the site may not be addressed at a basic level before it is launched.
Audiences for development research websites are often very diverse, with users seeking information from academic, aid worker, policy-maker, NGO and non-professional backgrounds. This rich range of users can be problematic when defining the way a site functions and communicates the information it has available.
Usability evaluations, when carried out at the very beginning of the site development process, can determine if those who will be using the platform can do so quickly and easily to accomplish their own tasks. Usability testing applies to every aspect of the website with which a person interacts, such as hardware, software, menus, icons, messages, community pages, documentation, search functions and help. It generally involves measuring how well test subjects respond in four key areas:
- Performance – How much time and how many steps are required for people to complete basic tasks? (For example, create a user profile, access and download a document, search for a key term.)
- Accuracy – How many mistakes did people make? (Were they fatal or recoverable with the right information?)
- Recall – How much does the person remember afterwards or after periods of non-use? (Is the site user intuitive?)
- Emotional response – How does the person feel about the tasks completed? (Is the person confident, stressed? Would the user recommend this system to a friend?)
Evaluations are designed to solicit feedback from participants, focusing on areas of concern within the site’s functionality. An evaluation typically involves several participants, each of whom represents a typical user from a different background.
The Usability Process
Results from usability testing process can easily become inaccurate if users are approached incorrectly. The following checklist has been put together as a series of step-by-step actions for the test administrator to go through with the participant as they carry out the usability test. The checklist should be carried out under the presumption that the test will be conducted in one place (i.e. not remotely), on one computer with screen-video capturing software. Ideally, this should be a quiet area, away from external distractions, where a camera can be set up to record the participants as they navigate the site. Participants should be asked to adopt a “think aloud” attitude to the process, talking through each of their navigational decisions so as to get a good impression of the way they would naturally find their way around the site. [1]
Activities to be completed with user participants:
Process overview, system under test overview and setting information
Test session checklist (completed by the test administrator)
- Greet user and introduce any observers, data loggers and others present in the room.
- Explain the test area and outline the screen-video capture software and video camera set-up, until the user is comfortable with the equipment.
- Reassure that there are no right or wrong answers and we are not testing them but the site.
- Seat user and sit close by to explain the test session.
- Have user complete a respondent profile questionnaire (ask for basic information relating to their job role and current website familiarity).
- Have user sign a video/audio tape and disclosure agreement.
- Ask the user if they have any questions.
- Review and demonstrate the “thinking aloud” process and complete practice exercise until the user is comfortable.
- Ask the user if they have any questions.
- Verify logging is ready to begin and start video-tape.
Activities to be completed with user participants:
Step-by-Step System Testing
Test session checklist (completed by the test administrator)
Journey One:
- Ask the participant to search the site with a key phrase or word in mind.
- Ask them to say aloud how and why they are searching for information (noting down the systems they are using).
- Enquire with short phrases why the participant is doing certain things (i.e. “You are using the archives search function – why?”)
- Ask what results they have found:
- Was it successful?
- Did they find what they wanted?
- Were there any frustrating elements to the search?
- What could have been different?
Journey Two:
5. Ask them to start again, searching for something different – repeat the same process asking if they found what they wanted.Journey Three:
6. Ask the participant to spend 10-15 minutes or so searching the site independently (leave the room during this time to allow them to search in an uninhibited way).
7. Re-enter the room and ask the participant if they noticed anything in particular about the functionality of the site i.e:
- Were there things that didn’t make sense?
- Could the navigation of the site be made easier?
- What are their overall thoughts on the layout and style (including colours, design, etc)?
Results and feedback are accumulated for each participant through the interviews and on screen navigation captures. The analysis is not a mandate for change; it provides a base of information from which decisions can be made to solve problem issues or to indicate areas in which the design has been successful.
[1] This checklist is based on document by Wendy W. Naughton, Usability & Technical Documentation Group, Xerox Corporation
For more information on usability testing visit:
Usability Testing: How to plan, design and conduct effective tests
8 Guidelines for Usability Testing
This blogpost was originally published on CommsConsult’s Website.
Paula Fray
Shubha Jayaram
Sue Martin
Maria Balarin
James Harvey
Emily Hayter
Susan Koshy
Ronald Munatsi
Ajoy Datta
One other alternative is to try a service like http://www.usertesting.com where your can set demographics for the type of user you want from their huge panel of everyday people. These custom selected people will record their thoughts for you to listen to as you watch them navigate your site. I was shocked and amazed at the things I learned.