Usability testing
🖐️

Usability testing

Tag(s)
Research & testing
Cook(s)
Rebecca Blakiston

Nutrition profile

Evaluate the usability of a product by asking participants to complete tasks and then observing their behavior.

Cooking time

Varies depending on the number and breadth of tasks. Rapid usability tests can take minutes, while more in-depth tests can take up to an hour per participant.

Perfect for

Evaluating the usability of a website or digital product at any stage. You can test the usability of products early in the design process to inform your design direction, as long as you have an interactive prototype. You can also test products that already exist out in the world to identify pain points and areas of improvement.

Prep work

Decide what to test

Identify your research questions. What do you want to learn? If you are testing a website for the first time and don't know where to start, try identifying your users' top tasks and prioritize those for testing. Web analytics data, user interviews, and surveys can help you identify the top 5-10 tasks.

We recommend testing tasks that:

  • Are important to users
  • Align with your goals of the project
  • You have control over (i.e. avoid testing things you can't fix)

Write tasks and scenarios

Tasks are what you want participants to do and should be succinct. These are for internal use and it's ok if they include jargon.

Scenarios are what you tell participants to do and should be believable and unambiguous. Scenarios should avoid leading language, such as the link label you expect the participant to click on. Try out your scenarios in a practice run to ensure they are worded clearly.

For each task, define what success looks like. This will be important to consistently measure success rates.

Example tasks and scenarios

TaskScenarioDefinition of success

You've heard the library has quiet rooms for grad students you can use for a whole year. Find out if you are eligible and if so, apply for a room.

Participant finds the form, fills out the form, and hits "Submit" button.

Find one scholarly article about climate change that interests you.

Participant identifies a scholarly article and opens the full text.

Identify your audience

Knowing how unique your audience is helps you recruit the right participants. For general products intended for everyone, you can recruit broadly. For products with more specialized audiences, more targeted recruitment is necessary. Consider the following when creating any screening questions:

  • Inclusion criteria: required characteristics (e.g. knowledge of specialized terminology)
  • Exclusion criteria: unwanted characteristics (e.g. not attending the university)

For more on defining your audiences, see personas and empathy mapping.

Example of a participant explaining what they are seeing on the screen during a remote session
Example of a participant explaining what they are seeing on the screen during a remote session

Decide on moderated or unmoderated testing

Moderated tests require scheduling, whereas unmoderated tests can be sent to participants to complete on their own time.

Usually, we recommend moderated testing, since you can observe users in real time and probe to understand the "why" behind user behavior. You can also run a moderated test without any fancy software. If you are interested in unmoderated testing, which can allow you to reach more participants, check out UserZoom, UserTesting, or other tools for unmoderated usability testing.

Pick a location and technology

For in-person sessions, you can conduct sessions "in the lab" or "in the field." In the lab, assuming you don't have a dedicated usability lab, find a quiet space that is easy for participants to find and will allow you to moderate the session with minimal distractions. You can also provide technology or ask participants to bring their own device. You will have more control over your technology (e.g. screen recording software) but participant technology will allow you to observe more authentic user behavior.

For remote, virtual sessions, use a video conferencing tool such as Zoom that allows the participant to share their screen. Remote sessions have the advantage of reaching users at a distance, but can have technical challenges.

Recruit participants

If you can test with a general audience, you might try intercept recruitment, where you intercept passersby for short testing sessions. This could be areas with high traffic like a library lobby, a scholarship office, or a food court.

Example of intercept recruitment at the University of Arizona Main Library. Participant in middle, notetaker on the right using Google Forms to capture observations.
Example of intercept recruitment at the University of Arizona Main Library. Participant in middle, notetaker on the right using Google Forms to capture observations.

For targeted recruitment, you might try sending out emails, posting on social media, or contacting individuals directly. In your recruitment message, include:

  • the purpose of the study (e.g. "help us improve the library's website")
  • the time (e.g. "no more than 30 minutes")
  • a description of the incentive (e.g. "you will receive a meal voucher to thank you for your time")

Offer incentives that reflect the amount of time you are asking of from participants. A cup of coffee might be a reasonable incentive for a 10-minute on-the-fly usability test, whereas a $25 gift card might be appropriate for a 45-minute scheduled session.

Ingredients

  • A moderator and a note taker
  • A live website or product to test (or interactive prototype)
  • Incentives for participants (e.g. gift cards, snacks)
  • Basic screen capturing software (e.g. QuickTime Player, Xbox Game Bar)
  • Usability testing software (optional)

Directions

Conduct an initial interview

At the beginning of the session, gather background information from the participant, such as their job, area of study, or prior experience with the product being tested.

Introduce the study

Adapt an introduction script. Remember to tell the participants:

  • "We’re testing the website, we're not testing you, so you can't do anything wrong."
  • "Think out loud as you’re working through the scenario, telling us what you are looking at and what you are thinking." (Assuming you're using the think aloud protocol).
  • Any rules (e.g. stay on this website)

If you are recording, be sure to ask for consent first.

Facilitate the study

Try to keep the participant talking without distracting them too much. Ask probing or clarifying questions as needed. Remain neutral. Use good judgment and adapt to the situation.

Take notes

What you'll capture for each scenario will depend on your research goals, but data that are often useful include:

  • Navigation path(s) (e.g. visiting → parking → how to pay)
  • Number of attempts (e.g. how many times they click the back button)
  • Success or failure
  • Time on task (i.e. how long it takes for them to reach success or give up)
  • Useful quotes

To make it easier to analyze data afterwards, we recommend using Airtable, Google Forms, or Notion to capture your results in a structured format.

Example of usability results in Airtable, including scenario, number of attempts, and success or failure
Example of usability results in Airtable, including scenario, number of attempts, and success or failure

Debrief and analyze results

When possible, set aside a time to debrief immediately following a usability test. While fresh in your minds, you can start to identify pain points and patterns, and generate ideas for potential solutions.

Example of usability testing results in Google Forms, where the notetaker captured participant responses to a question about library hours
Example of usability testing results in Google Forms, where the notetaker captured participant responses to a question about library hours

Plating

Depending on your audience, you might present your findings through a presentation, a report, or an informal dialogue. Here are some templates to get you started:

Example of a slide presenting a high-priority finding from a usability study.
Example of a slide presenting a high-priority finding from a usability study.

Pro tips

  • Rather than usability testing on a project basis only (e.g. a website redesign), try establishing an ongoing schedule for testing and iteration. If this is done well, you'll never need a website redesign again.
  • Listen to participant suggestions, but don't take them too seriously. They often don't have the full context or design expertise to make useful suggestions.
  • Create a template for note taking as well as standard codes, abbreviations, or symbols to more easily capture what is happening (it can go quickly!)
  • It's ok to stop a test when someone has failed to complete a task and you aren't learning anything new, or when the participant themself is getting frustrated.
  • Don't rush to conclusions based on one or two participants. If you are proposing significant design changes, make sure to validate your recommendations with additional usability tests.

Resources

Facilitator's guide for usability testing (adaptable Word template)

Introduction script (adaptable Word template)

Presentation of Findings from Usability Testing (adaptable Google Slides template)

Credits

Usability Testing: A Practical Guide for Librarians by Rebecca Blakiston

Rocket Surgery Made Easy: the DIY Guide to Finding and Fixing Usability Problems by Steve Krug