How to conduct usability testing in 8 steps

Table of contents

1. Define your goals
2. Find relevant participants
3. Create a usability testing scenario
4. Craft usability testing tasks
5. Define your metrics
6. Run a pilot test
7. Record your tests
8. Extract and analyse insights
9. Final words

1. Define your goals

The first step of prominent usability testing is defying your main goals. That encompasses having clear answers to some of these questions:

• What are you testing? Knowing exactly what you are testing and defining the scope of it.
• Why are you testing? What is the purpose of conducting usability testing in your company, and what are you trying to accomplish?

Taking time to comprehensively answer these questions will help you lay a framework for your usability testing. It will also help you stay on course while conducting the whole process.

However, to lay the strong foundations for your usability testing, you’ll need to dig under the surface and invest some additional time and effort to hone your usability testing goals. This requires effectively communicating and collaborating with stakeholders of the solution you’re testing. They can give you core elements you’ll lay your usability testing foundations on.

To maximise setting your usability testing goals, try these:

1. Ask the stakeholders about their major concerns, and which features and processes they consider to be critical. This will help you determine where to focus your testing efforts and which areas to prioritise.
2. Explore and analyse the data stakeholders already have on the solution you’re testing — such as analytics, feedback, etc. An example for digital agencies as stakeholders can be their website analytics records that can show if something is out of the ordinary — e.g. video metrics with high video plays, but low percentage watched.
3. Ask them for a demonstration of a particular feature & process stakeholders want to be tested. This can help both you and them to better understand what aspect’s you’ll be testing and acquire more meaningful information to supplement your testing.

2. Find relevant participants

Finding the right participants is essential because matching them with your usability testing goals will make sure you’ll get the most relevant insights.

In particular, how you’ll find the right participants for your usability study mainly depends on the solution you are designing.

If you target a wider audience then you can simply segment your audience by demographics (age, location, employment status, etc) and a few more parameters like devices they’re using (smartphones vs PCs, etc.)

Comparably, if you are developing a complex system, then you’ll need to be more specific and significantly narrow your participant pool down. This means filtering participants by their expertise, skills, previous experience working with similar solutions, etc.

You’ll also need to measure to which extent your participants’ traits match with responsibilities and knowledge the system you’re developing requires. This is where things become more specific, and your ability to drill down as much as possible will be of huge value.

2.1 Leverage screeners to find the most relevant participants

You can use screeners to make sure that the traits of your participants are being aligned with the desirable traits of your end-users. Screeners will help you ensure you’ll acquire the best possible participants per their skill set, expertise, and knowledge. They will typically consist of multi-choice questions.

Screeners can also measure the level of motivation and commitment of your participants, besides their expertise and desirable traits.

Some of the main types of screening questions include:

Demographics — e.g “How old are you?”; “What is your household income?”
Experience level — e.g. “How experienced are you with Amazon Drop-shipping?
Behavioural questions — e.g. “What online stores do you use to purchase hardware?”
Firmographics — e.g. “In which sector have you worked?”
Specific knowledge or skills — e.g. “Can you describe your experience with coding and to what extent are you familiar with it?”
Frequency of use — e.g. “How often do you use Amazon to purchase hardware?

Here are some valuable tips for creating screeners:

• Ensure your predefined answers are clear and concise
• Avoid leading or biased questions
• Ask your participants for consent to record their usability testing sessions via screeners
• Avoid using only Yes/No questions

In the video below, you can see an example of our question library:

The final, and maybe the most effective step is utilising user personas. A user persona is an imaginary persona that defines a user type that is most likely to use and engage with a certain solution. It should include background information, skill set, behavioural patterns, character traits, and demographics of your ideal user. User personas can hugely skyrocket your user-centricity commitment and help you develop an in-depth understanding of your users.

You can leverage personas to create cutting-edge screener questions that will boost your efforts to pick the most relevant participants.

Finally, you can use tools like Google Forms, Survey Monkey, or Hotjar user survey feature to create powerful, easy-to-share screeners. Advanced tools, which usually require a subscription, will include helpful features like qualifying logic, narrow downs, and analytics coupled with effective filtering and sorting capabilities.

3. Create a usability testing scenario

Something that will substantially improve the effectiveness of your usability testing tasks (we talked about in the previous step) is usability testing scenarios.

A test scenario will help participants understand the context and immerse themselves in your testing, as users. The purpose is to simulate real-life scenarios and environments and to observe how participants interact with them while trying to solve certain problems.

Usability testing scenarios give context to participants and help them intuitively understand the whole process. This can help them to almost experience a scenario you’re simulating, so they can tap into their specific behaviour they’d naturally manifest if it was a real one.

Scenarios can be also used to add a skill setal layer of details needed to motivate participants and help them effectively carry the task till the end.

If you want to get profound and authentic results when conducting usability testing, then use testing scenarios to empower your usability testing tasks.

3.1 Example of a user testing scenario

Usability testing task is something simple like ——> “Choose a drone for a family trip that suits you the best using [ name of the website] website”

Usability testing scenarios can empower it with context like this: “ It’s wintertime and you are going for a weekend trip to a beautiful London’s countryside abounding with streams, grass, and trees with your family.

You need to buy a drone with a camera attached for under 1500$ to capture all those wonderful sights and record that precious experience for your family.

The drone should have a long flight time and should be able to avoid obstacles (trees, hills, rocks) in this environment. A weather report also showed the temperatures might drop down below -6°, so it should also be able to operate in cold conditions.”

This was a long-form scenario, and it’s not always needed, but the whole point was to demonstrate how a scenario can add depth to a plain task.

Some tips to keep in mind while creating a testing scenario are:

• Use user language — That way you’ll avoid any misunderstanding and help participants better immerse
• Keep it short and simple, but informative
• Address concerns (with your product) through tasks participants should complete — By creating realistic, actionable tasks you’ll get accurate, easy-to-implement insights

4. Craft a usability testing task

To effectively conduct usability testing, you’ll need representative tasks your participants will complete. By representative, we mean that your tasks should be real, actionable, and effectively mirror your usability testing goals.

Each task should take participants a step closer to achieving a predefined goal. Their purpose is to effectively, but immoderately navigate participants through the whole user journey.

Here are some examples of properly formulated testing tasks:

• Demonstrate how you would sign up for a plan B insurance policy
• Compare the benefits from Gold and Silver insurance packages and choose the one that suits your needs the best
• Afterwards, add that package to your shopping cart
• Accept a friend request and add that person to your “close friends” list

In short, your tasks should:

• Be actionable
• Have their success criteria defined
• Be non-leading, unrestrained, and unbiased
• Be simple and concise
• Portray experiences real users would have
• Be specific

5. Define your metrics

It’s of utmost importance to carefully select and leverage the right metrics. First, you need to know what exactly you want to measure. That mostly accounts for factors that influence the user experience and the usability testing metrics should usually measure:

• Performance
• Ease
• Effort
• Satisfaction

The most common metrics for measuring performance are:

• Success rate — measures the percentage of correctly completed tasks by participants
• Time on task — measures and sums the total amount of time participants needed to complete each task
• Error rate — how many errors participants make

To measure the rest, you can leverage the following metrics:

• SUS (System Usability Scale)
• CES (Customer Effort Score)
• SEQ (Single Ease Question)
• CSAT (Customer Satisfaction)

6. Run a pilot test

When you’ve finally defined everything from previous steps and before starting your usability test with participants, it’s mandatory to run a pilot.

Pilot tests will help you prevent many errors or uncover flaws within the usability testing workflow, and they’ll ensure that the actual test runs frictionlessly.

Precisely, running pilot tests can help you uncover if:

• Your participants understand what they need to do
• Your tasks are biased
• You’re getting the right and relevant information you expected to obtain
• You need to eliminate any redundant or unnecessary tasks or questions
• Your tasks are actually matching your predefined objectives

It would be best if you could find a participant for piloting your test who matches the real participant profile.

7. Record your test

When you have a recording of your tests, you can easily review the test workflow, take better notes or use the recording as a resource to help you present test results. With recordings, you’ll be able to transfer the most valuable insights in an easy-to-understand manner. Not only that, but those recordings will be in the form of video format and it’s one of the most engaging and context-transferring media formats.

There are many screen recording software on the market, and even a plain, free version of almost any software will do its job pretty well. Here you can find the favourite tools of TestingTime’s customers for remote user research.

However, a caveat with recording is that you’ll need to have consent from each participant. Furthermore, you’ll probably need to protect participants’ personally identifiable information by removing sensitive details, blurring their faces, or changing their voices. Here is a guide about GDPR in UX research.

8. Extract and analyse insights

The last step is analysing all collected data and extracting valuable insights. Make sure not to focus only on insights that support your hypothesis, but to include everything, like:

• Negative participants’ comments
• Unusual behaviour
• Other random comments or observations from both participants and other team members involved in the research

You should also consider adding tags, writing additional notes, and structuring & organising your data concisely. Start by categorising similar usability issues you’ve discovered into one group.

You can use various techniques to present and structure your data, such as the Kano model, one of the powerful visualisation techniques.

Next, you should prioritise issues by their criticality and importance (low—critical).

When all of this is done, you should discuss the results with your and other teams responsible for product design (the production team, for example).

9. Final words

Conducting usability testing is essential for creating outstanding products that skill set the problems of users. Usability testing will help you understand those problems and unearth optimal solutions. This is the reason why you’ll need to invest enough time and effort into conducting usability testing effectively. This process can be cumbersome at times, but will certainly become easier and more accurate the more you repeat it and the more different products you’re testing.