Table of contents
The most common reasons for external test user recruitment
Before you begin: your starting point and goals
Establishing criteria for choosing a provider
Pre-selection and narrowing down recruitment agencies
Advice and final provider selection
Continue reading online or download the eBook as a PDF:
The most common reasons for external test user recruitment
For a long time now, it has been no secret that creating an excellent user experience has become a crucial competitive factor. The time seems to have come to unconditionally place users at the heart of product development and to break away from personal opinions and assumptions. In businesses, the transition to user-focused product development doesn’t just change the rules – it also creates a much greater understanding of the target groups.
Some companies conduct test user recruitment internally when carrying out user research studies, while outsourcing recruitment is becoming increasingly key for many UX researchers. One of the most common reasons for this is the huge amount of time saved: after all, internal recruitment and organisation are very likely to take much more time than actually conducting the study. Accordingly, internal recruitment also costs more. In addition, specialist test user providers have a much wider pool of people to draw from, which allows target group criteria to be defined more precisely – even for challenging test user profiles.
Avoid potential pitfalls when choosing a test user recruitment agency
Finding suitable test users in professional and personal environments isn’t just difficult, it’s also highly inefficient. However, when choosing a test user recruitment agency, it‘s important to think things through and be prepared. There’s a wide range of potential pitfalls, so it’s crucial to establish specific selection criteria and take a close look at potential providers. This is the only way to avoid the risk of making a wrong decision in advance. To increase your chances of a fruitful partnership with a recruitment agency, a clear understanding of the various selection criteria is required.
In this guide, you will find out how to go about selecting a test user recruitment agency to ensure that your user research project is a success and to generate useful insights for product development. You will learn how to handle typical pitfalls when selecting a provider, so you can ultimately be sure you made the right decision. After all, just like your customers, you ultimately just want to get what you really need.
Happy reading!
Before you begin: your starting point and goals
It might sound trivial, but a problem really is half solved when it‘s clearly formulated and linked to specific goals. Ask yourself at the beginning where the issue is and which problems and challenges need to be resolved through test user recruitment. A clear understanding of these will help you later when briefing your recruitment agency.
Simply ask yourself the following questions:
- How are the target group criteria for the UX project defined?
- What needs to be tested and what exactly do I want to find out in the UX project?
- Which aspects of the product are unsatisfactory?
- Which testing method is most suitable? Inhouse, remote, expert review or quantitative methods such as A/B testing or surveys?
- What will the test users be asked to do?
Before you start, make sure that you have a clear understanding of what the problem is, what exactly you want to achieve and which UX methods will best help you to meet your goals.
Establishing criteria for choosing a provider
To successfully find and choose the right test user recruitment agency, it’s important to critically assess in advance which criteria matter to you and how you weight the different criteria. Simply use the assessment matrix that you downloaded along with this guide to record and weigh your selection criteria. It may even be worthwhile to define the selection criteria weighting with managers or colleagues.
Examine the following selection criteria to assess their relevance to recruiting external test users for the UX project:
Provider screening process
What does the provider’s quality assurance process look like? How does the provider ensure that the test users are actually suitable and have the right motivation? Do they call and brief the test users?
Test user quality and impartiality
For regular tests, how do they ensure that the test users aren’t always the same people? Is study participation limited per test user or are they professional test users, increasing the likelihood of skewed results?
Test user reliability
How reliable are the provider‘s test users? What is the no-show rate? How does the provider handle no-shows and unqualified participants? Are new test users provided? If so, who covers the additional costs?
Coverage of research methods
Does the recruitment agency help to book test users for methods such as diary studies, indepth interviews, focus groups, in-home testing, remote testing, etc.? Do they support UX methods suitable for the research project?
Geographic and demographic coverage
Does the agency cover test users for national projects or international and multilingual projects too? Do you have to find a new provider every time for different geographic and demographic features or does the agency have a wide-reaching test user pool?
Understanding of UX and user research
Does the recruitment agency have an in-depth understanding of special test user requirements for user research, UX design and usability engineering? Or is the provider specialised in test users for general market research?
Recruiting challenging profiles
Can the test user provider recruit target groups with challenging profiles, such as managing directors of SMEs, people with disabilities, wealthy people, tradespeople etc.?
Incentives and schedule coordination
Who is charge of incentives and paying the test users who are recruited? Are the incentives handled by the agency or do you have to look after compensation yourself? Does the agency also coordinate participants’ schedules?
Reach and pool size
How many test users are available from the recruitment agency? Is the recruitment agency’s reach and pool big enough for them to consistently supply suitable test users over longer periods of time?
Cost transparency
Are testing recruitment costs transparent in advance? Is it possible to clearly calculate recruitment costs or do you get the impression that you are in for hidden costs at a later stage?
Delivery time from commissioning
How long do you have to wait after commissioning the recruitment for suitable test users to be supplied? Are there long lead times or can you also test with suitable people at short notice if need be?
Commissioning process
Do you need to contact the recruitment agency to brief them and request a quote? Or can you enter the relevant target group criteria for your UX project into a configuration tool and order online in a reliable process?
Recruitment status updates
Does the recruitment agency offer you a way to view the current status of the ongoing recruitment process at any time? Is it clear which test users have been invited, which have declined and which responses are still pending?
Personal contact
What happens if you have questions for a recruitment agency before, during or after booking test users? Do you have a personal contact who can provide you with competent advice? Do you feel as if you are in good hands?
Pre-selection and narrowing down recruitment agencies
Before you begin looking for potentially suitable test user recruitment agencies, as described in the previous section, you should critically examine which selection criteria are relevant for you and your business. After all, if all the key criteria are determined precisely, you are already in a position from the outset to look for potential providers with a sharpened focus.
Looking for suitable recruitment agencies
- Examine the test user recruitment agency websites
- Take a look at providers’ case studies and references
- Give recruitment agencies of interest a call to get a first impression
Create a long list of 7–10 recruitment agencies
In your initial research to create a long list, it is not necessary to assess the agencies you find in detail. The first step of the search process is more about identifying recruitment agencies that come into consideration according to your predefined selection criteria. Record the names of interesting providers in the assessment matrix and pre-select around 7–10 agencies.
Create your top three shortlist
In the next step, you take a closer look at your pre-selected recruitment agencies in order to define a shortlist with your top three favourites. As such, your shortlist should only contain agencies which mostly or entirely meet your requirements and criteria. Note down any outstanding queries and clarify these with the providers on the phone.
Advice and final provider selection
Depending on the complexity of your planned study, you may have outstanding queries that you would like to discuss over the phone with your preferred test user recruitment agencies.
To allow the agency to provide competent advice, it helps if you provide them with information on the following:
Test objects
Which product needs to be tested, redeveloped or optimised?
Goals and motivations
What do you need to find out? What do you expect from the studies?
Target groups
How are your target groups specifically defined? Which demographic or other characteristics do your target groups have?
Time frame
When should the next study with test users ideally be completed?
Challenges
What is the problem? Which challenges need to be dealt with?
Budget
What is your planned budget for recruiting suitable test users?
In an initial conversation over the phone, can you clarify any queries and get an idea of what working with the recruitment agency could look like?
If the provider’s website leaves questions to be answered and you decide to give them a call, you should be able to answer the following questions after the conversation:
Consulting skills
Did you find out more from the conversation? Were your queries dealt with?
Recruitment costs and process
Are the costs transparent and does the process meet your expectations?
Industry expertise
How well does the agency understand your challenges and requirements?
Critical questions
How does the agency react to critical questions? Do they avoid them?
References
Can the agency provide references for projects they have worked on?
Book test users online or request a quote
Once you have examined the website and potentially got to know the provider over the phone, you will have a good idea of which of the recruitment agencies appeals the most to you. Once you’ve chosen a provider, you can now either order test user recruitment online straight away or request a quote from the provider of your choice, depending on the provider.
Download the eBook so you always have a reference work:
Summary of key steps
By carefully preparing for and conducting the provider selection process, you increase the likelihood of a successful partnership with your preferred test user recruitment agency. Ultimately, you are also ensuring that you do not waste any valuable resources and you make a decision that feels right.
Here are all the key steps for success at a glance:
Starting point
Work out your starting point. Define your target group as specifically as possible and consider which UX testing methods are most suitable for your project.
Selection criteria
Decide which selection criteria are most important for you, so you can look for providers with a sharpened focus.
Assessment matrix
Use the assessment matrix. Enter the determined providers in the matrix and assess them based on the weighting.
Longlist and shortlist
Create a longlist with 7–10 providers that you think are worth considering. Then create a shortlist with your three favourites.
Recruitment agencies
Examine and compare recruitment agency websites based on your selection criteria. Clarify any outstanding queries over the phone.
Test users
Order the test users for your study online or request a quote if your project is more complex or you cannot order online.
