Table of contents
1. Location-independent test methods
2. Five-second tests
3. Three-minute tests
4. Moderated remote tests
5. Unmoderated remote usability tests (~ 20 Minutes)
6. Summary: The right method for all use cases
All methods presented here have one thing in common: They are carried out remotely, so that test users can proceed with the testing in their familiar environment, using their own devices. Thus, these tests are location-independent and closer to actual normal use conditions than in-house tests or lab tests.
Website visitors usually decide after just a few seconds whether the page or the offer is relevant to them. A good design is therefore characterised by the fact that the core message is conveyed within 5 seconds maximum. You can find out how your product is positioned in this respect with a 5-second test.
2.1 What questions are 5-second tests suitable for?
The 5-second test is suitable for the following questions:
- First impression: Is your core message understood? Does the test user’s first impression agree with your brand values?
- Relevance: Is your offer relevant to the target group? Does it meet their requirements?
- Design / Brand: Do the users find the design attractive? What emotions does it cause?
- Comprehension: Is the content understood at the first glance?
- Desirability: Does your website, your advertising materials or other design arouse the desire of the users to own / use your product? Does the product really solve the greatest pain of the user?
- Variant comparison: You can run multiple 5-second tests and present them to the testers in random order. This is particularly suitable for comparing different (competing) websites or design variants with one another.
2.2 Best practices and examples for test creation
Basically, the following applies: The 5-second test consists of three parts. In the introduction you prepare the test users about the content of the test. Next, the website or image file is shown. Finally, you can ask the test user your questions.
Clear introduction: To get the test users concentrated right from the start, you should point out that the page will close again after a short time. If it makes sense, you can also draw the tester’s attention to a certain aspect of the page or assign them a very specific task.
Take a quick look at the following website / ad / design.
Please pay particular attention to the design.
What would you click on to find out more about this company?
When you are ready, click on “Display page”.
Creation of image files of the test objects: Do you wish to test designs or promotional materials that are not available as URLs? With RapidUsertests you can simply upload them as JPEG, PNG, GIF files.
Subsequent goal-oriented questions: After the test object has been displayed for 5 seconds, you can assign a task to the test user. Below we provide some sample formulations for individual cases:
- First impression: What is your spontaneous first impression? What are the positive or negative aspects you noticed? What is this website’s effect on you at the first glance?
- Relevance: Does the offer displayed interest you? Would you take a closer look? Why (why not)?
- Design / branding: What does the design of the website/advertising material prompt you to do? If 1 means very bad and 10 very good, how would you rate the design on a scale from 1 to 10?
- Comprehension: Please describe in your own words what is being offered on the website.
Finally, no more endless discussions about small adjustments to the user interface or new functions: Simply create a 3-minute test in the morning and enjoy your lunch break. In the meantime, your target group will provide the answers to your most important questions.
The 3-minute test is a very short usability test with a single task only, so that it can be created and evaluated quickly.
3.1 What questions are 3-minute tests suitable for?
We recommend 3-minute tests for the cases below:
- Design comparison: Which design do your users like best? Where are your brand values conveyed best?
- Feature tests: What do your users think about a new feature? Is it user friendly?
- First impression: Get feedback on the first impression of individual pages (e.g. landing page, home page, advertisement).
- Advertising materials: What are the first impressions of ads, flyers, posters and other advertising materials? Are they convincing? Can they be understood?
- SEO: What are your customers looking for, what results would they choose and why?
3.2 Who are 3-Minute tests suitable for?
3-minute tests can be used by these teams:
- Designers / Design Department: With 3-minute tests, you can make and argue about design decisions based on user feedback.
- Marketing: The 3-minute test is particularly suitable to find out about the first impression generated by your advertising materials, which versions are preferred by the target group and whether your brand values are conveyed. Another advantage: They are faster and easier to do than online surveys.
- Product Management: Test new functions and features quickly and easily before launching them.
- UX: With a 3-minute test, you can also test small UX optimisations within just a few hours.
3.3 Best practices and examples for test creation
Concrete scenario: Especially if your test starts in the middle of the customer journey, it is essential to give the users a good briefing about the scenario. Where specifically are you now and what would you normally have done before?
Your car will not pass this year’s vehicle inspection. Since spending a lot of money on repairs doesn’t make sense, you would like to buy a new car.
Unfortunately, you don’t have that kind of money in your account right now, which is why you intend to take out a loan. A friend has recommended Bank X. Now you want to check out what the bank can offer.
Very precise tasks: So that the test users do their job quickly and you actually receive only a 3-minute video, you need to be very clear about what they should and what they should not do.
Try out the loan calculator and comment on how you go about it.
- What do you like and what do you dislike?
Finally, click on “Apply for a Credit”.
A more detailed UX test should be used for those questions that cannot be answered by a 5-second or 3-minute test. For these cases, we have to differentiate between moderated tests and unmoderated tests. In a moderated remote test, the test users also do the testing at their home, but you can speak to them live.
4.1 Which test cases are moderated remote tests suitable for?
Moderated tests are particularly suitable for the following cases:
- Low fidelity prototypes: With moderated tests, you can lead the test users in a better way and ask questions, so that they are also suitable for low-fidelity prototypes.
- In-depth interviews: In addition to usability tests, you can also use live sessions for user research.
- UX tests combined with interviews: Usability tests and interviews can of course also be combined. First have the website tested and then ask your interview questions.
- Products requiring an explanation: Do your test users need a longer briefing or induction period? Moderated remote tests are also suitable for this.
4.2 The advantages of moderated remote tests
The following are the advantages of moderated tests:
- Not location-bound: Test users, test coordinators and observers can participate from anywhere in the world.
- Realistic: During the test, test users stay in their familiar environment and use their own equipment. This causes their behaviour to be more natural than in a sterile lab setting.
- No long-term planning required: Moderated remote usability tests can be carried out faster than in-house tests. Usually, you can organise your first tests within just a few hours.
- No need for special premises: Only one (small) room is required. Observers can participate via video streaming.
- Result sharing with stakeholders: Tests are recorded, and you can cut highlight videos from them or share individual excerpts via a link.
- Easy integration into existing processes: Moderated remote usability tests are fast and agile due to the time they save. Thus, they can be better integrated into agile processes than in-house tests.
Almost anything can be tested by means of an unmoderated remote usability test. Below some suitable cases:
- The test of the entire user journey: How do customers find their way around your website? Where would they have left it? Do they successfully get to the checkout?
- Prototype testing: As long as the prototype is fully functional, it can be tested without moderation. However, moderated tests should be used for low-fidelity prototypes. You can find more best practices for testing prototype usability here (in German) and here (in English).
- Testing of competitors’ websites: What do the test users think about your website compared to competitors’ websites? You can find more information about this topic here (in German) and here (in English).
- Software testing: How do new or existing customers get along with your software?
You can also start with a 5-second test to get the first impression and then test the entire website.
3-minute tests are a great way to test small optimisations you’ve introduced based on the in-depth tests.
Let’s finish with a brief overview of all methods:
- 5-second test: Suitable for the first impression of a website, a design or an advertising material.
- 3-minute test: Suitable for small questions that need to be answered within a very short time.
- Unmoderated remote tests: Suitable for more extensive tests, e.g. for analysing the entire user journey.
- Moderated remote tests: Suitable for testing low-fidelity prototypes, for products that require explanation or for in-depth interviews.