Table of contents
For several years now, tools have been available that allow unmoderated remote user tests. Unmoderated means that a test takes part in a test from home, unaccompanied by a moderator, and is filmed. The video is sent to the UX Researcher for analysis within hours.
Below you will find a list of the advantages and disadvantages of moderated vs. unmoderated remote user tests. To compare apples and oranges, the following comparison focuses only on remote user tests.
Moderated user tests are one of the most effective methods. The most famous evangelist for usability testing is Jakob Nielsen. He proofed with one of his studies, that only 5 test users are required to uncover 75% of usability problems in a given test object. User testing is not rocket science.
- Preferred tools can be used: Skype, Hangout, GoToMeeting, etc.
- Ideal for testing prototypes at any level of maturity (LoFi – HiFi)
- Combination of user test and open interview. Especially the hacking in case of unexpected events is an advantage.
- Testing of unpublished apps and websites.
- Ideal for the RITE methode
- Time expenditure. A moderator and optionally an observer is necessary for the test.
- Difficult target groups are poorly available during normal working hours. Closing time appointments are often unavoidable.
- Suboptimal for testing touch interfaces.
Unmoderated user testing In an unmoderated user testing setting, there are no real-time interactions between the test person and the researcher. Instead, the tasks are created before the test and usually completed by the test participant through an online testing tool.
- Testing in a natural environment without the influence of a moderator.
- No moderator is necessary at the same time. Results come back within hours via video.
- Video analysis can be performed efficiently afterwards.
- Test object must be mature. Ideally an operational website or app.
- Precise writing of instructions/tasks is necessary, otherwise, the test subject gets lost on his own.
- No hacking possible during the test. I.e. interesting events cannot be traced back immediately.
- In case of technical problems with the test infrastructure, no support can be offered, since the test subject is on his own. It often blocks the execution of the complete test.
Which procedure you finally want to use depends strongly on the test design and the available budget. For the above-mentioned platforms, you can also buy the whole service (planning, execution and analysis) from the provider. For detailed instructions on moderated user tests you can download our guide:
If you have any questions about choosing the appropriate test method, please feel free to contact us at firstname.lastname@example.org.