Table of contents
The four criteria and key questions for identifying suitable UX methods
- Phase within the product development process
- Formulating the question
- Goals and project results
- Qualitative vs. quantitative UX methods
The most common UX methods – in a nutshell
- In-depth interviews
- Focus groups
- Online surveys
- Diary studies
- Field studies
- Card sorting
- Expert review
- Lab usability test
- Online/crowd usability test
- Lab eye tracking
- Virtual eye tracking
- A/B testing
UX methods sorted by phase in the product development process
Summary of the key steps for selecting methods
Continue reading online or download the eBook as a PDF:
Introduction
For some user researchers, involving both customers and users in a user-centred development process and creating the best user experience possible for a given product is already a reality. For others, it remains a difficult task. It is understandable that for many user researchers, the path to the right UX method is not always immediately clear. This is due to the large number of often complex UX methods.
This guide offers concise guidelines to quickly assess the general suitability of UX methods for your research project. As such, this guide contains a brief description of the most common UX methods in a condensed form. It summarises their typical areas of application, special features, strengths and weaknesses.
You will also learn about the four key selection criteria and core questions to quickly and easily identify appropriate UX methods. Which UX methods are appropriate at which stage of the product development process? And what type of results can you generate?
In addition to this guide, you will also receive as a bonus:
- a printable chart showing the different phases in the product development process. This includes the four criteria for identifying suitable UX methods.
- a printable spreadsheet of UX methods. It clearly summarises the typical areas of application and strengths and weaknesses of the respective methods.
Happy reading!
The four criteria and key questions for identifying suitable UX methods
Which research methods are most suitable for generating the most useful insights within UX projects? It’s a good question, and a seemingly difficult one at that. Essentially, identification of suitable UX methods is based on four simple criteria and core questions that build on each other:
The four most important criteria and core questions
- Phase in the development process – Where am I currently?
- Formulation of question – What is it that I want to find out?
- Goals and project results – What type and depth of knowledge do I need?
- Qualitative vs. quantitative – Which methods deliver the best insights?
Phase in the product development process
The first crucial factor is the phase of the product development process in which you are currently – because not every research method is suitable for every stage. So ask yourself at the very beginning: ‘What stage am I at right now?’ Are you in the idea, concept or design phase for the new development of a digital product? Or are you in the process of maintaining an already launched product and want to optimise it?
The four phases of the product development process
Formulating the question
After you have identified the current phase of the product development process, the second key question follows: What exactly am I trying to find out? Let’s assume that you already have a number of hypotheses about user needs or usability for your product. These hypotheses will ultimately lead to specific questions.
Example questions for each phase of product development
Note: The more specifically your hypothesis-driven questions are defined, the easier it will be to assign suitable UX methods to your project.
Goals & project results
What exactly would you like to achieve with your research project or product development in general? Do you want to understand the needs of your users better and develop personas for a better collective insight? Do you want to explore the market potential of a new product or business idea to minimise innovation risks? Would you like to find out how you can use the strengths of your relevant competitors to your advantage?
The possibilities are endless. Think about where you want the path to lead you.
Qualitative vs. quantitative UX methods
If you have hypothesis-driven, specific questions based on your goals and your current phase in the product development process, you are almost ready to start assessing the suitability of UX methods for your project. But first, we will discuss why and how a distinction is made between qualitative and quantitative methods.
The main differences between qualitative and quantitative UX methods are the observation methods, the type and extent of user researcher influence, and the type of insights and results generated.
In addition, some UX methods can be considered hybrid since they have both qualitative and quantitative properties. Depending on the focus of the research, qualitative and quantitative methods can also be combined; e.g. to validate knowledge gained about the behaviour of users (qualitative) with reliable figures and data (quantitative).
Direct comparison of the characteristics of qualitative and quantitative UX methods:
| Qualitative | Quantitative | |
| Sample size | Fairly small sample size (approx. 10-50 participants) | Larger sample size for statistically reliable results (from 50 participants) |
| Type of observation | Direct observation of behaviour by a UX researcher | Indirect observation through data acquisition and analysis tools |
| Direct influence by a UX researcher | The UX researcher can ask the user questions at any time during application of the method. | The UX researcher cannot ask the user any questions during application of the method. |
| Type of findings | Answers the ‘why’; i.e. the reasons behind the behaviour that leads to a certain effect. | Answers the ‘what/how much/how often/how large/how strongly’; i.e. the efficiency areas and the degree of efficiency. |
| Type of results | Behavioural analysis (needs, motivators, concerns, problems etc.) | Hard numbers and data |
| Data analysis | Not mathematical | Mathematical analyses |
| UX methods | In-depth interviews Focus groups Field studies Expert review Lab usability test | Online surveys Virtual eye tracking A/B testing Click tracking and analysis |
| Hybrid methods | |
| Online/crowd usability test, Card sorting, Diary studies |
The most common UX methods – in a nutshell
In-depth interviews
In-depth interviews are individual discussions with members of the target group. They are used to reveal users’ core needs, motivating factors and concerns, which in turn can be used to derive the added value of the product/tool and the concept and design requirements.
| Type | Qualitative |
| Areas of use | Understand, describe and become acquainted with the user. Create personas and customer journey maps. Define use cases, scenarios, define touchpoints, MVP requirements, assess market potential. |
| Special feature | Can be done in person, by phone or video call. |
| Strengths | Through targeted questioning, it is possible to gain a deep understanding of the person, their motives, problems and challenges. |
| Weaknesses | Demands a high level of interview skill on the part of the UX researcher. No validity due to the low number of interviews (additional statistical validation required). |
Focus groups
Focus groups are moderated discussions with up to 10 participants; their aim is to obtain a variety of user opinions on a particular subject. In order to better evaluate the results, the discussions are recorded with a video camera and/or a note-taker.
| Type | Qualitative |
| Areas of use | Understanding, describing and becoming acquainted with the user. Creating personas and customer journey maps. Defining use cases, scenarios, customer journey, defining touchpoints, MVP requirements. Assessing market potential. |
| Special feature | Particularly suitable for studies in the early idea/analysis phase in order to gain insights into the required features of the product. |
| Strengths | Uses group dynamics to creatively generate new perspectives, opinions and ideas. |
| Weaknesses | Individual opinions can get lost in the group; group gravitates towards the opinion leaders. |
Online surveys
Participants complete an individually tailored questionnaire online. Online surveys are suitable for quantification and validation of qualitative study results, for example. The statistical data enables a precise description and comparison of the target groups on the basis of mean values, proportions, correlations, etc. Differences between groups can be tested for statistical significance.
| Type | Quantitative |
| Areas of use | Validate hypotheses, test advertising effectiveness. |
| Special feature | Can be carried out at any location and across all devices (desktop and mobile), immediate availability of data, diverse analysis options. |
| Strengths | Provides statistically reliable results, allows explorative analysis, insights about sub-groups, relatively fast results. |
| Weaknesses | Not so suitable for generating hypotheses. |
Diary studies
These are long-term studies in which participants record their thoughts, opinions, attitudes and needs about the use of a product in everyday situations. Diary studies provide the opportunity to record changes in usage behaviour over time.
| Type | Hybrid |
| Areas of use | Record of the long-term use and optimisation potential of a product. |
| Special feature | Record of changes in usage patterns over time. |
| Strengths | Captures information on usage habits and changes in usage. |
| Weaknesses | Requires a comparatively large amount of time and effort. |
Field studies
The UX researcher observes directly how users and customers search for, try out, buy and use the product under real conditions. Among other things, field studies allow detection of undiscovered innovation potential, thus establishing decisive competitive advantages.
| Type | Qualitative |
| Areas of use | Testing and evaluating physical products to optimise the user experience. |
| Special feature | Find, use and buy a physical product; e.g. in a retail shop. |
| Strengths | Captures real, authentic user experiences. |
| Weaknesses | During covert observation, subjects may feel observed and behave differently. |
Card sorting
Card sorting is useful for the redesign and optimisation of navigation structures to ensure that relevant content from online shops, websites, apps and software can be found quickly. Test users define the top navigation category and optimise the names of the navigation options, grouping related options together. The result is a navigation structure that is understandable from the user’s point of view, which improves the user experience.
| Type | Hybrid |
| Areas of use | Development of user-friendly navigation for websites, online shops, apps and software. |
| Special feature | Maps the users’ mental category and concept systems. |
| Strengths | Possible to combine qualitative and quantitative data. Requires only a small number of users. |
| Weaknesses | Without card sorting software, documentation and evaluation of the results is very time-consuming. Online card sorting makes the evaluation easier, but there is no way to observe the behaviour of participants or clarify comprehension questions. |
Expert review
In an expert review, usability experts analyse websites and apps using established usability and design principles in order to identify weaknesses and to discover potential for optimisation. Usability experts use their knowledge to provide specific solutions for optimisation or redevelopment of the product.
| Type | Qualitative |
| Areas of use | ‘When usability tests are not possible. Suitable for complex user interfaces.’ |
| Special feature | Delivers comprehensive analyses of all relevant conversion factors, such as design, content, usability, etc. |
| Strengths | Combines the use-case-driven use of the interface with the experience and expertise of UX experts. |
| Weaknesses | No ‘real’ user feedback. Some usability problems will be found only by real users; thus, combination with a usability test with test users is recommended. |
Lab usability test
In a lab test, UX researchers uncover usability problems by directly observing and questioning users. The client can follow the test from an observation room. When combined with eye tracking, it also reveals which areas of websites or apps attract the most attention.
| Type | Qualitative |
| Areas of use | Test interfaces, products, prototypes, designs. |
| Special feature | Can be easily combined with eye tracking. |
| Strengths | Holistic observation of users and their behaviour. Users’ problems are better understood through targeted enquiries. |
| Weaknesses | Artificial usage environment in which users may behave differently. |
Online/crowd usability test
Participants test online shops, websites, apps, etc., directly from home in their natural environment. The screen and the voice of the user thinking out loud are recorded via a tool to uncover weak points and potential for optimisation in the user experience.
| Type | Hybrid |
| Areas of use | Test prototypes and user interfaces (websites and apps). |
| Special feature | ‘Up to 90% cheaper than traditional lab usability tests with the same results.’ |
| Strengths | Natural usage environment. Recordings can be accessed online at any time for subsequent viewing. Large sample sizes possible. |
| Weaknesses | Compared with a lab usability test, no influence or direct query is possible during execution of the online usability test. |
Lab eye tracking
Laboratory eye tracking measures users’ eye movements to determine whether relevant messages and interaction elements are perceived by users or whether unimportant elements distract from important ones. Eye tracking is often combined with a classic lab usability test.
| Type | Qualitative |
| Areas of use | For determining which elements are seen in the interface and receive the most attention. |
| Special feature | Shows whether important elements are adequately perceived. |
| Strengths | Allows detailed analysis of eye movement and behaviour. |
| Weaknesses | Eye tracking alone does not allow conclusions to be drawn as to WHY a respondent has or has not seen an element on the screen. Evaluating the results is very time-consuming. |
Virtual eye tracking
Unlike laboratory eye tracking, which is much more complex, the results of virtual eye tracking are simulated by modern software tools. The results obtained after a few seconds are accurate to over 90% compared with laboratory eye tracking, and therefore represent a cost-effective alternative.
| Type | Quantitative |
| Areas of use | Landing page and home page tests. |
| Special feature | Shows the visual perception and attention during the first seconds. |
| Strengths | Results are equivalent to a lab eye-tracking study. Cheaper than eye tracking in a lab. |
| Weaknesses | The results from virtual eye tracking are meaningful only for landing pages and home pages. Analysing scrolling behaviour and the corresponding eye movement is rather difficult. |
A/B testing
In A/B testing, also known as split testing, two or more variants of a website or tool are tested against each other to identify the version that performs best. However, in order to obtain valid results, a lot of traffic or a large group of respondents is required.
| Type | Quantitative |
| Areas of use | Conversion rate optimisation, testing of interface variants. |
| Special feature | Enables an increase in conversion or turnover with the same volume of traffic. |
| Strengths | Quantitatively measures the behaviour and behavioural preferences of users. Statistically sound results. Can be valued on a monetary basis. |
| Weaknesses | ‘Does not provide any insight into WHY users behave, just how they behave.’ |
UX methods sorted by phase in the product development process
Diagram of UX methods
Now, you have a good overview of the areas of application and the strengths and weaknesses of the most common UX methods. The following diagram shows the different methods broken down according to their suitability for each phase in the product development process.
Download the eBook so you always have a reference work:
Summary of the key steps for selecting methods
Finally, let’s review the four key criteria and questions that you can use to identify suitable UX methods for your project.
- Product development phase – Where am I currently?
Make yourself aware of which phase you are in. The phase will help you start to narrow down the UX methods you need to consider. - Formulating the question – What is it that I want to find out?
Define your hypothesis-driven questions as specifically as possible to further narrow down suitable UX methods for your project. - Goals & project results – What kind and depth of knowledge do I need?
Make yourself aware of what exactly you want to achieve with the research project or product development. - Qualitative vs. quantitative – Which methods provide the best insights?
Consult the method overview to see which observation methodology, and which type and extent of UX researcher influence generates which insights.
Of course, some real-world experience is also needed to select the right methods for your UX project quickly and reliably. But even with limited experience, an examination of these four criteria and core questions will make selecting suitable UX methods in your everyday project work much easier.
