Table of contents
1. Survalyzer provides answers and TestingTime test users
2. Methodological approach to market research and UX research
3. Quality in the offer, quality in the experience
4. Clients from market research, user experience and customer experience, as well as innovation departments
5. Templates for quantitative UX research to measure UX KPIs
6. UX Toolbox for free quantitative UX research
1. Survalyzer provides answers and TestingTime test users
Christian Hyka has been Managing Director of Survalyzer for almost 10 years. He is responsible for strategic product development and corporate strategy, but also supports sales. With its intuitive survey software, Survalyzer supports market researchers in survey creation, sampling and evaluation. The software has great features such as reports in the form of a ready-made PowerPoint presentation at the push of a button, as well as an online dashboard with various filtering options. The product covers all needs concerning the online data collection process to provide the desired insights fast and in an agile manner.
Reto Lämmler, Co-Founder and CEO of TestingTime, created the company in early 2015, together with Oliver Ganz, who acts as Chief Technical Officer. The idea behind TestingTime is to simplify the lives of UX and user researchers. For this purpose, TestingTime takes over the entire test user recruiting effort, thus pursuing the vision of creating a world full of happy users. As CEO, Reto ensures that TestingTime progresses in the right direction.
This is also the topic of the conversation between the two thought leaders from market research and UX. What is the next step in these fields? What is the next step for these companies? Christian and Reto appear to be at home in two different worlds, but actually they are extremely similar. Why? Read on and find out.
2. Methodological approach to market research and UX research
In general, market research is directed at understanding purchasing behaviour and preferences related to products and messages. Its aim is to understand and interpret the factors that lead to achieving a sale. UX, on the other hand, focuses on the interaction between customers and products. Market research reveals the target group and trends for a product or service. UX, on the other hand, explores how to achieve a more desirable user experience for the target group. When reading this, the immediate question that arises concerns the potentials that could be derived from a combination of both fields. And considering their development, could there be a merger sooner or later?
2.1 Quantitative or qualitative market research
Classic market research is a traditional field, while UX research is considered agile and modern. The differentiation between quantitative and qualitative market research has existed for several decades. And user experience research, which is now available in a wide variety of facets, historically originates in qualitative market research.
The quantitative market research that Survalyzer is doing, traditionally sneers a little at qualitative market research, according to Christian. Because when it comes to hard facts for a broad mass of people, quantitative market research, with its big sampling, is required. In this case, a qualitative study with just a few participants is simply not sufficient.
Christian believes that digitisation is a major game changer in connection with a possible combination of both methods. Research tools can bridge the gap, because some of their functions are the same. For example, the way a screener for test users is created is technically very similar to that of an online survey. Hence, Christian feels certain that the boundaries between quantitative and qualitative research will tend to disappear. Because suddenly people are working with the same tools and sharing their experiences. Both domains still have different tools in use, but they will be able to benefit more and more from each other.
2.2 UX researcher or classic market researcher
Reto’s position is similar. In past times, market researchers used to sneer at UX researchers and their approach. Exaggerating a little, quantitative market researchers did not seriously believe that it is possible to tell everyone how tomorrow’s product should be designed with just five user tests. At that time it was a very agile way of working, while market researchers had to rely on methodical correctness following the waterfall principle.
In addition, market research has existed as a study path for several years, and the concept is still the same as it was years ago. UX courses, on the other hand, are something very new. Reto himself obtained a master’s degree in Human Computer Interaction Design (HCID), something that has not been around for very long.
In the meantime, the methods in the world of UX have changed and improved. No longer is there just the simple 5-people usability test—currently various study methods are used. In addition, these have been supplemented by quantitative UX methods. Qualitative results are supported with quantitative validation.
Reto therefore agrees that the worlds of market research and UX research are getting increasingly closer. Not only are some of the same tools used, but there are also parallels in the methods. Ultimately, both intend the same: to offer or develop products and services that customers love, and as a consequence thereof, use and buy them.
3. Quality in the offer, quality in the experience
Quality is a central issue for both Survalyzer and TestingTime—but in a slightly different context. It is a key factor for a great customer experience.
3.1 High-quality results in online surveys
Survalyzer focuses on ensuring quality for the integration of their solution. Quality checks for this purpose are basically carried out automatically and by machine. However, they are constantly faced with challenges, such as the time required to complete a survey. How long does it take to complete the survey? What is appropriate here and what could indicate an unsuitable participant? Another issue are the so-called “straightliners”. Straightlining occurs when survey participants select identical (or almost identical) answer options. This reduces the data quality of the survey results.
Another issue are “trap questions”. Survalyzer provides its clients with a list of questions to intercept such people. It is used to identify respondents who do not pay much attention to the questions in the survey and not only answer the “trap question” itself in a non-optimal manner, but also other questions included in it.
Finally there are the “bad open answers”. This is the biggest problem for Survalyzer and its clients. Because participants sometimes write letter sequences like “afds” in the answer field. Uncovering and eliminating precisely these behavioural patterns is extremely crucial for Survalyzer and its clients to achieve quality.
3.2 High-quality panel with reliable test users
What TestingTime often hears from its clients is how they have had bad experiences with quantitative research panel providers in the past. They wished to use some of them for qualitative research purposes and were disappointed with the results. For Survalyzer, the above problems are a major challenge, and if unsuitable test users are added to the process, it has an even more negative effect on the result.
TestingTime, on the other hand, comes from the world of qualitative research, and it was necessary to make sure from the start that reliable test users were recruited. There was no way around checking people carefully beforehand. Because they have to appear personally for interviews—regardless of whether on site at the customer’s, at a research lab or remotely. When doing so, these people must match exactly the profile requested by the client. In addition, they have to be exactly what they said about themselves during the screening process.
TestingTime has always made the effort to carry out careful quality checks. However, at some point a database size was reached where it was no longer possible to do manual checks, for example, by calling the candidate in person. For this reason and to be able to continue to scale, machine learning was invested in. The team checked whether it was possible to use past responses and data to determine whether they could be rated correctly. You can find out more about this process in the blog posts “Pool quality at scale with the help of artificial intelligence“ and “Great quality and reliability are no coincidence“.
3.3 Survalyzer and TestingTime as a power duo
TestingTime has the same mechanisms in the screener as Survalyzer’s trap questions, response speed, etc. But the algorithm is based on all the information that the test users have provided so far. It automatically calculates whether a test user is reliable and suitable for the respective study.
Our offer would solve the problem for Survalyzer and its clients. If only well-rated and quality-tested people were used for studies, there would be no problem with the quality of the survey results and no time would be lost in sorting out straightliners, etc. The advantage of using Survalyzer in combination with TestingTime is that market researchers can access a qualitative panel like the one provided by TestingTime and use high-end survey software.
4. Clients from market research, user experience and customer experience, as well as innovation departments
Survalyzer serves clients from various industries in the DACH region as well as in the Netherlands. The company boasts a large number of clients from the banking industry, which has high requirements concerning compliance guidelines and Survalyzer specialises in these. Most of Survalyzer’s clients are still market researchers, but this is something the company wishes to change.
TestingTime is active across Europe, with a focus on the DACH region, France and the United Kingdom. At TestingTime, clients can be divided into three different fields: classic market research (the main focus of Survalyzer), user and customer experience, and innovation departments.
Six years ago, there was only the UX designer. Some time later, the UX researcher split off. The UX designer creates the experience and the UX researcher checks the experience. Or even before the UX designer creates the experience, the UX researcher checks what the experience should be like. Therefore, there was also a change at TestingTime, and the focus, although mainly on UX researchers, now also is on product managers, because these are often concerned with product-market fit.
However, innovation specialists and market researchers are also increasingly among TestingTime’s clients. People working in UX and innovation do small and regular projects with us. Market researchers, on the other hand, come with very big projects, but not so often. Agility has always been in the DNA of UX professionals, and market researchers are becoming increasingly aware of agility. For this reason, they also approach TestingTime, as we include agile practices in our offer.
Specialists from the three areas are looking for slightly different answers, and we are very excited about all of them at TestingTime. They have one thing in common though: they are looking for people to validate their idea, product or service. And they all need a survey tool to find answers.
5. Templates for quantitative UX research to measure UX KPIs
With its technology platform, Survalyzer provides easy-to-use templates as an option. This template approach requires application purposes. Last summer, Christian and Reto came up with the idea of creating standardised templates for UX.
5.1 System Usability Scale (SUS)
When evaluating the most important KPIs that are common in the world of UX, they quickly came up with the System Usability Scale (SUS). It offers a fast and reliable method for assessing the usability of design solutions. After a deep dive, Christian realized that it fits Survalyzer’s approach very well. Among other things, he read our various blog posts on how to measure UX (see the blog post in German Make UX measurable & strengthen the UX culture in the company).
5.2 Single Ease Question (SEQ)
After doing some further brainstorming, Christian and Reto decided to build together a UX toolbox with various measuring tools. Thus, the Single Ease Question (SEQ) was added to the SUS. It checks how easily a tool can be used and how easy it is to complete a task. As the name suggests, the SEQ focuses on just a single question.
5.3 Net Promoter Score (NPS)
Finally, they decided on the Net Promoter Score (NPS). The Net Promoter Score uses a standardised survey to determine the satisfaction of a company’s customers. It’s origins can be found in the world of marketing, but surprisingly, it is often used in the world of UX. The SUS is superior to the NPS for the purposes of UX research, but the NPS has become an industry standard because it allows a better communication with stakeholders outside the world of UX.
6. UX Toolbox for free quantitative UX research
With these three tools mentioned above, Survalyzer and TestingTime launched the UX Toolbox. Its purpose is to build the bridge between market research and UX research. We wish to give our customers an advantage by using Survalyzer and TestingTime in combination, thus getting the best out of their studies.
UX has matured extremely from a methodological viewpoint in recent years, and UX measurability is a topic increasingly focused on. The toolbox helps its users to measure UX and to show to their supervisors and other stakeholders that progress and investment in UX is important.
The UX Toolbox with its evaluation convenience feature can now be used free of charge.
