Table of contents
1. Product development with user research based on scrum
1.1 From Wayfinder to Product Researcher to Product Owner
1.2 Empowering other teams
2. Getting started with Research Ops
2.1 Discovery research and usability testing
2.2 Use of several resources
2.3 Presentation of the findings
3. Suitable and unbiased test users
3.1 Challenge: No-shows and misfits
3.2 Solution: Uncomplicated recruitment
3.3 Transparent communication and ongoing development
1. Product development with user research based on scrum
sipgate’s headquarters are in Düsseldorf, with 250 employees being currently active there. They develop innovative fixed line and mobile communications products both for companies and private individuals. As part of their business, they create the interfaces between the Internet and telephony.
Their products are developed following the Scrum Principle in 2-week sprints. A scrum team always consists of a product owner, a scrum master and a development team (dev team). In David’s case, the dev team consists of four researchers, and David himself is the product owner of the research team. As such, he has primary responsibility for the team’s research work. He is the one who decides what questions are to be answered and he is also responsible for the Backlog Refinement.
At the start of each sprint, everybody gets together for planning. When doing this, the team tries to plan as many tasks as can be managed realistically. The researchers are responsible for their implementation. They define how they want to answer the questions and which are the most sensible methods to answer them. At the end of each sprint there is a sprint review and a retrospective takes place to discuss how things went and what should be done differently next time.
1.1 From Wayfinder to Product Researcher to Product Owner
David started as a wayfinder at sipgate. As the job title suggests, he had to find his way at sipgate – in a completely open-ended manner. Previously, as a university student, he got involved intensively with Design Thinking and studied communication science. All the methods to be mastered by a researcher were therefore very familiar to him from an academic perspective. In addition to his studies, he also acquired start-up experience in the field of digital products.
He landed at sipgate because both he and the company thought they were a good match. Only the perfect role for him still had to be found. This crystallized in the Wayfinder programme within one year. With this programme, both employee and employer find out within one year which job makes the most sense. In David’s case, it was ultimately the newly created role of researcher.
Following the programme, David worked for six months in various teams as a product researcher. Four more people were hired at a later point and today they work as an independent research team.
1.2 Empowering other teams
Of the 250 people who work at sipgate, around 20 to 25 are employed as UX designers and 5 of them are researchers. Compared to other companies, this is a comparatively large number.
But at sipgate, many departments and roles are interested in user research. Especially designers, who regularly carry out usability tests. But also product owners wishing to have a better understanding of the use of their products and test the latest developments. They all approach the UX research team with diverse questions, causing it to consider what they are able to do by themselves and when they may have to empower other co-workers to conduct the research required. For this purpose, they support them in selecting the right method, provide input on questions to be posed and more.
sipgate’s team at work (Source: www.sipgate.de).
2. Getting started with Research Ops
Some research was already being done at sipgate before Davig started working there. However, at that time, this practice was extremely unstructured. David’s role as a UX researcher definitely brought more methodology to the company. Previously, there were no dedicated people responsible for UX research. The UX designers felt rather responsible for this area. But they had to focus on the design of the products and were not able to take care of the implementation of the research work.
David’s first responsibilities were therefore clear: To define strategies, set up structures and processes and select the right tools, i.e. to develop everything related to Research Ops (also called ResearchOps, ReOps, or Research Operations) and everything necessary to support the researchers of an organization in deployment and scaling. Without this cornerstone, it is very difficult to conduct good research.
2.1 Discovery research and usability testing
The research team conducts exploratory research for two to three months on larger issues. The purpose of exploratory research is the investigation of a problem that has not been defined clearly. For this kind of research, the team starts with a general idea and identifies topics that will be focused on in future research tasks.
An important aspect to be taken into account here is that the research team must be ready to change direction as new data or insights emerge. And this is something that works very well in the case of David and his team. They structure their work using a Discovery Canvas and work on it before it is released to the product team for implementation.
This entire research work involves six to seven weekly user interviews. And possibly this happens during four to five weeks. This is a very intense but also illuminating time for the team, which likes to include a twist in the user interviews. The research team asks users to arrange things on a Miro Board to see how they are imagining the structure. And currently they are thinking of starting to use Mentimeter for scale representation.
Each product team includes one or two UX designers who carry out usability tests after this stage. The research team provides support during this process. They make available everything needed and ensure that the UX designers can do their testing with suitable people. In addition, they check from time to time whether the usability tasks are meaningful and if not, they optimize them.
2.2 Use of several resources
The researchers at sipgate use various tools and services for their user research. Lookback is used for moderated studies and RapidUsertests for unmoderated ones. Miro is used when things get a little more interactive and Figma is used for prototypes. sipgate uses the TestingTimes recruitment platform to find suitable study participants. And for surveys they use SurveyMonkey.
David and his team do most of their work in pairs. It does not matter whether the task on hand is the creation of interview guidelines, the implementation of interviews or the evaluation of a study. They believe that the greatest advantage lies in that the quality of their research is significantly higher this way. In addition, it never happened yet that the research team could not continue working because of a lack of information. But this is something that can happen quickly if, for example, attention is not paid for a brief moment during an interview or if someone is out enjoying their holidays.
2.3 Presentation of the findings
As part of the sprint reviews, the research team gives presentations on the findings of the study. Subsequently, it will be decided what the next steps will be. To document the findings, David and his team have created a research repository using a Google Sheet. The research repository is a joint overview of all past studies with a reference to the insights gained. This allows all teams to easily recover insights from past projects.
In their dataset, the team describes the target group tested, summarizing the three most important findings. They also indicate for which team this research was conducted. Often, they create summary videos, because these are more likely to be viewed than a classic research report. Links to further details are also included. For data privacy reasons, they usually record themselves and talk about the findings, instead of showing the actual users.
With TestingTime, whatever documents that are required can be signed by the test users during the ordering process, before their participation in the study starts. For example, a non-disclosure agreement, or a consent to record the test. Subsequently we integrate the document to be signed directly into our recruitment process. Every document is legally signed, including the date, the full name of the test user and their electronic signature. All signed documents can be accessed in the order overview before commencing the test.
Presentation of the results during the review (Source: www.sipgate.de).
3. Suitable and unbiased test users
In the past, sipgate recruited study participants via Facebook, as well as family members and friends. At the start, this worked with some difficulty, but after a while it was no longer possible. It took too much effort and actually, for 2-week sprints, recruitment must be fast. In addition, these people were rarely part of the actual target group and were biased, since over time the same people participated once and again.
3.1 Challenge: No-shows and misfits
The biggest challenge for sipgate when doing research were the so-called “no-shows”, i.e. people who simply do not show up for the study. And then there were the “misfits”, i.e. the profile didn’t fit or the participant had technical problems when conducting online studies. Occasionally a microphone would not work or a browser would not be updated. For this reason, sipgate conducts technology checks or delivers goodie packages to the test users in advance, to ensure that they will turn up for the test and that everything will work fine.
3.2 Solution: Uncomplicated recruitment
Prior to using the TestingTime platform, sipgate worked with a recruiting agency. But they were not completely satisfied with the service and there was often a lot of going back and forth. The process leading to the order confirmation took far too long and was too complex. David values a lot the fact that with TestingTime many questions are dealt with straight away using the order form. There is no need for unnecessary email rounds to get to the desired test users.
sipgate considers that one of the great advantages of using TestingTime is the fact that it is possible to recruit people who do not know the company yet. Otherwise it would not be possible to reach the target group so easily or far too many financial and human resources would have to be invested in, the same as for the original attempts. Above all, they do not want to do the recruitment by themselves via Facebook & Co., because they are not good at it and it is not part of their core business either. David believes that they would never be able to keep up with TestingTime’s expertise and on top, they would have to sacrifice quality.
Nevertheless, sipgate manages its own test user panel with approx. 1,000 people – all of them sipgate clients. The people of its own panel are sipgate fans, which is the reason for their signing up for the panel. But for certain studies, this can be a disadvantage, and for new products, it makes sense to conduct tests also with potential new users. Therefore, sipgate uses TestingTime’s public panel for about 50% of its studies.
3.3 Transparent communication and ongoing development
What David particularly appreciates with TestingTime is transparency. He is constantly and correctly informed about the status of the recruitment via his client dashboard. He is also registered as a test user with TestingTime and knows how the whole process works from both ends.
Just as sipgate continuously develops its products, so does TestingTime. We regularly conduct user research on our own products to ensure the best possible user experience. This is another asset that David thinks is absolutely great. He recently noticed that it is now possible to create screener questions yourself. David values this product feature very much, because it sometimes takes telephony market expertise to create certain screener questions for the studies. He and his team are happy that they can take this over themselves if they want to.
As a test user in our customer pool , he recently participated in a TestingTime study. A new feature was being tested which he was very enthusiastic about and which will make life as a researcher easier for him and many others (register here to learn more about this).
David is curious and advises his UX colleagues to also follow this approach. His tip is to do research on a daily basis. Also in private life. Just ask an Uber driver why they are registered with Uber and not with another provider. Or when buying popcorn at a cinema, ask what is being sold more: sweet or salty popcorn? After all, practice makes perfect.
