Tabe of contents
2. From on-site studies to online tests
2.1 New design of the photo product for Android smartphones
2.2 Structured approach with a Product & User Experience Team
2.3 Online surveys and interviews to record customer opinions
3. Private Pool with customers for greater customer loyalty
3.1 A mini-project on her own initiative
3.2 TestingTime’s support for the management of the pool
3.3 Development and promotion of the pool
3.4 Less time and less stress with Private Pool
3.5 More insights thanks to more tests
3.6 Better response rates and more flexibility with a pool of their own
Sara is a passionate researcher. Before landing at ifolor, she worked for companies such as Migros, one of the forty largest retailers in the world, and Jung von Matt, an international advertising agency. At ifolor, she and another colleague are responsible for customer research. Before establishing this department, a conscious decision was made not to use the term “UX Research”, because ifolor wished to broaden the concept and analyse the entire customer experience, as well as brand perception (see: Comparison between CX and UX).
The international online service provider and specialist in photo books used to conduct customer research even before creating a customer experience research role, but mainly spontaneously, and only on a short-term basis. In addition, external agencies were called in to answer the questions. At that time however, ifolor was already working with TestingTime to recruit study participants, but focusing exclusively on user testing.
At some point this approach was no longer sufficient. ifolor wanted to conduct more extensive testing, learn more about its customers and expand customer research. Nevertheless, Sara first had to establish the research mentality within the company, before it reached its current level. She developed the research strategy, vision and goals. As a CX researcher, she drives the customer-first mentality and is responsible for customer feedback management to allow the development of better products.
By now, this mindset has become firmly anchored within the company, and the research team consists of two people. Sara’s focus is on surveys, workshops and interviews—everything having the aim of testing physical products. The second researcher focuses on usability tests of digital products, from project websites to ifolor applications. Both of them carry out all their studies by themselves. They get external support for very big research projects or to try out new study methods. And they are very familiar with moderated or unmoderated user tests, interviews and online surveys.
Before the pandemic, Sara’s main focus was on on-site user testing. For this reason, customers were regularly invited to have interviews at the premises of ifolor. Sara prefers on-site tests because, in her opinion, they are much more open, uncomplicated and, above all, more personal. To have contact with the customer is very important to her, and conversations are much easier held on site than remotely, especially when testing physical products. Emotions are completely different when private photo books are shown to her in person. Sara also believes that it is possible to obtain more and better information with this type of study. Subsequently, she uses surveys for the quantitative verification of the findings. However, due to the pandemic, they had to switch to online studies.
At the end of last year, the design of the photo product for Android smartphones was launched in the company’s app. Previously, creating photo books via the Android app was not possible. The interface of the mobile editor had to be completely redesigned. After the launch, the research team sent out a survey to all users who had already created a photo book previously. In the online survey, customers were asked for their feedback and input on the new product.
At the beginning of this project, there was a kick-off with the so-called PUX team (Product & User Experience Team). Sara was part of it, as well as a UX designer and a product manager. In total, ifolor has three designers who create the entire UI/UX concepts, two researchers, a digital product manager for mobile applications, and a product manager for physical products. Marketing people or developers are also part of the PUX team for certain projects and studies. After the kick-off and brainstorming, the questions were structured by Sara. Then the ball was kicked back to the team, which gave feedback on the design of the study.
After a few feedback loops, she sent the survey to her customers and newsletter subscribers via Selligent, a multichannel tool offering optimal services to manage B2C marketing campaigns. Sometimes Sara publishes the surveys on ifolor’s social media channels to get even more inputs. However, the most frequently used channel to recruit test participants and get user feedback, is TestingTime, either via the Public Pool or ifolor’s Private Pool. The test users for more than 50% of the studies are recruited via TestingTime.
Interviews are generally carried out over a period of two weeks via Microsoft Teams. Subsequently, Sara analyses the results and writes a summary of the findings. These are then discussed in a smaller group. The result is always an optimisation recommendation, prioritised within the team. The UX designer then makes improvements to the UX concept based on the optimisation recommendations and forwards it to the developers. Finally, a discussion in which the results (findings and improvements) are shown is held with other stakeholders. In the case of larger studies, the result is also presented to the entire company via email.
ifolor has had its own panel of test users for almost a year now. Sara and her colleagues decided to implement it because they wanted to test more complex products with ifolor customers. The idea was to build up a pool of people to accompany ifolor over a longer period and provide in-depth insights to the team. The time saved because there was no need to search for new users every time was also a decisive reason to set up a test user pool of their own. Thus, the so-called “Research Lab“ project was created (Private Pool). When it comes to product development, ifolor attaches great importance to the opinion of existing customers, besides having a loyal customer base. It pays off, because customers identify themselves with ifolor. They have voluntarily registered for the Private Pool or the Research Lab, as ifolor calls it.
Sara started the Research Lab project over a year ago, before the pandemic broke out. In those days she had enough time on her hands to deal intensively with the concept. Within a very short time, she set up a landing page. This is kept short and concise, with a call to its readers to participate in ifolor’s research projects. When designing the landing page, she got inspiration from other companies such as Google, Migros, Galaxus or Tamedia. TestingTime was not yet involved at this point.
All data was collected in an Excel file. At some point, Sara realised that there were more people wanting to participate than she had originally expected. She couldn’t and didn’t want to manage this number of people on her own and, above all, not manually. Screening the pool and manually dispatching invitations took a lot of effort.
At exactly this time, TestingTime launched its Private Pool product. Perfect timing! Practically immediately, Sara talked about this with her supervisor, suggesting the outsourcing of the management of the pool to TestingTime. Previously, she examined various private pool management solutions. All of them were either too big or they had too many features that she didn’t need. Or they were too expensive. Thus, TestingTime was practically a “no-brainer” when making the decision. Private Pool was the perfect solution with an exclusive focus on managing their own pool, i.e. the members of the Research Lab.
ifolor has implemented a large number of promotions to make its pool grow. A teaser with a link to the Research Lab landing page was included in the customer newsletter. Most recently, a QR code was created, printed on physical flyers and enclosed with the photo books when shipped. ifolor is a very emotional product, but ultimately still physical. The flyer is more popular with the older aged people within the target group than online advertising. The Research Lab was also promoted via social media (LinkedIn, Instagram, Facebook). Finally, there is the pool’s landing page, with a backlink on the main ifolor.ch page.
Sara already knew TestingTime from her time at Migros. In her opinion, saving time is the greatest benefit obtained by means of the service provided by TestingTime. She no longer has to worry about administrative tasks such as invitations or assigning study participants, nor about making payments. In addition, TestingTime has proven experience and knows very well:
- what works best and what doesn’t;
- what the best approach to write to people is;
- what the optimum number of people to invite to the study is (advises her).
Sara really appreciates TestingTime’s expertise. In addition to the Private Pool, access to an even larger pool is another advantage (TestingTime’s Public Pool). ifolor never has the resources needed to build up such a large, high-quality pool.
Since having a Private Pool, ifolor conducts tests much more often. The reason for the more frequent testing is that it is less complicated and faster than before—when they had to manage the pool themselves or it had not even been created. This was exactly the reason for setting up a Research Lab of their own: to do more testing, and more continuously. Even if it’s only short online surveys that last a maximum of five minutes.
In addition, they use unmoderated online user tests to obtain more frequent feedback on smaller prototypes. This means they conduct testing across all phases of the product development process. The designers are glad that they can get a brief feedback on the concept before they finish building the product. It did not happen very often in the past. The customers in their pool like to spend time testing and are very interested in being part of the development process. They think it’s great that ifolor conducts user studies and asks customers for their opinion. Sara receives this feedback again and again during the interviews. The customers want to share what they think is great and what ifolor could do better. Conducting more tests also means more insights for optimising the customer and user experience.
3.6 Better response rates and more flexibility with a pool of their own
By now there are 500 customers in the Private Pool. The response rate for surveys is above 50%. This is an extremely high figure, and compared to other channels, the Private Pool performs best. When they send a survey to all customers via the newsletter, the response rate is only about 2-3%. For Sara, this is a confirmation of the fact that there are absolutely loyal customers in the pool, wanting to participate and being extremely interested in helping to shape ifolor.
In addition, Sara enjoys a lot of independence and flexibility with the Private Pool. If she were to send the surveys only via direct mailings, she would have to coordinate with the marketing department every time. But this way she can easily send the survey to Private Pool members without being dependent on anybody, regardless of what other campaigns the company is running. This comes in very handy for Sara, especially for mini-studies in which the UX designer only needs very brief feedback.
Sara’s tip for all readers is that they should take the time to establish the minimum level of UX within the company. It is important to have a clear strategy and to clarify the following questions:
- What do you want to achieve?
- How much/often do you want to test?
- What is the purpose of these tests?
It is also important to esnure good communication about the studies currently taking place. You have to announce the result and implement the insights gained with the customers’ feedback. If you do this again and again, at some point there will be acceptance and a better understanding of the process within the company. Over time, more and more stakeholders at ifolor are also approaching the researchers. They have diverse questions and want to conduct studies to answer them. The importance of UX and customer research is being increasingly recognised.
A very important point is to always keep it rolling. It takes time to build up your own pool, and when you do, it should be used actively. It is important to find a balance between regular testing and not overtaxing the user. Sara’s advice: Make a plan of when to test something and with whom.