Table of contents
1. Why to embrace remote user research
2. What is a remote user research study?
3. Why is it important to leverage remote research tools?
4. Three case studies we completed recently
4.1 Case study #1: Uber Pro Beta Research
4.2 Case study #2: Target in the US
4.3 Case study #3: Uber Money
5. Getting practical, what are some use-cases remote research tools can be helpful?
1. Why to embrace remote user research
A big part of the world is spending more time at home, changing their routines, accessing more online services and missing human connection with those outsides of home. This is the perfect time for innovation. In order to continue to bring in user needs to our teams, we researchers need to also adapt our practice. As it becomes more challenging to meet participants face to face, I suggest we embrace remote user research. Here are some of the lessons learned along my remote UX Research journey.
2. What is a remote user research study?
A remote user study is a type of qualitative research where the researcher and the user (the participant we want to learn from) do not meet in person. Based on the relationship between user and researcher, there are two types of remote studies:
- Remote moderated studies are often video-calls where the researcher interviews the user and adapts the session based on the user’s answers or experience.
- Remote unmoderated studies are a type of research where the user completes a set of tasks, or, answers a set of questions based on fixed questions or instructions. Unmoderated studies can range from usability testing to diary study or co-creation. The user normally shares answers in the shape of open/closed text answers, pictures, videos or screen recording on how they use a product or prototype.
3. Why is it important to leverage remote research tools?
Remote user studies and remote research tools have many big advantages. Some of the important reasons why it should be taken into consideration are the following ones:
- Increase efficiency and reduce project overall time by up to 50% compared to in-person studies.
- Save money and travel while gaining a wide geographical reach.
- Ability to gather real-time and contextual feedback to help your stakeholders easily empathise with the users.
- Standardise replicability in order to compare across markets or user segments.
4. Three case studies we completed recently
There are some challenges we’ve experienced with recently conducted remote user studies and of course some lessons learned, which we would like to share with you.
4.1 Case study #1: Uber Pro Beta Research
Image Credit: Uber
The challenge: Uber launched Uber Pro in November 2018, a Rewards program aiming to recognise drivers’ quality and commitment. For the first time in Uber’s history, the program was fully integrated into the driver app, spread across more than 30 screens. The team knew that gathering early feedback from drivers in different tier-levels would help us learn and iterate the MVP (Minimum Viable Product) quickly.
The research process: Eduardo and his team carried out a multi-method research study to gather drivers’ feedback on the program. We launched a Diary study in dScout that lasted for 5 weeks to understand the impact of Uber Pro in drivers’ experience, motivation to drive and sentiment towards Uber. Drivers were prompted to answer three dScout parts each week (planning at the beginning of the week, reviewing at the end of the week, and a deep dive on a specific reward).
Despite how comprehensive it was, the most revealing part was a “bonus question” that was optional for drivers to answer any time: “Did you just have a moment related to the rewards program that you want to share with us? Describe where you are and what happened. Be as specific as you can. The more entries, the better!”
Drivers would share their delight and appreciation right after enjoying an airport priority dispatch, their frustration with a trip cancellation at 2am in the night or a reflection when getting home after a full day of driving.
The outcome: Based on the feedback gathered (in combination with other research methods), the team identified and fixed a number of UX problems, renamed some of the rewards, launched special email communications and a push notifications campaign aimed to complement product educational tooltips, adapted the eligibility criteria and prioritised product roadmap to improve program awareness and comprehension.
Lessons learned:
- The Diary study uncovered 60% of the issues we learned later via focus groups and in-depth interviews
- It took too long to analyse. We should have divided the study into smaller, lighter studies – it doesn’t have to be 5 weeks, 50 drivers all at once!
- We could have limited the duration to 3 weeks (instead of 5 weeks) because we did not see many incremental changes after the third week
- Limit the number of participants to make sure there is enough time to analyse their answers and compare them
- Give more accurate instructions on video quality so you can use their answers from day 1
- Encouraging drivers to answer questions via chat proved to be effective an insightful (demonstrating to users we care for their answers)
- The analysis should be done within the platform to avoid manual copy-pasting and increase efficiency. Consider having an analysis-focused training with all researchers involved
- Multiple choice questions helped us have quick results to share along the way
- If more than two people were analysing, it’s advisable to have 1-2 researchers overlooking the quality of the analysis
- We lost 35% (15 drivers out of 50) drivers from sign-up to download the app, so we had to recruit more users during the first week
4.2 Case study #2: Target in the US
Image Credit: Target
The challenge: The Tech team was based in India but they needed insights about Target consumers in the US and had constraints to travel due to lack of budget.
The research process: Naman Mathur created a process that combined data analytics and remote qualitative research to inform new brand launch and performance.
The outcome: Insights reports that combined data and qualitative insights to what wasn’t working and why. They were valid to help specific product teams but did not help in identifying broader product opportunities.
Lessons learned:
- Naman: “It was my first experience as a researcher and combining data with qualitative insights was very powerful to create user stories”
- These methods lacked the ability to build empathy with our users, especially regarding observation of broader product usage. This led to adding one new method: remote observations via a local US agency, which combined foundational questions like product discovery and motivations with participants using a website.
4.3 Case study #3: Uber Wallet
Image Credit: Uber
The challenge: Testing an early prototype for the Money team in the US.
The research process: Moderated remote usability testing. I used Zoom to have a conversation with screen sharing to show an early prototype that was quite new to them.
The outcome: It felt pretty natural because I facilitated the transitions between screens.
Lessons learned:
- The lack of familiarity with your product works better with moderated research
- Faster and more efficient, keeping your lifestyle more balance without traveling
- “I was so used to doing in-person research that oftentimes I didn’t consider remote research. Remote research becomes powerful when you do a lot of product iterations”
- After piloting it, I saw how participants used it from their cars and felt more contextual.
- Less preparation and less unknown “You can do a lot more in your day at office and did not require to mentally prepare for a whole day of research”
5. Getting practical, what are some use-cases remote research tools can be helpful?
In the following cases remote research tools can be the solution.
- Diary study to map the user experience of a product or service
- Diary study to gather contextual feedback after launching a new product/program/feature
- Diary study to sensitive and screen user participants for a particular study
- Video-conference to do a remote usability testing or test your design research approach
- Online communities to facilitate customer co-creation of new initiatives
These are some of our experiences. Have you used remote research tools? Share with us your experience or questions!
