\n2. What is a remote user research study?<\/a>
\n3. Why is it important to leverage remote research tools?<\/a>
\n4. Three case studies we completed recently<\/a>
\n4.1 Case study #1: Uber Pro Beta Research<\/a>
\n4.2 Case study #2: Target in the US<\/a>
\n4.3 Case study #3: Uber Money<\/a>
\n5. Getting practical, what are some use-cases remote research tools can be helpful?<\/a><\/p>\n
<\/a>1. Why to embrace remote user research<\/h2>\nA big part of the world is spending more time at home, changing their routines, accessing more online services and missing human connection with those outsides of home. This is the perfect time for innovation. In order to continue to bring in user needs to our teams, we researchers need to also adapt our practice. As it becomes more challenging to meet participants face to face, I suggest we embrace remote user research. Here are some of the lessons learned along my remote UX Research journey.<\/p>\n
<\/a>2. What is a remote user research study?<\/h2>\nA remote user study is a type of qualitative research where the researcher and the user (the participant we want to learn from) do not meet in person. Based on the relationship between user and researcher, there are two types of remote studies:<\/span><\/p>\n\n- Remote <\/span>moderated<\/strong> studies are often video-calls where the researcher interviews the user and adapts the session based on the user\u2019s answers or experience.<\/span><\/span><\/li>\n
- Remote <\/span>unmoderated<\/strong> studies are a type of research where the user completes a set of tasks, or, answers a set of questions based on fixed questions or instructions. Unmoderated studies can range from usability testing to diary study or co-creation. The user normally shares answers in the shape of open\/closed text answers, pictures, videos or screen recording on how they use a product or prototype.<\/span><\/li>\n<\/ul>\n
<\/a>3. Why is it important to leverage remote research tools?<\/h2>\nRemote user studies and remote research tools have many big advantages. Some of the important reasons why it should be taken into consideration are the following ones:<\/span><\/p>\n\n- Increase efficiency and reduce project overall time by up to 50% compared to in-person <\/span>studies.<\/span><\/li>\n
- Save money and travel while gaining a wide geographical reach.<\/span><\/li>\n
- Ability to gather real-time and contextual feedback to help your stakeholders easily empathise with the users.<\/span><\/li>\n
- Standardise replicability in order to compare across markets or user segments.<\/span><\/li>\n
- <\/li>\n<\/ul>\n
<\/a>4. Three case studies we completed recently<\/h2>\nThere are some challenges we\u2019ve experienced with recently conducted remote user studies and of course some lessons learned, which we would like to share with you.<\/span><\/p>\n<\/a>4.1 Case study #1: Uber Pro Beta Research<\/h3>\n![\"\"](\"https:\/\/www.testingtime.com\/app\/uploads\/2020\/03\/Case-study-1-480x296.png\")
Image Credit: Uber\u00a0<\/em><\/p>\nThe challenge<\/b>: Uber launched <\/span>Uber Pro<\/span><\/a> in November 2018, a Rewards program aiming to recognise drivers\u2019 quality and commitment. For the first time in Uber\u2019s history, the program was fully integrated into the driver app, spread across more than 30 screens. The team knew that gathering early feedback from drivers in different tier-levels would help us learn and iterate the MVP (Minimum Viable Product) quickly.<\/span><\/p>\nThe research process<\/b>: Eduardo and his team carried out a multi-method research study to gather drivers\u2019 feedback on the program. We launched a Diary study in <\/span>dScout<\/span><\/a> that lasted for 5 weeks to understand the impact of Uber Pro in drivers\u2019 experience, motivation to drive and sentiment towards Uber. Drivers were prompted to answer three dScout parts each week (planning at the beginning of the week, reviewing at the end of the week, and a deep dive on a specific reward).<\/span>
\n<\/span>
\n<\/span>Despite how comprehensive it was, the most revealing part was a \u201cbonus question\u201d that was optional for drivers to answer any time: \u201c<\/span>Did you just have a moment related to the rewards program that you want to share with us? Describe where you are and what happened. Be as specific as you can. The more entries, the better!<\/span>\u201d<\/span>
\n<\/span>
\n<\/span>Drivers would share their delight and appreciation right after enjoying an airport priority dispatch, their frustration with a trip cancellation at 2am in the night or a reflection when getting home after a full day of driving.<\/span><\/p>\nThe outcome<\/b>: Based on the feedback gathered (in combination with other research methods), the team identified and fixed a number of UX problems, renamed some of the rewards, launched special email communications and a push notifications campaign aimed to complement product educational tooltips, adapted the eligibility criteria and prioritised product roadmap to improve program awareness and comprehension.<\/span><\/p>\nLessons learned<\/b>:<\/span><\/p>\n\n- The Diary study uncovered 60% of the issues we learned later via focus groups and in-depth interviews<\/span><\/span><\/li>\n
- It took too long to analyse. We should have divided the study into smaller, lighter studies \u2013 <\/span>it doesn\u2019t have to be 5 weeks, 50 drivers all at once!<\/span><\/li>\n
- We could have limited the duration to 3 weeks (instead of 5 weeks) because we did not see many incremental changes after the third week<\/span><\/li>\n
- Limit the number of participants to make sure there is enough time to analyse their answers and compare them
\n<\/span><\/li>\n- Give more accurate instructions on video quality so you can use their answers from day 1<\/span><\/li>\n
- Encouraging drivers to answer questions via chat proved to be effective an insightful (demonstrating to users we care for their answers)<\/span><\/li>\n
- The analysis should be done within the platform to avoid manual copy-pasting and increase efficiency. Consider having an analysis-focused training with all researchers involved<\/span><\/li>\n
- Multiple choice questions helped us have quick results to share along the way<\/span><\/li>\n
- If more than two people were analysing, it\u2019s advisable to have 1-2 researchers overlooking the quality of the analysis<\/span><\/li>\n
- We lost 35% (15 drivers out of 50) drivers from sign-up to download the app, so we had to recruit more users during the first week<\/span><\/li>\n<\/ul>\n
<\/a>4.2 Case study #2: Target in the US<\/h3>\n![\"\"](\"https:\/\/www.testingtime.com\/app\/uploads\/2020\/03\/Case-study-2-1-480x296.png\")
Image Credit: Target<\/em><\/p>\nThe challenge<\/b>: The Tech team was based in India but they needed insights about Target consumers in the US and had constraints to travel due to lack of budget.<\/span><\/p>\nThe research process:<\/b> Naman Mathur created a process that combined data analytics and remote qualitative research to inform new brand launch and performance.<\/span><\/p>\nThe outcome:<\/b> Insights reports that combined data and qualitative insights to what wasn\u2019t working and why. They were valid to help specific product teams but did not help in identifying broader product opportunities.<\/span><\/p>\nLessons learned<\/b>:
\n<\/span><\/p>\n\n- Naman: \u201cIt was my first experience as a researcher and combining data with qualitative insights was very powerful to create user stories\u201d<\/span><\/li>\n
- These methods lacked the ability to build empathy with our users, especially regarding observation of broader product usage. This led to adding one new method: remote observations via a local US agency, which combined foundational questions like product discovery and motivations with participants using a website.
\n<\/span><\/li>\n<\/ul>\n<\/a>4.3 Case study #3: Uber Wallet<\/h3>\n![\"\"](\"https:\/\/www.testingtime.com\/app\/uploads\/2020\/03\/Case-study-3-1-1-480x296.png\")
Image Credit: Uber\u00a0<\/em>
\n<\/b><\/p>\nThe challenge:<\/b> Testing an early prototype for the Money team in the US.<\/span><\/p>\nThe research process: <\/b>Moderated remote usability testing. I used <\/span>Zoom<\/span><\/a> to have a conversation with screen sharing to show an early prototype that was quite new to them.<\/span><\/p>\nThe outcome:<\/b> It felt pretty natural because I facilitated the transitions between screens.<\/span><\/p>\nLessons learned:<\/b><\/p>\n
A remote user study is a type of qualitative research where the researcher and the user (the participant we want to learn from) do not meet in person. Based on the relationship between user and researcher, there are two types of remote studies:<\/span><\/p>\n Remote user studies and remote research tools have many big advantages. Some of the important reasons why it should be taken into consideration are the following ones:<\/span><\/p>\n There are some challenges we\u2019ve experienced with recently conducted remote user studies and of course some lessons learned, which we would like to share with you.<\/span><\/p>\n The challenge<\/b>: Uber launched <\/span>Uber Pro<\/span><\/a> in November 2018, a Rewards program aiming to recognise drivers\u2019 quality and commitment. For the first time in Uber\u2019s history, the program was fully integrated into the driver app, spread across more than 30 screens. The team knew that gathering early feedback from drivers in different tier-levels would help us learn and iterate the MVP (Minimum Viable Product) quickly.<\/span><\/p>\n The research process<\/b>: Eduardo and his team carried out a multi-method research study to gather drivers\u2019 feedback on the program. We launched a Diary study in <\/span>dScout<\/span><\/a> that lasted for 5 weeks to understand the impact of Uber Pro in drivers\u2019 experience, motivation to drive and sentiment towards Uber. Drivers were prompted to answer three dScout parts each week (planning at the beginning of the week, reviewing at the end of the week, and a deep dive on a specific reward).<\/span> The outcome<\/b>: Based on the feedback gathered (in combination with other research methods), the team identified and fixed a number of UX problems, renamed some of the rewards, launched special email communications and a push notifications campaign aimed to complement product educational tooltips, adapted the eligibility criteria and prioritised product roadmap to improve program awareness and comprehension.<\/span><\/p>\n Lessons learned<\/b>:<\/span><\/p>\n The challenge<\/b>: The Tech team was based in India but they needed insights about Target consumers in the US and had constraints to travel due to lack of budget.<\/span><\/p>\n The research process:<\/b> Naman Mathur created a process that combined data analytics and remote qualitative research to inform new brand launch and performance.<\/span><\/p>\n The outcome:<\/b> Insights reports that combined data and qualitative insights to what wasn\u2019t working and why. They were valid to help specific product teams but did not help in identifying broader product opportunities.<\/span><\/p>\n Lessons learned<\/b>: The challenge:<\/b> Testing an early prototype for the Money team in the US.<\/span><\/p>\n The research process: <\/b>Moderated remote usability testing. I used <\/span>Zoom<\/span><\/a> to have a conversation with screen sharing to show an early prototype that was quite new to them.<\/span><\/p>\n The outcome:<\/b> It felt pretty natural because I facilitated the transitions between screens.<\/span><\/p>\n Lessons learned:<\/b><\/p>\n\n
<\/a>3. Why is it important to leverage remote research tools?<\/h2>\n
\n
<\/a>4. Three case studies we completed recently<\/h2>\n
<\/a>4.1 Case study #1: Uber Pro Beta Research<\/h3>\n
Image Credit: Uber\u00a0<\/em><\/p>\n
\n<\/span>
\n<\/span>Despite how comprehensive it was, the most revealing part was a \u201cbonus question\u201d that was optional for drivers to answer any time: \u201c<\/span>Did you just have a moment related to the rewards program that you want to share with us? Describe where you are and what happened. Be as specific as you can. The more entries, the better!<\/span>\u201d<\/span>
\n<\/span>
\n<\/span>Drivers would share their delight and appreciation right after enjoying an airport priority dispatch, their frustration with a trip cancellation at 2am in the night or a reflection when getting home after a full day of driving.<\/span><\/p>\n\n
\n<\/span><\/li>\n<\/a>4.2 Case study #2: Target in the US<\/h3>\n
Image Credit: Target<\/em><\/p>\n
\n<\/span><\/p>\n\n
\n<\/span><\/li>\n<\/ul>\n<\/a>4.3 Case study #3: Uber Wallet<\/h3>\n
Image Credit: Uber\u00a0<\/em>
\n<\/b><\/p>\n