What is a diary study?
A diary study is a market research term and describes the diary-style recording, by (potential) customers, of their real-life user habits and behaviours. Data is recorded over a pre-determined period of time, during which the participants document – at regular intervals, via an app or in writing – how, when and where they use the product under test.
To illustrate how the process works, we have put together a case study with Zalando’s UX researcher, Carina Kuhr. The real-life scenario provides an insight into how Europe’s biggest e-commerce provider prepares, performs and evaluates diary studies.
More about diary studies and other UX methods:
How Zalando performs diary studies
A few months ago, Zalando asked us to provide them with testers to take part in a two-week diary study. The purpose of the study was to observe and understand shopping behaviour and decision-making processes among customers shopping for gifts in as realistic conditions as possible.
Planning phase – the checklist
Carina Kuhr, UX researcher at Zalando, recommends three to four weeks of planning before carrying out a study. During this time, the following happens:
- Internal discussion with stakeholders – what is the aim of the study? What does it aim to discover?
- Setting the budget – this determines the duration of the study and the number of participants.
- Determining the research plan and forming a hypothesis.
- Defining the participants – a mixture of ages, genders, locations and study-specific criteria.
- Determining the tasks for the testers – what do participants have to record and at what intervals.
- Preparing all materials for evaluation and spreading the workload across the duration of the study – this includes tracking the participants, determining the points of contact for participants on various channels, simultaneous analysis and motivating the participants.
Recruiting the testers
Two weeks before the start of the study, Carina assigned us the task of recruiting 12 testers for the diary study. We had to include a mixture of different ages, genders and other relevant criteria of existing and potential customers. The recruitment was commissioned via our order form.
Diary order form on TestingTime.com
In addition to the target group, the time period within which the diary study is to be performed must be defined, as must the estimated workload for the participants and a short description of the task.
A diary study typically starts with an email briefing and ends with an online questionnaire.
We provided Zalando with the list of recruited participants via our concise online line-up (Example). So, Zalando was already able to follow which testers were being selected during the recruitment process.
Once recruitment was complete, we passed the baton to Zalando, so that the diary study itself could begin. We only came back into play towards the end of the process, to pay the participants.
Setup of the diary study
Zalando uses the dscout tool internally. This software is used by testers to record diary entries electronically via an app. Zalando uses its straightforward web interface to manage and evaluate the entries.
The dscout website – a platform specially designed for diary studies.
During the study
On day one of the study, all testers were personally instructed over the telephone and taken through the requirements and daily tasks in detail.
The testers were given two missions:
- Search for gifts online and record at least 10 complete diary entries.
- Search for gifts offline and record at least 10 complete diary entries.
During the study, the testers used the downloaded app to record their thoughts. All the entries from each tester (text, video and images) were aggregated on a web interface by Zalando.
‘This phase was quite laborious,’ explains Carina. ‘Despite a clear briefing, the participants had some task-specific questions or technical problems that needed solving. Our team decided to analyse and categorise incoming diary entries on a continual basis. With hundreds of entries, this strategy proved very worthwhile, in hindsight.’
Evaluation and analysis
‘We categorised our data in multiple rounds. First, two researchers read through every entry and marked the texts with tags such as ‘Problem’ ‘Feature Request’ or ‘Decision Criteria’. In the second round, the less interesting insights were filtered out, so that only the really interesting ones remained. In subsequent rounds, the insights were condensed even further until we were able to observe viable, recurring patterns,’ Carina recalls.
Results of the gift study
The results of the study were very informative for the Zalando team.
‘We found out how the decision-making process works when shopping for gifts, that it consists of multiple steps and that it is not as straightforward as we imagined. We also discovered that we have to personalise the offer according to the various types of gifts, occasions and steps in the decision-making process. Finally, we came up with a special campaign based on these insights, which perfectly met the needs of our customers when shopping for gifts.’
More about diary studies and other UX methods:
Get further insights straight from Zalando’s UX/Usability & Technology Team on Twitter @ZalandoTech or the official Tech Blog. Many thanks to Carina Kuhr for her valuable input regarding this case study. Book test users for your own diary study with TestingTime.
