As you may have already heard, FreeAgent implemented a four-day week throughout the summer this year as a ‘thank you’ for the employees’ contributions during the pandemic and a year of working from home. Whilst Pat George’s blog post talks about measuring the success of the four-day week with “job satisfaction, business objectives and personal stress levels”, as an analyst, I wanted to see if I could find less anecdotal evidence! With the shortened work week period officially ending two weeks before our biannual Hack Days, it made sense to assemble a team and turn this into a project to obtain some data-driven insights into how this actually impacted us as a business…
The Project
To kick the project off, on the morning of Hack Days we all sat in front of a white board and after a short discussion came up with two overarching themes we wanted to analyse data on:
- The impact on productivity
- The impact on wellness
You’re probably wondering what we mean by productivity and wellness, but don’t worry, I’ll cover this later! Anyway, we created a Trello board to house any ideas or questions we had and tasks we had to complete. It also contained all our datasets so they were readily available to everyone within the team. We also created a Slack channel so we could easily and quickly share information with each other (remember this is Hack Days after all, we had to be efficient!)
Our plan of action was to spend the first day of hacking getting the data into the dream formats, analyse any trends and try to have at least three main headlines. The second day is usually only half a day as the afternoon is when everyone presents their projects, so we decided to have all our analysis done by day one. Then we could just spend the second morning pulling all our insights together and creating a great presentation (there are Hack Day prizes to be won, including best presentation, so you gotta make an effort if you want to be in with a chance of winning).
The Datasets
During our prep for Hack Days, we had already acquired or created several datasets covering a variety of information, and we found that each dataset could be used to help answer questions based on the themes above. To maximise efficiency, the datasets were distributed amongst the team and so we started analysing the data to see what trends we could find.
The data we had access to included:
- OKR Data 📊
- Whilst many subteams within FreeAgent use different ways to measure their day-to-day work (such as cards on a Trello board), the one consistent tool that every team uses each cycle to outline their work and progress is our Objectives and Key Results (OKR) sheet. You can read here about how we score OKRs at FreeAgent.
- We decided to create a dataset that included all of the cycle end scores for each project, including the department that was responsible for that project. This included data for the four-day week cycle, but also the two previous cycles and the same cycle from the previous year that could be used as comparison.
- Employee Data 👫
- This dataset included the total number of ‘contracted’ employees, the number of absent employees and number of employees on holiday, with this data presented on a per-day basis. This data would allow us to calculate how many employees were physically ‘in work’ during the periods of time we were analysing. With the total number of employees who work at FreeAgent regularly changing (especially when you look at year-on-year data) and also with an ongoing pandemic, where we might see an increase in absences, we wanted to understand productivity in relation to the number of employees we had working during each time frame we were measuring.
- Calendar Data 📅
- Before our four-day week had even started, there was a lot of emphasis from team leads on how to be more efficient with our time, and naturally the most obvious suggestion was to reduce or remove meetings wherever possible. We wanted to understand if we had actually achieved this as a business, and to do this we wanted to access the Google Calendar API (as this is the primary tool FreeAgent uses to book meetings).
- As this data was coming from an external API this was slightly harder to acquire (actually, if I’m being honest, this was the hardest data to get our hands on) and required an IT systems engineer, a software engineer and a senior business analyst just to get it into a raw format we could use to bring it into our ETL tool, Matillion. This data also required a lot of ‘cleaning’ – for example, many people use their calendar to enter task-based work or block out time for their lunch, which isn’t technically a meeting at all but was still coming into our data, so we wanted to remove single-occupant meetings and other things like that that might pollute our data.
- Anyway, once we got this data into the format we needed, this gave us access to the number of meetings and their durations for every employee throughout the business.
- Survey Results 💤🚲🧘
- Before our four-day week started, the People team sent out a three-question survey asking employees about their wellbeing, sleep and activity levels. These questions were multiple choice, giving users a scale to answer to (e.g. on a scale of 1 to 5 with 1 being ‘Poor’ and 5 being ‘Great’). On completion of the four-day week period, this same questionnaire was sent out to employees. This dataset contained the before and after responses to this survey.
- Covid Data 🦠
- If you hadn’t noticed, there’s a pandemic going on, and at the time of our four-day week period there were some significant spikes in Covid cases. We wanted to have access to Covid data to see if we could explain any unusual trends in absences that we might see in the data available within the employee dataset. We obtained data of the number of new cases per day that was openly available to the public.
- UX Interviews 🎤
- Whilst all the data above contained quantitative data, we also wanted to include some qualitative data into our analysis too. Luckily, one of our UX researchers had agreed to be part of the team (even though they were already working on another Hack Days project) and so she set about fitting in 30-minute interviews with five employees from around the business, with questions focused on both productivity and wellness.
I should mention now that any personally identifiable information (PII) was removed from these datasets and replaced with IDs (if we needed unique identifiers) before we even thought about analysing them.
The Analysis: Productivity
After some discussion, we decided that we would define productivity as a ‘Green’ OKR score. This included any project that achieved an end score of 7 or above. We uploaded our OKR dataset into Looker via Matillion so that we could manipulate the data into a format we could analyse. When plotting the percentage of OKR Green scores, we saw that there was an average increase by department of 29% compared to the previous cycle and 10% compared to the same cycle last year. This data suggests that we were better at achieving the targets we set out to reach at the beginning of the cycle.
% of Green OKR Scores by Department
To see if we were more efficient with our time (especially with an emphasis on fewer meetings), we wanted to analyse the calendar behaviour. As mentioned above, this data was the hardest to get into the right format, but thanks to Matillion (and senior business analyst Jack) we were able to transform the raw calendar data and split it up into three separate, more manageable tables. From here we could query this table using SQL to get some insight.
Matillion job converting raw data into three separate tables
During the four-day week, we found that employees were on average in 2.4 meetings a day compared to an average of 2.1 meetings during the same cycle in 2020 – a 14% increase. This was an unexpected result, considering there was a big emphasis on reducing meetings. However we did find that meetings were, on average, five minutes shorter, with the average meeting time reducing from 45 minutes to 40 minutes. Overall, employees spend an average of around 96 minutes in meetings each day, which is in line with the data we analysed for 2020.
The insight we gained from our UX interviews also backs the quantitative data up. Notes from the interviews show that some employees did mention that while the meeting times had reduced, the frequency of meetings did not, which meant proportionally less time outside of meetings to do work.
The Analysis: Wellbeing
To analyse the survey results, we decided to only look at data from employees who had responded to both questionnaires, so we could understand any correlations. For each question, we assigned each response (remember these questions were on a scale of 1 to 5) a specific score. This allowed us to give every employee a before and after score for the sleep, wellness and activity question, which in turn allowed us to see if there were any changes to these scores. We then grouped these difference scores into three categories: Poorer, No Change and Improved, and from there we were able to plot the results.
We found that, overall, 64% of respondents reported an increase in their wellbeing, with 33% of respondents reporting no change. For the sleep results, 53% of respondents saw an increase in the number of good nights’ sleep they got, waking up more refreshed, with 32% reporting no change. And in terms of activity levels, 38% reported no change over the four-day week, but 29% of respondents saw an increase in how active they were each week.
Whilst sleep and activity levels are quite easy to define, you might still be wondering what we mean by wellness. And so were we. So we decided to use our UX interviews to gain insight to what ‘wellness’ meant to employees.
When asked to reflect on what wellbeing meant to them, participants spoke about balance, presence and space. Our UX researcher, Emily, found that employees felt that having the extra day a week to relax and reset meant they felt more present on the weekends with family and friends, and also more focused at work.
The interviews found that instead of seeing the extra day off as providing a three-day weekend, it was actually more of a ‘free’ day outside the constraints of the working week and weekend. Emily says “This is an important distinction to make – while folks described planning for ‘long weekends’, i.e. activities, exercise, getaways, when they first heard about the trial, most participants ended up using it as a day for themselves. This explains the increase in Sleep and Wellbeing but lack of increase in Activity outlined in the quantitative research.”
Finally we looked to see if the four-day week had any impact on the number of absences we saw during the cycle. During our initial browsing of the data over 2021, we could clearly see an unusual uplift in absences around the four-day week cycle period. As mentioned above about the recent waves of infection being the highest recorded so far, we thought that these absences may be related to this. Luckily, the People team were able to provide us with a last-minute set of data which included the number of Covid-related absences per day, and we found that this confirmed our theory for the increase in absences. However, we were surprised to see that a large proportion of the absences were due to a reaction to the Covid vaccine (which was being rolled out during this time frame).
We decided that in order to do a like-for-like analysis, we had to exclude all absences related to Covid and compare these figures to the previous year to understand if absences had been impacted by the four-day week. Our analysis found that overall there was 0% change in absences compared to the previous year, but after the first six weeks we begin to see a sustainable decrease in absences. This could suggest that after working a four-day week for a while, employees could be less likely to take a day of absence; however, this theory can’t be proven without further, sustained research.
Summary
So our Hack Days analysis told us that teams were better at achieving the work they set out to do at the start of the cycle, yet somehow employees still managed to squeeze in more meetings into a shorter week. However, overall people felt more balanced and present whilst also seeing improvements in their sleep.
I think this was my favourite Hack Days to date, as it required using data I wouldn’t normally use (Google Calendar data) and it required bringing multiple datasets together to answer questions that people around the business were genuinely interested to know the answers to. It also meant I could work with some cool people, including people I don’t normally get the chance to. Overall it took around 10 people to bring these results together!
Whilst FreeAgent has no plans to implement a four-day week in the future, I think I can speak for all employees when I say the shortened work week was very much appreciated and much needed after what was a pretty crappy year last year (thanks Covid!)
PS: if you’re wondering, yes we won Hack Days – our presentation was epic.
