Our data science hiring process

Posted by on January 28, 2022

Introduction

In this blog post, I’m going to describe our hiring process for data science roles, starting from an advertised role.  I’m not going to go into how to decide what the job specification should be, how to write it or how to advertise the position, because these are large topics in their own right.

We don’t work by a principle of surprise so I don’t see why the same should be true of our recruitment process.  In any case, if you do apply we’ll tell you what the process is, so it’s not exactly a secret.

We aim to find qualified candidates for the role as soon as possible while minimising both the amount of time that candidates must commit to the process before receiving a decision, and our own time evaluating candidates.  As a result, the process is split into several filtering steps. The later steps require more time and effort than the earlier ones.

Hopefully this post will be of some use if you’re thinking about hiring in data science!

Some context

At the time of writing, FreeAgent employs around 250 people.  We have a recruitment team of two full-time staff, one of whom helps us with phone screens and process management for our data science and analytics roles.  (Thanks Katie!)

We have seven full-time data science and analytics staff including myself, one senior data scientist and one mid-level data scientist.  We’re currently recruiting for a second senior data scientist.

Here’s the disclaimer – if you’re smaller or larger than us or different in another way then this process may not work for you.  We’ve done different things in the past and will probably do different things in the future.  This works for us for now.

Our evaluation process

Application screen

We ask candidates to apply with a CV and covering letter.  This is pretty standard, so we expect most candidates will already have the required information to hand in this format.

The application form asks some standard questions about salary expectations and preferences for office or remote work, as well as one specific question related to the role. We’re currently recruiting for a senior data scientist to work with the team building models in a customer-facing environment, so we ask: “Please describe your experience creating and maintaining ML models in a customer-facing environment.”

This gives us enough information to know if the candidate is motivated by the role and meets the minimum eligibility criteria.  This step is fairly fast.  If you ever read an article that says hiring managers might spend ten seconds looking at your CV, they’re probably right.  So if you’re an applicant it’s good to make it snappy!

The rejection rate at this stage is typically around 50 per cent.

Phone screen

We invite qualified candidates to a half hour phone call with a member of our recruitment team.  (Thanks Katie!)

The questions are quite general and focus on understanding why the candidate is interested in the role, what they value in the workplace and clarifying their understanding of some of the technical requirements for the role.  It’s also a chance for the candidate to ask questions about how the process will work and for us to take note of notice periods or any other constraints.

Fundamentally we’re looking for evidence that the candidate is motivated, has good people skills and can explain their technical work clearly to someone in a non-technical role.

The rejection rate at this stage is typically less than 10 per cent.

Problem and review call

Candidates have to be able to prove that they have a practical understanding of data science to progress.

We send out a zip file containing a simulated dataset and a small working system to train a machine learning model and make predictions for new data in response to HTTP requests.  We then ask the candidates to book a 45-minute call about a week later to discuss the system with two members of the data science team. We make it clear that we don’t expect more than a couple of hours of investigation of the system before the call.

We ask the candidate to describe how the system works during the call, and we may delve into some pair programming to improve it or discuss any improvements the candidate has tried before the call.

There are a couple of aspects of this that are worth commenting on.

In the past, we had asked candidates to implement a small system from scratch.  This typically resulted in boilerplate solutions based on well-known patterns, which to be fair is perfectly reasonable given the time constraints, but not great as a test.  Providing a working system with a few specific design flaws and a well understood simulated dataset made for a better evaluation.

Ensuring that candidates book a call with the team means that even candidates who didn’t have the chance to write any code in advance still get to discuss the system, explain their thinking and do some pair programming during the call.

We’re looking for evidence that the candidate understands the principles behind what they’ve written on their CV and can work together with the team on a technical problem at a level commensurate with the requirements for the role.

The rejection rate at this stage is typically around 50 per cent.

Final interview

By this stage, we should be fairly confident in the applicant’s abilities, so we rarely take more than a handful of candidates to the final interview.  Well done if you got this far!

Our final interview comprises two parts: “people skills” and “technical skills”.  Each part takes around an hour, with a panel of two interviewers each and a standard set of questions agreed in advance.

Because we can have up to four interviewers, it’s a great chance to get input from a more diverse set of people from the wider team.  The technical skills interview panel should include a data scientist at least as experienced as the role we’re hiring for, but the people skills interview panel can include anyone from the wider engineering team who has experience participating in this type of interview.

We’re looking for a degree of assurance that the candidate has the required people skills to work well in the team as well as sufficient background technical knowledge and reasoning for the role.  I’m not going to give away the exact questions here, but we do expect candidates to be able to clearly explain their choices in their approach to the problem in the previous stage, or in past project work mentioned in their application, in response to questions from the panel.

Conclusion

Following the two final interviews, the panel will reconvene with the hiring manager to agree a final decision on the candidate.  All the panel members have the chance to express their feedback on the candidate during this meeting. There’s usually some discussion because different panel members might have picked up on different strengths or weaknesses.

So now we’ve made a decision, let’s recap some of the key points:

  • The process is designed to minimise both the amount of time that candidates must commit to the process before receiving a decision, and our own time evaluating candidates.
  • We assess practical ability as early as possible by giving qualified candidates the chance to talk through a well-defined problem with members of the data science team.  
  • Providing candidates with some data and code that we wrote and understand as a starting point for the conversation helps to keep the process fair and minimises the time candidates have to commit.
  • The final interview is a confirmatory step for good candidates rather than a test to try to trip anyone up.

Hiding Elements on the Web

Posted by on January 18, 2022

For many of us the word hiding has visual connotations. But remember that the web is perceived both visually and non-visually. In this talk, Laurence Hughes from our Design System team explains that hiding is a tricky front-end development technique, used for both user interface experiences and accessibility-related purposes. He also provides some tips on how we can avoid doing DIY hiding completely.

How I prepare for a tech job interview

Posted by on January 5, 2022

You’ve updated your CV, applied for a job in tech and been offered an interview. Fantastic! But what next?

Preparing for interviews is time consuming and can feel daunting, particularly in the earlier stages of your career. I’ve found that having a regimented process for this preparation can reduce stress and save valuable time. In this post I’m going to outline my own process and provide some examples of what each step might look like for you.

In some cases the examples I use are related to my own field of data analysis. However, the process I follow could be useful in preparing for any job interview in a tech-related field, and possibly beyond!

In Summary

Let’s assume you’ve already updated your CV and perhaps written a cover letter too. By this point, have both of these saved in a new folder (e.g. Jobs > 2022 > FreeAgent) so that you can refer back to them. From here, your preparation process is to put together two further documents:

  1. A list of key projects you’ve worked on (the Project List) and the skills/competencies that they demonstrate
  2. A Rolesheet about the organisation and role that you’re applying for, and why you’d be the right fit

The Project List

At every stage in an interview you’ll be expected to demonstrate particular skills or competencies. Some interviewers might be looking for interpersonal skills, such as handling disagreements or working under pressure, and other interviewers might be looking for technical skills, such as working with bad data or using a particular statistical technique. While you may be able to demonstrate these skills in the interview itself, it’s important that you can also give examples of where you’ve demonstrated these skills in the past.

Regardless of whether you’re a fresh graduate or a seasoned veteran, it’s likely that you’ve demonstrated most of these skills at some point or another in your life. However, it’s often difficult to recall these quickly in an interview. A well-prepared Project List can work wonders here. Although it’s quite time consuming to put together, it can help you prepare for interviews for multiple jobs – both now and in the future.

Step 1: Chuck some projects down

Open a blank document and make a bullet point list of projects you’ve worked on. Don’t worry too much about adding too much detail. Don’t worry if it was a really small piece of work. Going through your previous job history or education in chronological order can help ensure you don’t miss any out, and adding rough dates in front of each one keeps that organised.

It might look something like this:

  • 2019-08 Masters dissertation
  • 2020-01 Blog post on pivot tables
  • 2020-06 Ran a half marathon
  • 2020-06 Kaggle challenge – Titanic survival analysis
  • 2021-02 AB test – adding new homepage banner
  • 2021-05 Survey data text analysis
  • 2021-06 Ran SQL training

The term ‘project’ here doesn’t have to relate to something you’ve done in a job. If you’re applying for your first job, you might want to think of this as a list of personal achievements or academic assignments.

Nobody else is going to read this, so use whatever language feels most natural to you!

Step 2: Identify some skills

Go through each project and think about what skills you feel it demonstrated. Don’t worry about stating the obvious. Don’t worry if it was only a minor example of that skill. Don’t worry if you think somebody else might disagree with you. Do try to use consistent language: if you’re talking about the same skill across different projects, give it the same name.

Your list might expand into something like this:

  • 2019-08 Masters dissertation
    • Time constraints
    • Written communication
  • 2020-01 Blog post on pivot tables
    • Self-motivated
    • Written communication
  • 2020-06 Ran a half marathon
    • Self-motivated
    • Overcoming adversity
  • 2020-06 Kaggle challenge – Titanic survival analysis
    • Self-motivated
    • Python
    • Cleaning data
    • Statistical modelling
  • 2021-02 AB test – adding new homepage banner
    • Liaising with stakeholders
    • Tableau
    • Delivering bad news to somebody senior
  • 2021-05 Survey data text analysis
    • Python
    • Learning a new skill
  • 2021-06 Ran SQL training
    • Explaining a difficult concept
    • Presenting

Step 3: Cross-check against common competency questions

There are lots of resources online listing common interview questions designed to give you the opportunity to highlight your skills. This list from prospects is a nice starting point and it explains the STAR (situation/task/action/result) response structure really well.

Go through each competency question in order and find a project of your own that would fit your response. You may need to add the skill to your list to help you remember that link, or you may need to think of another project and add that to your list. Again, don’t worry if you feel you’re giving a really tenuous example of that competency! Here are some examples.

Tell me about a time when you showed integrity and professionalism.

I haven’t explicitly called out integrity as a skill under any of my projects, but I do remember being challenged on that AB test when somebody asked me if we could stop the test early. I said no and that the test needed to run for longer to reach statistical significance, which is an example of my integrity, so I’m going to add integrity to that list:

  • 2021-02 AB test – adding new homepage banner
    • Liaising with stakeholders
    • Tableau
    • Delivering bad news to somebody senior
    • Integrity

Give an example of a situation where you solved a problem in a creative way.

Again, I haven’t explicitly called it out, but I think that my text-mining in Python was a creative way to gauge our customers’ response to a new feature. I’ll add that to the list.

  • 2021-05 Survey data text analysis
    • Python
    • Learning a new skill
    • Creative problem-solving

How do you maintain good working relationships with your colleagues?

I don’t think any of the projects listed here are the best examples for this, but I can think of things I do to support good working relationships. There was that time I tried out a colleague’s suggestion even though I didn’t think it was likely to be suitable. It may feel like overkill to create a whole new project just to list this, but I find this project-oriented approach helps me to remember all of these small examples more easily. I’ll add that as a project to my list. 

  • 2020-06 Moving tools from Basecamp to Trello
    • Listening to others’ suggestions to support good working relationships

And so on. Ideally, you want your project list to be the one-stop-shop for answering any competency question that might be thrown at you. This sounds excessive – how can you be prepared to answer any possible question?! The trick is to find ways to frame your projects in the right way, and this step should help with that.

Step 4: Practise, practise, practise

It’s all well and good having a colossal list of things you’ve done and the skills they demonstrate, but it’s unlikely you’ll have this to hand at an interview. You need to practise recalling these as examples in order to embed them in your head.

Ask a friend to chuck some competency questions at you and try to answer them using the STAR structure and your examples above. If you’re embarrassed about asking a friend, create some flashcards and go through them in a random order. If you’re doing it by yourself, I’d still encourage practising your responses out loud.

To begin with, have your list in front of you. Once you’ve used a particular example in response to a question, highlight it. Challenge yourself by trying not to use the same project more than once. If, at the end, there are whole projects left untouched, see if you can think of ways you could’ve worked that project into a response – especially if it’s a project you’re particularly proud of.

Once you’re feeling a bit more confident, try it without the list in front of you and see how that goes. If you need to reach for your list at any point, do that!

Remember that one interview might only cover 20% of the examples in your list, and that’s fine. You’re over-preparing here. If you can find ways to mention other skills (“…and I was also particularly pleased with that project because of how I handled the conflict between Bill and Ben”) then you can mention those in passing too.

No interview is going to be perfect, and as such you’re not expected to think of the most appropriate example for any given question. However, having this list will help minimise the chance of having nothing to say, which is what you want to avoid.

The Rolesheet

In addition to questions about your skills, interviewers will also be asking questions to understand your motivations for applying for this role at this organisation. Your responses should show that you really want to work at this organisation, in this particular role. This is where the Rolesheet comes in.

While the Project List is generally transferable between different interviews, the Rolesheet should be heavily tailored to the organisation and role that you’re applying for. 

Step 1: Make notes on the organisation

Nobody expects you to recite an organisation’s Wikipedia entry in your interview, but it’s important to know a bit about the organisation’s background and the wider context it sits in. Do some desk research and make a bullet point list of what you find out. 

Some things you might want to consider:

  • When was the organisation founded?
  • How many employees does it have? 
  • How many customers/users does it have? Does it have different types of customer?
  • Do its customers pay? If so, how much?
  • Who owns the organisation? Has it ever changed hands?
  • What’s the organisation’s mission, or purpose?
  • Does the organisation have any competitors?
  • Has the organisation been in the news recently?
  • Does the organisation have its own blog or newsfeed?
  • How is the organisation perceived by its customers? Does it have reviews?

The job advert will probably contain a paragraph outlining some of this, but don’t forget other sources of information, such as:

  • The organisation’s website
  • Google (and the News section)
  • The organisation’s social media pages (Linkedin? Twitter? Facebook?)
  • Trustpilot
  • Glassdoor
  • Wikipedia

Step 2: Outline why you want to work for the organisation

You can pretty much guarantee that you’ll be asked “why do you want to work here?” in an interview. This is where your research above comes into play. What do you like about the organisation? What drives you, and how does that match up with the organisation? 

Try to prepare a rough outline of how you’ll answer this question with 3-5 key reasons as bullet points. Being able to mention specific things you’ve learnt about the company (“…and I liked what I read about your approach to prioritisation in your CEO’s blog post”) shows you’re really interested.

It might look something like this:

  • Market-leading product
  • Mission indicates that they genuinely want to make a difference for users
  • Size of business is perfect place to have an impact
  • Blog posts by previous interns indicate positive place to work

Step 3: Outline why you want to work in the specific role

Following on from the above, you’ll want to mention why this role appeals to you. If you’ve been in this field for a while, what led you into it? If you’re moving from a current role, what is it you like about this one? Try to frame this in a positive light. “This role looks really exciting because…” is much better than “I want to leave my current role because…” even though it might make the same point.

Again, try to make a 3-5 bullet point list. This is another opportunity to show you’ve done some research by mentioning specific elements of the role description if you can. You might not be asked explicitly why you want to work in this role, so be prepared to offer these points up in your response to the “why do you want to work here?” question.

It might look something like this:

  • Team is growing – opportunity to shape the team’s direction
  • Opportunity to work with variety of teams across the business
  • Potential opportunity to mentor more junior team members

Step 4: Outline what you can offer

Again, you might not be directly asked “what can you bring to this role?” in an interview, but you want to make it clear that you feel you can offer something. Consider the skills you listed in your Project List. What stands out? What do you think your relative strengths are? This is your chance to freestyle. Get these down in another bullet list. 

Check the job description and see how your strengths match up with what the organisation is looking for. In a dream world you’d address every listed requirement at some point throughout the interview, so go through the requirement list and highlight the points you’ve already covered in your responses. If there’s anything you haven’t addressed, make a note of these and think about ways that you could weave them in.

For example:

  • Previous experience building data-first culture
  • Accurate and keen eye for detail – crucial in a small business where everybody’s role is important
  • Love presenting/talking to people – a major part of a role involving so many stakeholders

Step 5: Practise, practise, practise

You should now have 3 short lists: reasons why you want to work at this organisation, reasons why you want to work in this role, and what you can offer. This is your Rolesheet:  9-15 points that you want to get across at some point in your interview.

Take a look at some typical interview questions and practise some responses. Similar to the competency questions, begin with your Rolesheet in front of you and tick each point off as you make it. If, after a handful of questions, you’ve missed out any of your points, think about how you could have woven those in. 

While it’s good to have key points you want to get across in the why you want to work here questions, there will be other questions that haven’t been mentioned so far: what are your weaknesses? How do you prioritise your work? These other questions aren’t ones to over-prepare for. Your Project List and Rolesheet will have plenty of points to draw from. If you have to think up examples outside of the ones you’ve already listed, add them to your list!

Find another list of questions and repeat with those. Once you’re feeling more comfortable, try each question without your Rolesheet and check it afterwards to see how you did. 

Wrapping Up

So that’s my prep process: write two documents and use them to practise answering questions. As mentioned, the Project List is reusable and so worth investing the time into. On the other hand, the Rolesheet doesn’t take too long to put together and should be started from scratch for each interview. Here’s a very basic template for starting these two documents off, based on the examples above.