Fast becoming my favourite event of the year is Rails Girls London. Every event is different as you get to spend it with a whole new group of interesting people from a huge variety of industries and backgrounds.
This year we headed to the offices of Simply Business in the City of London for the one-and-a-half day workshop, and happily with FreeAgent sponsoring for the first time. Between the pool table, ping-pong and tables piled high with freebies, around two hundred students and over a hundred coaches worked together on building the students’ very first Ruby on Rails app.
Presenting to a packed room at Rails Girls London
Installing Ruby on Rails en masse
Friday evening started with installing Ruby on Rails onto the students’ laptops and getting them ready for the main event on Saturday. I spent a while shuttling between students with overloaded wifi access points, “interesting” gem errors on Windows and the piles of pizza, but we eventually all had a working installation with which to try the main tutorial!
Olly talking about life as a software engineer at FreeAgent
Bright and early on Saturday morning, we headed back to London for breakfast, sponsor talks and kicking off the main workshop. It turns out to be hard to teach somebody all of Rails in one day, so the aim of the day is to show how Rails scaffolds can be used to build basic functionality, learn a few Ruby, HTML and CSS basics and get a feel for how to make and test changes to an app.
You’re the expert now
The really interesting part of the event is finding out about the students and their reasons for wanting to spend their weekend coding. This year I worked with students who had careers in business administration, fashion and teaching (which comes up often). At midday we heard some amazing lightning talks from coaches, who made the point repeatedly that everybody in the room has domain and real life experience of business problems.
A student working with Karen on their Rails app at Rails Girls London
Emma from Consonance explained how she’d proven this by starting a successful software company out of her expertise in the publishing industry, and it was a great reminder that this extends to businesses like ours too – a small business owner who’s done their own accounts likely knows far more about the problems our users face than a developer coming from a Ruby on Rails-only background!
I spent Saturday working with Yv through the tutorial and building out her Rails app together. By mid-afternoon we’d experimented with changes to views and then started to add in extra functionality like image uploads by adding gems to the project. It’s especially nice when students rediscover past experience in tools like HTML and CSS then are able to reapply and build on their knowledge when working in Rails!
What next?
For many at Rails Girls, this will be the start of an exciting journey and career in software engineering. Some of this year’s coaches were students at this very same event less than a year ago, then had gone on to join a bootcamp course and make their successful move into the industry.
Iain talking with students about Ruby
We wish all of the students at Rails Girls the very best of luck with their future coding endeavours and hope you may be able to join our team in Edinburgh in the not-too-distant future!
If you have a little bit of Rails experience and would like to coach too, then look out for Rails Girls events in your part of the country or world – they’re happening all over and volunteers are always needed.
Seven years ago we started planning our first major infrastructure migration. Nine months later we made the move, taking FreeAgent from our first home in Rackspace London to a new, co-located home in two data centres (DCs) run by The Bunker. FreeAgent has been happily humming along in Ash and Greenham Common ever since.
Co-locating has been a terrific win for us over the years, providing us with a cost-effective, high performance compute platform that has allowed us to scale to over 95,000 customers. I wrote all about it back in 2017, and our uptime since then has continued to be excellent:
It’s fair to say this co-located infrastructure plan has been a huge success for the business, measured not only by great uptime but also by excellent performance (an end-user Apdex score of > 0.93 and 1.2s average end-user page load time) and with infrastructure costs as a percentage of ARR well below 1%.
Everything changes
As we know – especially in tech – nothing stands still. Everything changes. Since 2017 we’ve seen dramatic change at FreeAgent:
Our company headcount has doubled, from ~120 in 2017 to > 240 today
Engineering staff numbers have grown from 40 to > 100
Customer numbers have increased from 65,000 to > 95,000 and are still growing quickly
This growth put us at a crossroads with respect to the infrastructure of the FreeAgent app. Co-location was undoubtedly the right decision for the business at the time, but we wanted to take the time to re-evaluate this position for a number of reasons:
Was there an opportunity to increase resilience? We had developed world-class, data centre-level redundancy, introduced second cabinets in each DC (and established a networking pattern for expanding into additional cabinets), but we were still operating through a single provider. We felt the option of a third DC with an alternative provider would allow us to reduce this risk exposure, and it would also allow us to experiment with a more resilient ‘always-on’ architecture
Reaching hardware limitations. We were starting to approach the limits of our existing physical hardware capacity that could only be resolved by manual upgrades and replacements (e.g. SSD pools, RAM, a third cabinet, third DC)
Coping with demand on our Ops engineers. A big increase in both customers and engineers on our team had resulted in a large increase in demand on our Ops team from dealing with general infrastructure expansion, the need for greater focus on security obligations, as well as new requirements from fast-growing engineering and data science teams. To keep up we would need to make a significant investment in both physical infrastructure, hiring additional Ops engineers and site reliability staff (for keeping the lights on – KTLO), and in-house tooling
Finding people with the right skills.It was proving to be challenging to hire Ops engineers who had existing skills – or had an interest in acquiring skills – in the technologies we were using, in particular Joyent’s Triton compute platform
Progress in the Infrastructure/Platform as a Service space.Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) offerings had evolved enormously over the past 5 years and we wanted to take the time to consider what benefits they might bring us
In addition to this, there were some “known knowns” about the future of FreeAgent that we felt should influence our technical direction:
Machine learning (ML) applications and services. We were already experimenting with an ML service and we knew we’d be building more in the future. An ML service would require its own hosting, deployment and monitoring if we hosted it internally with our other apps and services, so we wanted to have the option to leverage cloud-based services for this such as AWS SageMaker in order to deliver business value more quickly
We would see an increase in our use of serverless compute. We were already using AWS Lambda in production and could see other areas where serverless would be useful
We wanted to evolve our approach to deployments. Our deployment mechanism was fine, but we were already considering how to optimise our approach and unify it across all our services.
Data Warehousing and Business Intelligence requirements. We had experienced a large increase in our use of event-tracking and data warehousing, and the demand for business intelligence and data reporting was increasing across the business
Demand on our CI/CD pipeline was increasing fast. With more engineers on our team, the number of tests we were writing was growing quickly resulting in increased demand on our CI/CD platform. How could we scale this up, improve our average number of deploys per day per developer, and keep test run times as low as possible?
Scaling the database is a big challenge. The amount of data we were storing on behalf of customers was increasing, with increasingly large database tables. How would we scale our database, would we need sharding for instance?
Ultimately there were three options in front of us: continue with our co-lo approach, take a hybrid approach (co-lo plus some cloud), or go full cloud. A decision of this magnitude isn’t something you can make easily, and whatever we chose to do would have a huge impact on the business for the next 5-10 years.
We felt we understood how to scale co-lo and understood the challenges. A hybrid approach didn’t fill us with joy (you get the best of both worlds, but also the worst of both!), so we wanted to focus our efforts on understanding the implications of migrating FreeAgent and its supporting services to public cloud infrastructure. In order to do this, we decided to undertake a time-boxed R&D exercise, over 3 months, that was geared to provide enough data and insight to allow us to make the necessary decision for the long-term future of infrastructure.
Which mountain should we climb?
As with every engineering project at FreeAgent, we kicked it off by writing an RFD:
The recommendations that we proposed in the RFD were:
Choose AWS as the cloud provider
This should not be a “lift and shift” project. We would focus on leveraging managed services wherever possible (e.g. Aurora instead of MySQL, AWS Elasticsearch instead of hosting our own)
We would deploy the app and related services in containers using Docker
Despite only being able to commit two people full-time to the project, progress was rapid. After just six weeks, a report was written that went into detail on what had been learned, how our thinking had changed in some cases, and what progress we had made to date. Several weeks later we came to the end of the research stage of the project and a final, more detailed report was written that put forward a much clearer picture of what FreeAgent and our infrastructure would look like after a full migration to the cloud.
The outcome was encouraging. Being in a position to fully leverage compute on demand and take advantage of the increasing number of fully managed services that AWS offers was an extremely exciting prospect for all our engineering teams, as well as the wider business. Granted, any infrastructure migration would be expensive, the project complex and it would come with many challenges, but the advantages and opportunities that a full cloud migration would open up in the future were undeniable.
The decision was made to migrate to AWS! 🎉
Embarking on the journey
The next few months were spent writing up detailed reports and creating OKRs across multiple teams. We hired a project manager who was experienced in cloud migrations to help draw up the plan of precisely how we intended to undertake the migration and what resources – both people/time investment and cash – we would need for it to be successful. As well as recruiting staff to bring AWS experience to the team, we supported training for upskilling our existing team members. We also had to understand what additional work would be required during the migration period in our existing DCs, and what capacity we will need to bring online there (if any) in order to cope with interim demand. While we would ultimately be retiring our data centres, we absolutely had to retain our world-class service while the migration was happening.
Several months later, we are well on our way to our new home in AWS and we’re due to arrive around the middle of 2020!
It’s a fascinating time in the world of cloud computing and we’ve been presented with an opportunity to learn and take full advantage of cutting edge technologies that will allow us to move more quickly and take on challenges that have previously felt out of reach. We’re hoping to blog more regularly as we approach the big day, about the progress of the project and about the specific technical decisions we’re making along the way – for example, why didn’t we choose Kubernetes in the end? – so keep an eye out for that. And if you’re interested in getting involved and joining us on this exciting journey, we’re hiring!
