How we Use Dagster Automations in our Data Pipeline
Introduction The heart of a reliable data platform are robust and automated data pipelines. As our team migrates our data pipelines to Dagster, re-architecting our automation logic is a crucial task. Dagster offers condition-based approaches to creating or updating a data asset (table or file), moving us toward a modern, asset-centric view of data. This post details the three primary automation strategies we considered and implemented (so far) in Dagster—Schedules,… Continue reading
Waiting for it with Capybara’s synchronize method
Feature specs are notorious for their potential to flake. It’s possible for the results of feature specs to be inconsistent because they have to deal with asynchronous state. In a typical test environment, there’s a single Ruby process at play, so test code will be executed in order as written – we can reasonably expect one line to complete before the next is executed. But when it comes to feature specs… Continue reading
The Engineering Behind Salesforce at FreeAgent
For nearly 15 years, Salesforce has been the engine room for our customer data here at FreeAgent. Since we first brought it on board in 2011, it has grown from a straightforward CRM into a complex, deeply integrated platform. While one blog post cannot cover every detail, we hope this gives you a better understanding of how we leverage Salesforce and the engineering approach we use to maximise its impact.… Continue reading
The Other Copilot: Coding with AI
It seems Copilots are ten a penny these days. While our CoPilot accountant partners are a human way to support you using FreeAgent, the flight term has really taken off(!) as branding for AI tools, GitHub’s Copilot being just one of them. The rise of AI tooling for developers has sparked a lot of discussion and controversy as of late. It’s also seen rapid development and innovation, with several code… Continue reading
Type checking in Ruby – Part 1
Over the course of a career in software engineering, we learn to love elements of our tooling and dislike others - that’s perfectly natural. As requirements change, including our own need to improve as engineers, so does what appeals to us when reaching for a new framework, language or library. A common path for lots of engineers will have been to learn something like C or C++, both strongly typed… Continue reading
A brief introduction to ‘the cloud’ and managing infrastructure with code
Over the last decade the ‘cloud’ has become increasingly prevalent . A cloud based system allows a company to flexibly buy servers, storage, networking and various other services that are hosted externally rather than on-site, typically with a programmatic interface to allow large-scale use. According to a 2019 report, 94% of companies were utilising the cloud in one way or another. The market for cloud providers was valued at $200… Continue reading
Getting started with Jupyter Notebook
Jupyter Notebook is a development environment that runs in your web browser and can be used with several languages, including R and Python. In this blog post, we’ll look at some of the benefits of using Jupyter Notebook and how to start using it with Python. Benefits of Jupyter Notebook Chunking code into cells Instead of having to write code in large flat files, developers can use Jupyter Notebook to… Continue reading
In full flow: moving from Jenkins to Actions – Part 1
At FreeAgent a recent project to move our Continuous Integration/Continuous Delivery (CI/CD) workflows from Jenkins to GitHub Actions has brought some real benefits. In this post we’ll cover the background of our CI/CD pipelines, why we wanted to change how they run, and how we decided on GitHub Actions. In the next post we’ll cover how we handled the migration, and how we solved the challenges we encountered. But let’s… Continue reading
Tooled Up. A brief history of SaaS tools we’ve loved (and lost)
Distributed (remote) working has always been a foundation of the Engineering team at FreeAgent. Our entire company was founded as a distributed team and over the years we've managed to maintain a 50% distributed-to-office staff ratio in Engineering. The tools we use to work effectively as a distributed team have changed over the years, but there are a number of stalwarts that we've used since 2007 and still love today.… Continue reading
An Evolution of Bank Feeds: from Yodlee to Open Banking
We care a great deal about bank feeds. They provide the essential link between our users' FreeAgent accounts and their bank accounts, automatically importing transactions each day. From our customers' point of view they're probably the most valuable and most used feature of the software, so they're really important to us too. For many of our customers, the main purpose of using accounting software like FreeAgent is so they don't… Continue reading