Grinding Gears

Text classification is often done through fine-tuning of a pretrained foundation model with domain-specific data. In FreeAgent we use transformer based models to automatically classify incoming bank transactions. Specifically we use a DistilBERT model that is fine-tuned on hundreds of millions of bank transactions with customer-labelled accounting categories. The model inputs are currently text-based, built from a combination of bank transaction descriptions and amounts. In this post we describe an approach to fine-tuning the DistilBERT model and training the classifier including the numerical amount feature as a single network. Continue reading

One of the challenges of working with LLMs is getting them to respond with a consistent format, such as a given JSON schema. Anyone who has tried to solve this issue with prompt engineering knows how frustrating it can be. You add a ‘MUST’ here and an ‘always return JSON’ there, but still the output doesn't reliably parse. Maybe you're about to add a try-except block to handle parsing errors… Continue reading

It seems Copilots are ten a penny these days. While our CoPilot accountant partners are a human way to support you using FreeAgent, the flight term has really taken off(!) as branding for AI tools, GitHub’s Copilot being just one of them. The rise of AI tooling for developers has sparked a lot of discussion and controversy as of late. It’s also seen rapid development and innovation, with several code… Continue reading

Do you remember the first time you opened Excel and discovered pivot tables? Or perhaps that moment when you first held a digital camera after years of developing 35mm film? Maybe it was when you wrote your first SQL query and watched data magically arrange itself exactly as you needed it. These technological transitions in our lives often follow a strikingly similar emotional pattern. The initial rush of discovering something… Continue reading

These days Generative AI is being employed for everything from interpretation and summarisation of text to problem solving with a conversational natural language interface. What sort of conceptual model should we have in mind when thinking about LLM systems? Continue reading