All posts written by Ed Berry
Structured outputs with Pydantic AI
One of the challenges of working with LLMs is getting them to respond with a consistent format, such as a given JSON schema. Anyone who has tried to solve this issue with prompt engineering knows how frustrating it can be. You add a ‘MUST’ here and an ‘always return JSON’ there, but still the output doesn't reliably parse. Maybe you're about to add a try-except block to handle parsing errors… Continue reading
Using API Gateway, Lambda, SageMaker and DynamoDB to build a categorisation service in AWS
I’ve talked previously about the value of combining rules-based and machine learning approaches to categorisation. In short, rules-based approaches make it easy to do customer-level personalisation that complements a machine learning model trained to find patterns across customers. In this post I’ll talk about how we used AWS to build an expense categorisation service that combines machine learning with a rules-based approach. This service forms part of the Smart Capture… Continue reading
Combining machine learning with rules-based personalisation
One of the ways we use machine learning at FreeAgent is to help automate data entry. Keeping on top of your accounts can involve slightly tedious manual tasks like categorising bank transactions or managing your expenses. Machine learning can help here by automating aspects of these tasks so our users can nail their daily admin and focus on bigger things. Personalisation with rules In 2020 we launched our first operational… Continue reading
How we structure our data teams at FreeAgent
Since joining FreeAgent back in April I’ve been both impressed and interested with how the Data organisation is structured. I’ve come from an enterprise world where you have lots of Data Engineers, a team of dedicated Data Architects and a separate Business Intelligence org. A few things that immediately struck me at FreeAgent were: No one has the title ‘Data Engineer’ Data Analytics are part of the Engineering org No… Continue reading