Grinding Gears

Classification with transformer models A common approach for classification tasks with text data is to fine-tune a pre-trained transformer model on domain-specific data. At FreeAgent we apply this approach to automatically categorise bank transactions, using raw inputs that are a mixture of text, numerical and categorical data types.    The current approach is to concatenate the input features for each transaction into a single string before passing to the model.  For… Continue reading

This post describes our approach to fine-tuning a BERT model for multiclass categorisation with Hugging Face and Amazon SageMaker. Continue reading

Introduction For the past year, FreeAgent has been running a machine learning model in production that categorises customer bank transactions. This model takes transaction descriptions and transaction amounts as inputs, and attempts to predict the corresponding accounting category. This summer, I joined the data science team with the more specific goal of increasing model generalisation, which would allow it to make predictions for a larger fraction of incoming transactions. One… Continue reading