We’re Gonna Need a Bigger Boat

Posted by on March 9, 2021

Earlier this year, the FreeAgent marketing website www.freeagent.com was the target of a volumetric Distributed Denial of Service (DDoS) HTTP flood attack.

This was a relatively unsophisticated attack in that it targeted a particular static endpoint of our website with a massive number of HTTP GET requests from multiple remote IP addresses around the globe, as visualised on the map below.

Predominantly serving the UK small business base, FreeAgent wouldn’t normally get such a level of love and attention aimed at our website. The level of traffic directed at us was unexpected and so our standard content service and network protection resources were eventually overwhelmed, resulting in downtime and the website being unavailable for a short period.

The graphs below show a typical average level of traffic requests per minute, followed by a massive x40 jump in traffic at 22h45, triggering our automated alerting.

Our initial response was to drop traffic to the specific endpoint with a 404 Not Found HTTP response code at the load balancer layer, protecting the overall integrity of the rest of the website, while increasing the compute resource available to absorb additional capacity. This got us over the hump until traffic dropped back to normal levels at 2am – three hours after the attack began, with just the initial hour or so (77 minutes to be precise) offline.

An exact three hour attack window is revealing in itself. Taking some of the malicious IP addresses and entering them into shodan.io reveals a pattern of vulnerable MikroTik routers being exploited. Smells like a botnet-for-hire to me.

Around the same time, our Social Media Team received an inbound Tweet which seemed suspicious and certainly not coincidental. Further analysis of the Twitter account showed little activity, apart from “following” FreeAgent, and a previous history of trying to elicit a ransom demand from other online entities, using the same wording and tactics.

So, at what point does paying for a botnet-for-hire become financially unviable? Well, at an average of often less than $10 per hour, it’s a relatively cheap and easy attack to mount. However, if the victim does not engage (and neither they should!), and readily mitigates the attack, then any potential for a return soon evaporates.

But hey! Let’s up the traffic and try once more. So the attacker did later that morning, increasing the volume of traffic and pivoting to a different endpoint, including hitting our main customer application on an unauthenticated endpoint – the forgotten password page, as many a provider will have.

Only the static website was affected. As we had already completed the migration of our customer facing application to AWS in December, it remained unaffected with enough resource capacity and scalability available to meet increased traffic demands. We also already have AWS Shield in place. For the website, we increased resources further (luckily available having vacated the application from the hosted data centres earlier).

Prior to this DDoS attempt, we had already begun a project to migrate our website out of our hosted data centres. As a result of the second attack, we decided to expedite this process, so the website is now hosted with a new incumbent technology hosting provider, Netlify Edge with active DDoS mitigation in place.

The Response Team considered, and test implemented other alternatives, should our current mitigations be insufficient. A sterling effort, coolly and calmly considered during a time of potential crisis and stress – a great example of our Incident Response process working like clockwork. 

We have  alternatives in our back pocket in case of another rainy day. Some of these include:

  • Blocking non-UK IP addresses at the Layer 3/4 level.
  • Leveraging our Runtime Application Self-Protection (RASP) technology to block at the Layer 7 level.
  • Additional traffic routing via AWS, availing of AWS Shield.
  • Hosting directly in AWS, availing of AWS Shield. 
  • Implementing alternative CDN technology.

And back to our attacker. Whatever became of them? Well, we can only surmise based on the same Twitter handle and who they followed after realising that FreeAgent wouldn’t succumb to their games. But their antics have been reported to the Police Scotland Cybercrime Operations Unit and reported on the NCSC CyberSecurity Information Sharing Partnership (CiSP) portal. Even if they have “changed their name” (but not their number!).

FreeAgent employs a number of organisational and technical measures to help protect our systems and the data of our customers. We employ a secure coding framework and strategy, with both static and dynamic security testing, regular automated and manual penetration testing and runtime application protection.

But, nothing is perfect and we are always looking to learn, iterate and improve. We operate a Responsible Disclosure program, so if you are aware of a specific vulnerability, please do reach out.

Leave a reply

Your email address will not be published. Required fields are marked *