Over the last decade the ‘cloud’ has become increasingly prevalent . A cloud based system allows a company to flexibly buy servers, storage, networking and various other services that are hosted externally rather than on-site, typically with a programmatic interface to allow large-scale use.
According to a 2019 report, 94% of companies were utilising the cloud in one way or another. The market for cloud providers was valued at $200 billion by the end of Q2 2022, with expectations for it to increase drastically over the next decade.
There are a number of different cloud providers currently in operation, each constantly creating new innovations to standout from the rest. The main players are currently AWS, GCP and Microsoft Azure. These three companies alone capture 65% of the available market. While each one provides similar services, they all have very different ways of deploying and managing their resources.
What are the advantages of cloud systems?
Zero Wires Zero Worries
One of the main advantages for companies that run the majority of their services on the cloud is the reduced upkeep of the hardware for their servers, storage and networks. Instead, an office only has to worry about its own power and internet connections.
Autoscaling
Another benefit of using cloud hosting is the ability to auto-scale. Companies can experience various situations where a larger than usual volume of users wants to access their online resources. This can be due to DoS (denial of service) attacks or simply a large influx of customers from a new marketing campaign.
Cloud hosting enables providers to reduce the impact of these periods of high demand automatically by deploying more resources to maintain the normal functioning of their services.
Managing infrastructure effectively with code
Using a cloud provider’s UI to create infrastructure by hand is simple but it introduces risks. It’s easy to overlook how something should be configured and this method makes auditing difficult. In addition, you can end up introducing configuration drift over time which makes it harder to link different resources together. It’s also time-consuming to create many instances of a resource.
We can mitigate these issues by managing infrastructure as code.
What are the benefits of using Terraform to manage infrastructure?
Terraform is a declarative programming language that describes infrastructure. Unlike most standard languages, developers don’t have to describe how a resource will be deployed. Instead, they state the resource desired outcome in code and Terraform will work out what steps to run.
Resources to deploy are defined in a folder of files with the .tf extension. Terraform uses this configuration to create a state file when the command terraform apply is run, which is used to directly track the resources that have been deployed. This enables commands such as terraform destroy to be used to destroy the relevant resources that are being managed by Terraform.
Below is a part of a Terraform configuration that can be used to create a simple EC2 instance (virtual server).
This is a very simple example of what an EC2 instance will look like in Terraform. In practice, there will usually be a lot more configuration required to enable it to do very specific tasks and communicate with other resources.. The extent of the configuration capability can be found on Terraform’s website here. Terraform’s documentation is generally very comprehensive, making it very useful when developing code for it.
What if you want to change configurations once resources have been deployed with terraform? Terraform has made this extremely simple whereby you can alter the terraform files you have written. Once changes are made to the files, terraform apply will be run again and these changes will be made in your cloud service.
What other tools does Terraform have?
Terraform has a huge ecosystem of developers trying to improve and innovate the language and concepts behind it. There’s a variety of projects both open source and closed source that can be utilised to improve IaC (Infrastructure as Code) workflow efficiency.
Terragrunt by Gruntwork
Terragrunt is a thin wrapper that helps prevent the repetition of Terraform code.
Creating resources using Terraform requires various bits of information about your AWS account in order to run. Using Terraform alone means developers have to state this backend information every time they want to apply a configuration.
Terragrunt enables users to deploy resources from a terragrunt.hcl file. This file contains blocks of code that allow you to pull in configuration from other places e.g. enabling the backend configurations to be stated in a single place and re-used.
Below is an example of what a terragrunt.hcl file will look like.
The Terraform block points to the location of a module that contains Terraform code.
The ‘include’ block tells terragrunt to look through parent folders to find any other configuration. Typically this will include at least the backend configuration in a shared root configuration, which will save developers a lot of time.
The ‘inputs’ block can be used to alter any of the predefined configuration using the variables within the module. It’s the Terragrunt equivalent of Terraform’s tfvars files.
You would now run terragrunt apply to deploy the resources, instead of using Terraform directly. This will pull in all the Terraform code and insert the extra configuration provided by Terragrunt, and then run Terraform for you.
Terragrunt’s modules
In addition to Terragrunt augmenting Terraform usage, Gruntwork, the company behind the tool, has a huge number of Terraform modules that provide configuration for a wide range of best-practice cloud services. These can be deployed by inputting a git ssh link to the source section at the top of the terragrunt.hcl file. These modules can be used to create VPCs or load balancers, for example, which can then be linked together in your own code.
Account cleaner
Tools such as cloud-nuke and aws-nuke are becoming increasingly popular to implement within organisations. These tools will ‘nuke’ your account, meaning they will destroy all resources that the tool is configured to destroy.
When to use the nuke function
We mentioned the terraform destroy command above, which can be used for destroying resources managed by Terraform. However, this can’t necessarily remove everything you might want to destroy.
For instance, you might create a resource with Terraform that then creates another resource, or someone might use the console to add things to your system. These resources won’t be stored in the state file for Terraform to destroy. As resources in the cloud cost money to run, you’ll want to ensure they are tidied up. These tools will iterate over all the resources in an account and then delete them.
How to use the nuke function
The tool is configured using a configuration file such as nuke.yml. An example of an aws-nuke configuration file is shown below.
As you can see there is a large amount of customisation available in this tool. This example is for aws-nuke to find every eu-west-1 and global resource in AWS account 565872034739, excluding the named policy and any S3 objects from deletion. It will be impossible to nuke anything in account 467527364293.
Ensuring that careful testing is done when creating the configuration file is extremely important. Poor configuration can lead to entire production accounts being destroyed, and can have severe effects on the organisation.
Summary
Terraform and the tools surrounding it ensure developers are able to produce resources quickly and follow best practices. There can be a bit of a learning curve when getting started with these tools but, once your code is written once, it’ll be as simple as linking your module and providing some input variables for terraform apply to deploy your resources.
At FreeAgent, infrastructure is entirely managed following the IaC principles. Resources are never deployed in production accounts without Terraform code, and the tools outlined above are used for development, testing and deployment for reliable infrastructure.