Versioning, Branching and Testing Infrastructure as Code

zoom_housezoom_streetzoom_town

Infrastructure management is evolving. Instead of provisioning hardware and tracking infrastructure in Excel spreadsheets, we can now interact with virtualized infrastructure resources in programmatic ways. Enter Infrastructure as Code (IaC).

IaC allows us greater reliability and flexibility than traditional hardware. We can now treat the infrastructure architecture as a typical software project and apply the software development life cycle to IaC. Versioning, branching and testing infrastructure is not only possible, but an important part of deploying IaC.

Branching

Following the common Git branching strategy, gitflow, you can work independently without affecting others. Your templates can go through code reviews before being merged, and you can tag stable commits as releases for advanced versioning. However, you still might run into constraints, especially in testing.

Testing

Working in infrastructure, one realizes the importance and sacredness of the development environment. In this field a broken development environment can hinder the productivity of hundreds of engineers working diligently towards a deadline and completely derail the company — even if for only an hour. In this regard, the development environment is nearly as important as production and therefore no place to introduce infrastructure bugs.

AWS delivers IaC via CloudFormation. I’ve written about the importance, flexibility and speed of modularizing backend CloudFormation templates and building nested stacks. At RightBrain Networks I’ve advanced this idea and practice to modularizing Nested Stacks, to make up an environment. This provides the flexibility of building and testing infrastructure before it’s merged back into the mainstream development pipeline.

This concept takes the ideas of modularization explained in the nested stack blog post to another level, where even the nested stacks are modular and reusable. This idea breaks any given environment into many stacks. An environment, say Development, may consist of many stacks. For example, network, logging and application stacks might be built off the infrastructure development branch and another application stack built off some feature branch that someone’s working on. This enables the person working on a new feature for the application stack to work in their own isolated stack, making use of some common services like the log aggregation stack.

Versioning

Versioning infrastructure is just as important as versioning code. In fact, the entire suite should be grouped together as a release. Versioning infrastructure will mature your service or product offering. Versioning helps put hard numbers to integration testing to ensure that particular versions of code work with particular versions of infrastructure.

The Whole

In practice, I’ve found this methodology to work best for us as we rapidly build new infrastructure, alter existing rules, routes, sizes and the like. We use this methodology to ensure we do not interfere with the software development life cycle, by not only testing and checking our work, but to understand what changes will occur when a change is applied to a living stack via an update.

An example of what an environment might look like can be represented by the following diagram, where network and logging are bases of the environment being shared amongst the other stacks that sit on top. This diagram shows four engineers working separately on their own branches of a stack containing the application infrastructure before merging into development, which is always standing.

Testing IaC

Save

Save

Save

Save

Leave a Reply

Your email address will not be published. Required fields are marked *