Devops Contract

A common quality can be found in many devops efforts- from bare-bones CI scripts built by scrappy development teams all the way to mature CI/CD pipelines deployed and maintained by dedicated practices within an enterprise.

That quality is an intimate understanding of a project and its artifacts by the automation and infrastructure that is responsible for making it operational.

How can we avoid this tight coupling between the software we develop and the scripts and configurations operations requires to develop it?

When we are developing features and/or products, we often:

  • Make carefully crafted schemas of our entire domains, their interdependencies, aggregates and derivatives.
  • Define API specs so that they’re easier to test, validate, and become more distributable.
  • Devise elaborate patterns and hierarchies to bring order to our ever-sprawling codebase.

Defining and agreeing to these contracts allows us to interact across teams, avoid tight couplings, and adapt our systems over time in a reliable way.

An area that can be overlooked yet is a perfect candidate for these sorts of formalities is the intersection between development and operations.

We refer to the solution to this gap as a “Devops Contract”

Tenets of a Devops Contract

If we applied to the practice of devops the same principals and rigor that we collectively value when writing code - such as encapsulation, separation of concerns, dryness, etc… we’d end up with something we can refer to as a devops contract.

The primary components of a devops contract are an interface of commands + a lifecycle that knows when to run those commands:

  1. A multi-phase build system defines a contract’s lifecycle

    • Phases represent concepts such as initializing the environment, resolving external dependencies, performing checks on the source artifacts, building/deploying, and cleaning things up.
    • As commands are executed within each phase, errors should immediately short-circuit the lifecycle. This makes it easy to group the commands in a series from pulling source code to deployment.
    • AWS Codebuild/Codepipeline, Jenkins, and GitHub Actions are examples of multi-phased build systems.

  2. A contract’s interface is described by a collection of commands that can be associated with the phases of the build system and are trusted to perform a specific class of actions.

    • Commands represent abstract concerns such as installing libraries from a package manager, running lints and unit tests, building artifacts, deploying a stack, integration testing and removing a stack
    • The lifecycle understands how to run commands, and controls the order of their execution, but should have no knowledge or dependencies on what happens inside a command.
    • Commands should have as few inputs (args, etc…) as possible and none is best.
    • Parameters required by the inner workings of a command should be declarative, using environmental variables is a good way to achieve this.
    • Commands should be understood, configurable (and typically implemented) by the team that develops the software.
    • Commands may be shared and reused, but if you can’t fix it, you don’t own it.

Benefits

Once a lifecycle and interface are established, you enable considerable gains in the following areas:

Reusable Infrastructure and Automation

  • It becomes possible to develop generic pipelines that are composed of reusable components in configurations that best support the software’s needs.
  • Increased adoption of devops best practices by reducing the amount of ceremony and integration around standing up and configuring pipelines.
  • Moving to new CI/CD runtimes (or even supporting several simultaneously) becomes relatively trivial.

Developer/Team Autonomy

  • Developers can make adjustments to various aspects such as reconfiguring a test runner or adding code transpiler without the need to redeploy a pipeline.
  • Potential for innovation is increased by empowering teams to explore new infrastructure and related tooling.

Operational Knowledge Sharing

  • As teams become more responsible for defining how their software is tested and deployed, they’re able to help find and resolve issues faster and make more informed choices in their architecture.

Example code is worth a 1000 lines of markdown

Let’s say we have a build system(eg Jenkins, CodeBuild, etc…) that’s made up of three phases:

  1. Init
    • runs make deps
  2. Build
    • runs make test followed by make build if the tests are successful
  3. Deploy
    • runs make deploy

A real world contract will likely have more commands, but consider the following simple interface:

Interface

./Makefile

.PHONY: deps
deps:
	echo INSTALL DEPS
	
.PHONY: test
test:
	echo TESTING
	
.PHONY: build
build:
	echo BUILDING

.PHONY: deploy
deploy:
	echo DEPLOYING

Obviously this doesn’t do much real work, let’s implement the interface for a few use cases.

A static web application hosted on S3:
.PHONY: deps
deps:
	npm i
	
.PHONY: test
test:
	jest  --coverage
	
.PHONY: build
build:
	webpack -p

.PHONY: deploy
deploy:
	aws --region $AWS_REGION s3 sync dist $AWS_S3_BUCKET --delete
An NPM library project:
.PHONY: deps
deps:
	npm i
	
.PHONY: test
test:
	jest  --coverage
	
.PHONY: build
build:
	tsc

.PHONY: deploy
deploy:
	semantic-release
A python package:
.PHONY: deps
deps:
	pip install -r requirements.txt
	
.PHONY: test
test:
	python -m unittest discover -s tests
	
.PHONY: build
build:
	python setup.py bdist_wheel

.PHONY: deploy
deploy:
	twine upload dist/my_package-*-py3-none-any.whl
A serverless application:
.PHONY: deps
deps:
	npm i
	
.PHONY: test
test:
	jest  --coverage
	
.PHONY: build
build:
	tsc

.PHONY: deploy
deploy:
	sls deploy --stage=$STAGE --account=$ACCOUNT

It’s immediately obvious that the same pipeline, one that is soley concerned with issuing commands in a pre-defined order can be used for all sorts of applications.

It's so simple!