Module Composition in Terraform

Table of Contents

Working with Terraform, I have quickly stumbled upon the question: how can I avoid code duplication? This post recollects my findings so far.

Scenario #

Let’s imagine that we want to create a GitHub repository, then a team of administrators for the repository, and finally add a user to the team.

Disclaimer: This example wants to represent a situation in which keeping all the code in a single module is not desirable or possible; it may be not the most fitting one but I wanted to keep the resource-specific code to a minimum.

Option 1 #

We contextually create both repository and team, then add a user to the team.

provider "github" {
  token = var.token
  organization = var.organization
}

variable "collaborator_name" {
  type = string
}

variable "organization" {
  type = string
}

variable "repository_name" {
  type = string
}

variable "token" {
  type = string
}

resource "github_repository" "this" {
  name = var.repository_name
}

resource "github_membership" "this" {
  username = var.collaborator_name
  role     = "member"
}

resource "github_team" "admins" {
  name = "admins"
}

resource "github_team_repository" "this" {
  team_id    = github_team.admins.id
  repository = github_repository.this.name
  permission = "admin"
}

resource "github_team_membership" "this" {
  team_id  = github_team.admins.id
  username = var.collaborator_name
  role     = "member"
}

The above snippet does what we want but is only partly reusable because it can only add one user to the team; sure enough, we could change it to take as input an array of users to add to the team (and that could be enough, depending on your situation) but let’s take another route instead.

Option 2 #

In this scenario, we split the above snippet into:

one module responsible for setting up the repository and an admins team, and
another one responsible for adding members to the team

This approach is useful when the main resource (github_repository, in this example) is only created once and then used by other resources (github_membership, in this example), some of which may be added or removed after the resource creation.

# ./repo/main.tf

variable "organization" {
  type = string
}

variable "repository_name" {
  type = string
}

variable "token" {
  type = string
}

provider "github" {
  token = var.token
  organization = var.organization
}

resource "github_repository" "this" {
  name = var.repository_name
}

resource "github_team" "admins" {
  name = "admins"
}

resource "github_team_repository" "this" {
  team_id    = github_team.admins.id
  repository = github_repository.this.name
  permission = "admin"
}

output "repo" {
  value = {
    admins_team_id = github_team.admins.id
  }
}

# ./users/main.tf

variable "admins_team_id" {
  type = string
}

variable "collaborator_name" {
  type = string
}

resource "github_membership" "this" {
  username = var.collaborator_name
  role     = "member"
}

resource "github_team_membership" "this" {
  team_id  = var.admins_team_id
  username = var.collaborator_name
  role     = "member"
}

# ./main.tf

module "my_repo" {
  source = "./repo/"

  organization = "supercows"
  token = "xyz"
  repository_name = "my_repo"
}

output "my_repo" {
  value = module.my_repo
}

module "my_user" {
  source = "./users/"

  admins_team_id = module.my_repo.repo.admins_team_id
  collaborator_name = "foobar"
}

Note how we read admins_team_id from the output of my_repo module: we need to do so because team_id is a generated field, so we have no way to know its value before the resource is generated.

Option 3 #

Let’s now explore the third and last scenario, in which the users module reads admins_team_id from repo’s remote state; in this scenario, it’s important to note that:

since the remote state is only persisted when a configuration has already been applied, this approach only works when repo has already been created running terraform apply!
Terraform locks the remote state whenever a module is accessing it: other attempts to access it from other modules will fail until the lock is not released.

# ./repo/main.tf

terraform {
  backend "local" {
    path = "/tmp/terraform.tfstate"
  }
}

# Note: the rest of this file is unchanged

# ./users/main.tf

data "terraform_remote_state" "my_repo" {
  backend = "local"

  config = {
    path = "/tmp/terraform.tfstate"
  }
}

variable "collaborator_name" {
  type = string
}

resource "github_membership" "this" {
  username = var.collaborator_name
  role     = "member"
}

resource "github_team_membership" "this" {
  team_id  = data.terraform_remote_state.my_repo.outputs.admins_team_id
  username = var.collaborator_name
  role     = "member"
}

# ./main.tf

module "my_repo" {
  source = "./repo/"

  organization = "supercows"
  token = "xyz"
  repository_name = "my_repo"
}

module "my_user" {
  source = "./users/"

  collaborator_name = "foobar"
}

Note how ./repo/main.tf is now writing its state to /tmp/terraform.tfstate and ./users/main.tf is reading from it.

Conclusion #

Both Option 2 and 3 enable us to compose modules in order to avoid code duplication in our codebase
Whenever possible, Option 2 is preferable because it introduces less complexity than Option 3
Option 3 is a valid last-resort solution for when all other options are not viable