Setting up a monorepo in Gitlab

A monorepo is an alternative approach to individual repositories, where all your projects are stored in a single repository. In recent years, it has been promoted by big names like Google and Uber (just to name a couple of them) as a way to simplify dependency management, promote code reuse, and enable large-scala code refactoring.

Let’s dive right into it by looking at a monorepo containing two Scala projects:

/
|_ foo
  |_ project
  |_ src
  |_ build.sbt
  |_ .gitlab-ci.yml (*)
|_ bar
  |_ project
  |_ src
  |_ build.sbt
  |_ .gitlab-ci.yml (*)
|_ .gitlab-ci.yml

(*) Even though it’s not mandatory to have one .gitlab-ci.yml file per project, I prefer this approach: in this way, each .gitlab-ci.yml file is only aware of (and responsible for) its own project.

Let’s see how these files would look like, starting from the main one:

# file: /.gitlab-ci.yml

foo:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "web"'
    - if: '$CI_PIPELINE_SOURCE == "push"'
      changes:
        - foo/**/*
  trigger:
    include:
      - local: '/foo/.gitlab-ci.yml'
    strategy: depend

bar:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "web"'
    - if: '$CI_PIPELINE_SOURCE == "push"'
      changes:
        - bar/**/*
  trigger:
    include:
      - local: '/bar/.gitlab-ci.yml'
    strategy: depend

Let’s dig a bit into the above file, starting from the rules section:

  rules:
    - if: '$CI_PIPELINE_SOURCE == "web"'

What the above snippet says is: “Trigger this pipeline whenever a user requests it on Gitlab’s Pipelines page” (if the above snippet is absent in the configuration, trying to trigger the pipeline from the UI will ony get you an error saying “There is no pipeline to run”).

 - if: '$CI_PIPELINE_SOURCE == "push"'
   changes:
     - foo/**/*

What the above snippet says is: “Trigger this pipeline in reaction to changes pushed to the repository and any file in foo/ has been changed”; this is handy to prevent pipelines from running when unrelated changes have been pushed.

Let’s move on to the trigger section:

  trigger:
    include:
      - local: '/foo/.gitlab-ci.yml'

This snippet imports a local file (= a file that exists in the same repository) describing the child project’s CI/CD.

    strategy: depend

Finally, this means that the trigger job will wait for the child pipeline to complete before marking itself as completed (it otherwise completes as soon as the child pipeline has been created).

In essence, this file only orchestrates its child pipelines. Let’s now look at one child pipeline:

# file: foo/.gitlab-ci.yml

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "parent_pipeline"'

stages:
  - test
  - deploy
  - ...

...

The workflow section creates a set of rules that apply for all stages in this file: in this case, it says that we accept to be triggered by the parent pipeline. The rest of the script is defined according to the specific needs of this project.