node-gitlab-ci

Create dynamic GitLab CI pipelines in JavaScript or TypeScript for each project. Reuse and inherit instructions and avoid duplicate code!

Continuous Integration (CI) and Continuous Deployment (CD) are fantastic concepts for process automation in software development. We love GitLab CI because it implements the concept in an integrated solution with powerful configuration capabilities. However, pipeline configurations are stored in a static .gitlab-ci.yml file.

node-gitlab-ci allows you to develop pipeline configurations dynamically in TypeScript and avoid duplicates in the statements with programming concepts like inheritance or functions. This way you can perfectly integrate e.g. monorepos with many similiar projects into the CI/CD.

Installation

Navigate to your repository and install the package via yarn or npm:

1# Yarn
2yarn add -D node-gitlab-ci
3
4# NPM
5npm install --save-dev node-gitlab-ci

Afterwards, create a .gitlab-ci.yml file with the following content:

1# CI pipeline is dynamically created through `node-gitlab-ci`, please checkout `.gitlab-ci.ts`!
2
3ts config:
4    image: devowliode/node-gitlab-ci:latest
5    stage: build
6    script: node-gitlab-ci create-yml
7    artifacts:
8        paths:
9            - .gitlab-ci.ts.yml
10
11trigger pipeline:
12    stage: test
13    trigger:
14        strategy: depend
15        include:
16            - artifact: .gitlab-ci.ts.yml
17              job: ts config

What does this statement do? The first job creates the .gitlab-ci.ts.yml file dynamically and the second job triggers the child pipeline. Learn more about this in the GitLab documentation for child pipelines. It is recommended to add the .gitlab-ci.ts.yml file to your .gitignore file.

Usage

Your first .gitlab-ci.ts

It is a good practice to create a .gitlab-ci.ts in the root directory of your repository:

1import { Config, CreateConfigFunction } from "node-gitlab-ci";
2
3const createConfig: CreateConfigFunction = async () => {
4    const config = new Config();
5
6    config.stages("build", "test");
7
8    config.defaults({
9        image: "alpine:latest",
10    });
11
12    // Setting variables globally or per job
13    config.variable("DOCKER_DRIVER", "overlay2");
14
15    // Run a job only in production branch
16    config.job(
17        "only production",
18        {
19            only: {
20                refs: ["master"],
21            },
22        },
23        true // Creates a hidden job (prefixed with a dot)
24    );
25
26    // Allows you to include further configurations by glob patterns
27    await config.include(__dirname, ["devops/.gitlab/*.ts"]);
28    await config.include(__dirname, ["packages/*/devops/.gitlab/.gitlab-ci.ts"]);
29
30    return config;
31};
32
33export { createConfig };

The complete GitLab CI pipeline configuration is typed. Give it a try within your IDE and autocomplete!

Note: You can not import (ES6) or require (ES5) all your installed modules. At the time of creating the dynamic pipeline, it is executed within devowliode/node-gitlab-ci docker container and there are only the node modules like fs, path, ... available. Please read more about it below "Use installed modules".

Dry run locally

If you have successfully created the above file open a terminal session, navigate to your repository and:

1# Yarn
2yarn node-gitlab-ci create-yml
3
4# NPM
5npx node-gitlab-ci create-yml

A file .gitlab-ci.ts.yml will be created.

How include works

The most interesting part of node-gitlab-ci is how include works (for example you are using yarn workspaces or lerna). With Config#include you can dynamically include files by a glob pattern:

1// Do not forget the await!
2await config.include(__dirname, ["packages/*/devops/.gitlab/.gitlab-ci.ts"]);

The extension file packages/test/devops/.gitlab/.gitlab-ci.ts must look like this:

1import { ExtendConfigFunction } from "node-gitlab-ci";
2
3const extendConfig: ExtendConfigFunction = async (config) => {
4    // Create a job
5    config.job(/* [...] */);
6
7    // You can include further files
8    await config.include(__dirname, ["./stage-*.ts"]);
9};
10
11export { extendConfig };

How extends work

node-gitlab-ci resolves automatically the extends keyword for you so you can fully profit from nested jobs without limitations (e. g. nested extends with same keys like only are no covered by GitLab CI). This is done a deep merge mechanism:

1config.job(
2    "only production",
3    {
4        only: {
5            refs: ["master"],
6        },
7    },
8    true
9);
10
11config.extends(".only production", "my-job", {
12    script: ["echo This job runs only in production!"],
13});

You can also extend from multiple jobs:

1config.job(
2    "common files changed",
3    {
4        only: {
5            changes: ["common/**/*"],
6        },
7    },
8    true
9);
10
11config.extends([".only production", ".common files changed"], "my-job", {
12    script: ["echo This job runs only in production and when common files got changed!"],
13});

How macro works

With macros you can define callbacks and consume them with a set of parameters so you can dynamically create jobs with "hard coded" variables. The most excited use case is only in a monorepo:

1type EsLintMacroArgs = MacroArgs & {
2    prefix: string;
3};
4
5config.macro<EsLintMacroArgs>("lint eslint", (self, { prefix }) => {
6    config.extends([`.common files changed`, `.lint eslint`], `${prefix} lint eslint`, {
7        only: {
8            changes: [`packages/${packageName}/{lib,scripts,test}/**/*.{js,jsx,tsx,ts}`],
9        },
10    });
11});

And in your package you can use this macro as follow:

1config.from<EsLintMacroArgs>("lint eslint", { prefix: "utils" });

Interact with the GitLab REST API

This package comes with @gitbeaker/node bundled, so you can directly communicate with the GitLab REST API. The API handler is brought to you with the following functionality:

1// List last 500 jobs in your project
2config.api.Jobs.all(1 /* your project id */, {
3    maxPages: 5,
4    perPage: 100,
5});

Get changed files

If you need to detect changed file while child pipeline generation, you can use the following:

1const changed = config.hasChanged(); // returns string[]
2const specificFileHasChanged = config.hasChanged(/^packages\/my-package\//gm);

Use installed modules

As mentioned previously you can not import or require any module. If you want to do so, you need to consider the following:

• Open a Pull Request or Issue here and ask to install the module globally in the image
• Create your own Dockerfile with the modules installed globally (e. g. npm install --global fs-extra), extended from this dockerfile
• Modify the ts config job and install the modules globally or locally

Todo:

This repository is still in beta phase and the following things should be done:

• Use debug package instead of console.log
• Create GitLab CI with semantic-release to automatically publish the package to npmjs.org
• Create and push docker image through CI instead of hub.docker.com
• Write Tests

License

MIT