Automate CI/CD Pipeline with GitLab Runner and DigitalOcean Droplet

By Thilina Waasalage

Pipeline Structure

GitLab is an open-source collaboration platform that provides powerful features beyond hosting a code repository. You can track issues, host packages and registries, maintain Wikis, set up continuous integration (CI) and continuous deployment (CD) pipelines, and more.

In this tutorial you’ll build a continuous deployment pipeline with GitLab. You will configure the pipeline to build a Docker image, push it to the GitLab container registry, and deploy it to your server using SSH. The pipeline will run for each commit pushed to the repository.

You will deploy a small, static web page, but the focus of this tutorial is configuring the CD pipeline. The static web page is only for demonstration purposes; you can apply the same pipeline configuration using other Docker images for the deployment as well.

When you have finished this tutorial, you can visit http://your_server_IP in a browser for the results of the automatic deployment.

Prerequisites

Before we get started, ensure you have:

A GitLab account.
A DigitalOcean account.
A basic understanding of Git and GitLab.
A Linux-based DigitalOcean Droplet set up (Ubuntu 22.04 is recommended).

Step 1: Create a New Repository in GitLab

Log in to your GitLab account.
Click on the “New Project” button.
Choose a blank project or import an existing one.
Name your project and initialize it with a README file.

Creating a project

Create a new file index.html and add the following HTML to the file body:

<html>
    <body>
        <h1>My Personal Website</h1>
    </body>
</html>

Dockerfiles are recipes used by Docker to build Docker images. Let’s create a Dockerfile to copy the HTML file into an Nginx image.

Create a new file Dockerfile and add these instructions to the file body:

FROM nginx:1.18COPY index.html /usr/share/nginx/html

Step 2: Set Up Your DigitalOcean Droplet

Log in to your DigitalOcean account.
Click on “Create” and select “Droplets.”
Choose the Ubuntu image and select your desired plan.
Create the Droplet and note its IP address.

Step 3: Install GitLab Runner on the Droplet

SSH into your Droplet:

ssh root@your_droplet_ip

2. Install GitLab Runner:

In order to install the gitlab-runner service, you’ll add the official GitLab repository. Download and inspect the install script:

curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

To install the gitlab-runner package, run the following command in terminal:

sudo apt-get install gitlab-runner

After the curl command finishes execution, verify the installation by checking the service status:

systemctl status gitlab-runner

You will have active (running) in the output:

gitlab-runner.service - GitLab Runner
     Loaded: loaded (/etc/systemd/system/gitlab-runner.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2024-08-25 13:52:51 UTC; 3 weeks 4 days ago
   Main PID: 11511 (gitlab-runner)
      Tasks: 9 (limit: 4647)
     Memory: 24.0M
        CPU: 24min 28.128s
     CGroup: /system.slice/gitlab-runner.service
             └─11511 /usr/bin/gitlab-runner run --config /etc/gitlab-runner/config.toml --working-directory /home>

3. Register the GitLab Runner:

To register the runner, you need to get the project token and the GitLab URL:

In your GitLab project, navigate to Settings > CI/CD > Runners > Expand.

In the Project runners section, click on New project runner and follow the form to create a new runner for your project.

Once a runner is in place, you’ll find the registration token and the GitLab URL. Copy both to a text editor; you’ll need them for the next command. They will be referred to as https://your_gitlab.com and project_token.

Creating a Runner in gitlab

Back to your terminal, register the runner for your project:

sudo gitlab-runner register -n --url https://your_gitlab.com --registration-token project_token --executor docker --description "Deployment Runner" --docker-image "docker:stable" --tag-list deployment --docker-privileged

-n executes the register command non-interactively (we specify all parameters as command options).
url is the GitLab URL you copied from the runners page in GitLab.
registration-token is the token you copied from the runners page in GitLab.
executor is the executor type. docker executes each CI/CD job in a Docker container
description is the runner’s description, which will show up in GitLab.
docker-image is the default Docker image to use in CI/CD jobs, if not explicitly specified.
tag-list is a list of tags assigned to the runner. Tags can be used in a pipeline configuration to select specific runners for a CI/CD job. The deployment tag will allow you to refer to this specific runner to execute the deployment job.
docker-privileged executes the Docker container created for each CI/CD job in privileged mode.

After executing the gitlab-runner register command, you will receive the following

Output
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

Activated runner

Step 4: Creating a Deployment User

You are going to create a user that is dedicated for the deployment task. You will later configure the CI/CD pipeline to log in to the server with that user.

On your server, create a new user:

sudo adduser deployer

Add the user to the Docker group:

sudo usermod -aG docker deployer

This permits deployer to execute the docker command, which is required to perform the deployment.

Step 5: Setting Up an SSH Key

You are going to create an SSH key for the deployment user. GitLab CI/CD will later use the key to log in to the server and perform the deployment routine.

Let’s start by switching to the newly created deployer user for whom you’ll generate the SSH key:

su deployer

You’ll be prompted for the deployer password to complete the user switch.

Next, run the following command and confirm both questions with ENTER to create a 4096-bit SSH key and store it in the default location with an empty passphrase

ssh-keygen -b 4096

To authorize the SSH key for the deployer user, you need to append the public key to the authorized_keys file:

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

Step 5: Storing the Private Key in a GitLab CI/CD Variable

Start by showing the SSH private key:

cat ~/.ssh/id_rsa

Copy the output to your clipboard. Make sure to add a linebreak after — — -END RSA PRIVATE KEY — — -:

Now navigate to Settings > CI / CD > Variables in your GitLab project and click Add Variable. Fill out the form as follows:

Key: ID_RSA
Value: Paste your SSH private key from your clipboard (including a line break at the end).
Type: File
Environment Scope: All (default)
Protect variable: Checked
Mask variable: Unchecked

A file containing the private key will be created on the runner for each CI/CD job and its path will be stored in the $ID_RSA environment variable.

Create another variable with your server IP. Click Add Variable and fill out the form as follows:

Key: SERVER_IP
Value: your_server_IP
Type: Variable
Environment scope: All (default)
Protect variable: Checked
Mask variable: Checked

Finally, create a variable with the login user. Click Add Variable and fill out the form as follows:

Key: SERVER_USER
Value: deployer
Type: Variable
Environment scope: All (default)
Protect variable: Checked
Mask variable: Checked

You have now stored the private key in a GitLab CI/CD variable, which makes the key available during pipeline execution. In the next step, you’re moving on to configuring the CI/CD pipeline.

Step 6: Create a .gitlab-ci.yml File

In the root of your GitLab repository, create a file named .gitlab-ci.yml. This file defines your CI/CD pipeline.

stages:
  - publish
  - deploy

variables:
  TAG_LATEST: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:latest
  TAG_COMMIT: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:$CI_COMMIT_SHORT_SHA

publish:
  image: docker:latest
  stage: publish
  services:
    - docker:dind
  script:
    - docker build -t $TAG_COMMIT -t $TAG_LATEST .
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker push $TAG_COMMIT
    - docker push $TAG_LATEST

deploy:
  image: alpine:latest
  stage: deploy
  tags:
    - deployment
  script:
    - chmod og= $ID_RSA
    - apk update && apk add openssh-client
    - ssh -i $ID_RSA -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY"
    - ssh -i $ID_RSA -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker pull $TAG_COMMIT"
    - ssh -i $ID_RSA -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker container rm -f my-app || true"
    - ssh -i $ID_RSA -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker run -d -p 8080:80 --name my-app $TAG_COMMIT"
  environment:
    name: production
    url: http://your_server_IP
  only:
    - main

CI_REGISTRY_IMAGE: Represents the URL of the container registry tied to the specific project
CI_COMMIT_REF_NAME: The branch or tag name for which project is built.
CI_COMMIT_SHORT_SHA: The first eight characters of the commit revision for which the project is built.
image is the Docker image to use for this job. The GitLab runner will create a Docker container for each job and execute the script within this container. docker:latest image ensures that the docker command will be available.
stage assigns the job to the publish stage.
services specifies Docker-in-Docker — the dind service. This is the reason why you registered the GitLab runner in privileged mode.
docker build Builds the Docker image based on the Dockerfile and tags it with the latest commit tag defined in the variables section.
docker login Logs Docker in to the project’s container registry. You use the predefined variable $CI_JOB_TOKEN as an authentication token. GitLab will generate the token and stay valid for the job’s lifetime.
docker push Pushes both image tags to the container registry.
chmod og= $ID_RSA: Revokes all permissions for group and others from the private key, such that only the owner can use it. This is a requirement, otherwise SSH refuses to work with the private key.
apk update && apk add openssh-client: Updates Alpine’s package manager (apk) and installs the openssh-client, which provides the ssh command.
docker login Logs Docker in to the container registry.
docker pull Pulls the latest image from the container registry.
docker container rm Deletes the existing container if it exists. || true makes sure that the exit code is always successful, even if there was no container running by the name my-app.
docker run Starts a new container using the latest image from the registry. The container will be named my-app. Port 80 on the host will be bound to port 80 of the container (the order is -p host:container). -d starts the container in detached mode, otherwise the pipeline would be stuck waiting for the command to terminate.

Finally click Commit changes at the bottom of the page in GitLab to create the .gitlab-ci.yml file. Alternatively, when you have cloned the Git repository locally, commit and push the file to the remote.

You’ve created a GitLab CI/CD configuration for building a Docker image and deploying it to your server. In the next step you are validating the deployment.

Step 7: Validating the Deployment

Now you’ll validate the deployment in various places of GitLab as well as on your server and in a browser.

When a .gitlab-ci.yml file is pushed to the repository, GitLab will automatically detect it and start a CI/CD pipeline. At the time you created the .gitlab-ci.yml file, GitLab started the first pipeline.

Go to Build > Pipelines in your GitLab project to see the pipeline’s status. If the jobs are still running/pending, wait until they are complete. You will see a Passed pipeline with two green checkmarks, denoting that the publish and deploy job ran successfully.

Pipeline

Let’s examine the pipeline. Click the passed button in the Status column to open the pipeline’s overview page. You will get an overview of general information such as:

Execution duration of the whole pipeline.
For which commit and branch the pipeline was executed.
Related merge requests. If there is an open merge request for the branch in charge, it would show up here.
All jobs executed in this pipeline as well as their status.

Open http://your_server_IP in a browser and you should see the My Website headline.

Website deployed

Conclusion

In this article, you have configured a continuous deployment pipeline with GitLab CI/CD. You created a small web project consisting of an HTML file and a Dockerfile. Then you configured the .gitlab-ci.yml pipeline configuration to:

Build the Docker image.
Push the Docker image to the container registry.
Log in to the server, pull the latest image, stop the current container, and start a new one.

GitLab will now deploy the web page to your server for each push to the repository.

SL

MY

ID

Google reCaptcha:

SSL (Secure Sockets Layer):

W3C Compliance:

Accessibility Widget:

Carbon Neutrality: