Tutorials

Build your own pipeline platform

Deploy your own CI/CD platform on Breqwatr Cloud with OpenTofu: Gitea for your repositories, and Woodpecker CI to run pipelines, on two instances you own. Deploy from the pipeline then uses it to deploy to Breqwatr Cloud.

Well-Architected

Pillar: Operational excellence. The platform that runs your changes is itself built from code, so it can be reviewed, rebuilt and changed the same way as anything it deploys.

Also touches: Security and sovereignty (your code and pipeline secrets stay on instances you run, with HTTPS open only to you) and Reliability (repositories and databases on a volume that outlasts a rebuild, as in Rebuild, don't repair).

Gitea and Woodpecker CI are open-source software you deploy and run yourself. They aren't Breqwatr services: you own the instances and everything on them.

What you'll build

your browser ── HTTPS (443, your IP only) ──┐
Let's Encrypt ── HTTP (80, challenge and redirect) ──┐
                                             │      │
                              floating IP, git.<domain> and ci.<domain>
                                             │
ci-platform-01   Caddy ── Gitea ── Woodpecker server
                 PostgreSQL       data volume: repositories, databases, certificates
                       │
          private network: 9000 (Woodpecker), 5432 (PostgreSQL)
                       │
ci-agent-01      Woodpecker agent, running pipeline steps in containers

Instance	Flavour	Runs	Reachable on
ci-platform-01	v1.c2r8	Caddy, Gitea, the Woodpecker server and PostgreSQL, under Docker Compose	80 from anywhere, 443 and 22 from your IP, 9000 and 5432 from the agent
ci-agent-01	v1.c1r4	The Woodpecker agent, under Docker Compose	22 from your IP

Component	Version
Gitea	28.1.0
Woodpecker CI	3.19.0
Caddy	2.11.7
PostgreSQL	18.4
Docker Engine and Compose	From Docker's apt repository for Ubuntu 24.04

Caddy answers on git.<domain> and ci.<domain>, and gets their certificates from Let's Encrypt. Gitea calls Woodpecker over the platform's own network, and the agent reaches the Woodpecker server and PostgreSQL over the private network.

Before you start

• Completed: A three-tier web app and Rebuild, don't repair.
• A domain you control, where you can add DNS records. The platform uses two names in it, git.<domain> and ci.<domain>.
• OpenTofu 1.13.1, and your clouds.yaml entry set in OS_CLOUD as in Launch your first instances. The commands are the same in PowerShell and in a macOS or Linux shell.
• The SSH key from Launch your first instances, first-instance.pub.
• Quota for 2 instances, 3 vCPUs, 12 GB of memory, 70 GB of storage and 1 floating IP.
• Time: about an hour, much of it waiting for the instances to install their software.
• The code: every file in this tutorial is also in pipeline-platform/ in learn-projects, the course's code repository, at release v1.2.0.

Step 1: Create the project

Create a folder called pipeline-platform with a templates folder inside it, and these files. Set your values in terraform.tfvars (in learn-projects it's terraform.tfvars.example, to copy):

domain           = "<your-domain>"
my_ip            = "<your-ip>/32"
external_network = "<external-network>"
linux_image      = "<linux-image>"
admin_email      = "<your-email>"

terraform.tfvars will hold the Woodpecker client secret too, so keep it out of version control, as in OpenTofu fundamentals.

versions.tf:

terraform {
  required_version = ">= 1.13"
  required_providers {
    openstack = {
      source  = "terraform-provider-openstack/openstack"
      version = "~> 3.4"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.6"
    }
  }
}

provider "openstack" {}

variables.tf. woodpecker_client and woodpecker_secret stay empty until step 6:

variable "prefix" {
  description = "Prefix for every resource name"
  type        = string
  default     = "ci"
}

variable "domain" {
  description = "Your domain; the platform answers on git.<domain> and ci.<domain>"
  type        = string
}

variable "my_ip" {
  description = "Your public IP address in CIDR form, for example 203.0.113.10/32"
  type        = string
}

variable "external_network" {
  description = "External network that floating IPs come from"
  type        = string
}

variable "linux_image" {
  description = "Name of the Ubuntu 24.04 image"
  type        = string
}

variable "admin_username" {
  description = "Gitea administrator, also the Woodpecker administrator"
  type        = string
  default     = "platform-admin"
}

variable "admin_email" {
  description = "Email address for the Gitea administrator"
  type        = string
}

variable "woodpecker_client" {
  description = "Client ID of the Woodpecker OAuth application in Gitea; empty until you create it"
  type        = string
  default     = ""
}

variable "woodpecker_secret" {
  description = "Client secret of the Woodpecker OAuth application in Gitea"
  type        = string
  default     = ""
  sensitive   = true
}

variable "subnet_cidr" {
  description = "Address range of the platform network"
  type        = string
  default     = "10.40.0.0/24"
}

variable "public_key_path" {
  description = "Path to your SSH public key"
  type        = string
  default     = "~/.ssh/first-instance.pub"
}

Step 2: The network and the security groups

network.tf creates the platform's network, subnet and router:

data "openstack_networking_network_v2" "external" {
  name = var.external_network
}

data "openstack_images_image_v2" "ubuntu" {
  name        = var.linux_image
  most_recent = true
}

resource "openstack_networking_network_v2" "ci" {
  name = "${var.prefix}-net"
}

resource "openstack_networking_subnet_v2" "ci" {
  name            = "${var.prefix}-subnet"
  network_id      = openstack_networking_network_v2.ci.id
  cidr            = var.subnet_cidr
  dns_nameservers = ["1.1.1.1", "1.0.0.1"]
}

resource "openstack_networking_router_v2" "ci" {
  name                = "${var.prefix}-router"
  external_network_id = data.openstack_networking_network_v2.external.id
}

resource "openstack_networking_router_interface_v2" "ci" {
  router_id = openstack_networking_router_v2.ci.id
  subnet_id = openstack_networking_subnet_v2.ci.id
}

security.tf keeps the platform closed except where it has to be open:

• Port 80 from anywhere. Let's Encrypt checks your domain over HTTP before it issues a certificate, and Caddy redirects every other request on port 80 to HTTPS.
• Port 443 from your IP, for Gitea and Woodpecker in your browser and for git push.
• Port 443 from the router's own address. The agent clones from https://git.<domain>, and that traffic reaches the platform through the router's address.
• Ports 9000 and 5432 from the agent's group, for the Woodpecker server and PostgreSQL, over the private network.
• Port 22 from your IP, on both instances.

resource "openstack_networking_secgroup_v2" "platform" {
  name = "${var.prefix}-platform"
}

resource "openstack_networking_secgroup_v2" "agent" {
  name = "${var.prefix}-agent"
}

locals {
  # The router's own address: traffic from the agent to git.<domain> comes from it
  router_ip = "${openstack_networking_router_v2.ci.external_fixed_ip[0].ip_address}/32"

  platform_rules = {
    acme  = { port = 80, from = "0.0.0.0/0" }
    https = { port = 443, from = var.my_ip }
    agent = { port = 443, from = local.router_ip }
    ssh   = { port = 22, from = var.my_ip }
  }
}

# Port 80 is open to the internet for Let's Encrypt's HTTP-01 challenge and the
# redirect to HTTPS; everything else is limited to you
resource "openstack_networking_secgroup_rule_v2" "platform" {
  for_each = local.platform_rules

  security_group_id = openstack_networking_secgroup_v2.platform.id
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = each.value.port
  port_range_max    = each.value.port
  remote_ip_prefix  = each.value.from
}

# The agent reaches the Woodpecker server (9000) and PostgreSQL (5432) over the private network
resource "openstack_networking_secgroup_rule_v2" "from_agent" {
  for_each = { grpc = 9000, postgres = 5432 }

  security_group_id = openstack_networking_secgroup_v2.platform.id
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = each.value
  port_range_max    = each.value
  remote_group_id   = openstack_networking_secgroup_v2.agent.id
}

resource "openstack_networking_secgroup_rule_v2" "agent_ssh" {
  security_group_id = openstack_networking_secgroup_v2.agent.id
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = 22
  port_range_max    = 22
  remote_ip_prefix  = var.my_ip
}

Step 3: The platform instance

platform.tf builds the platform instance, with a fixed port so its address doesn't change when it's rebuilt, and a data volume protected with prevent_destroy:

resource "openstack_compute_keypair_v2" "ci" {
  name       = "${var.prefix}-key"
  public_key = file(pathexpand(var.public_key_path))
}

resource "random_password" "postgres" {
  for_each = toset(["admin", "gitea", "woodpecker", "tfstate"])
  length   = 24
  special  = false
}

resource "random_password" "gitea_admin" {
  length  = 20
  special = false
}

resource "random_password" "agent" {
  length  = 32
  special = false
}

# Gitea's repositories, the databases and Caddy's certificates live on this
# volume, so they outlast any rebuild of the platform instance
resource "openstack_blockstorage_volume_v3" "data" {
  name = "${var.prefix}-platform-data"
  size = 20

  lifecycle {
    prevent_destroy = true
  }
}

# A fixed address, so the agent and the pipeline always find the platform
resource "openstack_networking_port_v2" "platform" {
  name               = "${var.prefix}-platform"
  network_id         = openstack_networking_network_v2.ci.id
  security_group_ids = [openstack_networking_secgroup_v2.platform.id]

  fixed_ip {
    subnet_id = openstack_networking_subnet_v2.ci.id
  }
}

resource "openstack_compute_instance_v2" "platform" {
  name        = "${var.prefix}-platform-01"
  flavor_name = "v1.c2r8"
  key_pair    = openstack_compute_keypair_v2.ci.name
  user_data = templatefile("${path.module}/templates/platform.sh", {
    domain            = var.domain
    private_ip        = openstack_networking_port_v2.platform.all_fixed_ips[0]
    admin_username    = var.admin_username
    admin_email       = var.admin_email
    admin_password    = random_password.gitea_admin.result
    pg_admin          = random_password.postgres["admin"].result
    pg_gitea          = random_password.postgres["gitea"].result
    pg_tfstate        = random_password.postgres["tfstate"].result
    pg_woodpecker     = random_password.postgres["woodpecker"].result
    agent_secret      = random_password.agent.result
    woodpecker_client = var.woodpecker_client
    woodpecker_secret = var.woodpecker_secret
  })

  block_device {
    uuid                  = data.openstack_images_image_v2.ubuntu.id
    source_type           = "image"
    destination_type      = "volume"
    volume_size           = 20
    boot_index            = 0
    delete_on_termination = true
  }

  network {
    port = openstack_networking_port_v2.platform.id
  }

  depends_on = [openstack_networking_router_interface_v2.ci]
}

resource "openstack_compute_volume_attach_v2" "data" {
  instance_id = openstack_compute_instance_v2.platform.id
  volume_id   = openstack_blockstorage_volume_v3.data.id
}

resource "openstack_networking_floatingip_v2" "platform" {
  pool       = var.external_network
  port_id    = openstack_networking_port_v2.platform.id
  depends_on = [openstack_networking_router_interface_v2.ci]
}

templates/platform.sh runs at first boot, and again every time the instance is rebuilt:

1. It formats the data volume the first time only, and mounts it at /srv/data.
2. It installs Docker Engine and Compose from Docker's own apt repository.
3. It writes the Compose file and Caddy's configuration, starts PostgreSQL, and creates a database each for Gitea, Woodpecker and the pipeline's OpenTofu state if they don't exist yet.
4. It starts Gitea and Caddy, plus the Woodpecker server once woodpecker_client is set, and creates the Gitea administrator the first time.

Gitea's sign-up page is turned off (DISABLE_REGISTRATION): the only account is the administrator, until you add people yourself.

#!/bin/bash
set -e

# The data volume: format it the first time only, and mount it on every boot
while [ ! -b /dev/vdb ]; do sleep 2; done
blkid /dev/vdb || mkfs.ext4 -L platform-data /dev/vdb
mkdir -p /srv/data
echo 'LABEL=platform-data /srv/data ext4 defaults,nofail 0 2' >> /etc/fstab
mount /srv/data
mkdir -p /srv/data/postgres /srv/data/gitea /srv/data/caddy

# Docker Engine and Compose from Docker's own apt repository
apt-get update
apt-get install -y ca-certificates curl
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" > /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

mkdir -p /srv/platform
cd /srv/platform

# A database each for Gitea, Woodpecker and the pipeline's OpenTofu state,
# created if missing on every boot
cat > databases.sql <<'SQL'
SELECT 'CREATE ROLE gitea LOGIN' WHERE NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'gitea')\gexec
SELECT 'CREATE ROLE woodpecker LOGIN' WHERE NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'woodpecker')\gexec
SELECT 'CREATE ROLE tfstate LOGIN' WHERE NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'tfstate')\gexec
ALTER ROLE gitea PASSWORD '${pg_gitea}';
ALTER ROLE woodpecker PASSWORD '${pg_woodpecker}';
ALTER ROLE tfstate PASSWORD '${pg_tfstate}';
SELECT 'CREATE DATABASE gitea OWNER gitea' WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'gitea')\gexec
SELECT 'CREATE DATABASE woodpecker OWNER woodpecker' WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'woodpecker')\gexec
SELECT 'CREATE DATABASE tfstate OWNER tfstate' WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'tfstate')\gexec
SQL

cat > Caddyfile <<'CADDY'
git.${domain} {
    reverse_proxy gitea:3000
}

ci.${domain} {
    reverse_proxy woodpecker-server:8000
}
CADDY

cat > compose.yaml <<'COMPOSE'
services:
  postgres:
    image: postgres:18.4
    restart: unless-stopped
    environment:
      POSTGRES_PASSWORD: ${pg_admin}
    volumes:
      - /srv/data/postgres:/var/lib/postgresql
    ports:
      - "${private_ip}:5432:5432"

  gitea:
    image: gitea/gitea:28.1.0
    restart: unless-stopped
    environment:
      USER_UID: "1000"
      USER_GID: "1000"
      GITEA__database__DB_TYPE: postgres
      GITEA__database__HOST: postgres:5432
      GITEA__database__NAME: gitea
      GITEA__database__USER: gitea
      GITEA__database__PASSWD: ${pg_gitea}
      GITEA__server__DOMAIN: git.${domain}
      GITEA__server__ROOT_URL: https://git.${domain}/
      GITEA__server__DISABLE_SSH: "true"
      GITEA__service__DISABLE_REGISTRATION: "true"
      GITEA__security__INSTALL_LOCK: "true"
      GITEA__webhook__ALLOWED_HOST_LIST: private
    volumes:
      - /srv/data/gitea:/data
    depends_on:
      - postgres
%{ if woodpecker_client != "" ~}

  woodpecker-server:
    image: woodpeckerci/woodpecker-server:v3.19.0
    restart: unless-stopped
    environment:
      WOODPECKER_HOST: https://ci.${domain}
      WOODPECKER_EXPERT_WEBHOOK_HOST: http://woodpecker-server:8000
      WOODPECKER_GITEA: "true"
      WOODPECKER_GITEA_URL: https://git.${domain}
      WOODPECKER_GITEA_CLIENT: ${woodpecker_client}
      WOODPECKER_GITEA_SECRET: ${woodpecker_secret}
      WOODPECKER_AGENT_SECRET: ${agent_secret}
      WOODPECKER_ADMIN: ${admin_username}
      WOODPECKER_DATABASE_DRIVER: postgres
      WOODPECKER_DATABASE_DATASOURCE: postgres://woodpecker:${pg_woodpecker}@postgres:5432/woodpecker?sslmode=disable
    depends_on:
      - postgres
    ports:
      - "${private_ip}:9000:9000"
%{ endif ~}

  caddy:
    image: caddy:2.11.7
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - /srv/data/caddy:/data
    networks:
      default:
        # So Woodpecker reaches Gitea, and Gitea reaches Woodpecker, on this host
        aliases:
          - git.${domain}
          - ci.${domain}
COMPOSE

# Pull the images, retrying if a registry is briefly unreachable
until docker compose pull; do sleep 10; done

docker compose up -d postgres
until docker compose exec -T postgres pg_isready -U postgres > /dev/null 2>&1; do sleep 2; done
docker compose exec -T postgres psql -v ON_ERROR_STOP=1 -U postgres < databases.sql
docker compose up -d

# The Gitea administrator, the first time only
until docker compose exec -T -u git gitea gitea admin user list > /dev/null 2>&1; do sleep 5; done
docker compose exec -T -u git gitea gitea admin user list | grep -qw '${admin_username}' || \
  docker compose exec -T -u git gitea gitea admin user create --admin \
    --username '${admin_username}' --email '${admin_email}' --password '${admin_password}' --must-change-password=false

Step 4: The agent

agent.tf builds the agent instance. The agent connects to the Woodpecker server's private address and runs each pipeline step in a container:

resource "openstack_compute_instance_v2" "agent" {
  name            = "${var.prefix}-agent-01"
  flavor_name     = "v1.c1r4"
  key_pair        = openstack_compute_keypair_v2.ci.name
  security_groups = [openstack_networking_secgroup_v2.agent.name]
  user_data = templatefile("${path.module}/templates/agent.sh", {
    server       = "${openstack_networking_port_v2.platform.all_fixed_ips[0]}:9000"
    agent_secret = random_password.agent.result
  })

  block_device {
    uuid                  = data.openstack_images_image_v2.ubuntu.id
    source_type           = "image"
    destination_type      = "volume"
    volume_size           = 30
    boot_index            = 0
    delete_on_termination = true
  }

  network {
    uuid = openstack_networking_network_v2.ci.id
  }

  depends_on = [openstack_networking_router_interface_v2.ci]
}

templates/agent.sh:

#!/bin/bash
set -e

# Docker Engine and Compose from Docker's own apt repository
apt-get update
apt-get install -y ca-certificates curl
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" > /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

mkdir -p /srv/agent
cd /srv/agent
cat > compose.yaml <<'COMPOSE'
services:
  woodpecker-agent:
    image: woodpeckerci/woodpecker-agent:v3.19.0
    restart: unless-stopped
    environment:
      WOODPECKER_SERVER: ${server}
      WOODPECKER_AGENT_SECRET: ${agent_secret}
      WOODPECKER_MAX_WORKFLOWS: "2"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - agent-config:/etc/woodpecker

volumes:
  agent-config:
COMPOSE

# Pull the image, retrying if the registry is briefly unreachable
until docker compose pull; do sleep 10; done
docker compose up -d

outputs.tf:

output "floating_ip" {
  description = "Point git.<domain> and ci.<domain> at this address"
  value       = openstack_networking_floatingip_v2.platform.address
}

output "gitea_url" {
  value = "https://git.${var.domain}/"
}

output "woodpecker_url" {
  value = "https://ci.${var.domain}/"
}

output "gitea_admin_password" {
  value     = random_password.gitea_admin.result
  sensitive = true
}

output "pg_conn_str" {
  description = "Connection string the pipeline uses for OpenTofu state"
  value       = "postgres://tfstate:${random_password.postgres["tfstate"].result}@${openstack_networking_port_v2.platform.all_fixed_ips[0]}:5432/tfstate?sslmode=disable"
  sensitive   = true
}

Step 5: Build it and point your domain at it

tofu init
tofu plan
tofu apply

The plan shows 26 resources to add. Type yes when asked. When it finishes, read the floating IP:

tofu output floating_ip

At your DNS provider, add two A records for that address: git.<domain> and ci.<domain>. Check that they resolve:

nslookup git.<domain>

Caddy asks Let's Encrypt for a certificate as soon as the names resolve to the platform, and keeps retrying until they do. Allow a few minutes after the DNS change, then open https://git.<domain>/ in your browser. The page is Gitea's, with a valid certificate.

Step 6: Connect Woodpecker to Gitea

Sign in to Gitea as platform-admin, with the password from:

tofu output -raw gitea_admin_password

Then create the OAuth application Woodpecker signs in with:

1. Open your account menu at the top right, choose Settings, then Applications.
2. Under Manage OAuth2 Applications, open Create a new OAuth2 Application.
3. Set Application Name to Woodpecker and Redirect URIs to https://ci.<domain>/authorize. Leave Confidential Client ticked.
4. Choose Create Application. Gitea shows a Client ID and a Client Secret. The secret isn't shown again.

Add them to terraform.tfvars:

woodpecker_client = "<client-id>"
woodpecker_secret = "<client-secret>"

Apply again:

tofu apply

The plan replaces ci-platform-01, because its first-boot script now includes the Woodpecker server. The data volume, the port and the floating IP stay, so Gitea comes back with everything in it, and the certificates don't need to be issued again. Allow a few minutes for the rebuild.

Open https://ci.<domain>/. Choose git.<domain> under Sign in to Woodpecker with, and in Gitea choose Authorize Application. Woodpecker opens on an empty Repositories list.

Check it worked

• https://git.<domain>/ and https://ci.<domain>/ open in your browser with valid certificates.
• http://git.<domain>/ redirects to https://git.<domain>/.
• From any network other than yours, https://git.<domain>/ doesn't answer.
• Gitea's footer shows version 28.1.0, and Woodpecker's top bar shows 3.19.0.

Clean up

Deploy from the pipeline uses this platform. If you're going on to it now, keep everything.

Otherwise, prevent_destroy stops tofu destroy, as in Rebuild, don't repair. Remove the lifecycle block from the data volume in platform.tf, then:

tofu destroy

Type yes when asked. OpenTofu deletes the instances, the data volume with your repositories, and everything else it created. Delete the two DNS records at your DNS provider too.

What you learned

• A pipeline platform is infrastructure like any other: built from code on instances you own, and rebuilt from code when its configuration changes.
• Data that has to survive goes on its own volume, protected with prevent_destroy, and the first-boot script picks it up again; see Pets vs cattle.
• Security groups can keep a service open to the internet only for what has to be public, here the certificate challenge, and limit everything else to you and to the instances that need it; see Networking model.

Next

• Deploy from the pipeline: use the platform to deploy to Breqwatr Cloud from pull requests and merges.
• Gitea documentation and Woodpecker CI documentation: everything else they can do.