Tutorials
Deploy from the pipeline¶
Use the platform from Build your own pipeline platform to deploy to Breqwatr Cloud: every pull request runs tofu fmt, validate and plan, and every merge to main runs tofu apply. Then make a change and follow it from commit to the deployed site.
Well-Architected
Pillar: Operational excellence. Every change is a commit, checked and planned before it's merged, and deployed the same way every time.
Also touches: Security and sovereignty (a restricted application credential with an expiry, kept as a pipeline secret rather than in the repository) and Reliability (the workload is replaced from code on every change, as in Rebuild, don't repair).
What you'll build¶
A repository called web in Gitea, holding a small site: one nginx instance with its own network, security group and floating IP. Its pipeline in Woodpecker:
| Event | Steps |
|---|---|
| A pull request | check: tofu init, tofu fmt -check, tofu validate, tofu plan |
A push or merge to main |
check, then apply: tofu apply |
OpenTofu keeps the site's state in PostgreSQL on the platform, with the pg backend, so every pipeline run works from the same state.
Before you start¶
- Completed: Build your own pipeline platform, with the platform still running.
- On your computer: Git, OpenTofu 1.13.1 as in Launch your first instances, SSH and a browser.
- Windows: Git for Windows, from git-scm.com, and OpenSSH, which Windows includes.
- macOS: Git from the Xcode Command Line Tools (
xcode-select --install). SSH is included. - Linux: Git and the OpenSSH client from your distribution's packages.
- Quota for 1 more instance, 1 vCPU, 2 GB of memory, 20 GB of storage and 1 floating IP.
- Time: about 45 minutes.
- The code: every file you commit in this tutorial is also in
deploy-from-pipeline/in learn-projects, the course's code repository, at releasev1.2.0.
Step 1: Create the repository and an access token¶
In Gitea:
- Choose + at the top right, then New Repository. Set Repository Name to
web, tick Make repository private, and choose Create Repository. - Open your account menu, choose Settings, then Applications. Under Generate New Token, set Token Name to
laptop, set repository to Read and Write, and choose Generate Token. Copy the token: Gitea shows it once. You'll use it as your password forgit push.
Step 2: Activate the repository in Woodpecker¶
In Woodpecker, choose Add repository, then Enable next to platform-admin/web. Woodpecker adds a webhook to the repository in Gitea, so every push and pull request starts a pipeline.
Step 3: A credential for the pipeline¶
The pipeline needs its own application credential, so you can revoke it without affecting anything else. In the Breqwatr Cloud Portal (Strata), create one as in Application credentials and clouds.yaml: this project, the name pipeline, Expiration 90 days, and Unrestricted (dangerous) left unticked.
On the page shown after you create it, note the credential's ID and Secret, and from openrc.sh the values of OS_AUTH_URL and OS_REGION_NAME.
Step 4: Add the pipeline's secrets¶
In Woodpecker, open web, then Settings → Secrets. Add each of these with Add secret, ticking Pull Request under Available at the following events as well as Push, which is ticked already:
| Name | Value |
|---|---|
os_application_credential_id |
The credential's ID |
os_application_credential_secret |
The credential's secret |
pg_conn_str |
From tofu output -raw pg_conn_str in your pipeline-platform folder |
my_ip |
Your public IP address in CIDR form, <your-ip>/32; only it can open the site |
Pipeline steps read them as environment variables, through from_secret in the pipeline file, so the values stay out of the repository.
Step 5: Write the site¶
Clone the empty repository. When Git asks, sign in as platform-admin with the token from step 1 as the password:
Add these files to the web folder.
versions.tf. The empty backend "pg" block reads its connection string from PG_CONN_STR, which the pipeline sets from the pg_conn_str secret:
terraform {
required_version = ">= 1.13"
required_providers {
openstack = {
source = "terraform-provider-openstack/openstack"
version = "~> 3.4"
}
}
# State lives in PostgreSQL on the platform; the pipeline sets PG_CONN_STR
backend "pg" {}
}
# Reads the application credential from the OS_ environment variables
provider "openstack" {}
variables.tf:
variable "prefix" {
description = "Prefix for every resource name"
type = string
default = "site"
}
variable "my_ip" {
description = "Your public IP address in CIDR form; only it can open the site"
type = string
}
variable "external_network" {
description = "External network that floating IPs come from"
type = string
}
variable "linux_image" {
description = "Name of the Ubuntu 24.04 image"
type = string
}
main.tf. The page is part of the instance's first-boot script, so a change to index.html replaces the instance:
data "openstack_networking_network_v2" "external" {
name = var.external_network
}
data "openstack_images_image_v2" "ubuntu" {
name = var.linux_image
most_recent = true
}
resource "openstack_networking_network_v2" "site" {
name = "${var.prefix}-net"
}
resource "openstack_networking_subnet_v2" "site" {
name = "${var.prefix}-subnet"
network_id = openstack_networking_network_v2.site.id
cidr = "10.50.0.0/24"
dns_nameservers = ["1.1.1.1", "1.0.0.1"]
}
resource "openstack_networking_router_v2" "site" {
name = "${var.prefix}-router"
external_network_id = data.openstack_networking_network_v2.external.id
}
resource "openstack_networking_router_interface_v2" "site" {
router_id = openstack_networking_router_v2.site.id
subnet_id = openstack_networking_subnet_v2.site.id
}
resource "openstack_networking_secgroup_v2" "site" {
name = "${var.prefix}-web"
}
resource "openstack_networking_secgroup_rule_v2" "http" {
security_group_id = openstack_networking_secgroup_v2.site.id
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 80
port_range_max = 80
remote_ip_prefix = var.my_ip
}
resource "openstack_networking_port_v2" "site" {
name = "${var.prefix}-web-01"
network_id = openstack_networking_network_v2.site.id
security_group_ids = [openstack_networking_secgroup_v2.site.id]
fixed_ip {
subnet_id = openstack_networking_subnet_v2.site.id
}
}
# The page is part of the instance's first-boot script, so a change to
# index.html replaces the instance: rebuild, don't repair
resource "openstack_compute_instance_v2" "site" {
name = "${var.prefix}-web-01"
flavor_name = "v1.c1r2"
user_data = templatefile("${path.module}/web.sh", {
page = file("${path.module}/index.html")
})
block_device {
uuid = data.openstack_images_image_v2.ubuntu.id
source_type = "image"
destination_type = "volume"
volume_size = 20
boot_index = 0
delete_on_termination = true
}
network {
port = openstack_networking_port_v2.site.id
}
depends_on = [openstack_networking_router_interface_v2.site]
}
resource "openstack_networking_floatingip_v2" "site" {
pool = var.external_network
port_id = openstack_networking_port_v2.site.id
depends_on = [openstack_networking_router_interface_v2.site]
}
output "url" {
value = "http://${openstack_networking_floatingip_v2.site.address}/"
}
web.sh:
#!/bin/bash
set -e
apt-get update
apt-get install -y nginx
cat > /var/www/html/index.html <<'PAGE'
${page}
PAGE
index.html:
<!doctype html>
<title>Deployed by the pipeline</title>
<h1>Hello from Breqwatr Cloud</h1>
<p>Version 1</p>
.woodpecker.yaml is the pipeline. Replace the four placeholders with your values: OS_AUTH_URL and OS_REGION_NAME from step 3, and the external network and image names from openstack network list --external and openstack image list. Both steps run in the ghcr.io/opentofu/opentofu:1.13.1 image, the same version as yours, so you and the pipeline work on the state with the same OpenTofu:
when:
- event: pull_request
- event: push
branch: main
variables:
- &env
OS_AUTH_TYPE: v3applicationcredential
OS_AUTH_URL: <auth-url>
OS_REGION_NAME: <region>
OS_INTERFACE: public
OS_IDENTITY_API_VERSION: "3"
OS_APPLICATION_CREDENTIAL_ID:
from_secret: os_application_credential_id
OS_APPLICATION_CREDENTIAL_SECRET:
from_secret: os_application_credential_secret
PG_CONN_STR:
from_secret: pg_conn_str
TF_VAR_my_ip:
from_secret: my_ip
TF_VAR_external_network: <external-network>
TF_VAR_linux_image: <linux-image>
steps:
- name: check
image: ghcr.io/opentofu/opentofu:1.13.1
environment: *env
commands:
- tofu init -input=false
- tofu fmt -check -recursive
- tofu validate
- tofu plan -input=false
- name: apply
image: ghcr.io/opentofu/opentofu:1.13.1
environment: *env
commands:
- tofu init -input=false
- tofu apply -input=false -auto-approve
when:
- event: push
branch: main
.gitignore:
Step 6: Push, and watch the first deployment¶
git branch -M main names your branch main, whatever your Git calls a new repository's first branch, so the push lands on the branch the pipeline deploys from.
Woodpecker starts a pipeline for the push to main: clone, then check, then apply. Open it in Woodpecker to follow each step's log. The check step's plan shows 9 resources to add, and apply creates them. The site's address is in the apply log, under Outputs:, as url.
Open the address in your browser once the instance has installed nginx, a minute or two after the pipeline finishes. The page says "Version 1".
Step 7: Change the site through a pull request¶
Make the change on a branch:
In index.html, change Version 1 to Version 2, deployed from a merged pull request. Then:
Git's output includes a link to create a pull request in Gitea. Open it and choose Create Pull Request.
Woodpecker runs the pull request's pipeline: clone and check, without apply. The plan shows the instance to be replaced, because its first-boot script changed. Back in Gitea, the pull request shows "All checks were successful" with the Woodpecker check. That's the review: the change and what it will do to the infrastructure, before anything changes.
Choose Create merge commit, and confirm. The merge is a push to main, so Woodpecker runs check and apply. A minute or two after apply finishes, the site says "Version 2, deployed from a merged pull request".
Check it worked¶
- The pull request's pipeline ran
checkonly, and the merge's pipeline rancheckandapply. - The site shows the new version after the merge, with no command run on your computer.
- A push that changes nothing OpenTofu manages, such as a new
README.md, plans and applies "No changes."
Clean up¶
The pipeline's state is in PostgreSQL on the platform, which only the agent can reach. To destroy the site from your computer, open an SSH tunnel to PostgreSQL through the platform. In the pipeline-platform folder:
The address in it, after @, is the platform's private address. In a second terminal, open the tunnel and leave it running:
In the web folder, with your own clouds.yaml entry and the connection string pointed at the tunnel (localhost:15432 in place of the private address and port):
Type yes when asked. OpenTofu destroys the site's 9 resources. Close the tunnel with Ctrl+C.
Then revoke the pipeline credential on the portal's Cloud credentials page, and destroy the platform as in Build your own pipeline platform.
What you learned¶
- A pipeline turns a merge into a deployment: the plan is reviewed on the pull request, and the same code is applied on merge.
- The pipeline signs in with its own restricted credential, with an expiry, kept as a secret outside the repository; see Security and sovereignty.
- Shared state lets every run, and you, work from the same record of what exists; see OpenTofu fundamentals.
- A change to an instance's first-boot script replaces the instance, so every deployment starts from a clean build; see Rebuild, don't repair.
Next¶
- Woodpecker CI documentation: pipeline syntax, more events and more steps.
- OpenTofu's pg backend: how the state is stored and locked.