Tutorials

A three-tier web app

Build a small web application from code: a web tier and an app tier, each behind its own Octavia load balancer with a health monitor, and a PostgreSQL database whose data lives on its own volume. Each tier only accepts traffic from the tier in front of it.

Well-Architected

Pillar: Reliability. Two instances in each of the web and app tiers, behind load balancers whose health monitors stop sending traffic to an instance that fails.

Also touches: Security and sovereignty (each tier reachable only from the one in front of it), Operational excellence (the whole application in OpenTofu) and Performance efficiency (scaling a tier by changing a count).

What you'll build

internet
   |
public load balancer (port 80, health monitor on /health) + floating IP
   |
shop-web-01, shop-web-02   nginx, proxying to the app tier
   |
internal load balancer (port 8080, health monitor on /health)
   |
shop-app-01, shop-app-02   a small Python app, counting visits
   |
shop-db-01                 PostgreSQL, data on the shop-db-data volume

Tier	Instances	Accepts	From
Web	shop-web-01, shop-web-02	HTTP on 80	The subnet, where the public load balancer sits
App	shop-app-01, shop-app-02	HTTP on 8080	The subnet, where the internal load balancer sits
Database	shop-db-01	PostgreSQL on 5432	Instances in the app tier's security group only

All of it runs on its own network, shop-net, with a router for outbound traffic. Only the public load balancer has a floating IP; no instance is reachable from the internet.

The application is deliberately small: each request to / records a visit in the database and returns the app instance's name and the visit count, and nginx adds an X-Served-By header naming the web instance. That makes it easy to see each tier at work.

Before you start

• Completed: OpenTofu fundamentals.
• OpenTofu 1.13.1, and your clouds.yaml entry set in OS_CLOUD as in 4.1. The commands run the same way from Windows, macOS and Linux.
• Quota for 5 instances, 5 vCPUs, 10 GB of memory, 110 GB of storage and 1 floating IP.
• Time: about 45 minutes, most of it waiting for the build.
• The code: every file in this tutorial is also in three-tier-web-app/ in learn-projects, the course's code repository, at release v1.2.0.

Step 1: Create the project

Create a folder called three-tier-web-app with a templates folder inside it, and these files. Set your values in terraform.tfvars (in learn-projects it's terraform.tfvars.example, to copy):

external_network = "<external-network>"
linux_image      = "<linux-image>"

versions.tf pins the providers. The random provider generates the database password:

terraform {
  required_version = ">= 1.13"
  required_providers {
    openstack = {
      source  = "terraform-provider-openstack/openstack"
      version = "~> 3.4"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.6"
    }
  }
}

provider "openstack" {}

variables.tf:

variable "prefix" {
  description = "Prefix for every resource name"
  type        = string
  default     = "shop"
}

variable "external_network" {
  description = "External network that floating IPs come from"
  type        = string
}

variable "linux_image" {
  description = "Name of the Ubuntu image"
  type        = string
}

variable "flavour" {
  description = "Flavour for every instance"
  type        = string
  default     = "v1.c1r2"
}

variable "subnet_cidr" {
  description = "Address range of the application network"
  type        = string
  default     = "10.10.0.0/24"
}

Step 2: The network and the security groups

network.tf creates the application's network, subnet and router:

data "openstack_networking_network_v2" "external" {
  name = var.external_network
}

data "openstack_images_image_v2" "ubuntu" {
  name        = var.linux_image
  most_recent = true
}

resource "openstack_networking_network_v2" "app" {
  name = "${var.prefix}-net"
}

resource "openstack_networking_subnet_v2" "app" {
  name            = "${var.prefix}-subnet"
  network_id      = openstack_networking_network_v2.app.id
  cidr            = var.subnet_cidr
  dns_nameservers = ["1.1.1.1", "1.0.0.1"]
}

resource "openstack_networking_router_v2" "app" {
  name                = "${var.prefix}-router"
  external_network_id = data.openstack_networking_network_v2.external.id
}

resource "openstack_networking_router_interface_v2" "app" {
  router_id = openstack_networking_router_v2.app.id
  subnet_id = openstack_networking_subnet_v2.app.id
}

security.tf sets up the three tiers' rules. Load balancers reach their members from addresses on the subnet, so the web and app groups accept their ports from the subnet. The database group accepts PostgreSQL only from instances in the app group:

# Load balancers reach their members from addresses on the subnet,
# so the web and app tiers accept their port from the subnet.
resource "openstack_networking_secgroup_v2" "web" {
  name = "${var.prefix}-web"
}

resource "openstack_networking_secgroup_rule_v2" "web_http" {
  security_group_id = openstack_networking_secgroup_v2.web.id
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = 80
  port_range_max    = 80
  remote_ip_prefix  = var.subnet_cidr
}

resource "openstack_networking_secgroup_v2" "app" {
  name = "${var.prefix}-app"
}

resource "openstack_networking_secgroup_rule_v2" "app_http" {
  security_group_id = openstack_networking_secgroup_v2.app.id
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = 8080
  port_range_max    = 8080
  remote_ip_prefix  = var.subnet_cidr
}

# The database accepts connections only from instances in the app group.
resource "openstack_networking_secgroup_v2" "db" {
  name = "${var.prefix}-db"
}

resource "openstack_networking_secgroup_rule_v2" "db_postgres" {
  security_group_id = openstack_networking_secgroup_v2.db.id
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = 5432
  port_range_max    = 5432
  remote_group_id   = openstack_networking_secgroup_v2.app.id
}

Step 3: The database tier

database.tf keeps the data on its own volume, attached to the instance, and gives the database a port of its own so its address stays the same if the instance is rebuilt. Rebuild, don't repair relies on both.

resource "random_password" "db" {
  length  = 24
  special = false
}

# The data lives on its own volume, so it outlasts any one database instance
resource "openstack_blockstorage_volume_v3" "db_data" {
  name = "${var.prefix}-db-data"
  size = 10
}

# A port of its own keeps the database's address the same when the instance is rebuilt
resource "openstack_networking_port_v2" "db" {
  name               = "${var.prefix}-db"
  network_id         = openstack_networking_network_v2.app.id
  security_group_ids = [openstack_networking_secgroup_v2.db.id]

  fixed_ip {
    subnet_id = openstack_networking_subnet_v2.app.id
  }
}

resource "openstack_compute_instance_v2" "db" {
  name        = "${var.prefix}-db-01"
  flavor_name = var.flavour
  user_data = templatefile("${path.module}/templates/db.sh", {
    subnet_cidr = var.subnet_cidr
    db_password = random_password.db.result
  })

  block_device {
    uuid                  = data.openstack_images_image_v2.ubuntu.id
    source_type           = "image"
    destination_type      = "volume"
    volume_size           = 20
    boot_index            = 0
    delete_on_termination = true
  }

  network {
    port = openstack_networking_port_v2.db.id
  }

  depends_on = [openstack_networking_router_interface_v2.app]
}

resource "openstack_compute_volume_attach_v2" "db_data" {
  instance_id = openstack_compute_instance_v2.db.id
  volume_id   = openstack_blockstorage_volume_v3.db_data.id
}

templates/db.sh runs at first boot. It formats the volume only if it's empty, mounts it, installs PostgreSQL and keeps the database cluster on the volume:

#!/bin/bash
set -e
# Wait for the data volume, and format it the first time only
while [ ! -b /dev/vdb ]; do sleep 2; done
blkid /dev/vdb || mkfs.ext4 -L db-data /dev/vdb
mkdir -p /srv/db-data
echo 'LABEL=db-data /srv/db-data ext4 defaults,nofail 0 2' >> /etc/fstab
mount /srv/db-data

# Install PostgreSQL without its default cluster
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y postgresql-common
sed -i 's/^#*create_main_cluster.*/create_main_cluster = false/' /etc/postgresql-common/createcluster.conf
apt-get install -y postgresql
VERSION=$(ls /usr/lib/postgresql)

# The first time, create the cluster on the volume and move its configuration there too
if [ ! -d /srv/db-data/etc ]; then
  pg_createcluster -d /srv/db-data/"$VERSION"/main "$VERSION" main
  echo "listen_addresses = '*'" >> /etc/postgresql/"$VERSION"/main/postgresql.conf
  echo "host shop shop ${subnet_cidr} scram-sha-256" >> /etc/postgresql/"$VERSION"/main/pg_hba.conf
  mv /etc/postgresql /srv/db-data/etc
  sync
fi

# Every time, including on a rebuilt instance, use the cluster on the volume
rm -rf /etc/postgresql
ln -s /srv/db-data/etc /etc/postgresql
pg_ctlcluster "$VERSION" main start

sudo -u postgres psql -tc "SELECT 1 FROM pg_roles WHERE rolname = 'shop'" | grep -q 1 || sudo -u postgres psql -c "CREATE ROLE shop LOGIN PASSWORD '${db_password}'"
sudo -u postgres psql -tc "SELECT 1 FROM pg_database WHERE datname = 'shop'" | grep -q 1 || sudo -u postgres createdb -O shop shop

The password comes from random_password and is written into the instance's user data and the OpenTofu state. Treat the state as sensitive, as in OpenTofu fundamentals.

Step 4: The app tier

app.tf builds two app instances with count, and the internal load balancer in front of them. Its health monitor checks /health every 5 seconds and takes an instance out of the pool after 2 failed checks:

resource "openstack_compute_instance_v2" "app" {
  count           = 2
  name            = format("%s-app-%02d", var.prefix, count.index + 1)
  flavor_name     = var.flavour
  security_groups = [openstack_networking_secgroup_v2.app.name]
  user_data = templatefile("${path.module}/templates/app.sh", {
    db_host     = openstack_networking_port_v2.db.all_fixed_ips[0]
    db_password = random_password.db.result
  })

  block_device {
    uuid                  = data.openstack_images_image_v2.ubuntu.id
    source_type           = "image"
    destination_type      = "volume"
    volume_size           = 20
    boot_index            = 0
    delete_on_termination = true
  }

  network {
    uuid = openstack_networking_network_v2.app.id
  }
}

# An internal load balancer: the web tier reaches the app tier through it
resource "openstack_lb_loadbalancer_v2" "app" {
  name          = "${var.prefix}-app-lb"
  vip_subnet_id = openstack_networking_subnet_v2.app.id
}

resource "openstack_lb_listener_v2" "app" {
  name            = "${var.prefix}-app-http"
  loadbalancer_id = openstack_lb_loadbalancer_v2.app.id
  protocol        = "HTTP"
  protocol_port   = 8080
}

resource "openstack_lb_pool_v2" "app" {
  name        = "${var.prefix}-app-pool"
  listener_id = openstack_lb_listener_v2.app.id
  protocol    = "HTTP"
  lb_method   = "ROUND_ROBIN"
}

resource "openstack_lb_monitor_v2" "app" {
  name        = "${var.prefix}-app-health"
  pool_id     = openstack_lb_pool_v2.app.id
  type        = "HTTP"
  url_path    = "/health"
  delay       = 5
  timeout     = 3
  max_retries = 2
}

resource "openstack_lb_member_v2" "app" {
  count         = 2
  pool_id       = openstack_lb_pool_v2.app.id
  address       = openstack_compute_instance_v2.app[count.index].access_ip_v4
  protocol_port = 8080
  subnet_id     = openstack_networking_subnet_v2.app.id
}

templates/app.sh installs the app as a service on port 8080:

#!/bin/bash
set -e
apt-get update
apt-get install -y python3-psycopg2
cat > /opt/app.py <<'PY'
import socket
from http.server import BaseHTTPRequestHandler, HTTPServer
import psycopg2

DSN = "host=${db_host} dbname=shop user=shop password=${db_password} connect_timeout=3"

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == "/health":
            body = b"ok\n"
        else:
            with psycopg2.connect(DSN) as conn, conn.cursor() as cur:
                cur.execute("CREATE TABLE IF NOT EXISTS visits (id serial PRIMARY KEY, at timestamptz DEFAULT now())")
                cur.execute("INSERT INTO visits DEFAULT VALUES")
                cur.execute("SELECT count(*) FROM visits")
                count = cur.fetchone()[0]
            body = f"app: {socket.gethostname()}, visits: {count}\n".encode()
        self.send_response(200)
        self.send_header("Content-Type", "text/plain")
        self.end_headers()
        self.wfile.write(body)

HTTPServer(("", 8080), Handler).serve_forever()
PY
cat > /etc/systemd/system/app.service <<'UNIT'
[Unit]
Description=Shop app tier
After=network-online.target

[Service]
ExecStart=/usr/bin/python3 /opt/app.py
Restart=always

[Install]
WantedBy=multi-user.target
UNIT
systemctl daemon-reload
systemctl enable --now app

Step 5: The web tier and the public load balancer

web.tf builds two web instances that proxy to the internal load balancer's address, the public load balancer with its own health monitor, and the floating IP:

resource "openstack_compute_instance_v2" "web" {
  count           = 2
  name            = format("%s-web-%02d", var.prefix, count.index + 1)
  flavor_name     = var.flavour
  security_groups = [openstack_networking_secgroup_v2.web.name]
  user_data = templatefile("${path.module}/templates/web.sh", {
    app_address = openstack_lb_loadbalancer_v2.app.vip_address
  })

  block_device {
    uuid                  = data.openstack_images_image_v2.ubuntu.id
    source_type           = "image"
    destination_type      = "volume"
    volume_size           = 20
    boot_index            = 0
    delete_on_termination = true
  }

  network {
    uuid = openstack_networking_network_v2.app.id
  }
}

# The public load balancer, with a floating IP for internet traffic
resource "openstack_lb_loadbalancer_v2" "web" {
  name          = "${var.prefix}-web-lb"
  vip_subnet_id = openstack_networking_subnet_v2.app.id
}

resource "openstack_lb_listener_v2" "web" {
  name            = "${var.prefix}-web-http"
  loadbalancer_id = openstack_lb_loadbalancer_v2.web.id
  protocol        = "HTTP"
  protocol_port   = 80
}

resource "openstack_lb_pool_v2" "web" {
  name        = "${var.prefix}-web-pool"
  listener_id = openstack_lb_listener_v2.web.id
  protocol    = "HTTP"
  lb_method   = "ROUND_ROBIN"
}

resource "openstack_lb_monitor_v2" "web" {
  name        = "${var.prefix}-web-health"
  pool_id     = openstack_lb_pool_v2.web.id
  type        = "HTTP"
  url_path    = "/health"
  delay       = 5
  timeout     = 3
  max_retries = 2
}

resource "openstack_lb_member_v2" "web" {
  count         = 2
  pool_id       = openstack_lb_pool_v2.web.id
  address       = openstack_compute_instance_v2.web[count.index].access_ip_v4
  protocol_port = 80
  subnet_id     = openstack_networking_subnet_v2.app.id
}

resource "openstack_networking_floatingip_v2" "web" {
  pool       = var.external_network
  port_id    = openstack_lb_loadbalancer_v2.web.vip_port_id
  depends_on = [openstack_networking_router_interface_v2.app]
}

templates/web.sh configures nginx:

#!/bin/bash
set -e
apt-get update
apt-get install -y nginx
cat > /etc/nginx/sites-available/default <<'NGINX'
server {
    listen 80 default_server;
    location /health {
        return 200 "ok\n";
    }
    location / {
        proxy_pass http://${app_address}:8080;
        add_header X-Served-By $hostname;
    }
}
NGINX
systemctl reload nginx

outputs.tf gives you the address to open:

output "url" {
  value = "http://${openstack_networking_floatingip_v2.web.address}/"
}

Step 6: Build it

tofu init
tofu plan
tofu apply

The plan shows 32 resources to add. Type yes when asked. OpenTofu creates the network and security groups first, then the database, the app tier and its load balancer, and the web tier last, because each tier needs the address of the one behind it. The apply takes several minutes; the instances need a few more after it finishes to install their software.

Check it worked

Open the url output in a browser, or request it a few times:

WindowsmacOSLinux

$url = tofu output -raw url
1..4 | ForEach-Object { $r = Invoke-WebRequest $url; "$($r.Headers['X-Served-By']): $($r.Content.Trim())" }

for i in 1 2 3 4; do curl -s -D - "$(tofu output -raw url)" | grep -i -E '^x-served-by|^app:'; done

for i in 1 2 3 4; do curl -s -D - "$(tofu output -raw url)" | grep -i -E '^x-served-by|^app:'; done

Each response names the web instance and the app instance that handled it, and the visit count goes up by one each time. The names alternate as the load balancers spread the requests across both instances in each tier; the count keeps going up because both app instances use the same database.

To see a health monitor at work, stop one app instance:

openstack --os-cloud <cloud-name> server stop shop-app-01
openstack --os-cloud <cloud-name> loadbalancer member list shop-app-pool

Within a minute, the member shows ERROR and every response comes from shop-app-02. Start it again with openstack --os-cloud <cloud-name> server start shop-app-01, and it rejoins the pool once its health checks pass.

Change it

Each tier's size is a count. To run three app instances, change count = 2 to count = 3 in both the openstack_compute_instance_v2.app and openstack_lb_member_v2.app resources, then tofu apply. OpenTofu adds one instance and one pool member, and the load balancer starts sending it traffic once it passes its health checks.

Clean up

Rebuild, don't repair starts from this application. If you're going on to it now, keep everything. Otherwise:

tofu destroy

Type yes when asked. OpenTofu deletes everything it created, including the database volume and its data.

What you learned

• Two or more instances behind a load balancer, with a health monitor, keep a tier serving when one instance fails; see Reliability.
• Security groups can chain tiers together, so each accepts traffic only from the one in front of it; see Networking model.
• Data that must outlast an instance goes on a volume of its own; see Pets vs cattle.
• A whole application, network to database, can be built and removed from code.

Next

• Rebuild, don't repair: replace this application's instances from code, and keep the data.
• HTTP and TCP load balancers: listeners, pools, members and a floating IP.
• Health monitors: health monitor options.
• Create a volume and Attach a volume: creating and attaching volumes.