Automated Multi-Site Deployment
A Jenkins build publishes each site's image to ECR, then refreshes pull credentials and performs a readiness-gated rolling restart on K3s behind nginx TLS.
The Challenge
Several public websites, including a consultancy site and this resume, needed to run economically on a single cloud server, each with its own content database, and updates had to ship without hand-building images or editing the server.
A private container registry adds a quieter problem: ECR login tokens expire every 12 hours, so anything that pulls images needs its credentials refreshed continuously. An earlier incident showed how silently this can fail. A token-refresh job reported success while refreshing the wrong namespace, the apex domain had no server block, and its DNS record still pointed at a retired server. Each problem hid the others.
The Solution
Each site is its own Wagtail repository with a Dockerfile and a Jenkinsfile, deployed into its own namespace on a single-node K3s cluster on a DigitalOcean droplet, with a dedicated PostgreSQL instance per site.
A Jenkins build packages the site, pushes the image to ECR, and then calls a shared Kubernetes pipeline to refresh the registry pull secret and perform a rolling restart gated on readiness probes. nginx on the droplet terminates TLS with certbot-managed Let's Encrypt certificates and routes each hostname to its site. After the incident, credential refresh was made self-discovering and the routing and DNS gaps were closed.
Implementation Details
Every site uses the same pipeline shape; only the image repository, namespace and hostname change.
- Image build: Python 3.12 slim base, static assets collected at build time, and gunicorn serving the application. Images are tagged with the build number and
latest. - Registry push: ECR login runs in a short-lived
amazon/aws-clicontainer, so build agents need no AWS tooling, and the local image is removed after the push. - Deploy by delegation: the pipeline triggers a shared Kubernetes job to refresh the pull secret and restart the deployment. These waiting stages run without an agent so they cannot tie up build executors.
- Schema migrations: an init container runs
manage.py migratebefore the new pod starts, and readiness and liveness probes gate traffic to it. - Per-site isolation: each site gets its own namespace, PostgreSQL 16 instance and persistent media volume.
- Edge: nginx on the droplet proxies each hostname to its service, certbot issues and renews Let's Encrypt certificates, and the apex domain redirects to the main site.
- Self-discovering token refresh: a scheduled job on the droplet finds every ECR pull secret in every namespace and refreshes it, reporting per-target errors, so a new site is covered without any configuration change.
- Builds on demand: deployments are started deliberately from Jenkins rather than on every commit, keeping releases of public sites an explicit decision.