Back to Projects

Containerized Infrastructure Toolchain

Docker AWS Secrets Manager Amazon ECR Bash kubectl Terraform Ansible Jenkins Job DSL Alpine Linux Proxmox API
project-infra-toolchain-2026-09

Tool containers carry no secrets: they fetch credentials from Secrets Manager at runtime and are distributed through ECR, the same pattern Jenkins uses.

The Challenge

Operating four Kubernetes clusters, a Proxmox cluster, Terraform, Ansible, Jenkins, DNS and Cloudflare requires a long list of command-line tools, each with its own version and credentials. Installing all of them on a workstation invites version drift, leaves kubeconfigs and API tokens on disk, and makes it easy to run a command against the wrong cluster.

The Solution

Each tool domain gets its own small container image behind a single run.sh entry point. Images contain no secrets; at startup the entrypoint pulls exactly the credential it needs, whether a cluster's kubeconfig, an SSH key or an API token, from AWS Secrets Manager. Because each cluster has its own image, the target cluster is determined by which toolkit you run, not by local context.

Images are published to ECR so any host or Jenkins agent can use them, and Jenkins follows the same pattern: a Job DSL seed generates build jobs from an application list, and pipelines run their tooling as short-lived containers.

Implementation Details

There are 12 toolkits: one kubectl image per cluster plus Terraform, Ansible, Proxmox, Jenkins, Cloudflare, DNS, network-debugging and Atlassian environments, with newer toolkits built on a shared base image.

  • Standard interface: every run.sh supports one-shot and interactive use, pulling from ECR or building locally, and mounts its own directory as the workspace. AWS credentials are passed to the container at run time only.
  • Runtime secret retrieval: ten of the twelve entrypoints fetch their credentials from Secrets Manager and write them inside the running container only, never into an image layer.
  • Shared base image: Alpine with the AWS CLI and a helper library for secret retrieval, notifications and required-variable checks, so a new toolkit needs little more than its own scripts.
  • Publishing: build and push scripts create the ECR repository on first use, push each toolkit's image and support scheduled rebuilds.
  • Jobs from a list: a Job DSL seed turns an application list into build jobs with per-application parameters, dedicated image-rebuild jobs and entries for applications that live in other repositories.
  • No AWS tooling on agents: a short-lived amazon/aws-cli container supplies registry logins for pipelines, a pattern reused across more than a dozen Jenkinsfiles.
  • Cluster operations in CI: a parameterized Jenkins job runs a kubectl runner image to check status, apply manifests, restart workloads or refresh registry credentials on any cluster, with dry run as the default.