Skip to content

Back to the catalog

IDP Zero-Dollar
github.comrepo

IDP Zero-Dollar

A companion GitOps playbook/repo for building a free, self-hosted "AWS-equivalent" Kubernetes platform from three bare-metal Linux boxes using k3s, Cilium, cert-manager, Traefik, Argo CD, Forgejo, Vault, Keycloak, kube-prometheus-stack, Loki, Kyverno, and Velero. It walks through the full security-relevant stack end to end: OS hardening (SSH lockdown, ufw, fail2ban), Cilium NetworkPolicy as CNI, Let's Encrypt DNS01 wildcard certs, Vault as the source of truth for secrets synced via External Secrets Operator (never committing secrets to Git), Keycloak OIDC for centralized identity across Argo/Grafana/Forgejo/Vault, Kyverno pod-security policies in audit-then-enforce mode, and Velero + Vault Raft snapshot backup/DR with quarterly restore drills. It closes with a candid list of what self-hosting gives up versus managed cloud (CVE patch cadence, DDoS exposure, compliance evidence, etcd/HA operational risk).

TL;DR End-to-end GitOps playbook for a free self-hosted Kubernetes platform (k3s/Cilium/Argo CD/Vault/Keycloak/Kyverno/Velero) covering secrets, identity, network policy, and DR.
Open original ↗
Document

What it is

The companion playbook for IDP Zero-Dollar, giving every command to go from bare Linux boxes to an Argo-managed k3s cluster. Tested on Ubuntu 22.04 LTS and Debian 12; should work on any systemd distro. The full stack is thirteen tools: k3s, Cilium, cert-manager, Traefik, Argo CD, Forgejo, Vault, Keycloak, Prometheus, Loki, Grafana, Kyverno, Velero. The fetched text covers OS hardening, k3s install, Cilium, and the start of cert-manager.

Preconditions

  • 3 Linux boxes, bare-metal or VPS, 8 vCPU / 32GB RAM / 200GB NVMe minimum each (a Hetzner AX102 or refurb Dell R640 clears this).
  • A domain you control; subdomains for Argo, Grafana, Forgejo, Vault, Keycloak under one wildcard cert.
  • DNS API access (Cloudflare, Route53, Hetzner DNS, deSEC) if boxes sit behind NAT — required for cert-manager DNS01.
  • Laptop with kubectl, helm, argocd.

What to do

The OS layer is load-bearing — the page warns "the whole platform inherits whatever you don't fix here." Key non-obvious steps: swap off (k8s requirement), br_netfilter/overlay modules, and SSH lockdown before anything else. Firewall baseline:

ufw default deny incoming
ufw default allow outgoing
ufw allow 22/tcp
ufw allow from 10.0.0.0/8
ufw allow from 192.168.0.0/16
ufw --force enable

k3s server install pins a version and disables four bundled components deliberately:

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.35.4+k3s1 sh -s - server \
  --cluster-init \
  --disable=traefik \
  --disable=servicelb \
  --flannel-backend=none \
  --disable-network-policy \
  --tls-san=k8s.example.com

The rationale matters: --disable=traefik (deploy your own controlled release), --disable=servicelb and --flannel-backend=none and --disable-network-policy because Cilium replaces kube-proxy, servicelb, flannel CNI, and NetworkPolicy — one tool instead of four. Nodes show NotReady until Cilium is installed; that is expected.

Cilium install (v1.19.3) turns on kube-proxy replacement and L2 announcements:

helm install cilium cilium/cilium --version 1.19.3 \
  --namespace kube-system \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=$API_SERVER_IP \
  --set k8sServicePort=6443 \
  --set l2announcements.enabled=true \
  --set externalIPs.enabled=true \
  --set devices='{eth0,enp+}'

cert-manager is v1.20.2 with crds.enabled=true, issuing Let's Encrypt wildcard certs via DNS01.

Caveats

  • Version pinning is a stated constraint: pick one Kubernetes minor below latest GA, since k3s typically lags by one minor. v1.35.4+k3s1 was current at writing — do not copy it blindly.
  • SQLite backend is "fine for dev"; use embedded etcd (--cluster-init with etcd) for an HA control plane.
  • L2 announcements are for LAN/NAT setups. On Hetzner with per-node public IPs, skip L2 and use BGP or expose Traefik via hostNetwork: true.
  • The fetched text cuts off inside the cert-manager ClusterIssuer secret; Vault, Keycloak, Kyverno, Velero and Argo steps are not shown here.

Technologies named here

Techniques named here

  • SSH hardening
  • Default-deny firewalling
  • DNS01 ACME challenge
  • Secrets-out-of-git pattern
  • Kubernetes auth backend for Vault
  • Admission-time image signature verification
  • Policy audit-before-enforce rollout
  • Restricted CNI network policy

Related in DevSecOps