bismigel

The design of the homelab

· 3 min read

I finally have the homelab shaped the way I want, so this first post is a tour of how it's put together and why the pieces are where they are.

Two machines, two jobs

There are two physical boxes:

  • The hosting box: an AMD Ryzen 5 4600G (6c/12t). This was leftover parts from my old pc I replaced a gen 2 ryzen with 4600g (which I heard was better for power. bless the AM4 platform) It serves the websites and any experimental apps that I want to deploy.

  • The media box: an Intel i5-7500T. I bought this for 100$ from a guy from facebook marketplace. I've read somewhere that this generation supports quicksync and can support hardware decoding. It will run my media stack.

Based on that, I deliberately have split their apps The 4600g having more cores means its better for more general app and experiment and the 7500t supporting quicksync will contain my whole media stack.

Everything is NixOS

Initially this was deployed in proxmox lxc but i didnt like the additional layer that I have to work with before getting to the container. Using baremetal will remove that layer. I chose nixos specifically for the ease of configuration and avoiding drift from spec.

Both run NixOS, declared in a single flake. The OS is code: hosts, modules, disko disk layouts, and a deploy-rs{.verbatim} hook so I can push a rebuild over SSH:

shell 
nix run github:serokell/deploy-rs -- .#<host>

There is no half-configured box drifting over time. If I need to rebuild a machine, I reinstall from the same flake and I am back to a known state.

Kubernetes, but boring

This would have worked with just docker instead of a single node k3s but I specifically chose this in order to experiment and troubleshoot and experience managing a kubernetes architecture. This would also mean I can use existing gitops tools like Flux and ArgoCd instead of having something like an ansible job that deploys to servers.

Each box runs a single-node k3s. I disabled Traefik and the built-in service load balancer on purpose and use ingress-nginx instead, so ingress is explicit and identical on both machines.

GitOps with ArgoCD

Cluster state lives in git and ArgoCD reconciles it. One repo (bakuran-gitops) is the source of truth: an app-of-apps root points at a folder of Application{.verbatim} definitions, and infra add-ons (ingress-nginx, sealed-secrets) sit alongside the apps.

I do not run deploys from CI. Instead:

  1. I push application code.
  2. CI builds a container image and pushes it to GHCR.
  3. CI bumps the image tag in the manifests.
  4. ArgoCD notices and rolls it out.

CI never holds cluster credentials. It only pushes a tag. That is the whole trick, and it is why a compromised build cannot touch the cluster.

Applications are reproducible

Every app repo ships a deploy/k8s/{.verbatim} folder (Deployment, Service, Ingress) and a GitHub Actions workflow that builds and pushes its image. To start something new I use homelab-app-template, which wires all of that up. I just write the app.

The edge is one Cloudflare tunnel

Nothing is port-forwarded. A single cloudflared tunnel connects out to Cloudflare, and public hostnames (like this one) route to ingress-nginx on the hosting box. TLS terminates at Cloudflare.

Secrets

Cluster secrets are Sealed Secrets: encrypted in git, decrypted by a controller in the cluster. Host level secrets, such as the tunnel credentials, are *SOPS*/age encrypted in the NixOS repo and decrypted at activation, keyed off each machine's SSH host key.

What it looks like end to end

example 
push -> CI (build+push image+tag bump) -> ArgoCD -> k3s -> ingress-nginx
                                                        -> cloudflared -> internet
homelab design nixos