HRNPH BLOG

Designing Multi-Tenancy for a B2B SaaS Healthcare Startup

Fri, 28 Aug 2026 00:00:00 GMT

Introduction

Every healthcare B2B sale I've been in reaches the same moment. The demo goes fine, the doctors like it, and then the hospital IT director leans back and asks:

"So where exactly does our patient data live?"

Whatever you answer in the next thirty seconds decides the deal faster than any feature you showed them.

This post is about the tenancy design I ended up with for our rehab platform: a silo model, where every hospital customer gets its own full stack (own database, own cache, own storage, own network), all running the same codebase, with a central control plane watching over all of them. I'll walk through why I picked it, what it costs, and how to reason about the choice for your own SaaS, because the right answer for you is probably different.

:::note This is an engineering perspective from building one product. I'm not claiming silo is universally correct. It isn't. There's a whole section at the end about when you should not do this. :::

What "multi-tenancy" actually means in practice

People throw the word around like it's one thing. It's a spectrum, and where you sit on it decides most of your operational life.

Pooled. One database, one app, every row has a tenant_id column. Cheapest to run. But every query you ever write carries WHERE tenant_id = ? forever, and the one time somebody forgets it, you've leaked one hospital's patients into another hospital's screen. One bad migration takes down every customer simultaneously, in the same minute, on the same outage page. Restoring a single customer's data means surgically extracting it from a live shared database, which nobody enjoys at 2am.

Bridge. Shared cluster, database (or schema) per tenant. Middle ground. Isolation at the data layer, shared everything else. Smaller blast radius, still shared compute and shared meltdowns.

Silo. Every tenant gets the whole stack: their own database, their own cache, their own object storage, their own containers. Most expensive per tenant. Also the only model where "this hospital's data" has a physical answer you can point at.

I ended up at silo.

How to pick a model for your SaaS

These are the questions I actually asked. In rough order of how much they mattered.

1. What does your worst day look like?

Write down the sentence "we leaked hospital A's patient records to hospital B" and imagine sending it to your customers. In healthcare that sentence ends the company. Not hurts it, ends it. PDPA liability, hospital contracts, reputation, done. If your worst day is "some users saw a slower dashboard", pooled is fine and you should save the money.

2. Who decides to buy?

Consumer SaaS: users decide, in seconds, and churn in seconds. B2B healthcare: an IT director and a director of medicine decide, over months, and the deployment review is part of the product. When the buyer is the person asking "where does our data live", isolation is a sales feature, not overhead.

3. Will you ever need to run on the customer's infrastructure?

We knew the answer was eventually yes. Some hospitals want (or procurement requires) the system running inside their own network, on their own metal. That requirement forces single-tenant-shaped deployment whether you like it or not, so your architecture has to be a single-tenant unit you can stamp out N times anyway. Once that's true, "cloud" and "on-prem" become the same artifact deployed in different places, which is exactly what we wanted.

4. Can your team operate N stacks?

This is the real cost gate. If the answer is no and you can't automate your way to yes, silo will eat you. More on that in the costs section.

Why silo won for us

The product is a rehab exercise platform: doctors prescribe exercise programs, patients do them through a LINE mini app with on-device pose detection, doctors watch progress. B2B, hospitals and clinics, Thailand.

Four things pushed me to silo:

We're B2B and every customer runs the same codebase. No hospital needs a custom fork. So the unit of deployment could be "the whole product, once per customer" with zero product divergence.
Healthcare people are correctly paranoid. Thai hospitals care about PDPA, about data residency, about who can see what. The sentence "your data lives in your own Postgres, on a machine you can point at, and no other hospital's data has ever been in it" does more for a deal than any roadmap slide. Call it easing geezer minds if you want; the geezers sign the contracts.
Blast radius. One tenant's runaway job, one botched migration, one noisy neighbor. It's that tenant's problem, at that tenant's pace. Everyone else doesn't know it happened.
On-prem had to exist eventually, and as question 3 above says, that forces the shape anyway.

There was also a quieter reason, operational sanity. Per-tenant backup and restore is a whole-stack snapshot, not surgery. Per-tenant upgrade is a rolling decision per customer. If a hospital wants to stay on an older version for a month while their IT reviews, that's their stack, their choice, no feature flags required.

The architecture

One codebase, one container image, stamped out once per tenant, plus a central control plane ("the tower") that provisions and monitors all of them.

Each tenant stack is a compose deployment with its own Postgres, Redis, object storage, the API, the doctor dashboard, and the patient mini app, all inside a Docker network named after the tenant. Nothing inside that network needs to talk to another tenant, ever. The tenant gets its own domain pair (API and app), its own TLS, its own everything.

The detail that makes on-prem work: tenants call home, the tower never dials in

The control plane never initiates connections to tenants. Every communication is tenant-initiated:

Each tenant sends a heartbeat every 60 seconds (with exponential backoff if the tower is unreachable). The heartbeat carries health status and metrics.
The tower's response to a heartbeat is where commands ride back: "push the latest exercise catalog", "rotate your key", "force a heartbeat now". Commands piggyback on a connection the tenant already opened.
Tenants push aggregate stats hourly: counts only, no personal data ever leaves the tenant.

A tenant deployed inside a hospital network is behind NAT, behind firewalls, in rooms you will never see. If your control plane needs to reach in, on-prem is dead on arrival. If tenants call out, the same artifact works in your cloud, in the customer's datacenter, and in a hospital basement with one outbound rule. HTTP(S) outbound only.

:::tip If you take one architectural idea from this post, take this one. Pull-based control beats push-based control the moment customers host things themselves. Your monitoring, your config distribution, and your remote commands should all ride connections the tenant opens. :::

The command mailbox

Control rides the channel that's already open, sixty times an hour.

A command is a row in a per-tenant queue, nothing more. When a heartbeat arrives, the tower attaches any due commands to the response. The tenant executes them, and the next heartbeat carries the result back as an ack. "Push the latest exercise catalog", "rotate your signing key", "force a heartbeat now" are all mail in a mailbox, picked up on a schedule the tenant controls.

Three details keep it honest:

Every command has a TTL. If it isn't acked within a few beats it expires and surfaces as failed. A command aimed at a tenant that went down shouldn't sit in the queue forever, and the operator should see that it never landed.
Handlers are idempotent. Delivery is at-least-once on a good day: the tenant can execute a command and then lose the connection before the ack goes out, so the same command can arrive twice. Commands are therefore written against desired state ("you should be on catalog version 12"), never against counts of actions ("apply 12 updates").
Acks carry structured results. What version was applied, what failed, how long it took. The tower's UI shows command history per tenant with timestamps, without anyone grepping logs on a box they can't reach.

That history is a side effect of the design, and in healthcare it's half the value. Auditors like knowing who told which tenant to do what, when, and whether it worked.

Trust, but signed

Since tenants are the ones talking to the tower, the tower needs to know a tenant is who it claims. Every tenant request is Ed25519-signed: headers carry the tenant code, a signature over a canonical request string, a timestamp, and a single-use nonce. The tower rejects stale timestamps and replays. Revoking a tenant's key is how you suspend a tenant. There is no shared static API key sitting in every environment, which was the first thing I wanted gone.

One image, any tenant

We build one set of container images, versioned by tag. The same image runs for every tenant; anything tenant-specific is runtime configuration injected at deploy time (domains, LINE app ids, feature knobs). A per-tenant image would have been a maintenance nightmare, so there isn't one.

Migrations run N times, so they only go forward

Every tenant stack boots the same way: a small migration sidecar runs the database migrations first, then the API starts. With N tenants on N schedules, "roll back the migration" stops being a real strategy, so it isn't one:

Deploys are upgrade-only. Migrations roll forward, including ones that fix a failed deploy.
The scary-sounding "rollback" button in the control panel is actually teardown: destroying that job's resources, deliberately destructive, gated behind suspending the tenant first. Recovery from a bad deploy means rolling forward to a newer tag, not version time travel.

Provisioning is automation or it is death

I learned this the hard way. Hand-deploying tenant number three sucks, and tenant number ten is a resignation letter. Our control plane drives the hosting API end to end: create the project, create the database, create the cache, create the isolated network, push the compose definition with the right image tag and env, wire up domains. It's a resumable, step-by-step job with a timeline you can watch in the UI, because when it fails (it will, somewhere, someday) you need to see exactly which step and why.

If you can't get to "new tenant in minutes, by clicking", silo's costs stop being worth it. That automation is load-bearing.

One provisioner, several deployment routes

The tower doesn't hardcode a single way to make a tenant exist. Every provisioning job runs against a small provisioner interface (create the stack, report status, tear it down), and the deployment type is just data on the tenant record. Two routes exist today, and they look nothing alike under the hood.

Managed. The tower drives a cluster API end to end: create the project, the database, the cache, the isolated network, push the compose definition with the right image tag, wire up the domains. Then, before anything boots, the tower injects the callback configuration (tower URL, tenant code, signing key material) as runtime environment. The stack's first act on startup is to call home. Nobody SSHes anywhere, nobody edits files on a box, and the tenant-to-tower connection is wired by the same automation that wired the tenant.

Manual. For on-prem, customer metal, or a hosting setup with no API worth the name. The tower still owns the bookkeeping: it issues the tenant code and credentials and holds the record, but the stack is deployed by hands (ours or the customer's) in a network the tower will never see. The stack boots with a bootstrap config, and the first heartbeat doubles as registration: the tenant tells the tower it's alive, which URLs actually work, what versions it's running. The tower learns the topology from the tenant, in the only direction that was ever going to work.

After that first heartbeat the routes converge completely: same monitoring, same command queue, same stats pipeline. The rest of the tower doesn't know or care which route a tenant arrived by. Adding a third route (another cluster API, a marketplace install, whatever next year's procurement demands) means implementing an interface, not redesigning the plane.

What's bad about this control plane

Everything is at least one heartbeat late. A command lands within a minute if the tenant is healthy, and its confirmation arrives a beat later. For "push the catalog" that's irrelevant. For "something is wrong, act now" it's an eternity, and there is no faster path, by construction.
Silence is ambiguous. If a tenant stops heartbeating, it might be down, partitioned, or just backed off after a network blip. You can't probe it to find out, because you can't probe it at all. First-line diagnosis is comparing timestamps and hoping; real diagnosis means a human at the hospital.
You are hand-rolling a message queue. TTLs, retries, idempotency, acks are the vocabulary of a message broker, and so are the bug classes. The protocol only stays small because commands stay rare and boring. The day someone wants chatty remote control, this design fights them.
No interactive debugging. No SSH, no log tailing, no port-forward into a tenant. You see exactly what the tenant chooses to send. Debugging an on-prem stack means asking the customer to paste command output into an email.
The manual route is a second code path with worse visibility. A tenant somebody deployed by hand can sit on an old image for months (their right, our blindness), and every difference between how the two routes configure a stack is a difference that can bite during an incident.
Key rotation is a careful dance. The signing key authenticates the very channel you'd use to fix the signing key. Rotation has to overlap old and new keys long enough for the tenant to pick the new one up, or you've suspended your own tenant and the recovery is a car ride.

Why I did it anyway

Because the alternatives lose worse:

It works behind anything. NAT, hospital firewalls, a basement network with one outbound rule. A push-based control plane needs the customer to open holes toward me; polling needs nothing I don't already have.
"Your control plane cannot connect to our network" ends the security review. Inbound-zero is a one-line answer that procurement can verify. Nobody has to trust my promises about firewalls.
One protocol for every tenant. Cloud, on-prem, manual, managed: same heartbeat, same queue, same dashboard. No special tooling for special customers, and that kind of entropy is what kills small teams.
The scope is bounded on purpose. Commands are rare, idempotent, and low-frequency, so the mailbox stays small and boring. Small and boring is exactly what you want in a control channel.
Degradation is graceful. If the tower is down, tenants back off and retry. Nothing cascades, nothing times out loudly, and when the tower comes back the fleet walks in over the next minute or two.

The costs, honestly

Everything above was the sales pitch. Here's the bill.

You operate N of everything. N databases to migrate, N stacks to patch, N sets of logs. Mitigation is the heartbeat-and-stats pipeline plus disciplined structured logging (single-line JSON with an event name, one summary line per request), so "what's happening across the fleet" is a dashboard, not an SSH loop. But the fleet reality doesn't go away.

Every tenant has a floor cost. A database, a cache, storage, app containers each. Our per-tenant footprint is modest (this isn't Kubernetes-per-customer), but it's nonzero, and it shapes pricing.

Cross-tenant queries do not exist. There is no "average across all patients" query, because there is no place where that data lives together. That's a privacy feature until product asks for a benchmark report, and then it's an engineering project. The answer is tenants pushing aggregates (which we already do for stats), never a shared warehouse of raw rows.

More moving parts to test. Signing, replay protection, "the tower never probes tenants", migration idempotency are invariants now, and invariants need tests or they slowly rot.

The control plane is now a product. Monitoring UI, provisioning jobs, commands, audit trail. That's real time that pooled-SaaS competitors spend on features. You're buying isolation and on-prem capability with your engineering hours; make sure that's the trade you want.

When you should not pick silo

To be clear about the flip side:

Consumer or prosumer product, self-serve, price-sensitive: pooled. Silo's floor cost per tenant will eat your margins.
Your team is one or two people with no infra automation appetite: pooled or bridge until that changes.
You will realistically never deploy on customer infrastructure and your buyers don't ask about data isolation: you'd be paying for paranoia you can't sell back.
You need heavy cross-tenant analytics on raw data as a core feature: shared data is your product, and silo fights it.

Takeaway

Pick the cheapest tenancy model your worst day can survive. Write the leak headline, the outage headline, the "restore one customer" runbook, and read them back to yourself. If your buyers are hospitals, geezers with signing authority who ask where data lives before asking what the product does, the cheapest model that survives those questions is a silo, and the only way to afford it is to automate the stamping.

The design that made it work was three boring rules: one image for every tenant, tenants call home (never the reverse), and migrations only move forward. Everything else was making those rules convenient to live with.

How to Run a Fabric Minecraft Server with Docker - Zero to Hero Guide

Sat, 06 Dec 2025 00:00:00 GMT

Introduction

This is a "Zero-to-Hero" guide designed for someone who has never touched a server, doesn't know what a terminal is, and just wants to play Minecraft with friends using the Fabric mod loader.

We are using Docker. Think of Docker as a "robot" that runs software for you in a neat, isolated box so it doesn't mess up your actual computer.

Phase 1: Get the Tools

Before we do anything, you need the engine that runs the server.

Download Docker Desktop:
- Go to docker.com/products/docker-desktop.
- Download it for your OS (Windows or Mac).
- Install it just like any other program (Next, Next, Finish).
- Crucial Step: Once installed, OPEN Docker Desktop. It must be running in the background. If it asks you to log in, you can usually skip it or create a free account.
Open your "Command Center" (Terminal):
- Windows: Press the Windows Key, type PowerShell, and hit Enter.
- Mac: Press Command + Space, type Terminal, and hit Enter.

Phase 2: Create the Server Folder (The "cd" stuff)

You need a specific place on your computer where your server files will live. We will do this using the terminal to get you comfortable with it.

Type this command to create a folder named mc-server (and press Enter):
```
mkdir mc-server
```
(Note: mkdir stands for "Make Directory".)
Type this command to "walk inside" that folder:
```
cd mc-server
```
(Note: cd stands for "Change Directory". Your terminal prompt should now look different, showing mc-server at the end.)

Phase 3: The Blueprint (docker-compose.yml)

We need to give the Docker robot a set of instructions. This is called a "Compose file". It tells Docker: "I want Minecraft, I want Fabric, and I want it on this port."

Create the file:
- Windows: Type notepad docker-compose.yml and press Enter. It will ask if you want to create a new file. Click Yes.
- Mac: You can use TextEdit, but ensure it saves as plain text. Or just run touch docker-compose.yml in the terminal to create it, then open that file with any text editor.

Copy and Paste the specific code: Copy the code block below exactly. Don't change anything yet.

services:
  mc:
    image: itzg/minecraft-server
    container_name: my-fabric-server
    tty: true
    stdin_open: true
    ports:
      - "25565:25565"
    environment:
      EULA: "TRUE"
      TYPE: "FABRIC"
      VERSION: "1.20.1"
      MEMORY: "4G"
    volumes:
      - ./data:/data
    restart: unless-stopped

Understand what you just pasted:
- image: itzg/minecraft-server: This downloads a pre-made Minecraft server created by a legend named itzg.
- TYPE: "FABRIC": Tells the server to use Fabric, not normal Minecraft.
- VERSION: "1.20.1": Important. This dictates the game version. Change this to whatever version you and your friends want to play.
- MEMORY: "4G": Gives the server 4 Gigabytes of RAM. If your computer is weak, lower this to "2G".
- volumes: This saves your world. It creates a folder called data in your mc-server folder so your world doesn't vanish when you turn the server off.
Save the file and close the text editor.

Phase 4: Start the Engine

Now we tell Docker to read that file and build your server.

Go back to your terminal (make sure you are still inside the mc-server folder).
Run this command:
```
docker compose up -d
```
- up: Start everything.
- -d: Detached mode. This means "run in the background, don't lock up my terminal window."
Wait. The first time you run this, it will take a while (5-10 mins) because it is downloading Minecraft, Fabric, and Java.
Check if it's working: Type this command to watch the logs (the server "thinking"):
```
docker compose logs -f
```
- You will see a lot of text scrolling.
- Wait until you see a line that says: Done (X.XXs)! For help, type "help".
- Press Ctrl + C to stop watching the logs (this does not stop the server, it just stops the scrolling text).

Congratulations! You now have a vanilla Fabric server running.

IP Address: localhost (for you).
IP Address: Your real IP (for friends—you will likely need to Port Forward your router for friends to join, but that is a separate networking topic).

Phase 5: Installing Mods (The "Asshole Beginner" Friendly Way)

You probably want mods. Here is the safest way to do it.

Step 1: Turn off the server You cannot add mods while the server is running.

docker compose down

Step 2: Locate the mods folder

Open your regular file explorer (Finder on Mac, File Explorer on Windows).
Find the mc-server folder we created in Phase 2.
Inside, you will see a new folder called data.
Inside data, you will see a folder called mods.

Step 3: Download your mods

Go to Modrinth or CurseForge.
Download the Fabric version of the mod.
CRITICAL: The mod version MUST match your server version (e.g., if server is 1.20.1, mod must be 1.20.1).
CRITICAL: You must also download the Fabric API mod. Almost all Fabric mods require this. Put it in the folder too.

Step 4: Drag and Drop Drag the .jar files you downloaded into that mc-server/data/mods folder.

Step 5: Turn the server back on Go back to your terminal:

docker compose up -d

Watch the logs again (docker compose logs -f) to make sure the server doesn't crash. If it crashes, one of your mods is probably the wrong version.

Phase 6: You are the Admin (OP Yourself)

You want to be able to ban people or switch to creative mode. You need to access the server console.

Run this command to execute the admin tool inside the server:
```
docker exec -i my-fabric-server rcon-cli
```
Your terminal prompt will change to something like >. You are now talking directly to the Minecraft server.
Type:
```
op YourMinecraftUsername
```
(Replace YourMinecraftUsername with your actual in-game name).
It should say "Made YourMinecraftUsername a server operator".
Type exit to leave the admin console.

Phase 7: The "Cheat Sheet" for Daily Use

Save this list. These are the only commands you need to know.

Turn Server ON: docker compose up -d
Turn Server OFF: docker compose down
Restart Server (after adding new mods): docker compose restart
Check Server Status/Logs: docker compose logs -f (Press Ctrl+C to exit logs)
Access Admin Console: docker exec -i my-fabric-server rcon-cli

Summary of Common Mistakes

Client/Server Mismatch: You need to install Fabric and the exact same mods on your own computer's Minecraft Launcher to join the server.
Wrong Java Version: If you change the Minecraft version to an old one (like 1.12), the server might fail. The itzg image usually handles this automatically, but stick to versions 1.18+ for the smoothest experience with this guide.
EULA: If you edited the file and didn't put EULA: "TRUE", the server will immediately shut down.

Isolated Kubernetes Cluster on Proxmox with Terraform and Ansible - part 1

Sun, 27 Apr 2025 00:00:00 GMT

Introduction

In this blog, I show you how to built a self-hosted Kubernetes cluster on Proxmox using Terraform, Ansible, MicroK8s, Rancher, and ArgoCD.

::github{repo="hrnph/Proxmox-k8s-Terraform"}

What you will get

A fully functional Kubernetes cluster with Rancher and ArgoCD
A self-hosted solution with no external dependencies
An easy to modify and extend templates
A clear? understanding of how to set up a Kubernetes cluster on Proxmox
A clear? understanding of how to use Terraform and Ansible for infrastructure automation

:::note I did this project for my r/homelab, so I don't have a lot of resources like r/homedatacenter. with a single Proxmox node with 32GB of RAM and 12 CPU cores. I love HP Prodesk for this job and you should too for a low budget homelab. :::

Overview Diagram

Design Decision & Planning of what we want to achieve

Before building the cluster, the following goals were set

✅ Complete network isolation: The Kubernetes cluster should not pollute or consume home LAN IP addresses unnecessarily.
✅ Controlled external access: Only intentional, limited ingress into the cluster (e.g., Rancher/ArgoCD UIs) should be exposed.
✅ Infrastructure as Code (IaC): Both VM creation (Terraform) and configuration (Ansible) must be fully automated and reproducible.
✅ Lightweight Kubernetes: Using MicroK8s for a minimal, fast, HA-capable cluster.
✅ Modern management UIs: Deploy Rancher and ArgoCD for visual cluster and app control.
✅ SSH Gateway (Jumpbox): Centralized, secured SSH access via a single VM, without needing to expose cluster nodes individually and we can turn this off without affecting the cluster, like if you don't need local dns access via to master-nodes anymore ex using tailscale or cloudflare tunnels/warps or whatever you want to use.

Key Components and Flow

Component	Purpose
Proxmox	Hypervisor that hosts all VMs.
Terraform	Automates VM provisioning inside Proxmox.
VMs	Cluster nodes (Masters, Workers, Jumpbox) created via Terraform.
Jumpbox	Secure SSH gateway into the cluster private network.
Ansible	Automates installation and configuration of MicroK8s, Rancher, and ArgoCD.
MicroK8s	Kubernetes cluster running across master and worker VMs.
Ingress Controller	Routes traffic internally from user DNS requests to services.
Rancher & ArgoCD UIs	Management UIs for the cluster and GitOps applications.
DNS (Pi-hole or /etc/hosts)	Local resolution of friendly service names to the exposed master node IP.
Cloudflare Tunnels	Secure external access to services without exposing IPs, All still defined in Terraform. (for now you can actually just use cloudflare tunnels on jumpbox or master-1)

Security Choices

Only master-1 exposes a LAN IP (192.168.1.250) for ingress traffic.
No internal cluster node can initiate outbound connections into the home network.
Jumpbox can be destroyed after provisioning for improved security hardening.

Why MicroK8s?

Fast deployment and updates via Snap packages
Lightweight but HA-capable Kubernetes
Built-in Helm 3, Ingress, and Cert-Manager
No etcd dependency (uses Dqlite HA by default)

:::tip This design replicates cloud provider Private Subnets inside a homelab environment. :::

Step 0: Prerequisites

Clone my template repository

::github{repo="hrnph/Proxmox-k8s-Terraform"}

Step 1: Proxmox Networking

Create vmbr0 (LAN) and vmbr1 (Cluster):

# /etc/network/interfaces

auto vmbr0
iface vmbr0 inet static
    address 192.168.1.100/24
    gateway 192.168.1.1
    bridge_ports eno1
    bridge_stp off
    bridge_fd 0

auto vmbr1
iface vmbr1 inet static
    address 192.168.3.1/24
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s 192.168.3.0/24 -o vmbr0 -j MASQUERADE

✅ Cluster nodes have isolated internet access.

Step 2: Terraform VM Provisioning

Prepare your terraform.tfvars file with your Proxmox login and IPs.

:::tip 2 Master 3 Workers Setup (should go to 3 for Acutal HA, but whatever, this is a good start) :::

:::important You need to also modified the inventory.ini file if you modified the cluster_vms :::

pm_password    = "your-password"
template_id    = "your-cloudinit-template-id"
target_node    = "your-proxmox-node"
ssh_public_key = "your-ssh-public-key"

cluster_vms = [
  { name = "master-1", ip = "192.168.3.11", cores = 2, memory = 4096, disk = 32 },
  { name = "master-2", ip = "192.168.3.12", cores = 2, memory = 4096, disk = 32 },
  { name = "worker-1", ip = "192.168.3.21", cores = 2, memory = 4096, disk = 32 },
  { name = "worker-2", ip = "192.168.3.22", cores = 2, memory = 4096, disk = 32 },
  { name = "worker-3", ip = "192.168.3.23", cores = 2, memory = 4096, disk = 32 },
]

terraform init
terraform plan -out ./plans/1-init
terraform apply ./plans/1-init

What is happening

Terraform creates VMs in Proxmox using the specified template and cloud-init configuration. By using the cloud-init template, it sets up the VMs with the specified IPs and resources.

2 Masters
3 Workers
1 Jumpbox (for SSH)

Step 3: Ansible Cluster Configuration

ansible-playbook -i inventory.ini playbook.yml

What is happening

Ansible configures the VMs by accessing the VMs via SSH through the jumpbox. Each node and also te jumpy is configured in the inventory.ini file that ansible use. Ansible then ssh into the jumpbox and then into the master-1 node to install microk8s. and continues to install microk8s on the other nodes.

Install MicroK8s
HA cluster setup (Join the cluster to the cluster)
Rancher + ArgoCD deployment
Expose via Ingress

Step 4: Access Rancher and ArgoCD

DNS Mapping: This can be done in your router or /etc/hosts file. by adding the following lines to your /etc/hosts file for linux and macOS

:::note help yourself if you are using Windows, not my forté :::

192.168.1.250 rancher.madoka
192.168.1.250 argocd.madoka

Then access:

Rancher: https://rancher.madoka
ArgoCD: https://argocd.madoka

✅ Full UI access secured.

📂 Important File Overview

This section explains the key files in this repository, what they do, and why they matter.
If you're new to Terraform or Ansible, read this carefully before editing!

File / Folder	Purpose
`main.tf`	Defines the infrastructure Terraform will create: VMs for Kubernetes masters, workers, and the jumpbox.
`terraform.tfvars.example`	A sample file showing what variables you need to define for Terraform (Proxmox login, cluster IPs, VM specs). Copy to `terraform.tfvars` and edit your values.
`terraform.tfvars`	Your actual Terraform variable configuration. Sensitive information like Proxmox API password, SSH public key, cluster VM details are placed here.
`inventory.ini`	Ansible inventory file. Lists all VM IPs and configures SSH access via the jumpbox. Must match the IPs used in `terraform.tfvars`.
`playbook.yml`	Main Ansible playbook. Automates the setup of MicroK8s, Rancher, and ArgoCD across the cluster. Fully sequential and clean.
`/plans/`	Terraform plans (output from `terraform plan -out`). Used to pre-approve and manage infrastructure changes safely.
`.terraform/`	Terraform working directory (created by `terraform init`). Contains plugins, backend data. No need to touch manually.
`terraform.tfstate`	Terraform state file. Tracks everything Terraform created. Do not edit manually. Keep it safe to manage future changes.
`/roles/` (optional)	Empty or deprecated for now. Originally for modular Ansible roles but merged into `playbook.yml` for simplicity.
`README.md`	Full documentation of the project, setup steps, file explanations, and additional tips for modification.

Final Thoughts

✅ Secure Kubernetes cluster
✅ Rancher and ArgoCD fully operational
✅ Infrastructure as code
✅ Minimal home network exposure

:::tip Next upgrade will be adding Cloudflare Tunnels for public network access. This is a great way to expose services without exposing your home network IPs.

for now you can actually just use cloudflare tunnels on jumpbox or master-1, This is also applied to tailscale or whatever you want to use :::