Kubernetes

Kubernetes schedules containers across a set of machines from declarative manifests, restarts what dies, and gives you ingress, scaling rules, secrets and rollouts as first-class objects. It is very good at what it does. It also assumes you have several long-running services that must stay up, which is a shape a lot of products do not have, and the assumption is expensive when it is wrong.

How we use it

Starterflare is the build that calls for it. The backend is a NestJS modular monolith with ten domain modules, and behind it sit PostgreSQL 16, Redis, NATS JetStream for event-driven side effects and OpenSearch for full-text search. Four stateful services and a Node process is not something you deploy to a serverless platform, so the repository carries Kustomize manifests with horizontal pod autoscaling and ingress, and a Docker Compose file so a developer can run the same set locally. CI runs lint and tests against Postgres and Redis service containers, which is the part of the setup that has earned its keep.

Being straight about the state of it: the manifests exist, there is no continuous deployment workflow and no evidence the codebase has been deployed anywhere. The cluster design is a design, not an operated system, and the live domain runs something else entirely. We would want a first deployment and a fortnight of real load before making claims about how it behaves.

What it costs you

The floor is the problem. Four stateful services bill by the hour whether the product has ten thousand users or none, and on top of the workloads themselves come the control plane, the node pool, a load balancer, persistent volumes, backups and the metrics stack that autoscaling depends on. We put actual numbers against this in server stack versus serverless cost, and the gap at zero traffic is the whole argument.

Then there is the ongoing time. A cluster wants upgrading a few times a year, API versions are deprecated on someone else's schedule, the ingress controller and certificate manager have their own release cycles, and none of that work is visible to a customer. Autoscaling needs metrics, which needs observability running before the scaling rules mean anything. Local development drifts from production unless someone maintains the Compose file. Debugging spans pod, service, ingress and application layers, so the first hour of an incident goes on working out which one you are in.

For a small team this is a part-time job that nobody was hired for. That is the honest reason Starterflare stalled where it did.

When we would choose something else

For request and response work that can scale to zero, Workers removes the entire category of problem: no nodes, no upgrades, no idle bill, and the trade is a runtime with limits you design around rather than a machine you own. Where coordination and per-entity state are what pushed someone towards a cluster in the first place, Durable Objects often cover it without any of the operations.

If the stack really does need Postgres, we would take managed PostgreSQL over a database inside the cluster every time; running your own primary and replicas is the part of this that goes wrong at three in the morning. And a single long-running process with a managed database behind it belongs on a container host or a plain VM, not on an orchestrator built for fleets.