The basement cloud

How a home lab became a working miniature of enterprise AI infrastructure — a model gateway, a secrets vault, zero-trust ingress, agentic CI with human review gates, and an eval harness — with no cloud bill.

I wanted to build and ship AI products without paying a cloud provider to think. Not because the numbers are huge at prototype scale — they aren't — but because the meter changes how you work. When every experiment has a marginal cost, you experiment less. When compute is something you already own, you leave the loop running overnight.

So the products run at home — real apps with real users (mostly me, my collaborators, and my dogs). Two of them I'll refer to only by code name, because they're the ones with a future outside this essay: Project Sigma, a conversational assistant, and Project Pi, a transactional API. Alongside them: a water-conditions app, a gated writing assistant, and a client-intake portal with an SMS follow-up loop. And somewhere along the way, the lab stopped being a network with some apps on it and became a small, complete instance of the platform problem every enterprise AI team is wrestling with right now. The same questions, at 1:87 scale: who holds the API keys, who's allowed in, who reviewed the code the robot wrote, and how do you know the model is any good?

The layout

Three layers. The edge is Cloudflare: it terminates TLS, applies the WAF, and — this is the important part — enforces identity before a request ever reaches my house. The DMZ is an isolated subnet holding three machines: hermes, a fanless ARM board that dials an outbound tunnel to Cloudflare and holds it open; iris, a small container host running the services that should never sit on the trusted network; and argus, a red-team box whose entire job is to attack the network from the inside. The LAN is where the real assets live — the git server, the local models, the backends, the backups — and the DMZ cannot reach it except through individually-approved, one-service-wide firewall exceptions.

I don't take that segmentation on faith. That third box, argus, sits in the DMZ and continuously probes the path to the trusted network — and every attempt times out into a wall. The boundary isn't documented, it's tested, on a schedule, from the exact vantage a compromised edge box would have. That's the same shift large security programs are making right now: away from writing a policy that says the networks are segmented, toward continuously proving it — the discipline the vendors sell as "breach-and-attack simulation." Mine is a small ARM board with a port scanner and a grudge.

There are no inbound ports. Not forwarded, not filtered — none. Every public hostname rides back down a tunnel the house initiated. You can point a scanner at my connection all day and get nothing; you can visit athena.botelle.org right now and get exactly as far as my single-sign-on wall lets you, which is to say: not far.

Everything has a name from Greek mythology, because if you're going to run a pantheon of services you might as well commit. Hermes carries messages in. Janus guards the doorway where clients file requests. Oracle answers questions — for a metered price. The names started as whimsy and turned out to be useful: nobody on a demo forgets which box is the messenger.

One keyholder, many apps

The centerpiece — and the piece that maps most directly onto what large organizations are building right now — is oracle, an internal AI gateway. None of my applications hold a frontier-lab API key. They call oracle; oracle holds the one key, routes each request by tier — basic goes to a local Gemma model, premium to Claude Sonnet, elite to Claude Opus — and meters every call per application and per user. Routing defaults live in an admin panel, not in app code, so swapping a route's model is a configuration change, not a deploy.

Selectivity is the point, and it's easiest to see in janus, the intake service that fields project requests from collaborators through a web form and an SMS follow-up loop. Most of what janus does — classify a request, draft a clarifying text, summarize a thread — doesn't need a frontier model, so oracle routes it to the local one: free to run, and the request never leaves the building. Only the work that actually needs a stronger model draws a paid tier. And every call is metered under the janus identity, so what the service costs to run is a number I can read, not a line item I find later. Cheap-and-local by default, a paid model only where it earns its keep — that's what keeps the bill both predictable and small.

If you have spent any time near an enterprise AI program, you will recognize this as the LLM-gateway pattern: central key custody, model routing, usage metering, chargeback. Banks and insurers are standing up entire teams to build exactly this. Mine fits in one container. Obviously theirs must handle a thousand times the concerns — but the shape of the problem, and most of its sharp edges, are identical: apps will hardcode keys if you let them, "just give everything the best model" bankrupts you, metering has to exist before the bill arrives, and the gateway instantly becomes the most security-critical service you run.

The secret that built a vault

For a while, secrets lived the way they do in most young systems: as .env files sitting next to the code. API keys, service tokens, database passwords — plaintext, on disk, copied from host to host. It worked until it didn't. Two incidents, close together, made the pattern untenable. A service token turned up in plain view inside a session transcript I was reviewing. And while recovering from a build that had wiped a production secrets file, a gateway token was briefly exposed to a terminal. Neither was a catastrophe — I rotated both — but rotating by hand, with no audit trail and no single place a secret lived, isn't a security posture. It's luck with extra steps.

So the incidents produced a control. I stood up cerberus — a secrets vault (OpenBao, the open-source fork of HashiCorp Vault) on the trusted network. Now no service holds its own secrets. Each one authenticates to the vault at startup and pulls what it needs into memory; nothing durable sits in a file. Rotating a key is one command in one place, and every read lands in an audit log. And the rule that falls out of the trust tiers: the vault never leaves the trusted network. Since the DMZ can't reach in, a timer on the trusted side renders each exposed service's secrets and pushes them out over SSH, refreshing whenever they change. I'll be honest about where that stops short: the secret still lands in a locked-down file on the exposed box, so compromising that box still costs me a real rotation. Making those pushes short-lived and self-expiring, with a broker that injects credentials without ever revealing them, is the phase I haven't built yet.

This is secrets management — Vault, Secrets Manager, the thing every regulated org runs a team for — and building it at a scale one person can hold makes the real lesson unmissable: your secret-hygiene posture is decided by the architecture, not by the incident response. The leaks were cheap to survive because of choices made before them — per-client tokens, independent revocation — and they stopped recurring because they produced a system, not just a rotation.

Identity at the edge, twice

Every public service is gated by Cloudflare Access before traffic enters the tunnel. Humans authenticate with SSO; machines authenticate with service tokens — one token per client, individually revocable. The gated apps then verify the signed identity assertion again at the origin, so compromising the network path alone doesn't get you in. One route needs to accept calls from a third-party webhook that can't do SSO; that route gets a narrow carve-out protected by cryptographic request signing instead. Machine identity, human identity, and an exception process — the entire zero-trust conversation, in miniature.

This architecture was forced on me, and that's the best part of the story. The original design was conventional: port-forward 443, load-balance across two ISPs. It never worked, because my primary residential ISP silently drops inbound 443 upstream — no error, no notice, just a health check that never goes green. The fix was to stop accepting connections entirely and pivot to the outbound-dialed tunnel, which turned out to be strictly better: immune to ISP filtering, no exposed ports, and failover comes free because the tunnel rides whichever uplink is alive. Enterprises land on the same pattern for the same reason — you can't always open ingress, but you can almost always dial out.

Robots write the code; humans merge it

The lab's development loop is agentic, and has been for a while. A planning session files a task in a private, self-hosted Git repo; an executor on the compute box picks it up, does the work with a local coding model, and opens a pull request under a bot identity — minotaur, half man, half bull, all commits. The entire loop stays on the trusted network: the source, its full history, and the model writing it never leave the building. At my scale that's just tidy; for a regulated shop, where source code is itself sensitive IP and can't be handed to a third-party SaaS, it's the whole point. Then CI runs — and when a build comes back red, a repair agent takes the failure and tries to fix it before a human is ever paged — a self-healing pipeline, at least for the easy breaks.

For product work the split is deliberate: a mid-tier frontier model implements one scoped PR per change, a stronger frontier model reviews the diff, and a human — me — approves and merges every single one, usually from my phone through Minos, a little PR-review app I built because judging the dead requires the right tools.

That three-way split — cheap model implements, expensive model reviews, human gates — is, I'd argue, the correct default for AI-generated code anywhere, and it's the governance structure large engineering organizations are converging on under the name "human in the loop." Running it end-to-end, even at my scale, teaches you the parts the think-pieces skip: provenance only works if bot commits are attributed as bots; review quality collapses if PRs aren't scoped small; and the human gate is only real if the human can actually exercise it in under two minutes, which is a tooling problem, not a policy problem.

Trust the model? Measure it.

Two more pieces round out the platform. A bakeoff harness pits local models against each other on real product tasks — same prompts, tiered difficulty, scored results — which is how the lab picked its workhorse coding model instead of going by vibes and leaderboards. And an eval harness replays ~40 scripted personas against one of them, Project Sigma, to catch regressions in behavior, not just in code. The subtlety is in what's actually under test. It isn't the model in the abstract — it's the model plus its safety policy, stood up as a real service behind a fixed contract and driven turn-by-turn by an orchestrator running on a separate machine, so the eval exercises the thing users actually hit. The policy that decides when to escalate, refuse, or ask a human sits behind a single function, which means I can swap the model underneath without rewriting a single scenario. That distinction — you evaluate the system, not the weights — is one organizations tend to learn late and expensively, usually right after a model upgrade quietly changes a behavior nobody was measuring.

Model selection and behavioral evaluation are the least glamorous and most valuable parts of an AI platform; they're also the first thing skipped when the compute is borrowed and the meter is running. When the GPU is yours, evals are free, so you actually run them.

What it costs

Cheap is beside the point, honestly. What matters is that the development loop has zero marginal cost — branches, CI runs, model experiments, eval sweeps, overnight agent jobs — while the expensive things (frontier model calls) are metered through exactly one place where I can see them. That's the cost posture every platform team wants and few achieve: free iteration, observable spend.

What broke, and what it would mean at scale

The lab earns its keep as a discovery instrument precisely because things go wrong in it. A sample, each with its enterprise translation:

• An invisible Unicode character broke authentication. A pasted credential picked up a stray line-separator byte that no amount of squinting would reveal. Translation: machine identity fails in ways human identity doesn't; build detection for the dumb cases.
• The monitoring host couldn't see its neighbors. An OS privacy feature silently blocked the uptime monitor from probing other machines on the same network — everything looked down while everything was up. Translation: monitor through the front door (the public path users take), not the back.
• Every new service is a boundary negotiation. Each published route needs its own firewall exception, identity policy, and monitor — by design. The friction is the governance. Translation: if adding a service to your platform is frictionless, your platform has no security model.
• The backups had a single point of failure. Nightly dumps went to one NAS in the same building as everything else. The honest fix was writing "no offsite copy yet — PLANNED" in the architecture doc where I have to keep reading it. Translation: a DR posture you haven't written down truthfully is one you don't have.

Why bother

You could read all this as an elaborate way to avoid a modest cloud bill, and fair enough. But the bill was never really the point. Building the whole stack — gateway, identity, agent pipeline, evals, monitoring, backups — at a scale where one person can hold all of it in their head is the fastest way I know to understand which enterprise AI-platform problems are essential and which are incidental. The essential ones showed up here, uninvited, in a basement: key custody, identity for machines, provenance for generated code, evaluation before trust, metering before the invoice. The incidental ones (most of what the vendors sell) conspicuously didn't.

The lab documents itself, too — the architecture reference these notes are distilled from is written and maintained with Claude against the live hosts, then served, naturally, by the lab it describes. But that's another essay.

If your organization is building an AI platform and any of this rhymes with your whiteboard, I'm happy to compare notes — justin@botelle.net.