Self-built vs platform: the real cost of in-house agent infra
Engineers love this question and they are usually wrong about it. The first time we built our own agent infrastructure it took six months. The second time we did it across two companies and it still took five. By the third time we knew the right answer was "do not build it."
This post is the build-vs-buy spreadsheet you should be running, but with the line items engineers tend to skip.
The build pitch
The build pitch is reasonable on the surface. LLM SDKs are open. Most of the components, retrieval, prompt orchestration, basic logging, are commodity. A senior engineer can stitch them together in two weeks. Why would you pay a vendor for what is essentially glue code?
The build pitch is wrong because the things you actually need are not the glue code. The glue code is two weeks. The things you need are:
- The eval harness.
- The review console.
- The audit pipeline.
- Connectors to the systems your ops team uses.
- The on-call rotation when the model swap regresses.
Each one of those is a multi-month project. Most teams discover this five months in.
What we underestimated
In our second build, the line items we underestimated were:
- The review console. We thought it was a CRUD UI. It is not. It is the most important UX surface in the entire system. Every supervisor click matters because every click is training signal.
- Per-segment metrics. Aggregate accuracy lies. We did not realize how much it lies until a single payer's traffic shifted and aggregate stayed stable while one segment fell off a cliff.
- The replay pipeline. Validating a model swap requires replaying production decisions against the candidate. Building that pipeline correctly (deterministic, fast, with diff tooling) was three engineer-months we did not budget for.
- The on-call. When OpenAI ships a deprecation notice, you have 90 days to migrate. The migration project is real engineering work and it happens at the wrong time, every time.
What surprised us
The thing that actually surprised us was not the technical complexity. It was the political complexity. Once the agent is in production, every team wants to add a workflow, every workflow wants its own thresholds, every threshold change becomes a change request, and the small infrastructure team becomes a bottleneck for the whole company.
A platform absorbs this complexity. A self-built system passes it directly to the engineer who built it.
The question to ask first
Before you write a build-vs-buy spreadsheet, answer this: is the agent infrastructure your differentiation, or is the workflow on top of it?
If the answer is "the workflow," buy the platform. If the answer is "the infrastructure," then build is fine but the bar is much higher than your engineers are quoting you.
Almost every ops team we have talked to has the workflow as the differentiation, not the infrastructure. Treat the infrastructure as commodity, buy it back, and put the saved engineering hours into the workflow itself. That is where the moat is.