Putting Claude into production without it embarrassing you
An LLM in a demo and an LLM in a feature real users depend on are different engineering problems. Here is how we put Claude into production systems so that it is dependable, not just impressive.
Claude is our default for the AI layer in the systems we build. It is genuinely good. But "the model is good" and "the feature is production-ready" are far apart, and the distance between them is ordinary engineering. A model in a demo answers to nobody. A model in a feature your users depend on answers to everybody. This is how we close that gap.
Treat the model as an unreliable but capable service
The mental model that works: Claude is a service call that is unusually capable and unusually unpredictable. You would never call an external API and assume the response is always present, always well-formed, always fast, always correct. You apply the same discipline here.
That means: timeouts, because a call can be slow. Retries with backoff and a cap, because a call can fail. Schema validation on the response, because the shape can vary. A fallback for when all of that is exhausted, because sometimes it will be. None of this is exotic. It is what you already do for any external dependency. The mistake is exempting the model from it because it feels like magic rather than infrastructure.
Validate the output, every time, at the boundary
We have said this in other posts and we will keep saying it because it is the lesson that costs people the most: the model's output is untrusted input.
Claude will usually return what you asked for. Usually is not a contract. If a feature depends on a specific structure, validate that structure the moment it comes back — strictly, at the boundary, before it touches anything else. A malformed response should fail right there, loudly, with a clear cause. The alternative is that it slips downstream and surfaces as a confusing error far from its origin, and you spend an afternoon tracing a bug that strict validation would have caught in a millisecond.
Make it produce artifacts you can check
Wherever the cost of a wrong answer is real, we do not ask the model for a verdict, we ask it for work. "Draft the thing" beats "rate the thing", because a draft is inspectable and a rating is just a confident number. If you need a quality judgement, get the model to produce the actual output and then check that — with a rule, a second pass, a constraint, or a human, depending on the stakes. Build the checking into the flow. Do not assume the first answer is the final one.
Budget it like the resource it is
Every Claude call has a latency cost and a money cost, and a feature that calls it carelessly is a problem waiting to surface.
On latency: a model call in a user-facing path is part of that path's performance budget. If the interaction needs to feel instant, either the call is not on the critical path, or the experience is designed around the wait. "Correct but slow" is still a failed requirement.
On cost: the bill is a function of how you call the model — how often, how much context, how many retries, how much fan-out. We use prompt caching for the stable, repeated parts of prompts, keep context to what the task actually needs, and cap retries and any fan-out. An AI feature with no cost ceiling is a budget incident that has not happened yet.
Log the whole interaction
When Claude does something strange in production — and at enough volume, it will — you need the full picture: exactly what was sent, exactly what came back, which path the code took afterward. Without that trace, every incident is a guess, and "the model did something weird" is not a debuggable statement. We log the full interaction, because an AI feature you cannot inspect is one you cannot really operate.
Pin the model, treat upgrades as changes
The model version is a dependency. A new version is, almost always, an improvement — and still a change in behaviour that your validation, your prompts and your expectations were tuned against. We pin the model version explicitly and treat moving to a new one as a real change: test it, compare it, roll it out deliberately. "Always use the latest" is not a strategy, it is an unannounced change in production.
What we would tell you
Putting Claude into production is not an AI problem. It is the same problem as integrating any powerful, external, somewhat-unpredictable dependency:
- Treat it as a service — timeouts, retries, fallbacks, the usual.
- Validate every output at the boundary. It is untrusted input.
- Ask for artifacts, then check them. Do not trust a verdict you cannot inspect.
- Budget latency and cost explicitly. Cache the stable parts, cap the rest.
- Log the full interaction. You cannot operate what you cannot see.
- Pin the version. Upgrades are changes, not background events.
The model is the impressive part and, honestly, the easy part — Claude does its job well. Making it dependable is the work, and the good news is that it is work you already know how to do.
Working on something similar?
We have probably hit the exact scaling or architectural bottlenecks you are navigating. Book a call and we'll walk through the technical trade-offs.