Operator notes on building AI for production.
Architecture decisions we’d defend, failure modes we keep watching teams hit, and the parts most vendor pitches leave out. Written by the founders, in the voice we actually use.
- 8 min readrag · retrieval · evaluation · memory
Similarity cannot tell you which fact is current
Plain RAG handles one-off corrections. It fails when an entity has a long history of near-identical facts. A cheap recency lane cuts the stale-fact leak.
Read the note - 7 min readlatency · real-time · fraud · architecture
Sub-10ms decisioning: where the model isn't
In a real-time decisioning system, the language model is not the thing making the decision. It is the system around the decision. Put it in the hot path and you turn a risk engine into a latency incident.
Read the note - 9 min readagents · security · prompt-injection · production
Your agent's tools are the attack surface
The thing that goes wrong with a production agent is rarely the model saying something rude. It is the model being talked into misusing the tools you handed it. Every tool you give an agent is a permission you give to whatever can talk to it.
Read the note - 8 min readagents · multi-agent · architecture · production
Most multi-agent systems should be one agent
The multi-agent demo looks incredible. The multi-agent production system is where teams go to debug nondeterminism for a quarter. A multi-agent system is a distributed system where the nodes can hallucinate.
Read the note - 8 min readmemory · context · agents · architecture
The context window is not your memory
Million-token context windows did not remove the need for memory architecture. They hid the bill for a while. A context window is what the model can see right now. Memory is what it can get back later.
Read the note - 7 min readfine-tuning · rag · models · ai-engineering
Fine-tuning answers a narrower question than you think
When a team says they want to fine-tune, the next question is usually 'to fix what?' The answers cluster, and most of them are not fine-tuning problems. Fine-tuning changes how a model behaves, not what it knows.
Read the note - 9 min readrag · retrieval · evaluation · production
Why most RAG systems fail before retrieval
The retrieval algorithm is rarely the problem. Most RAG failures happen earlier, at stages the team isn't looking at. Here's the failure shape we keep seeing and the order we'd actually debug it in.
Read the note - 10 min readevaluation · production · ai-engineering
Why eval harnesses belong in week one
Most teams treat evaluation as a post-launch optimisation. By the time launch happens, the team is debugging with vibes and reverting changes based on hunches. The eval set is week-one work, not week-six work.
Read the note - 10 min readagents · cost · operations · production
The real operational cost of AI agents
Token bills are the visible part of the cost. The bigger numbers are hidden in retries, fallbacks, conversation context growth, and cost accounting nobody set up. Cost discipline is an architecture decision, not an optimisation.
Read the note - 9 min readarchitecture · infrastructure · production
Boring on purpose: the stack that survives a year in production
Every framework you adopt is migration risk you accept on day one. The cheapest production system is the one made of components that have been in production for years. Boring is a feature.
Read the note - 9 min readvoice-ai · latency · performance · production
The latency budget you didn't know you had
Many voice AI and real-time agent projects ship with no explicit latency budget. They discover the budget exists when users start hanging up. The median number is the lie; the p95 is the system.
Read the note