Topic
19 articles on LLMs, written from production systems rather than documentation.
Asking a model to reason step by step improves some tasks and slows every one of them. The trade is latency and tokens, paid on every single call.
Teams pick one model and route everything through it. Most systems have several jobs with different requirements, and one answer fits none of them well.
A surprising share of what matters in an AI feature is mechanically checkable. Those checks are fast, free and boring, which is why teams skip past them.
Examples are the fastest way to fix a format problem and a slow way to fix a reasoning one. Adding more has a ceiling that arrives earlier than expected.
Teams reach for fine-tuning to make a model know their data. It is much better at teaching behaviour, and facts have a cheaper home.
Grading output with another model scales in a way humans cannot. It also has biases of its own, and a judge nobody validated is just a number.
The capability gap narrows every few months. What does not narrow is the difference in who controls deprecation, data residency and your cost curve.
Caching the stable prefix of a prompt cuts cost and time to first token substantially. It only works if the stable part is genuinely first.
A prompt is a load-bearing configuration string with no type checker behind it. Changing one line can break behaviour nobody remembers depending on.
Once you have sent a 200 and started writing tokens, there is no status code left for the failure that happens halfway through the answer.
Asking politely for JSON works most of the time, and most of the time is a poor guarantee for something a parser sits behind in production.
A system prompt is the one piece of a feature everybody edits and nobody owns. Structure decides whether it stays coherent or becomes sediment.
Whether it needs designing from scratch or rescuing from its own success, tell me what you’re building and I’ll tell you how I’d architect it.
Open to remote and hybrid work worldwide