Writing, page 3
Mostly the things I wish someone had told me before the incident, not after. Distributed systems, event pipelines and AI that has to survive real users.
A second agent multiplies the coordination problem, not the capability. What decides whether it holds together is who owns the shared plan.
Storing embeddings beside the rows they describe removes a sync problem and makes filtering a join. That is worth more than raw index throughput.
Caching the stable prefix of a prompt cuts cost and time to first token substantially. It only works if the stable part is genuinely first.
Instructions and data arrive in the same channel, so the model cannot tell them apart. No wording fixes that, because it is a property of the design.
A prompt is a load-bearing configuration string with no type checker behind it. Changing one line can break behaviour nobody remembers depending on.
Retrieval optimises for recall and the model needs precision. A reranker sits between them and fixes the mismatch without touching the index.
Approximate indexes want to be in memory, and memory is the bill. Sizing that before you commit is the difference between a plan and a surprise.
Transcription accuracy on clean audio is a solved problem. Production audio is noisy, overlapping and full of the terms your domain depends on.
Once you have sent a 200 and started writing tokens, there is no status code left for the failure that happens halfway through the answer.
Asking politely for JSON works most of the time, and most of the time is a poor guarantee for something a parser sits behind in production.
A system prompt is the one piece of a feature everybody edits and nobody owns. Structure decides whether it stays coherent or becomes sediment.
Three settings control the same thing from different angles. Turning all of them is how teams get output that is either rigid or quietly unhinged.
Whether it needs designing from scratch or rescuing from its own success, tell me what you’re building and I’ll tell you how I’d architect it.
Open to remote and hybrid work worldwide