Topic

System Design

11 articles on System Design, written from production systems rather than documentation.

3 min read

Content moderation for a feature that generates text

Moderating what users send is the familiar half. Moderating what your own product says, in your name, is the half that carries the liability.

AISecurityEvaluationSystem Design
Read post
3 min read

Fine-tuning teaches form, retrieval supplies facts

Teams reach for fine-tuning to make a model know their data. It is much better at teaching behaviour, and facts have a cheaper home.

AIRAGLLMsSystem Design
Read post
3 min read

Human review that earns the time it costs

People are the only judges who catch what nobody thought to check for. Spending that attention on random samples wastes most of it.

AIEvaluationObservabilitySystem Design
Read post
3 min read

Open weight or closed model: the decision is mostly not quality

The capability gap narrows every few months. What does not narrow is the difference in who controls deprecation, data residency and your cost curve.

AILLMsArchitectureSystem Design
Read post
4 min read

What embeddings actually encode, and what they miss

Embeddings capture how text is used, not what it means. That distinction explains most of the retrieval failures that look inexplicable from the outside.

AIEmbeddingsRAGSystem Design
Read post
4 min read

When not to build an agent

Most problems described as agent problems are retrieval, a workflow, or a form. Agents earn their cost in a narrow band, and it pays to know where it starts.

AIAgentsArchitectureSystem Design
Read post
4 min read

Why evaluations come before scale, not after it

Without evals you cannot tell an improvement from a regression, so every change becomes an argument. That is a team problem before it is a quality one.

AIEvaluationLLMsSystem Design
Read post
3 min read

Leader election: making sure only one node does the dangerous thing

Cron jobs and reprocessors are trivial with one replica and dangerous with three. Leader election lets multiple replicas run without duplicating one-off work.

Distributed SystemsConsensusSystem Design
Read post
3 min read

CAP theorem, explained the way it actually shows up in production

CAP theorem is usually taught as an abstract triangle. In production it's one question: what should this service return when it can't reach another node?

Distributed SystemsDatabasesSystem Design
Read post
4 min read

Latency vs throughput: you cannot optimise for both at once

Batching helps throughput and hurts latency. Caching helps latency and can hurt consistency. Most performance debates are about which number matters more.

PerformanceSystem DesignDistributed Systems
Read post
4 min read

The sidecar pattern in system design

Attaching a second container to handle logging, mTLS, retries and observability, so your main service doesn't have to. When a sidecar earns its overhead.

System DesignMicroservicesKubernetes
Read post

Have a system to build?

Whether it needs designing from scratch or rescuing from its own success, tell me what you’re building and I’ll tell you how I’d architect it.

Open to remote and hybrid work worldwide