Topic, page 2
Page 2 of articles on LLMs.
Three settings control the same thing from different angles. Turning all of them is how teams get output that is either rigid or quietly unhinged.
Reason, act, observe, repeat is the shape almost every agent uses. Its weaknesses are structural, and they show up in the same three places every time.
Cost, latency and the point at which text gets cut off are all counted in tokens. Reasoning about the system in words will mislead you in all three.
A million-token window suggests you can stop being selective about context. Attention is uneven across a long window, and cost scales with what you send.
Running a model locally is free in the sense that a server is free. The bill is idle capacity, and the arithmetic decides this more than principle does.
Without evals you cannot tell an improvement from a regression, so every change becomes an argument. That is a team problem before it is a quality one.
Most teams only ever write servers. Building the client side is what shows you where the protocol stops and your own orchestration has to take over.
Whether it needs designing from scratch or rescuing from its own success, tell me what you’re building and I’ll tell you how I’d architect it.
Open to remote and hybrid work worldwide