Intro: The Illusion of Control — When Software Stops Running Like Clockwork

I. From Vending Machine to Chatbot #

Software used to run like clockwork: same input, same result. AI and distributed systems have broken that promise. This series explains why, and how to build fences so a system only fails within limits set in advance.

Put a 10,000-dong note into a vending machine and out drops a can. Ask a chatbot to regenerate an answer and you usually get a different one. A vending machine doing that gets a repair call; a chatbot doing it is working as designed.

For sixty years, software worked like that vending machine. Code is instructions written in advance, if-else style: if this holds, do that. Same input, a thousand runs, same result; when something broke, you could trace it to one line of code.

Lately, for hard problems like understanding language, people have stopped writing if-else by hand. Instead, they train AI models by having them predict over huge amounts of data, then nudging billions of internal numbers so they are less wrong next time. Those numbers are weights; nobody writes them by hand, not even the model’s trainers.

That is the jump from Software 1.0 to Software 2.0¹: behavior moves from code a person wrote to weights learned from data.

An LLM is the kind of AI model behind chatbots like ChatGPT. Each time it extends its answer, it gives every possible next word piece a probability, then draws one. It picks by likelihood, not by checking what is correct, so reading the code no longer tells you what the system will say.

So software teams are losing control. A prompt is the instruction you type to an LLM. Prompts now often get tuned by feel: add “please answer accurately,” run it a few times, decide it seems better, never measure how much. People in AI call this habit vibe-tuning².

Worse, the model can invent a fluent, confident, wrong answer, with no line of code to trace the cause. This is called hallucination.

The Illusion of Control is a series about taking that control back, differently. Controlling every output is an illusion. What you can take back is deciding in advance how wrong a system may be and capping it there, instead of designing only for the smooth path. But AI is only half the story.

II. The Series in Four Stages #

The other half needs no AI. Behind a food-delivery app, many services usually talk to each other over the network. A service is a program on a server that does one job, such as payments or finding a driver. A system built like this can freeze with no part broken.

Rush hour in Ho Chi Minh City, with its packed motorbike traffic, works the same way: one hard brake can stop the whole stretch, with no accident ahead. On paper, every service is correct, just as every rider follows the rules. On the road, the whole stretch still does not move.

I work from the ground up, what engineers call First Principles³, borrowing tools from math and physics to explain things like this. Every time, I come back to one question: what does it mean for software? The current plan has 16 papers in four stages:

• Stage I — Taking Old Beliefs Apart (Papers 01–03). LLMs break the promise of “same input, same result.” Splitting a system into many services makes the number of situations to anticipate shoot up. Big incidents rarely have a single root cause.
• Stage II — The Toolkit (Papers 04–07). Why bigger systems tend to produce bottlenecks that almost everything else depends on. Why systems drift to the edge of collapse on their own, and freeze under overload instead of slowing down. Paper 07 covers how failures spread across a network.
• Stage III — Design (Papers 08–11, planned). Why automatically resending failed requests can take a system down, how to add and remove servers automatically without overdoing it, and Chaos Engineering. Chaos Engineering means injecting failures on purpose, such as garbage input or a machine switched off mid-run, to find weak points before users do.
• Stage IV — People and Organizations (Papers 12–16, planned). How team structure shapes software structure. The limits of AI when it must understand root causes. A closing paper on switching roles: from builder, who places every brick, to gardener, who sets conditions and prunes.

Six papers are out. So why read them if you don’t write code?

III. Why Read This If You Don’t Write Code #

The series is for everyone who works on a system: PMs, BAs, testers, designers, students, career switchers, junior developers. You may never run a server, but you are in the room when old intuition drives a wrong decision:

• Discussing an AI feature. Ask “how often may it be wrong, and what stops it then?” instead of “is it correct?”
• Reading a dashboard. The server shows 80% CPU and zero errors. Ask me how much room it has left, and my first reflex is still “20%.” Papers 05 and 06 explain why that does not mean 20% of safety left.
• Reading an incident report. Paper 03 explains why one root cause rarely tells the whole story.

You need no background in statistical physics, graph theory or calculus. Equations and code sit in collapsible “Going deeper” blocks, with a plain reading and a concrete number. Skip them all and the argument still reads end to end. The math is there to check the argument, not to understand it.

Looking for a ChatGPT tutorial? Wrong place. Looking for how to fence in a system that is only right with some probability? Right place.

IV. Start With Paper 01 #

Read Paper 01 first; it shares the series title. It argues that hallucination, and a model repeating one sentence endlessly, follow from how an LLM draws its answer one word piece at a time, and are not bugs one more if could fix. Each new piece builds on the earlier ones, so one wrong piece can drag the rest along, and nothing guarantees how many draws pass before the sequence stops.

The paper then builds three layers of fencing: block badly formatted output, commit to an error rate as a number, and put a circuit breaker inside the model. The second layer uses Conformal Risk Control. This statistical technique lets you say “the output is wrong at most 5% of the time,” on average, and check that claim against data. The price is that answers the model is not sure enough about go to a person instead.

“Wrong at most 5% of the time” sounds like a pre-written excuse. The difference: this number can be measured and rechecked, and “it’s probably fine” cannot.

The job of writing code is turning into the job of designing systems.