What you will learn How the self-healing loop runs, and the dangerous case where the loop manufactures a "pass."
The tests defined in IntentOps become the judge here.
All pass means done. One failure means not done.
There is no "almost there" or "good enough." That clarity is what makes automation possible — you can move to the next stage without a person judging.
On failure it does not call a person — it fixes itself.
HyperTeamsgraph TD
T["Run the tests"] --> F{"All pass?"}
F -->|"no"| D["Diagnose the failure"]
D --> R["Regenerate the code"]
R --> T
F -->|"yes"| N["To deployment"]One cycle looks like this:
The human is not in this loop. Rather than turning the loop, people look at whether the loop's criterion is correct.
The greatest weakness of this methodology.
When tests are shallow, the loop manufactures a "pass."
An example of what that means:
An extreme example, but the principle is the same. The loop's goal is passing tests, not satisfying intent. When those diverge, the loop goes towards the tests.
| When a person builds | Self-healing loop | |
|---|---|---|
| If tests are shallow | the person fills the gap with common sense | it finds the gap precisely |
| Result | generally as intended | satisfies the tests only |
A person knows "that's not it." The loop does not. That is why shallow tests are more dangerous with AI than with people.
The moment you fit tests to the code, the premise of this methodology collapses.
One layer cannot distinguish passing from correct.
| Layer | What it catches |
|---|---|
| Unit | the behaviour of one function |
| Integration | whether the parts work together |
| E2E | whether it actually works from the user's view |
With E2E, something like "total = original − 1" gets caught, because it is visibly wrong on the actual screen.
Problematic code runs somewhere isolated before it reaches production. While the loop is turning it must not touch real data.
Turning forever only burns cost. If five attempts do not fix it, the problem is in the specification or the tests and a person needs to look.
Everything above is tests for code. If what you are building is the AI response itself (customer support wording and so on), you use evals, not tests. Do not confuse the two.
1. What does the human do in a self-healing loop?
Not turn the loop, but check that the loop's criterion is correct. Whether the tests properly express the intent is the human's job.
2. Why are shallow tests more dangerous with AI than with people?
A person fills gaps with common sense, but the loop's goal is passing tests, so it finds the gaps precisely. You get code like "total = original − 1" which passes while violating the intent.
3. Why must you not edit the test to make it pass?
Because the test is the only line of defence. The moment you fit tests to code, the premise that "tests judge done" collapses, and then regenerating code can no longer be verified.
Stand up gates on one of your own outputs. Forty minutes → Design your validation gates
Turn 1: test 3 fails — "expired coupon message differs" → diagnosis: wording does not match the specification → fix and rerunTurn 2: test 5 fails — "total does not update" → diagnosis: missing state update → fix and rerunTurn 3: all pass → next stageTest: "after applying a coupon, the total must be less than the original"Passing code AI produced: total = original - 1→ The test passes. The intent is completely violated.✗ the code doesn't pass, so edit the test ← the defence line is gone✓ the code doesn't pass, so fix the code✓ the test differs from the intent, so fix the specification first□ maximum retries (e.g. 5)□ call a person when exceeded