Where does the agent loop go when the system already runs a durable execution engine? The fork looks like a tool choice — agent graph versus engine — but both are orchestrators: each has its own loop, its own state, its own retries. The question is not which one wins. It is where to draw the line between them, and what that line costs.
What follows is a month of running a bench where the line is drawn: Temporal on Python, a model gateway, call tracing, seven scenarios including a developer agent and an automated SRE investigation.
The claim it started from
The rule was adopted on day one: one model call equals one activity, and the graph belongs to the durable execution engine, not to the agent framework.
The reasoning sounded solid. Wrap a whole agent loop into a single activity and the engine outside sees one opaque block: a failure on step eight of fifteen replays it from step one. Step-level durability, the reason the engine was picked, disappears, and two layers of retries appear that know nothing about each other.
An honest caveat: that is reasoning, not measurement. The alternative was never built on this bench, so the month tested the price of the decision rather than the claim itself. The price is the data.
What cannot be claimed
“Agent frameworks give you no durability” is factually wrong. Modern agent graphs persist state, with a choice of mode: on graph exit only, asynchronously, or synchronously before every step. The real difference runs along a different line.
First, granularity: what counts as a step that is not replayed. Second, whose budget caps the attempts. Third, what a resume does — nodes after the recovery point execute again, including model calls and outbound requests. The recovery point is the unit you pay for twice.
The official integration between the engine and the agent SDK addresses exactly this: the agent loop, tool selection and handoffs live in the workflow, while each individual model call runs as an activity and is therefore not repeated on replay. An agent framework locked inside one activity is bad. An agent loop under an orchestrator with a turn-per-activity is a supported, ordinary design. The naive version of the claim erases that distinction.
Three retry layers you cannot stack
In practice it came down to a hard split of activity profiles, and the difference between them matters.
| Profile | Timeout | Attempts | Why |
|---|---|---|---|
| code: validation, rendering, writes | 20–30 s | 3 | failure is cheap and usually transient |
| model call | 180–240 s | 1 | failure is not transient and costs money |
| production effect | from policy | 1 | a silent replay of a mutation is a decision, not a default |
Retry-with-feedback — validation errors fed back into the prompt — lives inside the step. An infrastructure retry on top of it would be a second, invisible layer: it knows no contract, sees no episode budget, and pays for every attempt. That is what the original claim warned about, except the layers turned out to be “step retry versus activity retry” rather than one framework against another.
The effect deserves its own line. It is idempotent by key and still gets no infrastructure retry: idempotency makes a repeat safe, it does not make it wanted. Such an activity takes its deadline from the policy file rather than a constant in code, so the timeout written in the policy is the real one.
Where agency ends
The formula the bench arrived at: freedom inside the episode, determinism between episodes.
The only place where the model decides what to do next is a child episode. It picks which files to open and in what order, returns a typed proposal, and dies. Outside it nothing changes: the snapshot, the attempt budget, whether the result is applied and whether another pass happens all belong to the parent. The child has no write tool and no shell, and the read tools resolve every model-supplied path and refuse anything that lands outside.
Budgets live as constants in code, not as wishes in a prompt: a turn limit within a pass, a limit on passes, a separate cap on rejected answers — counted per episode, not per round. A per-round cap multiplies out: three rounds of three attempts is nine calls on the worst path, each carrying whole files in context.
The turn limit earns its place for another reason worth knowing in advance. A model that keeps re-reading the same file will spend the entire budget without moving. That is exactly how a reasoning model behaves when its own history is rewritten underneath it — by a sanitiser tokenising paths in its past tool calls, for instance.
The failure no layer sees
The bench's main finding, and the reason “wrap it and forget” does not work.
A generation that ends in a provider-side error arrives as HTTP 200 with no content and no tool call. It burns tens of thousands of output tokens and carries no price header — nobody charges for it.
| Layer | Fires on | Why it stays quiet |
|---|---|---|
| gateway fallback chain | HTTP error | the response is 200 |
| orchestrator activity retry | an exception | there is no exception |
| agent SDK | a malformed response | the response is well-formed, just empty |
Where a turn is required to call a tool, an empty turn eats one of ten attempts, and the episode can spend its whole budget answering nothing. The bill looks cheap meanwhile — hence a dedicated metric for priceless calls: it makes the loss visible instead of free-looking. Which token-saving techniques actually work and which are oversold, I covered separately.
The detector had to read the parsed response rather than the HTTP body: consuming the body in a transport hook breaks the SDK's own parsing. By the time the SDK hands back a result the finish reason is gone, so what is left to test is the consequence — the turn produced nothing. The boundaries of that test matter. “No tool call” is wrong for agents that answer in prose; “zero token usage” would call a legitimately empty completion a failure.
One measured run on the cheap tier: 167 turns, of which 97 came back priceless. Re-sending the request to the same model saved 0 of 36 — not once. The strong tier answered all 36, none empty. The share of answered cases rose from 79 to 91 percent. The price per turn between tiers differs by roughly seven times.
What matters more than the numbers: the fix routes around the failure, it does not reduce it. The cheap tier still produces an empty turn on more than half its calls. A third path — change the request rather than the tier, with a smaller tool schema or a relaxed obligation to call a tool — was never measured here and remains a guess.
The price: a step cannot be deleted
A direct consequence of every step being an activity. Replay runs against recorded history, so removing a call a live workflow has already made is a compatibility break.
On the bench this showed up during a metrics migration: two activities lost their purpose once the data went straight into a durable table — and stayed in the code as empty stubs carrying the comment “name and signature kept deliberately”. The metric is not worth a break, and the call cannot be removed.
The other side of that coin is the whole reason for the design: the step runs inside an activity, so replay never calls the model again — the result is already in history. A worker restart, a recovery after a crash, a replayed history: none of them pay for tokens twice. What you buy is not atomicity but repeatability without paying twice — the same logic as in a deploy where the state record is written last.
When to pick what
| Situation | What to take | What you accept |
|---|---|---|
| pipeline already under a durable execution engine | model turn = activity, loop in the workflow | a step cannot be deleted; signatures live forever |
| autonomous agent, steps cheap and reversible | agent graph with its own persistence | nodes after the recovery point replay, model calls included |
| effects on the outside world | engine with explicit retry profiles | more code around every step, retries under control |
| prototype, a mistake only costs time | neither; a plain loop in code | no recovery; rewrite at the first production use |
What this experience does not prove
An agent graph owning the loop was never stood up here: everything said about its persistence modes comes from its documentation, not from a run of my own. No replay non-determinism errors occurred in a month — the data cannot separate “the discipline held” from “it was never stressed”, and the conclusion “decomposition protects you from non-determinism” does not follow. The numbers are one bench and one or two runs: between-session variance was never measured.
That the expensive part of an agentic system is not the harness itself but everything around it has its own write-up. This bench adds one correction: the expensive thing is the boundary. Draw it once, before the first production use, and then live with every step staying in history forever.