Round-Trips Pile Up Latency and Tokens
The classic tool-calling loop is a chain of round-trips: the model picks one tool, the harness runs it, the result is fed back, and the model picks the next. For multi-step tasks with strong sequential dependencies — query, filter, aggregate — those round-trips accumulate linearly, and because the entire prior conversation is re-sent as the prompt at every step, latency and tokens balloon together. The Build 2026 materials, published on June 3, 2026, named exactly this per-step round-trip accumulation as a primary driver of runaway latency and cost.
CodeAct: Don't Pick — Batch in Code
CodeAct in the Microsoft Agent Framework (MAF) ships as an alpha. Rather than choosing one tool at a time, the model generates a Python program that calls call_tool(), and that program runs all at once inside a Hyperlight micro-VM sandbox. Multiple tool calls, along with the branching and looping between them, are handled inside a single program, so the model-to-harness round-trip collapses to one. The published benchmark reports a 52.4% latency reduction (27.81s → 13.23s) and a 63.9% token cut (6,890 → 2,489).
The Supporting Cast: Agent Harness and Hosted Agents
Announced alongside it, Agent Harness ships context compression, file-based memory, and task management out of the box, so prompts don't grow without bound on long tasks. Foundry Hosted Agents reached general availability (GA) with scale-to-zero and per-session VM isolation — the foundation for running CodeAct-generated code in an execution environment isolated per session.
From Design to Operations: A CodeAct Adoption Checklist
Fix the acceptance bar in numbers before you migrate. Tool round-trips per task cut to half of the pre-adoption count, tokens per task down at least 30% (half of the benchmark's 63.9%), code-execution success rate at or above 95%, and zero sandbox-escape attempts detected make a reasonable starting point. The 27.81s → 13.23s latency figure came from one specific workload, so treat it as a reference line, not a target, until you re-measure on your own golden set.
Not every task should go to CodeAct. Read-and-aggregate workloads with strong sequential dependencies and no side effects fit well; workloads that need a human approval between steps become a poor fit, because batching them into one program erases the intervention points. Fix the routing rule in the table below and classification of an incoming task finishes instantly.
| Fits CodeAct | Poor fit for CodeAct |
|---|---|
| Multi-step query / filter / aggregate with strong sequential dependency | Per-step human approval required (payments, permission changes) |
| Combining, looping, or branching over several tool results | Every call triggers an irreversible side effect |
| Bulk retrieval then summary, where round-trips accumulate linearly | Intermediate output needs human review before proceeding |
Failure patterns come in three forms. First, not adopting CodeAct at all, so per-step round-trips accumulate and latency and cost explode — the original problem this release targets. Second, executing model-generated code with no micro-VM isolation like Hyperlight, which turns into an incident the moment arbitrary code touches the filesystem or network. Third, having no fallback path to the single-call tool loop when code execution fails, so one code error cascades into total task failure.
Wire the recovery branches accordingly. When generated code fails with an exception or timeout, fall back automatically to the single-call tool loop to carry the task to completion, and record each fallback in a dedicated counter as a regression signal for code-generation quality. On a detected sandbox-escape attempt, do not fall back — halt immediately, and quarantine the program and prompt for post-mortem analysis.
Lock the operational log schema before the migration. Tool round-trips per task, tokens per task, code-execution success rate, and sandbox-escape-attempt counts are the minimum fields that let you compare the CodeAct path against the single-call path on one dashboard. Run execution on top of Hosted Agents' per-session VM isolation so state doesn't leak between sessions, and keep idle cost pinned to zero with scale-to-zero.
The improvement loop turns on the fallback rate. Periodically gather the logs where code execution failed, rank the tool signatures and prompts that produced bad code, and reinforce tool descriptions and examples to shrink the top offenders. If the fallback rate won't drop below its target (say, 5%), that signals CodeAct is not realizing its latency and token gains on real traffic — so tighten the fit table for eligible workloads.
Takeaways at a Glance
The crux of collapsing round-trips into one program is classification and fallback, not the benchmark number. Route only read-and-aggregate tasks with strong sequential dependency to CodeAct, execute code only on top of a sandbox like the Hyperlight micro-VM and per-session VM isolation, and always keep a fallback to single-call tools when code fails — then the 52.4% latency and 63.9% token savings have a chance to reproduce outside the benchmark. Putting the four metrics — round-trips, tokens, code-execution success rate, and escape-attempt detections — on your dashboard is where that starts.