Efficiency & cost

A workflow that works can still cost more than it needs to. The model is almost always the biggest line on the bill, so most of the savings come from spending tokens deliberately: the right model for each call, a tight prompt, and a shape that lets the platform reuse work instead of redoing it. This is not premature optimization. It is the difference between a workflow you run once and one you run on a schedule for months.

Match the model to the job

The model is chosen per agent() call (see Inference), so match each call to its work: a small, fast model for routine steps (classify, route, short summaries), and a stronger model for the genuinely hard ones. When you're not sure which to pick, reach for Auto: omit model (or pass model: "auto") and the managed lane routes each call to a fitting model, with no routing fee, and it keeps improving as better models ship. reasoning is the other dial: a high effort spends a lot of extra thinking tokens and latency, so raise it only on the steps that need careful multi-step reasoning.

// Routine triage: a small, fast model, no deep reasoning.
const { category } = await agent<Label>(`Classify: ${msg}`, {
  model: "google/gemini-3.5-flash",
  schema: LABEL,
});

// The hard step: a stronger model and more reasoning, only here.
const plan = await agent(`Work out the migration plan: ...`, {
  model: "openai/gpt-5.5",
  reasoning: "high",
});

// Not sure which fits? Let Auto route it.
const summary = await agent(`Summarize this thread: ...`); // model defaults to Auto

Scope tools and output

By default an agent carries the full tool belt. A step that only reads, or only needs to return an answer, does not need all of it:

• Narrow the built-ins. Set builtins: "read-only" for analysis, "none" for a pure classifier or judge. Every tool definition is tokens the model re-reads on each turn, so a smaller tool set means a smaller prompt and fewer chances for the agent to wander into extra turns. See Equipping agents.
• Ask for structured output. Pass a schema instead of prose you then re-prompt to reformat: one call, a typed object, no cleanup pass.
• Do the deterministic work in code. Parsing, filtering, deduping, and routing are cheaper and more reliable as plain TypeScript than as another model call.
• Save the model for the judgement only a model can make.

// A judge needs no tools and one structured answer.
const { verdict } = await agent<Verdict>(`Is this correct? ...`, {
  builtins: "none",
  schema: VERDICT,
});

Prompt caching

Managed inference caches the stable front of your prompt automatically, with nothing to configure. The win shows up inside a single agent() call that takes more than one turn (a tool-use loop): the instructions, tool definitions, and early context stay the same across turns, so after the first turn the model reads that prefix from cache at a small fraction of the normal input cost, and only the new tokens are charged in full. A long agentic step can be a great deal cheaper than its raw token count suggests.

To earn that discount, keep the front of the prompt stable and put the part that varies last. Lead with fixed instructions and shared context; append the specific item, the latest message, or the file under review at the end. Anything baked into the instructions that changes every run (a timestamp, a random id, "today is ...") makes every turn look new, so nothing is reused.

// Good: the instructions are a stable prefix the loop reuses every turn.
const review = await agent(`${RUBRIC}\n\nReview this PR:\n${diff}`);

// Avoid: a timestamp in the instructions changes the prefix on every turn,
// so the cache never hits.
const bad = await agent(`Run at ${new Date().toISOString()}\n${RUBRIC}\n${diff}`);

A standalone call with no follow-up turn (a one-shot classifier or judge, especially with builtins: "none") is a single model turn, so there is no later turn to read a cache. That is expected. Do not engineer caching for those; make them cheap with a small model and a tight prompt instead. And when many items share the same large context, prefer one agent loop that works through them, so the shared context is a cached prefix across its turns, over many separate one-shot calls that each resend the whole thing fresh. Very short prompts are not cached at all, which is fine.

Parallelism and patterns

Independent work should run concurrently. parallel() runs a batch of tasks at once on the same held machine, so the run finishes in the time of the slowest task rather than the sum. It is concurrency on one machine, so the cost is the tokens and compute you would have spent anyway, just sooner.

// Concurrent, not sequential: finishes in the time of the slowest item.
const results = await parallel(items.map((item) => () => handle(item)));

The multi-agent shapes in Patterns & loops (a panel of verifiers, a fan-out of researchers, a tournament) buy quality and coverage, and they spend real tokens to do it: five verifiers is five times the tokens of one answer. Reach for them when a task is genuinely large, parallel, or adversarial, and use a single focused agent() call for everything an everyday task handles well.

Don't pay to wait

Use sleep() instead of polling. A long sleep suspends the run and releases its machine, so you are not billed while it waits, then it resumes where it left off. A loop that wakes every minute to check on something keeps a machine (and often a model) busy the whole time; a single sleep until the next check, or until a known time, costs nothing in between. The full suspension semantics, including why budget caps count active compute only, are on Human-in-the-loop.

// Releases the machine while it waits, then resumes in place.
await sleep({ until: "2026-08-01T09:00:00Z" });

Guardrails and reuse

• Set a budget. max_usdstops a runaway loop before it burns money. A breach pauses the run and asks for approval (it never hard-kills mid-thought), so a false alarm costs a click, not the run's progress; usage.get() reads the live state if you want to self-govern before the pause.
• Right-size the machine. The default is small and fits most workflows, so ask for a larger runs_on only when a step is actually CPU or memory bound, since a bigger machine costs more per second.
• Persist expensive setup. If a run rebuilds the same thing every time (a cloned repo, a downloaded model, a built index), name it in workspace: { persist: [...] } so the next run reuses it.
• Never redo finished work. Put work you must not repeat behind workflows.call(), which re-attaches to a finished child on restart rather than running it again.

{
  "slug": "nightly-audit",
  "triggers": [{ "kind": "cron", "expr": "0 2 * * *" }],
  "budget": { "max_usd": 5 },            // pause for approval if cost runs away
  "workspace": { "persist": ["repo"] },  // reuse the clone next run
}