Agentic Loops and the SDK

Situation

A code-review assistant checks three fixed aspects for every pull request — style, security, and docs — each requiring reads, analysis, and its own report section. The workflow never changes from PR to PR.

Recommended approach

Use prompt chaining: analyze style, then security, then docs as sequential steps, then synthesize a final report.

Why it works

The workflow is fixed and always the same three steps. Prompt chaining — a predetermined sequence where each step produces a focused output before a final synthesis — matches a known, repeatable pipeline and keeps each step reliable and easy to inspect.

Weaker approaches people try — and why they fall short

• Use an orchestrator-workers pattern that decides which checks are needed dynamically.Orchestrator-workers is built for dynamic task sets; here the checks are always the same three, so dynamic routing adds unneeded complexity.
• Route to different prompts based on PR type (feature, bugfix, refactor).Routing by PR type isn't needed — all three checks run for every PR regardless of type.
• Use one comprehensive prompt that handles all three checks at once.One giant prompt tends to do each check less thoroughly and is harder to debug.

Situation

A metadata-extraction tool must run before two enrichment tools that depend on its output (a document ID). For a combined request ('extract the metadata and tell me how old it is'), the agent sometimes calls an enrichment tool first, which then fails for lack of the ID.

Recommended approach

Force tool_choice to the specific metadata-extraction tool for the first turn, then let the enrichment tools run in later turns once that data exists.

Why it works

A guaranteed ordering is needed: metadata first, enrichment after. Forcing tool_choice to name that specific tool guarantees the first step produces the needed ID, and enrichment happens naturally afterward — enforcing the dependency deterministically rather than hoping the model sequences correctly.

Weaker approaches people try — and why they fall short

• Set tool_choice to 'any' and add a system-prompt note prioritizing the metadata tool.'any' forces some tool to be called, but not necessarily the right one — an enrichment tool could still be picked first.
• Leave tool_choice on 'auto' and simply list the metadata tool first in the tools array.The order tools appear in the array doesn't reliably determine which one gets selected — this is a common misconception.
• Force the metadata tool on every single call in the pipeline.Would block the enrichment tools from ever being allowed to run.

Situation

On complex disputes needing many tool calls, an agent sometimes hits its turn limit after gathering data but before resolving the issue or escalating it. The goal is to guarantee every interaction ends in either resolution or a human handoff, no matter how the loop ends.

Recommended approach

Add orchestration-layer code that checks the outcome after the loop terminates; if neither resolution nor escalation happened, it programmatically calls the escalation tool with the accumulated context.

Why it works

A true guarantee can't depend on model judgment or hitting a specific threshold, because the loop can terminate for many different reasons. Wrapping the agent in code that inspects the final state and force-escalates whenever the outcome isn't 'resolved or escalated' catches every termination path — guarantees belong in the code around the agent, not inside its reasoning.

Weaker approaches people try — and why they fall short

• Add a system-prompt instruction telling the agent to escalate when it can't finish in its remaining actions.The agent may still run out of turns before it acts on that instruction — no real guarantee.
• Add a pre-tool-use hook that auto-escalates once 80% of the turn limit is used.An 80%-threshold hook helps in one specific case but doesn't cover terminations from other causes, like errors or unexpected stops.
• Split the work into two sequential agents — one to gather data, one to act — each with its own turn budget.Both agents still have their own budgets and can each terminate mid-way without a guaranteed final handoff.

Situation

During a multi-step identity verification, after the customer answers the third question, the agent asks for their name again — as if the earlier exchange never happened.

Recommended approach

Check whether the full conversation history is actually being included in each subsequent API request.

Why it works

The API is stateless — the model only 'remembers' what gets resent to it. If the agent appears to forget earlier answers, the near-certain cause is that the prior messages simply aren't being included in the next request. The full history has to be resent every turn; there's no server-side memory to fall back on.

Weaker approaches people try — and why they fall short

• Assume the verification tool is clearing its own internal state after each step.A tool doesn't wipe the model's context — the context is simply not being resent.
• Assume there's a default 'remember the last two turns only' setting that needs to be extended.There's no such default — memory is whatever history you choose to include, nothing more.
• Add prompt instructions telling the agent to remember across exchanges.Instructions to 'remember' can't help if the information isn't actually present in the request being sent.

Situation

A warranty-claim workflow needs a sequence of calls — look up the customer, then the order, then either process a refund or escalate. What should primarily decide whether the loop keeps going or stops?

Recommended approach

Check the response's stop_reason after every call: keep looping while it's 'tool_use', and exit once it becomes 'end_turn' or another terminal value.

Why it works

The agentic loop is governed by the API's stop_reason field. While it's 'tool_use', the model wants to call a tool, so the loop executes it, appends the result, and continues. Once it's 'end_turn', the model considers itself done — that field is the intended control signal for the loop, not a workaround.

Weaker approaches people try — and why they fall short

• Manually set tool_choice to 'none' once the last expected call has happened.A manual override, not a way to actually detect that the model considers itself finished.
• Exit the loop as soon as the response contains any text block.Text can appear alongside a tool-use request, so its mere presence isn't a reliable stop signal.
• Count tool calls and stop once a preconfigured maximum is reached.A max-call counter is a safety cap, not the primary mechanism for deciding when to continue or stop.

Situation

Mid-dispute, an agent discovers a promotional-pricing error that needs manager approval — beyond what it's authorized to resolve on its own.

Recommended approach

Compile a structured handoff — customer details, order information, and the specific issue identified — before calling the escalation tool.

Why it works

A good escalation hands the human a concise, structured package of exactly what matters: who the customer is, what the order was, and what the specific issue is. That lets the human act immediately without having to re-investigate from scratch, which is the whole point of escalating with context attached.

Weaker approaches people try — and why they fall short

• Attempt the refund anyway, and only escalate if the system rejects it.Attempting an unauthorized refund risks a real policy or compliance violation.
• Escalate with only the customer's original message attached.Forces the human to redo all the diagnosis the agent had already done.
• Persist the full conversation and tool history to a database, then escalate with just a reference ID.Burying the key facts behind a reference ID adds friction; a focused summary is more immediately useful (a full log can still complement it, but isn't the core need).

Situation

An order-lookup tool returns 'purchased 45 days ago.' How does the agentic loop actually decide whether to process a refund or escalate to a human next?

Recommended approach

The order details get added to the conversation, and the model reasons over them to choose which action makes sense.

Why it works

In an agentic loop, tool results are appended to the conversation and the model reasons over them to pick the next action. Seeing '45 days ago,' the model itself weighs whether policy allows a refund or requires escalation — flexible reasoning over results, not a hard-coded router, is the defining behavior of this kind of loop.

Weaker approaches people try — and why they fall short

• Use a pre-configured decision tree that maps order attributes directly to specific tool calls.A fixed decision tree removes exactly the reasoning that agentic loops are built to provide.
• Have the agent execute a tool sequence that was fully planned out at the very start.Pre-planning the whole sequence upfront contradicts the react-to-results nature of the loop.
• Have the orchestration layer route based only on the order's status field.Hard-coding routing on a single field bypasses the model entirely and is brittle to any nuance.

Situation

Deciding when an agent should escalate to a human. Which trigger most reliably catches the cases that genuinely need a person?

Recommended approach

Escalate when the customer explicitly requests a human, the issue requires a policy exception, or the agent genuinely can't make further progress.

Why it works

Genuine need for a human is contextual, so the criteria should capture the real underlying reasons: an explicit ask, a situation needing authority the agent lacks, or a real impasse. These judgment-based conditions map to actual 'needs a human' situations far better than rigid proxies.

Weaker approaches people try — and why they fall short

• Use a rules engine mapping issue types, customer segments, and products directly to escalation, removing model judgment entirely.A pure rules engine can't anticipate every situation and mishandles novel cases that genuinely need judgment.
• Trigger escalation from a sentiment-analysis frustration score threshold.Frustration isn't the same as needing a human — a calm customer can still need escalation, and an annoyed one may not.
• Escalate automatically after three consecutive failed tool calls.A fixed failure count is a crude proxy that both over-escalates and under-escalates depending on the actual situation.