Worked examples

Situation

A shared venue-lookup MCP server should be available to the whole team; a personal experimental playlist server should be visible only to you.

Recommended approach

Put the shared venue server in the project-level .mcp.json, and the personal playlist server in your own ~/.claude.json.

Why it works

Project-level .mcp.json lives in the repository and is shared with everyone who checks it out, so the team-wide server belongs there. ~/.claude.json lives in your home directory and is private to you, so the experimental server belongs there. Each server ends up matching who should actually see it.

Weaker approaches people try — and why they fall short

• Put both servers in your local ~/.claude.json.Teammates never receive the shared venue server.
• Put the venue server in ~/.claude.json and the playlist server in .mcp.json.This is backwards: the private experiment becomes team-visible, and the shared tool becomes invisible to everyone else.
• Put both servers in the project-level .mcp.json.Your private experiment gets exposed to the whole team.

Situation

CLAUDE.md says 'use 4-space indent and run Prettier,' but about 30% of generated files are still mis-formatted. Adding IMPORTANT/MUST language only cuts that to about 15%.

Recommended approach

Add a PostToolUse hook (matching Edit/Write) that automatically runs Prettier on every file Claude modifies.

Why it works

Formatting is a mechanical, deterministic task. A PostToolUse hook runs real code every time a file is edited or written, so formatting is applied 100% of the time regardless of what the model remembered. Deterministic problems deserve deterministic tools, not stronger wording.

Weaker approaches people try — and why they fall short

• A Stop hook with a prompt-based check asking Claude to fix violations.Still relies on the model's judgment to detect and fix issues — probabilistic, so violations slip through.
• Split rules into path-scoped rule files.Still instructions the model may or may not follow; organizes guidance without enforcing it.
• Extract the rules into a dedicated file with more examples.More examples and emphasis is the same probabilistic approach that already plateaued around 15% failure.

Situation

A new payment module should mirror patterns already used in three existing files. It's a one-off task, and the patterns are already clear in the code itself.

Recommended approach

Use @ references to pull the three existing modules directly into the prompt.

Why it works

@file references bring the exact code into context, so Claude sees the real patterns rather than a paraphrase of them. Because this is a one-off, there's no reason to pay the ongoing cost of documenting it project-wide — concrete, immediate examples are enough.

Weaker approaches people try — and why they fall short

• Ask Claude to explore the codebase to find the patterns first.Costs time and tokens and risks finding the wrong files, when the right files are already known.
• Add each pattern to CLAUDE.md as a project convention.CLAUDE.md loads into every session forever — overkill for a one-off, and it bloats context permanently.
• Describe the patterns in natural language in the prompt.Lossy — re-describing code in prose when Claude could just read the real thing directly.

Situation

You want Redis caching with a 5-minute TTL on a product endpoint, but you're new to production caching and unsure what else a robust implementation needs.

Recommended approach

Ask Claude to interview you first, surfacing invalidation strategy, caching layers, consistency guarantees, and failure modes before anything is built.

Why it works

You don't know what you don't know. Interviewing surfaces considerations you'd otherwise miss — cache invalidation, layering, consistency, failure modes — converting unknown-unknowns into explicit decisions before code is written.

Weaker approaches people try — and why they fall short

• Use plan mode to analyze the endpoint, then give requirements after Claude explains the code.Analyzing existing code doesn't teach you the caching considerations you're actually missing.
• Write a spec with TBD markers and let Claude propose solutions per TBD.You can only write TBDs for gaps you already recognize — the dangerous gaps are the ones you can't name.
• Make a minimal request now, add features via follow-ups as problems surface.Discovering requirements through production failures is the slowest, riskiest path for something you don't understand yet.

Situation

Add a date-validation check (event dates must be in the future) — a single conditional in one function in one file.

Recommended approach

Just make the change directly.

Why it works

The change is tiny, local, and unambiguous. Direct execution is the right amount of ceremony; planning modes and extended reasoning are for complex, multi-file, or uncertain work, and using them here just adds overhead.

Weaker approaches people try — and why they fall short

• Enter plan mode first for a detailed strategy.Overkill for a single-line conditional.
• Enable extended thinking to reason thoroughly.Wastes effort on genuinely trivial logic.
• Use plan mode to analyze impact on the wider reservation flow.There's no meaningful cross-flow impact to analyze for a self-contained check.

Situation

A migration script mishandles null values in optional fields, and needs a fix.

Recommended approach

Provide a concrete test case — an example null input plus the expected output — and ask Claude to fix it against that target.

Why it works

A concrete input/expected-output example defines success unambiguously and lets Claude make a targeted fix. It also becomes a regression check you can rerun to confirm the fix holds.

Weaker approaches people try — and why they fall short

• Add 'think harder about edge cases' and request a full rewrite.Vague feedback plus a rewrite discards working code and doesn't specify the correct null behavior.
• Fix the null handling yourself, then continue.Doesn't teach the model the pattern and loses the benefit of iterating together.
• Describe the null problem and ask for a full regeneration.Risky and wasteful when a small, well-specified fix would do.

Situation

CLAUDE.md contains three rules: never edit the migrations folder, prefer a custom logger over console.log, and always Prettier-format TypeScript after edits. Claude edited a migration file anyway.

Recommended approach

Use permissions.deny for the migrations folder, keep the logging preference as a CLAUDE.md instruction, and use a PostToolUse hook for Prettier — a different mechanism for each requirement type.

Why it works

Each requirement has a different nature. A hard prohibition needs a permissions.deny rule that physically blocks the action. A soft preference is genuinely advisory and belongs in CLAUDE.md. A mechanical guarantee belongs in a PostToolUse hook. Matching mechanism to requirement type is the core skill being tested here.

Weaker approaches people try — and why they fall short

• Rewrite all three rules in CLAUDE.md with stronger language and examples.Stronger wording is still probabilistic — it can't guarantee migration files are never touched.
• Use hooks for all three rules.Using a hook for the soft logging preference is over-engineering; a preference doesn't need a code gate.
• Move all three into path-scoped rule files.Still model-followed guidance, so the absolute prohibition still wouldn't be guaranteed.

Situation

Upgrading an auth library across a major version with breaking changes (callbacks becoming promises, a restructured type, removed methods), used across 45 files in multiple modules.

Recommended approach

Use plan mode: explore usage, map affected paths, and create a migration strategy before touching any code.

Why it works

This is a large, high-risk, interdependent change. Plan mode lets Claude map where and how the library is used and produce a reviewable strategy before implementation, so mistakes get caught on paper rather than in production.

Weaker approaches people try — and why they fall short

• Build a custom slash command and run it on each file without exploring first.A blind command can't handle usage patterns it never inspected.
• Paste the breaking changes and direct-execute across all 45 files.Direct-executing 45 interdependent files at once is error-prone with no plan to review.
• Bump the version, run tests, and fix each failure as it appears.Test-driven fixing only catches what tests cover; silent breakages in untested paths slip through.

Situation

A custom MCP server exposes MCP prompts (like a deploy checklist and an incident-response template) plus tools. How do those prompts become usable inside Claude Code?

Recommended approach

They appear as slash commands in the form /mcp__servername__promptname, with arguments passed after the command name.

Why it works

MCP prompts are user-invoked templates. Surfacing them as slash commands keeps them explicit and user-triggered — distinct from tools, which the model calls on its own, and resources, which are attached via @-mention.

Weaker approaches people try — and why they fall short

• Added to the tool registry and invoked automatically by the model.That describes MCP tools, not prompts — prompts are invoked by the user, not auto-called by the model.
• Auto-prepended to every conversation as system context.Prompts aren't silently injected into every conversation.
• Surfaced as @-mentionable resources attached when referenced.That describes MCP resources, not prompts.

Situation

A team wants Claude to follow an 8-item code-review checklist for pull requests, but also uses Claude heavily for features, debugging, and docs. Right now devs paste the checklist in by hand every time.

Recommended approach

Create a /review slash command that contains the checklist.

Why it works

The checklist is only needed sometimes (during reviews), and the same Claude session is used for many other kinds of work. A slash command injects the checklist on demand, exactly when needed, without burdening every other session — replacing the copy-paste ritual cleanly.

Weaker approaches people try — and why they fall short

• Create a dedicated review subagent with the checklist embedded.A subagent is heavier than needed; the task is 'apply this checklist,' not 'run an isolated specialist with its own context.'
• Make plan mode the default for reviews.Plan mode isn't a checklist mechanism and doesn't carry the 8 items.
• Add the checklist to CLAUDE.md under a Code Review heading.CLAUDE.md loads into every session — the checklist would pollute feature, debugging, and docs work where it's irrelevant.

Situation

A team migrates React components to Vue often, has a step-by-step workflow for it, wants everyone to be able to invoke /migrate-component, and wants that workflow to stay in sync as it evolves.

Recommended approach

Put a SKILL.md at .claude/skills/migrate-component/SKILL.md, committed to version control.

Why it works

A team-shared, evolving workflow belongs in a project skill that's committed to version control. Everyone gets it automatically, it's invokable, and Git keeps it in sync as the team iterates — one source of truth.

Weaker approaches people try — and why they fall short

• A big instruction block in the root CLAUDE.md.CLAUDE.md always-loads and isn't the right home for an invokable, self-contained workflow.
• A copy of the skill file under each person's home directory.The home directory is per-machine and private — it wouldn't be shared or stay in sync across the team.
• An override entry inline in settings.json.There's no proper mechanism for this — skills live in SKILL.md files, not inline in settings.

Situation

A PDF report has three interacting problems: narrow columns truncate content, dates aren't formatted, and page breaks land wrong — and changing one affects the others.

Recommended approach

Fix column width first (with specific measurements), verify, then fix dates within the corrected columns, then adjust page breaks — testing after each step.

Why it works

Because the issues interact, changing everything at once makes it impossible to tell what fixed or broke what. Fixing one at a time and verifying after each isolates cause and effect, so each change lands on a known-good base. Column width first makes sense because both dates and page breaks depend on it.

Weaker approaches people try — and why they fall short

• Show a correct example and ask Claude to match it, without listing the issues.Matching an example doesn't supply the precise measurements the layout needs and hides which sub-issue is being solved.
• Start fresh with all requirements listed upfront.Discards working query logic and still bundles the interacting fixes together.
• Give all three issues at once with exact specs and fix them together.Fixing all three simultaneously tangles the interactions so results can't be attributed to any one change.

Situation

An infrastructure-as-code repo has separate Terraform, Kubernetes, and pipelines directories. The root CLAUDE.md is 500+ lines, and Terraform rules load even when editing Kubernetes files, wasting context.

Recommended approach

Split the guidance into subdirectory CLAUDE.md files — one inside the Terraform folder, one inside the Kubernetes folder, and so on.

Why it works

Claude Code loads a directory's CLAUDE.md based on where you're actually working. Splitting guidance this way means only the relevant directory's rules load into context — Terraform guidance stays out of the way when you're editing Kubernetes. This is the built-in, token-efficient pattern.

Weaker approaches people try — and why they fall short

• Files in a rules directory with path-scoping metadata.Plausible, but the established, built-in mechanism for locality is nested CLAUDE.md files, not a custom rules-frontmatter pattern.
• Reorganize the root CLAUDE.md into labeled sections with headers.Headers improve readability but everything still loads — no token savings.
• Keep the root CLAUDE.md and use an import mechanism for tool-specific files.An import still pulls the imported content into the same always-loaded root context.

Situation

Adding real-time updates to a product — the choice is between WebSockets, server-sent events, or polling, each with different complexity, browser support, and infrastructure needs.

Recommended approach

Use plan mode to explore the architecture, weigh the trade-offs, and present the options for the team to approve.

Why it works

This is an architectural decision with real trade-offs and organizational impact. Plan mode lets Claude lay out the options so the team can approve a direction before code is committed — big directional choices should be decided by people, informed by analysis.

Weaker approaches people try — and why they fall short

• Direct-execute polling first, planning to upgrade later.Committing to polling prematurely may require a costly rewrite down the line.
• Direct-execute WebSockets and refactor if infrastructure issues arise.Committing to WebSockets before assessing infrastructure risks the same kind of rework.
• Direct-execute and let Claude silently pick the 'best' approach.Letting the model choose silently hides a decision that stakeholders should actually own.

Situation

Implementing a graph-traversal algorithm with performance requirements and edge cases: disconnected nodes, cycles, and weighted edges.

Recommended approach

Write a test suite first — covering behavior, edge cases, and performance — have Claude work to pass it, and iterate on whatever fails.

Why it works

The requirements here are precise and verifiable, which is exactly where writing tests first shines. It turns 'correct behavior' into an objective target; each edge case becomes a concrete pass/fail, and feedback is automatic and unambiguous.

Weaker approaches people try — and why they fall short

• Give a reference implementation, ask Claude to restyle it and add edge cases, then compare outputs.A reference implementation may not match your actual constraints, and comparing outputs is looser than a real test suite.
• Do extended-thinking research and produce one complete implementation from the plan.A one-shot implementation gives no iterative verification loop for the edge cases.
• Give a natural-language spec, review outputs manually, and describe changes each time.Manual review with prose feedback is slower and more subjective than tests, and doesn't verify performance.

Situation

After 25 minutes exploring a rendering subsystem, the agent starts citing 'typical rendering patterns' instead of the specific classes it actually found earlier. It's now asked to study how physics and rendering integrate.

Recommended approach

Summarize the key rendering findings, then spawn a subagent for the physics work, seeded with that summary as its starting context.

Why it works

Generalizing to 'typical patterns' instead of specific classes is the tell that context is degrading — useful detail is being crowded out. Summarizing preserves the valuable findings, and a fresh subagent regains a clean working context while still starting from what was already learned.

Weaker approaches people try — and why they fall short

• Clear the context entirely and start the physics work fresh from file paths in CLAUDE.md.Throws away the hard-won rendering findings still needed to relate physics to rendering.
• Continue in the same context with more targeted prompts naming the classes.Staying in the degraded context won't restore the lost detail; it will keep drifting.
• Spawn a subagent for physics independently, then manually synthesize with the rendering knowledge afterward.Exploring independently and hand-merging afterward risks losing the specific rendering linkages.

Situation

An MCP tool for analyzing code dependencies exists and works fine, but the agent keeps using a general search tool instead for 'code dependencies' questions — because the MCP tool's description is terse while the search tool's description is detailed.

Recommended approach

Expand the MCP tool's description to spell out its capabilities and outputs in detail.

Why it works

The model chooses tools largely from their descriptions. A detailed competing description makes that tool look more capable than a vague one. Rewriting the description to state exactly what it returns (direct imports, transitive dependencies, cycles) makes it the obviously better match — fix the description, fix the selection.

Weaker approaches people try — and why they fall short

• Split the dependency-analysis tool into three more granular tools.Adds more surface area and new overlap without addressing the underlying vague description.
• Add routing instructions in the system prompt to send dependency questions to the MCP tool.System-prompt routing is a brittle patch layered on top of the real cause.
• Remove the general search tool whenever the MCP server is connected.Cripples a generally useful tool just to mask a description problem.

Situation

Caching logic spans 15 files and roughly 8,000 lines across decorators, middleware, and service classes. The goal is to understand it well enough to add an invalidation trigger, while keeping context usage under control.

Recommended approach

Analyze imports and class hierarchies to find the base cache class, read that to learn the shared interface, then trace the specific invalidation implementations from there.

Why it works

8,000 lines won't all fit usefully in context, so a structural map has to come first. Following imports and class hierarchies to the base class reveals the interface everything depends on, giving a targeted anchor to trace from — rather than brute-force reading.

Weaker approaches people try — and why they fall short

• Search for 'invalidate'/'expire' everywhere and read only those line ranges.Keyword-only slices miss the architecture and give a fragmented, potentially misleading picture.
• Sequentially read all 15 files for complete understanding.Blows the context budget and buries the signal in noise.
• Find files with 'cache' in the name and read the largest ones first.'Largest first' is an arbitrary heuristic unrelated to architectural importance.

Situation

Yesterday's two-hour deep-dive session into a legacy monolith's auth code needs to continue, but three other codebases have been worked on since, and the specific session needs to be found by name.

Recommended approach

Use --resume <name> with the session's name to load that specific session directly.

Why it works

The session's name is already known, and that specific session is what's wanted — resuming by name targets it directly, which is the natural fit.

Weaker approaches people try — and why they fall short

• Use a session-id flag with the UUID pulled from a transcript file.Works, but only if you go hunting for the exact ID — unnecessary when you already know the name.
• Use --continue to pick up the most recent conversation.--continue resumes the most recent session, which by now is one of the three other codebases — the wrong one.
• Start fresh and re-read the files.Discards two hours of accumulated context for no reason.

Situation

Intermittent 500 errors on one endpoint in a 200+ file codebase, with no idea yet which components are involved — could be routing, middleware, business logic, or the database.

Recommended approach

Dynamically generate investigation subtasks based on what's discovered at each step, adapting as the error path emerges.

Why it works

The bug's location isn't known yet, so the investigation has to be adaptive — each discovery narrows the next step. Generating subtasks as evidence emerges follows the real trail instead of committing to a plan built on ignorance.

Weaker approaches people try — and why they fall short

• First create a comprehensive plan mapping all code paths before any exploration.You can't comprehensively map paths you don't yet have the knowledge to plan around.
• Run parallel agents on all four layers simultaneously, then synthesize.Blind parallel investigation of every layer wastes effort and struggles to pinpoint an intermittent root cause.
• Follow a fixed step sequence regardless of findings.Ignoring intermediate findings is the opposite of what debugging an unknown actually needs.

Situation

Two requests: a mechanical rename of one function across the whole codebase, versus an open-ended request to improve error handling across a module (try/catch coverage, meaningful messages, no silent data corruption). Which one benefits from an analyze-then-propose-then-implement-with-review workflow?

Recommended approach

The error-handling request benefits from the multi-phase workflow; the rename does not.

Why it works

Improving error handling is open-ended and judgment-heavy — many valid designs exist, and the risk (silent data corruption) is real, so a review step before changes land is valuable. The rename is mechanical and unambiguous, so multi-phase ceremony adds little.

Weaker approaches people try — and why they fall short

• Assume both tasks benefit equally from the multi-phase workflow.They don't — one is deterministic, the other is design-laden.
• Apply the multi-phase workflow to the rename instead.A rename is exactly the kind of well-defined task that doesn't need phased review.
• Assume neither task benefits from it.The error-handling task clearly does benefit, so 'neither' undersells the judgment-heavy case.

Situation

Before removing a function, all its callers need to be found — but it's defined in a core library and re-exposed under a different, renamed wrapper name in another module.

Recommended approach

Read the library and its wrapper modules first to find every exposed alias, then search for each of those names.

Why it works

Because the function is re-exported under a different name, searching only for the original name misses every caller that goes through the alias. Reading the wrappers first to enumerate every name is the only way to catch calls made through renamed wrappers.

Weaker approaches people try — and why they fall short

• Search documentation for intended usage and navigate the documented integration points.Docs may be incomplete or stale — not a reliable way to find all callers.
• Search only for the original function name across the codebase.Misses every caller that goes through the renamed wrapper.
• Search for files importing the library or its wrappers, then read each one to check usage.Import-based scanning is noisy and still won't reliably tie usage back to the specific renamed function.

Situation

Resuming yesterday's auth analysis: the session is still valid, but 3 of the 12 previously-read files were changed overnight by a teammate's merge.

Recommended approach

Resume the session, and explicitly tell the agent which 3 files changed so it can re-analyze just those.

Why it works

Resuming keeps the valuable accumulated context — only what actually changed needs correcting. Naming the exact 3 changed files prompts a targeted re-read: accurate about the new reality, efficient because the other 9 files aren't redone.

Weaker approaches people try — and why they fall short

• Resume and immediately re-read all 12 files.Wastes effort re-reading 9 files that never changed.
• Start a fresh session to avoid stale assumptions.Throws away an hour of correct context over a 3-file delta.
• Resume without mentioning that anything changed.Leaves the agent reasoning from stale versions of those 3 files.

Situation

Adding comprehensive tests to a 200-file legacy codebase with minimal existing coverage, with no priorities specified up front.

Recommended approach

Map the codebase's structure, find the heavily-coupled (high-risk, high-value) modules, plan to test those first, and revise the plan as the dependency structure becomes clearer.

Why it works

With no priorities given, effort should follow impact. Mapping the structure and starting with high-coupling modules puts testing effort where risk and value are highest, and staying adaptive means the plan improves as more is learned.

Weaker approaches people try — and why they fall short

• Follow a fixed schedule by directory, with equal effort per directory regardless of complexity or importance.Ignores that some code is far riskier and more central than others.
• Start alphabetically at the first file and discover related files organically.Alphabetical order is arbitrary and unrelated to impact.
• Read all 200 files to inventory every function before writing any tests.A huge upfront cost that delays any actual value.

Situation

Yesterday's analysis surfaced two refactor approaches — extracting a microservice, versus refactoring in place. Today, concrete code changes are wanted for both, before deciding which to pursue.

Recommended approach

Fork the session from yesterday's analysis, exploring one approach in each branch.

Why it works

Both explorations should start from the same accumulated analysis but proceed independently. Forking branches the existing context into two, so each fork inherits yesterday's findings yet develops its own approach without the two ideas bleeding into each other.

Weaker approaches people try — and why they fall short

• Resume for approach one, and manually recreate the context in a brand-new session for approach two.Manually recreating context is error-prone and loses fidelity.
• Resume and explore both approaches sequentially in the same thread.Lets the two approaches bleed into each other and bloats the context unnecessarily.
• Start two fresh sessions and manually summarize yesterday's findings into each.Hand-written summaries lose detail and duplicate effort.

Situation

An agent has already analyzed a 23-file service, understanding its request flows and error patterns. Now two different testing strategies — end-to-end with mocks, versus snapshot tests — need to be developed independently for comparison.

Recommended approach

Resume with a session fork, giving each testing strategy its own branch.

Why it works

Both strategies should build on the same 23-file analysis but stay independent. Forking gives each strategy its own branch seeded with the full prior context, with no re-reading and no interference between them.

Weaker approaches people try — and why they fall short

• Export the key findings to a file, then create two new sessions that reference it.Exporting and re-seeding loses the rich in-context understanding and adds friction.
• Continue in the original session, doing one strategy fully before starting the other.Sequential work in one thread entangles the two strategies and grows context unnecessarily.
• Start two fresh sessions that each re-read all 23 source files.Re-reading 23 files twice is wasteful and starts from zero understanding both times.

Situation

Investigating untested paths across a 45-file payment module. After about 8 files, accuracy starts dropping — the agent forgets earlier patterns and hasn't yet found all the test files or traced the critical flows.

Recommended approach

Spawn subagents for specific narrow questions (find all the test files, trace the refund dependencies) while the main agent coordinates and keeps the high-level picture.

Why it works

Context is overflowing. Each subagent gets a fresh window to answer one narrow question and returns a concise result, so detailed exploration happens in disposable contexts while the coordinating agent preserves the overall understanding.

Weaker approaches people try — and why they fall short

• Summarize findings, clear the context, and use the summary report as the sole reference going forward.A single summary as the sole reference loses detail that's still needed and forces re-discovery.
• Clear the context, then selectively re-read critical files while writing findings to a scratchpad file.Helps, but is more manual and still reloads a lot of detail back into the main context.
• Switch to keyword search only, to reduce how much content gets loaded.Reduces tokens but sacrifices the deep understanding that tracing flows actually requires.

Situation

An exploration subagent spent 30 minutes across 47 files documenting data flows, then got interrupted. Meanwhile, a merged pull request renamed two utility functions. The exploration needs to continue.

Recommended approach

Resume the subagent from its transcript, and explicitly inform it about the two renamed functions.

Why it works

Resuming preserves the full 47 files of accumulated context. The only actual change is two renamed functions, so resuming and naming those two renames keeps everything else intact while correcting the one factual drift precisely.

Weaker approaches people try — and why they fall short

• Start a fresh subagent with only a summary of the prior findings.A summary loses most of the detailed data-flow context that was already built up.
• Start a fresh subagent with the prior transcript pasted into the initial prompt.Re-feeding the whole transcript into a fresh agent is redundant when resuming does the same thing more directly.
• Resume without mentioning the renames, since the overall architecture understanding still holds.Leaves the agent reasoning with outdated function names.

Situation

A code-review assistant checks three fixed aspects for every pull request — style, security, and docs — each requiring reads, analysis, and its own report section. The workflow never changes from PR to PR.

Recommended approach

Use prompt chaining: analyze style, then security, then docs as sequential steps, then synthesize a final report.

Why it works

The workflow is fixed and always the same three steps. Prompt chaining — a predetermined sequence where each step produces a focused output before a final synthesis — matches a known, repeatable pipeline and keeps each step reliable and easy to inspect.

Weaker approaches people try — and why they fall short

• Use an orchestrator-workers pattern that decides which checks are needed dynamically.Orchestrator-workers is built for dynamic task sets; here the checks are always the same three, so dynamic routing adds unneeded complexity.
• Route to different prompts based on PR type (feature, bugfix, refactor).Routing by PR type isn't needed — all three checks run for every PR regardless of type.
• Use one comprehensive prompt that handles all three checks at once.One giant prompt tends to do each check less thoroughly and is harder to debug.

Situation

When source documents lack certain specs, the model invents plausible values to satisfy required schema fields — for example, fabricating a specific weight when the source only lists dimensions.

Recommended approach

Change the fields that may legitimately be absent from required to optional, letting the model omit them.

Why it works

The model fabricates because the schema forces a value — a required field must be filled, so it makes one up. Making those fields optional removes that pressure: the model can legitimately omit what isn't in the source. This fixes the incentive at its root rather than catching the fabrication afterward.

Weaker approaches people try — and why they fall short

• Add a self-reported confidence field and filter out low-confidence values afterward.Self-reported confidence is unreliable and doesn't stop the model from inventing the value in the first place.
• Instruct the model to use placeholder text for missing values.Placeholder text still writes something into a required field and can itself pollute downstream data.
• Add semantic validation that checks each value against the source afterward.Catches errors after the fact but doesn't remove the incentive to fabricate in the first place.

Situation

Extracting resume data (name, contact info, skills, experience, education) that must strictly conform to a JSON schema, where downstream validation currently fails on missing or mistyped fields.

Recommended approach

Define a tool whose input schema is the target JSON structure, and read the resulting tool-use input directly.

Why it works

Defining a tool whose input schema is the target structure makes the model emit data that conforms to that schema by construction — types and required fields are enforced by the tool-use mechanism itself, making this the most reliable path to schema conformance.

Weaker approaches people try — and why they fall short

• Make two calls: extract as free text first, then reformat that text as JSON.Two calls add cost and a second failure point, and the reformatting step can still drift from the schema.
• Give detailed JSON instructions and a template in the system prompt, and ask for JSON only.Prompt instructions are advisory — the model can still produce malformed or non-conformant JSON.
• Regex-parse the free-text response, with retry logic on failure.Regex parsing of free text is brittle, and retry loops paper over a problem that's avoidable in the first place.

Situation

After three months of full human review, extractions with 90%+ confidence show 97% accuracy overall. Before automating those high-confidence cases, what validation matters most?

Recommended approach

Break the accuracy number down by document type and field to confirm it holds consistently across every segment, not just in aggregate.

Why it works

An aggregate 97% can hide pockets of poor performance — some document types or fields might sit at 80% while others hit 99%. Confirming accuracy holds across every segment prevents auto-approving exactly the slices where the model is actually weak.

Weaker approaches people try — and why they fall short

• Verify that 97% meets all downstream requirements.Assumes the 97% is uniformly valid across segments — which is exactly what hasn't been checked yet.
• Compare accuracy at several different confidence thresholds to find an optimal cutoff.Threshold tuning optimizes a single number that could still mask segment-level failures.
• Run a two-week pilot routing a quarter of high-confidence cases downstream and monitor for errors.Useful, but launching it before checking segment consistency risks pushing an already-weak slice straight to production.

Situation

Event-metadata extraction uses an all-nullable schema, but the model still invents plausible values for fields not actually present in the source article — like a specific attendee count with no attendance information given.

Recommended approach

Add an explicit prompt instruction: return null for any field not directly stated in the source.

Why it works

The schema already allows null, so the fix is simply telling the model to use it. This directly authorizes omission and is the simplest effective correction when the schema is already nullable but the model still isn't using that option.

Weaker approaches people try — and why they fall short

• Make all fields required with strict validation.The opposite of the fix — it forces the model to invent values instead of allowing omission.
• Add a second model call to verify each value actually exists in the source.Adds cost and latency to patch behavior that a simple instruction can prevent.
• Upgrade to a more capable model tier.A bigger model may hallucinate less, but it doesn't address the missing instruction, and it's more expensive without fixing the root cause.

Situation

A contract's original terms say '30-day payment,' but an amendment says '45 days.' The model inconsistently returns one value or the other, with no indication of which one currently applies.

Recommended approach

Redesign the schema so amended fields capture multiple values, each with its source location and effective date.

Why it works

The real information here is that the value changed over time. A schema storing multiple values with source location and effective date captures that reality faithfully, letting downstream consumers see both the original and amended terms and know which applies when.

Weaker approaches people try — and why they fall short

• Add post-extraction pattern matching to detect amendments and flag them for manual review.Flagging for manual review scales poorly and still discards the structured history.
• Preprocess documents to classify and remove superseded sections before extraction.Deleting superseded sections destroys information that may still be legally required to retain.
• Instruct the model to always extract the most recent amendment and ignore superseded terms.'Take the latest' loses the original term and can be wrong when amendments are conditional or partial.

Situation

Recipe extraction hits informal measurements like 'a handful' or 'a splash' — the model either invents a specific amount or leaves the field empty, accounting for about a quarter of all corrections needed.

Recommended approach

Add few-shot examples showing informal measurements extracted verbatim — kept as-is, not converted to a number or dropped.

Why it works

The model doesn't inherently know the desired convention for this recurring case. Few-shot examples that demonstrate the exact behavior wanted — keep 'a handful' as written rather than inventing a number or dropping it — teach the convention cheaply and directly.

Weaker approaches people try — and why they fall short

• Add a measurement-type field (precise vs. informal) to the schema.Labels the type but doesn't actually tell the model to preserve the phrase verbatim.
• Add post-processing pattern matching to detect informal phrases and fill in empty values.Pattern matching is brittle across the endless variety of informal phrasing.
• Fine-tune a model on hundreds of corrected extractions.Heavy, slow, and overkill for a fix that a few well-chosen examples solve directly.

Situation

About 12% of extractions fail schema validation — for example, a quantity field expecting a number instead receives a range like '2 to 3.' Retrying without any change just reproduces the same failure.

Recommended approach

Send a follow-up that includes the specific validation error, asking the model to correct its output based on that error.

Why it works

Retrying without new information yields the same result. Feeding the specific validation error back gives the model the missing context to fix its own output — it now knows exactly what to correct, and this resolves most format failures within a couple of attempts.

Weaker approaches people try — and why they fall short

• Set the temperature to 0 for more consistency.Makes output more deterministic, but doesn't teach the model that a range isn't a valid number.
• Reprocess the failures with a larger, more capable model tier.More expensive, and may still stumble on ambiguous source values without any feedback.
• Pre-process documents to standardize every possible messy format in advance.A huge, brittle undertaking compared to a simple feedback-based retry.

Situation

A skills list field shows three separate problems at once: compound phrases like 'Python and SQL' get split inconsistently, implied skills sneak in that were never actually stated, and the number of extracted items varies wildly between documents.

Recommended approach

Add few-shot examples that demonstrate compound-phrase handling, an explicit-mention-only criterion, and the right level of granularity, all at once.

Why it works

All three problems are really judgment calls about how to extract, not separate bugs. A well-chosen set of few-shot examples can demonstrate how to split compounds, that only explicitly-mentioned skills count, and the right granularity simultaneously — far more precisely than a short instruction could.

Weaker approaches people try — and why they fall short

• Enrich the schema with confidence and source-quote metadata fields.Metadata fields don't fix the underlying inconsistency in what actually gets extracted.
• Add hard constraints like a fixed count range and one skill per entry.Hard numeric caps are arbitrary and can truncate legitimately long, valid lists.
• Do post-extraction normalization against a canonical taxonomy with deduplication.Helps with deduplication but doesn't fix compound-splitting or implied-skill inclusion at the source.

Situation

Retry-with-error-feedback is fixing most extraction failures within two or three attempts. Comparing a few failure types — a nested object where a flat array was expected, a comma in a number, a date in the wrong precision, and 'et al.' standing in for a full author list that only exists in an external document — which one will retrying help the least?

Recommended approach

The 'et al.' case, where the full author list simply isn't present anywhere in the input, is the one retries can't fix.

Why it works

The other cases are reformatting problems — the needed data is present, it just needs reshaping, which retries handle well. The author-list case is different: the information was never provided at all, and no amount of retrying can extract information that doesn't exist in the input.

Weaker approaches people try — and why they fall short

• Assume a nested object being flattened to a list is equally hard to fix.Flattening a nested object into an array is a straightforward, fixable reshape — retries handle this well.
• Assume a comma-formatted number is equally hard to fix.Stripping a comma to get a plain integer is trivial reformatting — retries handle this well.
• Assume a full datetime being truncated to a date is equally hard to fix.Truncating a datetime down to a date is straightforward reformatting — retries handle this well.

Situation

A metadata-extraction tool must run before two enrichment tools that depend on its output (a document ID). For a combined request ('extract the metadata and tell me how old it is'), the agent sometimes calls an enrichment tool first, which then fails for lack of the ID.

Recommended approach

Force tool_choice to the specific metadata-extraction tool for the first turn, then let the enrichment tools run in later turns once that data exists.

Why it works

A guaranteed ordering is needed: metadata first, enrichment after. Forcing tool_choice to name that specific tool guarantees the first step produces the needed ID, and enrichment happens naturally afterward — enforcing the dependency deterministically rather than hoping the model sequences correctly.

Weaker approaches people try — and why they fall short

• Set tool_choice to 'any' and add a system-prompt note prioritizing the metadata tool.'any' forces some tool to be called, but not necessarily the right one — an enrichment tool could still be picked first.
• Leave tool_choice on 'auto' and simply list the metadata tool first in the tools array.The order tools appear in the array doesn't reliably determine which one gets selected — this is a common misconception.
• Force the metadata tool on every single call in the pipeline.Would block the enrichment tools from ever being allowed to run.

Situation

For invoices, about 18% of extractions show summed line items that don't match the extracted grand total — sometimes from OCR errors, sometimes from extraction mistakes — and downstream systems reject any mismatched total.

Recommended approach

Add a calculated-total field (the model's own sum of the line items) alongside the stated total, and flag the record for human review whenever they differ.

Why it works

Capturing both the computed sum and the stated total turns a hidden inconsistency into a visible, checkable signal. It never silently alters financial data, and it surfaces exactly the ambiguous cases — whether caused by OCR or extraction — for a person to actually resolve.

Weaker approaches people try — and why they fall short

• Add few-shot examples of correctly-summing invoices.May reduce model errors, but does nothing about mismatches actually caused by OCR misreads.
• Add post-processing that proportionally adjusts line items to match the stated total.Auto-adjusting financial figures fabricates numbers and can hide real errors — dangerous in an accounting context.
• Extract independently, then use a separate model call to reconcile which value is correct.A reconciliation model guessing the 'right' value can silently pick wrong — money discrepancies should be adjudicated by a person.

Situation

Extractions at 85%+ confidence are routed away from human review, yet about 12% of those still contain errors — often around comparison tables, appendices, or ambiguous phrasing. A sustainable way is needed both to catch these and to measure improvement over time.

Recommended approach

Use stratified random sampling: review a fixed percentage of high-confidence extractions every week, on an ongoing basis.

Why it works

Both ongoing detection and a measurable error rate over time are needed. Stratified random sampling of high-confidence outputs gives an unbiased, repeatable error-rate estimate, surfaces recurring error patterns, and lets you tell whether changes actually reduce errors — a sustainable audit loop, not a one-off patch.

Weaker approaches people try — and why they fall short

• Add heuristic rules flagging documents with comparison tables or appendices, regardless of confidence.Heuristics only catch the error types already anticipated, and don't measure the overall error rate.
• Add a verification pass that re-extracts each high-confidence document and flags disagreements between the two attempts.Doubles extraction cost and only catches disagreements, missing cases where the model is confidently wrong both times.
• Lower the confidence threshold and review a larger volume.Floods reviewers and defeats the purpose of automation, without actually measuring anything.

Situation

Extracting menu items to JSON (item, description, price, dietary tags), where source menus vary wildly — prices written as '$12' or '$12.00', dietary info shown as icons in one menu and plain text in another.

Recommended approach

Define a strict output schema, and include explicit format-normalization rules directly in the prompt (e.g. always output price as a number with two decimals, map icons to standard tag names).

Why it works

Combining a strict schema (which guarantees structure and types) with normalization rules in the prompt gets consistent, normalized output in a single pass — the model handles the messiness of the input while the schema enforces the shape of the output.

Weaker approaches people try — and why they fall short

• Make multiple extraction attempts per document and pick the most common format.Costly, and voting across attempts may still agree on a wrongly-formatted value.
• Extract as-is and normalize everything in post-processing code.Can work, but pushes normalization logic into brittle code and misses the schema's built-in guarantees.
• Make a separate extraction call per field.Multiplies cost and latency without actually improving normalization.

Situation

Extracting calendar-invite data (title, date, time, location, attendees) into strict JSON, where the downstream system rejects any malformed or non-conformant output.

Recommended approach

Define a tool with the target schema as its input parameters, and have the model call it with the extracted data.

Why it works

Just as with resume extraction earlier, a tool whose input parameters are the schema makes the model produce data that conforms to it by construction — types and required fields are enforced by the tool-use mechanism, making this the most reliable path when the downstream system rejects any non-conformance at all.

Weaker approaches people try — and why they fall short

• Write detailed JSON instructions and a schema description in the prompt, then parse the text response.Prompt-only JSON guidance is advisory and can still emit malformed output.
• Pre-fill an opening brace to nudge the model toward JSON, then complete and parse it.A prefill nudges the format but doesn't enforce the schema's specific fields or types.
• Instruct 'output only valid JSON matching the schema' and retry on parse failure.Instruction-plus-retry treats symptoms and is less reliable than schema-enforced tool use.

Situation

Reviewers only have capacity to handle about 5% of total extraction volume. What should decide which extractions they actually see?

Recommended approach

Route the extractions where the model itself indicates low confidence, or where the source document is ambiguous or contradictory.

Why it works

With scarce review capacity, it should be spent where errors are most likely — low-confidence outputs and documents with ambiguous or contradictory content. This targets the slice most likely to actually be wrong, maximizing the value of limited human attention.

Weaker approaches people try — and why they fall short

• Randomly sample 5% of all extractions for review.Wastes most reviews on extractions that were already correct.
• Route by entity type (like financial figures or dates), regardless of confidence.Ignores confidence entirely — reviewing many correct high-confidence financial fields while missing risky non-financial ones.
• Only route extractions after downstream systems report a problem.Reacting only after failures means the error has already caused downstream damage.

Situation

A support agent receives uniform errors from its tools — just 'isError: true, Operation failed' — so it can't tell a temporary glitch from a validation problem from a permissions issue, leading to over-retrying, escalating too soon, or asking the wrong clarifying question.

Recommended approach

Enhance error responses with structured metadata: an error category (transient, validation, or permission), a retryable boolean flag, and a short cause description.

Why it works

The agent behaves inconsistently because the errors are opaque. Structured metadata gives the agent exactly what it needs to choose the right action — retry transient errors, don't retry validation or permission errors, and escalate appropriately. Fixing the information fixes the behavior.

Weaker approaches people try — and why they fall short

• Add a separate error-analysis tool the agent calls after any failure.An extra round-trip to derive information the failing tool could simply return directly.
• Add few-shot examples showing how to interpret error message patterns.Parsing free-text error patterns is brittle and still guesses at categories the tool already knows internally.
• Retry every error with exponential backoff at the server level.Wastes time retrying permanent errors, like a non-existent order, that will never succeed no matter how many times they're retried.

Situation

On complex disputes needing many tool calls, an agent sometimes hits its turn limit after gathering data but before resolving the issue or escalating it. The goal is to guarantee every interaction ends in either resolution or a human handoff, no matter how the loop ends.

Recommended approach

Add orchestration-layer code that checks the outcome after the loop terminates; if neither resolution nor escalation happened, it programmatically calls the escalation tool with the accumulated context.

Why it works

A true guarantee can't depend on model judgment or hitting a specific threshold, because the loop can terminate for many different reasons. Wrapping the agent in code that inspects the final state and force-escalates whenever the outcome isn't 'resolved or escalated' catches every termination path — guarantees belong in the code around the agent, not inside its reasoning.

Weaker approaches people try — and why they fall short

• Add a system-prompt instruction telling the agent to escalate when it can't finish in its remaining actions.The agent may still run out of turns before it acts on that instruction — no real guarantee.
• Add a pre-tool-use hook that auto-escalates once 80% of the turn limit is used.An 80%-threshold hook helps in one specific case but doesn't cover terminations from other causes, like errors or unexpected stops.
• Split the work into two sequential agents — one to gather data, one to act — each with its own turn budget.Both agents still have their own budgets and can each terminate mid-way without a guaranteed final handoff.

Situation

As a tool set grew from 4 tools to 10, tool-selection accuracy dropped noticeably. The errors cluster around semantically overlapping tools — one tool for issuing a credit and a separate one for processing a refund, and a delivery-status tool that duplicates data another lookup tool already returns.

Recommended approach

Merge the overlapping tools: combine the credit and refund tools into one tool with an action parameter, and fold the delivery-status tool into the existing lookup tool behind an include-tracking flag.

Why it works

Merging semantically overlapping tools into one tool with a parameter (and folding a redundant tool into an existing one) removes the ambiguous choice entirely — there's no longer a 'wrong' overlapping tool to pick, because there's only one right tool for the job. Removing the fork beats teaching the agent to navigate it.

Weaker approaches people try — and why they fall short

• Split the tools across two sub-agents with a coordinator.Relocates the ambiguity behind a coordinator without removing the overlapping tool definitions themselves.
• Enable deferred/lazy loading for the newer tools.Manages how many tools are visible at once, but the overlapping semantics remain once they're loaded.
• Add few-shot examples for each ambiguous tool pair.Helps the agent choose more often, but doesn't structurally eliminate the overlap itself.

Situation

During a multi-step identity verification, after the customer answers the third question, the agent asks for their name again — as if the earlier exchange never happened.

Recommended approach

Check whether the full conversation history is actually being included in each subsequent API request.

Why it works

The API is stateless — the model only 'remembers' what gets resent to it. If the agent appears to forget earlier answers, the near-certain cause is that the prior messages simply aren't being included in the next request. The full history has to be resent every turn; there's no server-side memory to fall back on.

Weaker approaches people try — and why they fall short

• Assume the verification tool is clearing its own internal state after each step.A tool doesn't wipe the model's context — the context is simply not being resent.
• Assume there's a default 'remember the last two turns only' setting that needs to be extended.There's no such default — memory is whatever history you choose to include, nothing more.
• Add prompt instructions telling the agent to remember across exchanges.Instructions to 'remember' can't help if the information isn't actually present in the request being sent.

Situation

A warranty-claim workflow needs a sequence of calls — look up the customer, then the order, then either process a refund or escalate. What should primarily decide whether the loop keeps going or stops?

Recommended approach

Check the response's stop_reason after every call: keep looping while it's 'tool_use', and exit once it becomes 'end_turn' or another terminal value.

Why it works

The agentic loop is governed by the API's stop_reason field. While it's 'tool_use', the model wants to call a tool, so the loop executes it, appends the result, and continues. Once it's 'end_turn', the model considers itself done — that field is the intended control signal for the loop, not a workaround.

Weaker approaches people try — and why they fall short

• Manually set tool_choice to 'none' once the last expected call has happened.A manual override, not a way to actually detect that the model considers itself finished.
• Exit the loop as soon as the response contains any text block.Text can appear alongside a tool-use request, so its mere presence isn't a reliable stop signal.
• Count tool calls and stop once a preconfigured maximum is reached.A max-call counter is a safety cap, not the primary mechanism for deciding when to continue or stop.

Situation

Mid-dispute, an agent discovers a promotional-pricing error that needs manager approval — beyond what it's authorized to resolve on its own.

Recommended approach

Compile a structured handoff — customer details, order information, and the specific issue identified — before calling the escalation tool.

Why it works

A good escalation hands the human a concise, structured package of exactly what matters: who the customer is, what the order was, and what the specific issue is. That lets the human act immediately without having to re-investigate from scratch, which is the whole point of escalating with context attached.

Weaker approaches people try — and why they fall short

• Attempt the refund anyway, and only escalate if the system rejects it.Attempting an unauthorized refund risks a real policy or compliance violation.
• Escalate with only the customer's original message attached.Forces the human to redo all the diagnosis the agent had already done.
• Persist the full conversation and tool history to a database, then escalate with just a reference ID.Burying the key facts behind a reference ID adds friction; a focused summary is more immediately useful (a full log can still complement it, but isn't the core need).

Situation

A refund tool returns both transient technical errors (like a timeout, about 5% of calls) and permanent business errors (like 'exceeds the 30-day window', about 12% of calls) — both as plain text, so the agent wastes several turns retrying the permanent business errors that will never succeed.

Recommended approach

Return structured errors with an explicit is_retryable:false flag for business errors, plus a customer-friendly explanation the agent can use directly in its reply.

Why it works

This solves two problems at once: the explicit retryable flag stops the agent wasting turns on permanent business errors, and the customer-friendly explanation improves what the customer actually sees — marking retryability explicitly rather than making the model guess at it.

Weaker approaches people try — and why they fall short

• Auto-retry technical errors at the tool level, and pass business errors through untouched.Helps reliability for the technical errors, but leaves the customer-facing message quality completely unaddressed.
• Add a mandatory eligibility-check tool that must run before the refund tool.Adds an extra round-trip, and still needs structured results itself to actually be useful.
• Add few-shot examples teaching the model to distinguish retryable from non-retryable errors by parsing the text.Parsing error text is brittle, and doesn't improve the customer-facing explanation either.

Situation

A compliance rule requires that refunds over $500 always auto-escalate to a human rather than being left to model discretion. Despite clear prompt instructions, about 3% of high-value refunds are still being processed directly.

Recommended approach

Add a hook that intercepts the refund tool call: if the amount exceeds $500, block it and invoke human escalation instead.

Why it works

A hard compliance rule needs a deterministic gate. A hook intercepting the tool call and blocking any refund over the threshold enforces the rule 100% of the time, independent of the model's judgment — compliance guarantees belong in code, not in prompt wording.

Weaker approaches people try — and why they fall short

• Add few-shot examples showing escalation at amounts just below, at, and above the threshold.Examples reduce the failure rate but never eliminate it entirely — still probabilistic.
• Use even stronger, more emphatic system-prompt language.The same probabilistic approach that already failed to reach 100%, just phrased more forcefully.
• Modify the refund tool itself to return an error above the threshold, asking the agent to escalate.Closer, but the agent could still choose not to call the tool at all or mishandle the resulting error — a hook intercepts the action itself for a firmer guarantee.

Situation

A customer returns 4 hours later to the same dispute. The old 32-turn session still has a stale 'pending' status recorded from a tool call; on resume, the agent keeps citing that outdated status even after fresh tool calls return updated information.

Recommended approach

Resume with the full history, plus a system instruction to always prefer the most recent tool result whenever the same tool has been called more than once.

Why it works

The goal is to keep conversation continuity while making sure current data wins. Resuming with full history plus an explicit instruction to trust the newest duplicate tool result resolves the staleness directly while preserving everything else — targeting the exact failure (old data vs. new) without discarding useful context.

Weaker approaches people try — and why they fall short

• Resume, but filter out all previous tool-result messages so the agent is forced to re-fetch everything.Loses useful context and forces re-fetching everything, not just the specific stale bits.
• Start a brand-new session and inject a structured summary of the issue, actions, and status before making fresh tool calls.Discards a rich 32-turn history unnecessarily when the fix could be much smaller.
• Resume and automatically re-call every previously-used tool at the start of the session.Wasteful, and may re-run tools that aren't even relevant to the current question.

Situation

An order-lookup tool returns 'purchased 45 days ago.' How does the agentic loop actually decide whether to process a refund or escalate to a human next?

Recommended approach

The order details get added to the conversation, and the model reasons over them to choose which action makes sense.

Why it works

In an agentic loop, tool results are appended to the conversation and the model reasons over them to pick the next action. Seeing '45 days ago,' the model itself weighs whether policy allows a refund or requires escalation — flexible reasoning over results, not a hard-coded router, is the defining behavior of this kind of loop.

Weaker approaches people try — and why they fall short

• Use a pre-configured decision tree that maps order attributes directly to specific tool calls.A fixed decision tree removes exactly the reasoning that agentic loops are built to provide.
• Have the agent execute a tool sequence that was fully planned out at the very start.Pre-planning the whole sequence upfront contradicts the react-to-results nature of the loop.
• Have the orchestration layer route based only on the order's status field.Hard-coding routing on a single field bypasses the model entirely and is brittle to any nuance.

Situation

For a request like 'refund for my recent purchase,' the agent calls the refund tool immediately with a fabricated order ID instead of first looking the order up — and the refund fails on the fake ID.

Recommended approach

Update the refund tool's description to state explicitly that its order ID parameter must come from a prior lookup call, and must never be assumed or invented.

Why it works

The root cause is that the tool's description never told the model where that ID actually has to come from, so it invented one. Stating the dependency directly in the description instructs the model to call the lookup tool first — fixing the tool's contract fixes the behavior at its source.

Weaker approaches people try — and why they fall short

• Pre-parse incoming messages for order IDs and inject any found into context.Only helps when an ID happens to be present in the message — here there isn't one to extract.
• Switch tool_choice from 'auto' to 'any' so some tool call is always forced.Just forces some tool call — it wouldn't stop the model from fabricating the parameter itself.
• Add server-side validation that the order ID exists, returning an error if it doesn't.Catches the bad ID after the fact, but doesn't stop the agent from fabricating and failing repeatedly.

Situation

An order-lookup tool catches every exception and returns the same flat message: 'Tool execution failed.' The agent either retries identically until it hits the turn limit, or escalates immediately — neither of which fits every situation.

Recommended approach

Return error-type-specific messages, such as 'Order not found — try looking up by customer or searching by phone' versus 'Database query timeout (transient) — retry should succeed.'

Why it works

The documented recommendation is to write instructive error messages that say what went wrong and what to try next. Type-specific messages — a 'not found' error suggesting an alternative path, versus a 'transient timeout' error indicating a retry will likely work — give the model the exact cues needed to choose the right recovery.

Weaker approaches people try — and why they fall short

• Remove the error flag and return the message as normal content instead.Hides the fact that a failure actually occurred, which can confuse how the loop handles it.
• Add a loop step that intercepts errors, classifies them into categories, and appends a recommendation.Works, but isn't what the documentation actually prescribes, and moves judgment out of the message the model directly reads.
• Retry with exponential backoff inside the tool itself, surfacing only the final failure.Handles transient errors fine, but leaves an 'order not found' case with no actionable guidance for the model at all.

Situation

Deciding when an agent should escalate to a human. Which trigger most reliably catches the cases that genuinely need a person?

Recommended approach

Escalate when the customer explicitly requests a human, the issue requires a policy exception, or the agent genuinely can't make further progress.

Why it works

Genuine need for a human is contextual, so the criteria should capture the real underlying reasons: an explicit ask, a situation needing authority the agent lacks, or a real impasse. These judgment-based conditions map to actual 'needs a human' situations far better than rigid proxies.

Weaker approaches people try — and why they fall short

• Use a rules engine mapping issue types, customer segments, and products directly to escalation, removing model judgment entirely.A pure rules engine can't anticipate every situation and mishandles novel cases that genuinely need judgment.
• Trigger escalation from a sentiment-analysis frustration score threshold.Frustration isn't the same as needing a human — a calm customer can still need escalation, and an annoyed one may not.
• Escalate automatically after three consecutive failed tool calls.A fixed failure count is a crude proxy that both over-escalates and under-escalates depending on the actual situation.

Situation

Three separate issues have unfolded over 45 turns of a conversation — a refund, then a subscription question, then a payment update. At turn 48, with context nearly full, the customer asks 'what happened with my refund?'

Recommended approach

Extract and persist the key structured facts for each issue — order IDs, amounts, statuses — into a separate, durable context layer as the conversation goes.

Why it works

To keep all three issues answerable even as context fills up, the key structured facts per issue need to survive independently of the raw transcript. Those compact records let the agent answer 'what happened with my refund?' precisely, even after the original turns describing it have aged out of the window.

Weaker approaches people try — and why they fall short

• Rely on tools to re-fetch information on demand whenever an earlier issue gets referenced again.Only works if a tool can actually retrieve it and the right identifiers are still known — exactly what risks being lost.
• Use a sliding window that keeps only the most recent 30 turns.A 30-turn window would drop the refund issue (turns 1 through 15) entirely — the very thing being asked about.
• Summarize earlier turns into a narrative, keeping full history only for the currently active issue.A narrative summary is lossy and less reliable than structured fields for exact IDs, amounts, and statuses.