Getting an AI Agent to Actually Work: A Step-by-Step Guide

6 min read · Last updated: October 25, 2026

Finding an AI agent takes minutes, but getting an AI agent to work takes testing. Demos run on clean inputs and short tasks. Real work has messy inputs and many steps, and each step can fail. To make an agent work, narrow its job, limit its tools, add checks between steps, and measure it with a small eval set.

Came here from Instagram, LinkedIn, X or YouTube? Here is the full source list you came for. Everything below is built from these, nothing else.

The engineering guidance

• Building effective agents, Anthropic, December 19, 2024
• Building an App With AI Is Easy. Building an Agent That Actually Works Is Hard, Toolpod, May 11, 2026

The market data

• Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027, Gartner press release, June 25, 2025

Key takeaways

• Gartner forecast on June 25, 2025 that over 40% of agentic AI projects will be canceled by the end of 2027.
• Gartner also said only about 130 of the thousands of vendors claiming to sell agentic AI are real. The rest relabel chatbots and RPA.
• Errors compound. A step that works 95% of the time gives you about 60% success over 10 steps (my arithmetic, not a benchmark).
• Anthropic's advice is to find "the simplest solution possible" and accept that this "might mean not building agentic systems at all."
• Anthropic says it spent more time optimizing tools than the overall prompt.
• Toolpod suggests starting with a 20 to 30 case eval set and logging from day one.

Why is picking an AI agent like hiring from a résumé?

A demo is a résumé: it shows the best day, on a task someone chose for you. Hiring from a résumé tells you what a person can do, not what they do on your messy Tuesday. An agent works the same way. The demo proves it can work once. Your job is to find out how often.

Imagine you find an agent that looks perfect. It books meetings, writes code or answers support tickets, and the video is flawless. You connect it to your real data. It reads an odd date format, calls the wrong tool, and carries on with total confidence.

Toolpod describes this exact failure. Agents "can proceed confidently in the wrong direction while producing reasonable-looking output," in the article's wording. The output looks fine. That is why it is dangerous.

What is an AI agent, and how does it work step by step?

An AI agent is a language model that decides its own next step. Anthropic defines agents as systems where "LLMs dynamically direct their own processes and tool usage." A workflow, by contrast, runs "through predefined code paths." The agent loops until it thinks it is done.

Here is the loop in plain steps:

1. Goal. You give it a task, such as "refund this customer."
2. Plan. The model decides the next action.
3. Tool call. It calls a tool: search, database, email, code runner.
4. Observe. It reads the tool's result.
5. Check. It decides whether the goal is met.
6. Repeat or stop. If not met, it goes back to step 2.

Toolpod describes the same loop of goals, steps, tool use and self-checks, and says this is where the difficulty lives. Every pass through the loop is a chance to be wrong. The model also grades its own work at step 5.

Why do AI agents fail after the demo?

Agents fail after the demo because mistakes stack up. Anthropic notes that the autonomy of agents means "higher costs, and the potential for compounding errors." Each tool you add is another place to fail, with wrong parameters, wrong order or a misread result, according to Toolpod.

Here is the compounding effect. These are my own calculations (success rate to the power of steps), not measured benchmarks:

Success per step	After 5 steps	After 10 steps
99%	95%	90%
95%	77%	60%
90%	59%	35%

A 95% step looks excellent in isolation. Over a 10-step task it fails four times in ten. That is the gap between a demo and a product.

The market agrees. Gartner's analyst Anushree Verma said most agentic AI projects today are early experiments, mostly driven by hype and often misapplied. Gartner says this can hide the real cost and complexity of running agents at scale. It also reports "agent washing", where vendors relabel assistants, RPA and chatbots as agents.

How do you get an AI agent to work?

To get an AI agent to work, shrink the problem until you can measure it. Narrow the task, cut the tools to the few that matter, validate every step, write a small eval set, and log everything. Toolpod names narrow scope as the most common fix for struggling agents.

Use this checklist:

1. Narrow the job. One task, one input type. "Triage refund emails" beats "handle support."
2. Cut the tools. Give it three tools, not thirty. Fewer tools means fewer wrong calls.
3. Design the tools for the model. Anthropic says to "put yourself in the model's shoes" and to "poka-yoke your tools", meaning make mistakes hard to commit. Clear names, clear parameters, clear errors.
4. Validate between steps. Check structure and sanity in code, not by asking the model if it did well.
5. Write 20 to 30 test cases. Toolpod suggests this size to start. Include the ugly inputs.
6. Log every step from day one. You cannot fix a loop you cannot see.

Anthropic also recommends "extensive testing in sandboxed environments, along with the appropriate guardrails." Do not point an untested agent at production data.

When should you use an agent, and when not?

Use an agent when the path cannot be written down in advance and you can afford errors and extra cost. Use a plain workflow when the steps are known. Anthropic says agentic systems "trade latency and cost for better task performance", and workflows offer "predictability and consistency for well-defined tasks."

Situation	Better choice
Steps are fixed and known (parse, validate, save)	Workflow in plain code
One model call with good context is enough	Single prompt
The path depends on what the model finds, at scale	Agent
A mistake is costly or irreversible	Workflow with human approval
You have no way to measure quality yet	Do not ship an agent yet

If you are hunting for a ready-made agent, ask the vendor one question: what is the success rate on my inputs, and how do you know? A good answer has a number. A bad one has a demo.

For real products to study, see our explainers on OpenClaw, the personal AI agent and Cindy, the AI agent.

What does a small working example look like?

A working agent is mostly the boring code around the model: a step cap, a validator and a log. This TypeScript sketch shows the shape. callModel and runTool stand in for whatever SDK you use, so treat it as a pattern, not a library.

type Step = { tool: string; args: Record<string, unknown> } | { done: string }

const MAX_STEPS = 8
const ALLOWED = new Set(['search_orders', 'issue_refund'])

async function runAgent(goal: string) {
  const log: unknown[] = []
  for (let i = 0; i < MAX_STEPS; i++) {
    const step: Step = await callModel(goal, log)
    log.push(step)
    console.log(JSON.stringify({ i, step }))

    if ('done' in step) return { ok: true, answer: step.done, log }
    if (!ALLOWED.has(step.tool)) return { ok: false, reason: 'bad tool', log }
    if (step.tool === 'issue_refund') return { ok: false, reason: 'needs human approval', log }

    log.push(await runTool(step.tool, step.args))
  }
  return { ok: false, reason: 'step limit', log }
}

Three habits do the work here. The step cap stops runaway loops. The allow-list rejects tools you never meant to expose. The refund tool stops for a human instead of acting alone. Then run your 20 to 30 cases through runAgent and count how many pass.

FAQ

Why do most AI agents fail in production?

Because errors compound across steps, tools add failure points, and demos use clean inputs. Anthropic warns of "compounding errors", and Toolpod says agents can proceed confidently in the wrong direction. Gartner forecasts over 40% of agentic AI projects will be canceled by end of 2027, citing cost, unclear value and weak risk controls.

What is the difference between an AI agent and a workflow?

A workflow follows code paths you wrote in advance. An agent chooses its own steps and tools as it goes. Anthropic recommends starting with the simplest option and adding autonomy only when needed, which may mean not building an agent at all.

How do you test an AI agent?

Build a small set of 20 to 30 realistic test cases, including messy ones, and run the agent against them after every change. Add tracing and logging from the start, and test in a sandbox before real data. Toolpod and Anthropic both stress evals and sandboxed testing.

What is agent washing?

Agent washing is when vendors relabel chatbots, assistants or RPA as agentic AI. Gartner said on June 25, 2025 that only about 130 of the thousands of agentic AI vendors are genuine. Check what the product actually decides on its own before you buy.

Before you adopt any agent, write down its ten ugliest real inputs and run them first. Then browse the free tools on Art of Code, and read AI is not ready: why we're being sold an unfinished product for the wider picture.

Written by Adithya Guttha, Founder of Art of Code.