Why agent projects get cancelled: it's the harness, not the model

The press release is dated 25 June 2025. The headline number is the 40%. The detail underneath is more useful. Gartner calls most current projects early experiments driven by hype. It describes "agent washing", where vendors rebrand assistants, chatbots and RPA as agents, and estimates that only about 130 of the thousands of agentic AI vendors are real. Its analyst also says many use cases sold as agentic don't need an agent at all.

Other numbers point the same way. Deloitte's 2026 State of AI in the Enterprise survey, with over 3,000 director to C-suite respondents, found that only 25% had moved 40% or more of their AI pilots into production. A year after Gartner's forecast, Robert Szczerba argued in Forbes that the failing projects die of management problems: no success metric, no access to the right data, no plan for when the agent goes wrong.

I agree with the diagnosis. But "manage it better" isn't something an engineering team can ship on Monday. So here's the same list, translated into things you can build.

One term first. By harness I mean the code and rules around the model: tool definitions, permissions, approval steps, budgets, evals, traces and the workflow engine that runs it all. The model writes the next step. The harness decides whether that step is allowed to happen.

Agent costs don't grow in a straight line. They grow in loops. The model calls a tool, the result goes back into the model, the model calls another tool. A tool error triggers a retry. A person reviews the output, and those minutes are a cost too. If your dashboard shows the price of one model call, you're looking at a small slice of the bill.

Two parts of the harness deal with this.

Budget caps. Every task gets a spending limit that covers all of its model calls, including retries. When a run hits the limit it stops and asks for help. A confused agent with no cap can keep looping until someone notices the invoice.

Tracing with a price on it. Every model call is logged with a task id, the tool involved, the tokens used and what it cost. Then you can add up one task from start to finish and see which step eats the money.

Here's a made-up example. A support triage agent reads a ticket, looks up the order history and drafts a reply. One day tickets start arriving without an order id, and the agent searches again and again. With a cap, the run tries a few lookups, stops, and marks the ticket "needs info". With traces, you see that lookup is the expensive step, and you fix the lookup. The model never changed.

The number I watch is cost per successful task: what you spent on a task type, divided by the runs that succeeded. A cheap run that fails half the time isn't cheap. I wrote that up separately in cost per successful task.

A demo proves the agent can write a good paragraph. Production needs something to be different afterwards: a ticket closed, one row written, an email waiting for approval. If "done" means "the text looked right", every run looks fine and nobody can say what the project is worth.

A checkable definition of done. Write down what must exist after the run, what must not exist, and what must stay untouched. For a reminder agent that might be: one draft linked to the right invoice, status draft, no second draft, nothing sent. Text quality gets graded only where a person reads the text.

Evals before every prompt change. Seed a known starting state, run the agent, check the state afterwards. Do it several times, because agents don't behave identically from run to run. What comes out is a pass rate per task. That number is what you take to the budget meeting, and it's the denominator in cost per successful task. The full recipe is in evals that check the end state.

Once value is a pass rate, the arguments get shorter. "The new prompt feels better" turns into "the new prompt took reminders from 84% to 93% and cost per success went down". Those two figures are invented, but that's the kind of sentence you want to be able to say.

An agent is risky in proportion to what it can reach. Give it one broad tool with open access and it can read the wrong customer's records or send a message it was only supposed to draft. If nobody recorded what it did, you can't even explain the incident afterwards.

Typed tools with permission scopes. Each tool does one thing, with defined inputs and outputs, and checks permissions before it runs. The scope names the records, mailboxes or repositories it may touch. The agent can't go around the tool, and every call is traced.

Human checkpoints for high-impact actions. Anything hard to undo or visible outside the company waits for a person. The agent prepares, a person approves, and the system sends only after that. Client email is the obvious case.

Explicit memory scopes. What an agent remembers for one customer or one task shouldn't leak into another. Decide where memory lives and who can read it, and write that down in code.

Durable execution. Run the work on a workflow engine that retries failed steps under a policy and parks what keeps failing in a dead-letter queue, so a person can look at it. Retries only help if repeating a step is safe, which means a retried send must not produce a second email.

A way back. Keep prompts and tool definitions in version control so you can revert a bad change in minutes. For the agent's writes, prefer drafts and status flags over hard deletes, so there's something to restore.

If a vendor can't show you these in a running system, you've probably met the agent washing Gartner describes.

Parker AI is a creative strategy platform for consumer brands, some of them spending up to $2M a month on Meta ads. What I can say about the stack is what's public on the project page, so I'll stick to that.

The agents are built on Mastra, with planning, memory, tool use and human-in-the-loop checkpoints. Ingestion and heavy jobs run as Temporal workflows with retries and dead-letter queues. Langfuse handles tracing and evals, and every prompt change ships with traces, cost and quality metrics. Tool calls are typed and traced. Agents have budget caps and explicit memory scopes. High-impact actions, like emails to clients, sit behind a human checkpoint. Model orchestration is cost-aware, with prompt budgets.

Now map that onto Gartner's list. Cost: budget caps, prompt budgets, and a cost figure attached to every prompt change. Value: evals and quality metrics on every change. Risk: typed tools, memory scopes, human checkpoints, retries and dead-letter queues.

None of this depends on those three products. If your team uses a different agent framework or a different queue, keep the functions and swap the names. What I'd resist is dropping a function because the framework you picked doesn't ship it.

You don't hand an agent full autonomy on day one. I'd move one workflow at a time through three stages.

Stage 1: suggest. The agent reads and proposes. It writes nothing and sends nothing. You need tracing and a small eval set. Move on when the pass rate on its suggestions holds steady.

Stage 2: act with approval. The agent prepares writes and drafts, and a person approves each one. Tools get narrow write scopes. Budget caps are on and evals run before every prompt change. Move on when cost per success is stable and the approvers almost never need to correct anything.

Stage 3: act alone, inside narrow limits. The agent acts without approval on low-impact steps only, with typed tools, caps and a tested way to revert. Anything a client sees still goes through a person.

If the pass rate or the cost per success slips, drop back a stage. Dropping back is what the stages are for.

Paste this into a coding agent such as Claude Code, opened in the repository that holds your agent. It only reads. I'd run it before the next budget review, not after.

Keep the table it produces. When finance or security asks what controls exist, that table and your pass rates are the answer. If you'd like a second pair of eyes on the result, this is the work I do under agentic platforms and cost and safety rails.

Goal: Audit our agent harness for cost, value and risk controls. Tell me what exists, what is missing, and what to fix first.

Inputs: This repository: agent configuration, tool definitions, prompts, eval sets, workflow and queue code, and any docs on tracing or approvals. Read only. Do not change, create or delete files. If you cannot see something (for example a tracing dashboard), say so and do not guess.

Constraints: Use only what you find in the repository. Do not invent numbers. Mark each finding as present, partial or missing, and give the file path as evidence. Call out any feature described as an agent that is a single prompt with no tools, no permission scopes and no evals.

Definition of done: One table with a row for each of these: typed tools, permission scopes, human checkpoints for high-impact actions, staged autonomy, budget caps per task, evals run before prompt changes, tracing with cost per task, retries and dead-letter handling, memory scopes, a way to revert prompt changes. Each row has a status, the evidence location, and the risk if it is missing. Below the table: the three fixes to do first, ordered by risk, and a list of open questions where you needed access you did not have.