YOUR AGENT'S REAL PRICE IS COST PER TASK.

WHAT HAPPENED

Gartner published a prediction on 17 August: inference cost per agentic workflow will rise more than fivefold through 2028, at the same time as the price per token keeps falling. It calls this the inference paradox. Senior director analyst Will Sommer's framing is that each new generation of capability needs more tokens, and often more expensive ones. The recommended fix is tiering, routing and orchestration. A day later Snowflake announced one piece of that: dynamic model routing in its Cortex AI Gateway, choosing a model per request against complexity and cost instead of sending everything to the largest one available.

THE UNIT EVERYONE IS BUDGETING IN IS WRONG

Per-token pricing is the metric vendors quote because it is the one that keeps improving. It is also nearly useless for planning, because it prices an input you do not control while hiding the variable you do.

A chatbot answers in one call. An agent reads a ticket, queries three systems, drafts, checks itself, retries a failed tool call, and writes back — twenty calls, some of them long. Halve the token price and triple the call count, and the bill still rises by half. That is what capability costs. The mistake is measuring it in the wrong denominator.

The number that survives contact with a CFO is cost per completed task: total inference spend for a period divided by the number of workflows that finished correctly without human rescue. It is the figure you can set against what the agent replaced: the loaded cost of a person doing the same task.

MEASURE COST PER TASK FROM THE FIRST PILOT

• Instrument at the workflow level. Tag every model call with a workflow ID from the first day of the pilot. Retrofitting this later is painful, so teams keep postponing it.
• Count failures in the cost and leave them out of the task count. A run that a person had to redo adds two costs and one completed task. Programs that exclude these runs report numbers that fall apart at scale.
• Watch the distribution as well as the mean. Agent costs are long-tailed. A handful of pathological runs (retry loops, oversized context, a tool that times out and gets called again) carry a disproportionate share of the bill, and they are usually the cheapest thing to fix.

Once the number exists, the fix is unglamorous. Most steps in a workflow do not need a frontier model. Classification, extraction, formatting, routing decisions and summaries of short inputs usually run well on a small fast model, especially when a schema check or a business rule validates the output. Keep the expensive model for the few steps that need reasoning over long context.

Platforms have started to build this in. Snowflake says its dynamic routing picks a model for each request on quality, speed and cost. If your stack sends every request to the largest available model instead, you pay that premium on every run, and it grows with your volume.

This is also where a written decision boundary pays for itself twice. A small model is fine where a mistake is cheap: the output is checked before it takes effect, by a validation rule or by the person approving a draft, or the step is low-impact and easy to reverse. A step that acts unattended on anything that matters needs more care, because nobody is there to catch the miss. Give it a stronger model, checks around it and a log someone reviews, or put a person back in the loop. The governance document and the cost model are the same document.

ASK VENDORS TO PRICE YOUR WORKLOAD

Three questions, in order. What is the cost per completed task on a workload like mine? Can I route steps to different models, or is the model fixed at the platform level? What happens to that number when volume goes up 10x — does anything about the architecture change, or does the bill simply multiply?

If you cannot state an agent's cost per completed task, what you have is a demo with a good week behind it.