Last year I gave an agent read access to a warehouse, a schema catalogue, and a brief to explain movements in a set of business-critical numbers. It produced an invoice I had not anticipated. Accuracy was never the difficulty; the model was largely correct. What consumed the budget was interrogation. Before it could attempt anything useful it had to work out which table was authoritative, which definition applied, which of two systems to believe. Most of that I had already told it. The meter ran throughout.

Databricks described the same problem at industrial scale in June. Annualized revenue reached $6.9 billion, up more than 80% year on year from $5.4 billion the previous quarter. Ali Ghodsi named the cause without much hedging: “the agents are generating way more queries.” He declined to give a current gross margin, and said it would go lower. The last figure the company disclosed was in its Series J announcement of December 2024: non-GAAP subscription gross margins above 80%.

Snowflake has to report, and shows the same direction with audited numbers. Non-GAAP product gross margin fell from 76.3% in fiscal 2025 to 75.8% in fiscal 2026 and 75.0% this year. Three consecutive declines, in the business sitting closest to this workload.

Neither company is in trouble. Databricks has reported positive trailing free cash flow across successive quarters while growing at 80%, which describes a company spending margin on growth rather than one losing control of its costs. It is still the wrong direction for a business that makes money when you consume more. The arriving workload costs more to serve per dollar than the workload already there.

Data-driven AI is iteratively expensive

LLM APIs are stateless. The orchestration layer resubmits the system prompt, the tool schemas and the accumulated transcript on every call, so step twenty carries steps one through nineteen with it. Double the steps and you roughly quadruple the tokens.

Prefix caching does most of the work people assume it does. It reduces the price of the repeated portion by roughly ninety percent, so the curve bends more gently than the raw token count suggests. It does not flatten it. The volume still grows with the square of the step count, the cache expires on a short timer, and any variation in the prefix costs you the hit. The blended rate in the model below already assumes you are getting most of that discount.

This matters more for analytics than for most work, because analytical questions require a substantial number of steps before anything useful emerges. The closest public measurement is not even from analytics, which indicates how rarely anyone measures this. AgenticRAG ran agentic retrieval over enterprise knowledge bases and financial filings. It reported 52,300 tokens per query on one dataset and 114,800 on the other, at roughly 4.5 to 4.8 tool calls per query.

Divide those totals by the accumulation the loop creates and each step contributes something like 3,800 genuinely new tokens. That figure comes from documents, and warehouse steps are fatter than document steps. In my own runs, a single schema fetch against a wide fact table consumed several times that on its own. Read 3,800 as a floor and scale accordingly.

Cost modelling

That was one question. Put it at volume.

Model a finance analytics agent answering 2,000 questions a working day, half a million a year. On the establishing path that is roughly $133,000 a year in model tokens. On the inherited path it is roughly $47,500.

Nobody should be alarmed by $133,000. Half a million answered questions is more analytical throughput than a large team produces, and at that volume the agent is inexpensive against any comparison you care to run. The figure worth attention is the $85,500 between the two paths. That is 64% of the bill, and you cannot presently say which requests it belonged to.

Raise the per-step figure to something warehouse-shaped, say 15,000 tokens, and the same model puts that gap past a third of a million dollars a year on a single workflow.

There is a second bill underneath the first. Seven steps means seven round trips into the warehouse where a human asked once, so half a million questions becomes three and a half million queries. Concurrency slots, scan volume and autoscaling behaviour were all sized for the first number. This is the part that reaches the platform before it reaches you, which is what Ghodsi was describing. It is also why, at that same summit, Databricks put governed metric definitions, a business glossary and scoped domain objects in front of agents, while Snowflake’s $6 billion AWS commitment buys cheaper units to serve the same workload. One is lowering the number of steps and the other the price per step. Neither is spending its energy on a better model.

Checking costs more than asking

The step count is therefore the variable worth watching. Two properties inflate it in query work.

The first is that verification is expensive. Coding agents thrash as well, and for the same structural reason, but a test suite either passes or it does not, so the thrashing remains merely costly. Analytics has verification too: certified datasets, reconciliation against the close, a human who checks the board pack before it goes out. All of it takes days and people. Nothing in the loop can tell the agent it has arrived, and nothing immediately afterwards tells you the number was right.

The second is what the agent spends those steps doing. Before it can approach a question about why renewals moved it must determine which dataset is authoritative, which definition of renewal applies, whether the region hierarchy changed, which report finance trusts, and whether partner renewals are included.

Ask a ten-year veteran why the company uses one report rather than the other and the answer is usually a let-down. “That’s just how we do it.” I have heard that sentence in more meetings than I can count and I have said it myself. It carries an enormous amount of weight, because it almost invariably means the question was raised once, years ago, people argued, somebody made the call, and everybody then stopped asking. The organization had completed the thinking and had no need to repeat it.

That decision was expensive the first time. It has been free ever since. A new joiner absorbs it in an afternoon and then relies on it for years.

Agentic AI cannot absorb it. It derives the whole thing again on every request, and derivation is measured in steps.

Two questions locate any agent use case on the same curve. How many steps before it produces something, and how expensive is the check afterwards?

Query work has the worst combination available: a long path, and a check that costs days. When the loop has no affordable stopping signal it continues until something external intervenes, and that something is usually a person or a spend alert. This is a property of the work rather than a limitation of the model, which is why a more capable model does not resolve it.

Count what it already knew

I should be straight about where this argument originates, because it points suspiciously close to my own job. I work in data infrastructure, so telling you the answer involves definitions, ownership and metadata is telling you to invest in precisely what I spend my days on. Check it against your own experience rather than mine. If your last stalled AI project died on a model limitation, I am wrong. If it died in a room where nobody could agree what the number was supposed to be, I am not.

Nor is every extra step waste. Agents investigate more anomalies and more possible causes than anyone could previously justify, and a larger bill is the right outcome when it buys that. My claim is narrower. Some proportion of those steps recovers facts the organization settled years ago, nobody measures that proportion, and it is priced as new work. The figures here are modelled, not measured.

So measure yours. It takes an afternoon rather than a project, and needs no instrumentation you do not already have.

1. Pull the trace from a single agent run.
2. Mark every step where the agent established something a competent analyst on your team already knew. Which table is authoritative. What the metric means. Which system wins when two disagree.
3. Count the marks against the total.

That ratio is the share of your bill spent on rediscovery. The step count tells you where you sit on the curve, and because the curve bends, removing three steps is worth considerably more than three steps.

Very little of what those steps recover is new. It is the organization itself: its definitions, its ownership, the determinations of authority that were contested once and then filed away. That deliberation was financed years ago and has cost nothing since. Direct an agent at it and you resume purchasing your own institutional memory, per request, on a meter.

I have watched more than one meeting slide from “can the AI do this” to “hang on, what is our definition of that metric again,” and then go quiet. The model is not failing in that room. It is presenting an invoice for work the organization never finished.

If you trace one of these workflows and the ratio surprises you, I would like to hear about it. I am on LinkedIn.