Cost Estimation When Usage Is Uneven and Inputs Are Messy

The question

A cost estimate is almost always built from two numbers: a price per token, and an assumed token count per request. Multiply by volume, add a margin, present as a monthly figure.

The estimate is wrong whenever the assumed token count is a constant, because the token count per request is a distribution, and its shape is dominated by the task, not by the model. The price is published. The token count is the variable the estimate invents. When usage is uneven and inputs are messy, the invented constant is the error, and the error compounds in ways the price page never suggests.

This post is about how to estimate cost when the per-request token count cannot be assumed, the record that replaces the assumption, and the four amplifiers that make a reasonable-looking estimate diverge from the invoice.

The short answer

A cost estimate is defensible when it is built from measured usage metadata, not from an assumed per-request token count. The bill carries the counts. The estimate's job is to reproduce them, not to guess them.

The estimate is only as good as the usage shape. A shape built from a sample of clean inputs describes the sample. Production inputs are messy, and the messiness is what moves the bill.

The price is two prices, and the date decides which

The first estimation error is using the introductory rate for a period that includes the standard rate.

Argon launched at an introductory price of $2 per million input tokens and $10 per million output tokens, with cached input tokens at 95% off. On December 1, 2026, the standard rates apply: $4 per million input and $20 per million output. Cached input follows the same transition — $0.20 per million at standard rates, up from $0.10 introductory.

An estimate that spans November and December using one rate is wrong by a factor of two on every line. The correct treatment is a dated price table, with the estimate computed per billing period rather than per annualized average:

The price table is the easy part. It is the only part of the estimate that is published, and it is the part most estimates get right. The error is almost never in the rate. It is in the token count the rate is multiplied by.

Thinking is billed at the output rate, and it is the first amplifier

The second estimation error is treating output tokens as “the answer.”

On Argon, thinking tokens are billed at the output rate. The thinking phase produces no user-visible tokens, consumes output budget, and appears on the invoice in the same line as the visible answer. A cost model that counts the visible answer and stops there will undercount by the ratio of thinking to answer, which for a reasoning model is not small.

The ratio is not stable. It depends on thinking_level (low/medium/high, default medium) and on the task. The practical form of this observation: a classification step asked for one word can still bill thousands of thinking tokens. The visible output is one token. The billable output is the thinking plus the one token.

This is the amplifier that makes “per-token price” a misleading unit of comparison. Argon is cheaper per output token than some alternatives at introductory rates and more expensive at standard rates, but the more important variable is tokens per completed task. An observed comparison put Argon at roughly 62,000 output tokens per completed task against roughly 27,000 for a competing model — a ratio that inverts the cost ranking once the standard rate applies. The price per token and the cost per task are different measurements, and the estimate needs the second one.

The output ceiling is not a budget, and the per-request cap is not the model cap

The third estimation error is reading the headline output ceiling as the per-request budget.

Argon’s single-response output ceiling is 1,000,000 tokens, up from 64,000 in the previous generation. That is a capability statement, not a cost statement. The per-request ceiling that applies to a given call path is lower:

• 32,768 tokens per request for non-streaming direct HTTP.

• 65,536 tokens for streaming and synchronous Interactions calls.

• Long decode continuation extends beyond that toward the model ceiling.

The estimation consequence is that the cap your integration actually runs against determines the worst-case cost per request, not the headline. A request at a 65,536 streaming ceiling and a standard output rate of $20/1M has an output cost ceiling of about $1.31 per request before input. A request at a 2,048 max_output_tokens safeguard has a ceiling of about $0.04. Both are Argon requests. The estimate that uses one ceiling for all requests describes neither.

The 1M ceiling also interacts with the reasoning amplifier. A long decode that runs toward the ceiling is not a single visible answer; it is thinking plus output across continuation calls, each billed at the output rate. The cost of a request that “used the 1M ceiling” is measured in the tens of dollars at introductory rates and higher at standard rates, and the estimate needs to know whether such requests occur — not whether the ceiling exists.

The cache is a separate line, and the estimate usually omits it

The fourth estimation error is treating caching as a discount with no cost.

Cached input tokens are discounted heavily, but cache storage is billed separately: $0.50 per million tokens per hour** through December 31, 2026, rising to **$1.00 per million tokens per hour from January 1, 2027. For Argon, explicit context caching is not supported; the caching is implicit, applied to shared prompt prefixes of 6,144 tokens or more. The storage line still exists in the billing structure, and it is a function of how long prefixes remain resident, not of how many requests hit them.

An estimate that models cached input as “input price minus 95%” and omits storage will be close for a workload with short-lived prefixes and wrong for one that keeps large prefixes resident. The error is proportional to prefix size × residency time, which is exactly the quantity a per-request estimate does not see.

The estimation rule is the same as the pricing rule: the cache is two lines (read and storage), not one discount. Model both or state that storage is assumed negligible and record the assumption.

The methodology: read the shape from the bill, not from a sample

The defensible estimate does not assume a per-request token count. It reads the distribution from usage metadata and builds the estimate from the distribution.

The procedure:

1. Extract per-request usage from the responses. Each response carries input, output, and cached-token counts. These are the measured usage, not an estimate. Export them for a period that includes messy inputs, not a period selected for clean ones.

2. Segment by task type and configuration. A single distribution over all requests is a mixture, and the mixture’s mean describes no actual request. Segment by task (classification, extraction, generation, agentic step), by thinking_level, and by call surface. The segments are the unit of estimation.

3. Build the per-segment distribution, not the mean. Report p50, p90, p99, and the maximum for output tokens per segment. The mean is pulled by the tail; the tail is where the bill is. A segment whose p50 is 400 output tokens and whose p99 is 30,000 has a mean that describes neither.

4. Count the amplifiers as separate line items. Retried requests that were billed but discarded, truncated requests that consumed the budget without producing usable output, and cache storage hours. These are not adjustments to the per-request estimate; they are additive lines, and each has a count from the logs.

5. Apply the dated price table. Introductory rates for the introductory period, standard rates after. Cached input at its own rate. Output including thinking at the output rate.

6. State the estimation interval. The estimate is not a number. It is an interval derived from the per-segment distributions and the amplifier counts, and the interval width is the honest statement of how much the input messiness matters.

The step that makes the estimate checkable is step 1. The bill already contains the measured usage metadata. The estimate’s job is to reproduce the bill for a period, and then to project the next period from the measured shape plus a stated volume assumption. An estimate that cannot reproduce a past invoice is not an estimate; it is a guess with a spreadsheet.

The amplifiers, quantified

The four amplifiers are not equally large for every workload. This is the table I use to decide which ones need a measured count and which can be assumed negligible for a given task:

The reasoning amplifier is the one most often missing from estimates, because the visible output is the thing the developer sees. The retry amplifier is the one most often missing from budgets, because the success rate hides it. The truncation amplifier is the one that produces both a cost line and a quality line — the money is spent and the output is unusable. The cache amplifier is the one that is simply a different billing dimension.

None of these requires a model of the model. Each requires a count from the logs. The count is the estimate.

Failure cases observed

The single-rate annualization. An annual estimate computed at the introductory $2/$10 rates, presented in October as the year-one cost. Seven weeks of the year are at the introductory rate; the rest are at $4/$20. The estimate understated the annual figure by roughly 45%, and the error was entirely in the rate table, not the usage shape.

The visible-answer count. A cost model that counted output_chars / 4 as output tokens. The thinking tokens were invisible to the model and billed at the output rate. For a classification workload, the visible answer was a few tokens per request and the thinking was in the hundreds to low thousands. The estimate undercounted the output line by more than an order of magnitude.

The ceiling confusion. An estimate that used the 1M output ceiling as the worst-case per request. The integration’s actual path was non-streaming direct HTTP with a 32,768 cap. The worst-case estimate was 30× higher than the path could produce, which made the estimate look conservative and the actual cost look safe — both wrong, in opposite directions.

The omitted storage line. A workload with a 40K-token shared prefix held resident across a business day. The estimate modeled cached reads at the discounted input rate and omitted storage. At $0.50 per million tokens per hour, 40K tokens resident for 10 hours is about $0.20 per prefix per day. At 50 prefixes, that is about $10 per day, or roughly $300 per month, entirely absent from the estimate.

The harness-dependent cost. The same model, the same benchmark, two CLI harnesses: Antigravity CLI at a listed $7.16 and 56.5 minutes** versus **Gemini CLI at a listed $32.08 and 85.1 minutes. The difference was attributed to fewer tokens and more cache hits in the cheaper harness. The model price was identical. The usage shape was not, and the usage shape was a property of the harness.

The retry that nobody counted. A 99% reported success rate over a 6% first-attempt failure rate. The billing record showed the successful requests; the abandoned first attempts were in the invoice but not in the estimate. The cost per logical request was roughly 6% higher than the per-successful-request estimate, and the gap grew with the failure rate.

None of these are pricing failures. All of them are usage-shape failures, and each maps to one of the amplifiers.

For AI tool features, rules and availability, refer to their official sites.