The Token Meter: The Cost of Extra AI Output

I want to be careful about the word "designed," because it carries too much of the argument people share about coding agents. The strong claim says their makers deliberately tune them to waste tokens and increase the bill. I cannot prove intent. What I can examine is a narrower concern: token billing makes unnecessary output costly, some training methods reward length, and agent loops can carry that output into later requests. These mechanisms deserve scrutiny. They do not, by themselves, establish that vendors deliberately encourage waste.

Start with the bill, because its arithmetic is straightforward. These examples use standard API list prices in US dollars per million tokens, recorded for 25 June 2026. They compare uncached text input with output, excluding batch discounts, fast modes, tool charges, and long-context premiums. Gemini 3.1 Pro uses its tier for prompts of at most 200,000 tokens; GPT-5.4 uses its tier up to 272,000 input tokens. These are selected models, not a complete market survey.

The rates are documented in the announcements for Opus 4.8 (28 May 2026), Sonnet 4.6 (17 February 2026), and GPT-5.4 (5 March 2026). Google lists Gemini 3.1 Pro in its API pricing documentation; that live page was rechecked on 15 October 2026. In this table, output costs five to six times as much per token as uncached input. More output therefore increases a usage-based bill. Subscription allowances, cached input, and provider costs complicate what that means for the seller.

The output premium is not a measure of profit. Prompt processing can handle many tokens in parallel, while ordinary autoregressive generation produces successive tokens one at a time. Hardware use, batching, and caching affect the cost of each. List prices alone do not reveal a vendor's margin or prove that longer answers are more profitable. My concern is the customer's bill: unnecessary output costs money whether it was encouraged deliberately or simply produced by a poor workflow.

Training can add another pressure toward length. Singhal et al. (2023) found that response length explained much of the reward improvement in the reinforcement learning from human feedback setups they studied. A reward based only on length reproduced much of those measured gains. That is evidence of a failure mode in those experiments, not proof that every current coding model rewards verbosity. A longer answer can be better, and training or evaluation can also penalise needless length. We need evidence about a particular model before assigning it the same bias.

The third piece is the agent loop. A coding agent reads information, takes an action, receives a result, and calls the model again. Later requests may include earlier conversation and file contents. The harness decides what to retain, summarise, retrieve, or drop. Anthropic's August 2024 prompt-caching announcement describes savings of up to 90% on cached input. Caching reuses computation for a matching prompt prefix; it does not mean that all those tokens are recomputed from scratch on each call. Cache writes and reads also have their own billing rules.

Long output can still create a recurring cost when the agent includes it in later model requests. A generated file does not automatically enter every later prompt, and creating fourteen files does not by itself mean paying to reread all fourteen. The cost depends on what the harness actually sends and how much qualifies for caching. The practical question is whether the extra text helps later work enough to justify keeping it in context.

Let me make that concrete, with every assumption on the table so you can argue with the numbers. The following is a back-of-the-envelope illustration. Your real usage will differ, which is why I am showing the working. Take a one-hour agent session on a premium model at the Opus 4.8 rate of $25 per million output tokens and $5 per million input. Suppose the agent does its job in a reasonably tight way: it generates 40,000 tokens of code and explanation over the hour, and because of the re-read loop it processes 600,000 uncached input tokens across all its turns. This deliberately excludes caching, subscription allowances, and tool fees. That is 40,000 × $25/M plus 600,000 × $5/M, which is $1.00 of output and $3.00 of input, four dollars for the session.

Now suppose the session produces 50 percent more output, 60,000 tokens, and also processes 50 percent more uncached input, 900,000 tokens. The bill becomes 60,000 × $25/M plus 900,000 × $5/M: $1.50 of output and $4.50 of input, six dollars in total. That is a 50 percent increase over four dollars. Both increases are assumptions in this illustration; extra output does not necessarily cause a matching increase in input. This is billing arithmetic, not a measured estimate of how much agents waste.

There is evidence of quality problems, but the studies measure different things. GitClear's 2025 analysis covered 211 million changed lines from 2020 through 2024. It reported copy-pasted lines rising from 8.3% to 12.3% and moved lines falling from about 25% to below 10% between 2021 and 2024. This observational trend does not establish that AI caused the changes. In the 2025 Stack Overflow survey, 84% said they used or planned to use AI tools. Among respondents to the frustrations question, 66% selected almost-correct solutions and 45% selected more time-consuming debugging of AI-generated code. That second answer does not measure whether their total workflow saved or lost time.

METR did measure completion time. Its July 2025 randomised study involved 16 experienced open-source developers and 246 tasks in familiar, mature repositories. Developers expected AI to reduce time by about 24% and afterwards believed it had reduced time by about 20%. With AI allowed, tasks actually took 19% longer. That is a change in completion time, not a directly interchangeable percentage change in productivity. The result applies to this sample, task mix, and early-2025 tooling.

The 2025 DORA report does not overturn that experiment. Google's research programme surveyed nearly 5,000 technology professionals worldwide; they were not a sample of Google employees. More than 80% believed AI increased their productivity. That figure is self-reported sentiment, not a measured percentage improvement in task completion time. It cannot show that newer tools fixed the perception gap METR found.

There is separate measured evidence from Google. In a 2024 randomised trial with 96 Google engineers, participants using three internal AI coding features completed a standardised task in an average of 96 minutes, versus 114 minutes without them. That is about 16% less time in the raw averages. The authors' preferred adjusted estimate was about 21% less time, but its 95% confidence interval included no effect. This was one task with particular coding features, not a test of today's autonomous agents or proof that productivity improved between the two studies.

METR's February 2026 follow-up also matters. Its later estimates suggested speedups, but participant and task selection, along with difficulties timing concurrent agents, made the size unreliable. The authors explicitly called the data very weak evidence for how much the effect had changed. I use agents because they help on some of my tasks. The research gives me a reason to measure that benefit, not infer it from how productive a session feels.

Cheap tokens can make waste easy to ignore. CNBC reported in September 2024 that OpenAI expected roughly a $5 billion annual loss on $3.7 billion of revenue. These were reported projections, not published audited accounts. My earlier spending ratio was wrong: $5 billion divided by $3.7 billion is about $1.35 of loss per dollar of revenue. A simplified revenue-minus-cost calculation would imply about $2.35 in costs per revenue dollar. Neither calculation tells us the marginal cost of serving one API request or proves that each token is subsidised.

Pricing models can still change. GitHub moved Copilot plans to usage-based AI Credits on 1 June 2026, with included usage and controls for additional spending. That establishes a billing change. It does not establish that every vendor must raise token prices or that present prices are below inference costs.

The risk I take from this is practical. A habit built around generous allowances may become expensive if usage grows or the billing terms change. An agent that generates fourteen files where four would do can leave more code to maintain as well as more tokens to pay for. I do not need a forecast of inevitable price rises to want that waste visible now.

I use agents and find them genuinely useful on the right task. I also want the tool I am paying for to be optimised for my work rather than its meter. Terseness should be the default. An agent should write four files when four files are right, say "done" without three paragraphs of ceremony, and treat my context window as a scarce resource I pay to refill on every turn. Cheaper models exist on the same pricing pages, but price alone does not tell me how concise or effective they will be on my task. Vendors still have to choose incentives that favour the user's benefit over the invoice.

I want tools that are terse by default, measured on the user's outcome, and careful with the tokens the user pays for. Until that becomes normal, the most useful habit you can build while tokens are still cheap is the one these incentives quietly train out of you. Read the output. Count the files. Watch the meter. The bill you ignore today may be the business model someone is counting on tomorrow.