The Economics of Inference: Why Tokens Keep Getting Cheaper

One of the most reliable trends of the past few years has been the falling price of a token. The cost to run a given query through a capable model has dropped repeatedly, and analysts expect the decline to continue. Understanding why reveals a lot about where the field is heading and what it means for the people building on top of these models.

The forces pushing prices down

Several forces compound. The first is hardware and serving efficiency: providers have become far better at packing many requests onto the same accelerators through batching, so the fixed cost of expensive hardware is spread across more work. The second is model architecture — mixture-of-experts designs, covered elsewhere on this site, let a model deliver large capacity while only activating a fraction of its parameters per request. The third is quantization, which represents a model's numbers with fewer bits so it runs faster and cheaper with, in many cases, little loss in quality. Layered on top is relentless competition, including from open-weight models that can be run for the cost of the hardware alone.

Batching and quantization, briefly

Two of these deserve a plain explanation. Batching means processing many users' requests together rather than one at a time; because the hardware is most efficient when kept busy, grouping work drives the per-request cost down. Quantization means storing and computing with lower-precision numbers — trading a sliver of accuracy for substantial gains in speed and memory. Neither is magic, and both have limits, but together they have squeezed real cost out of every request.

Competition does the rest

Perhaps the most powerful force is simply that there are many capable providers. When several closed labs and a thriving open-weight ecosystem all offer strong models, no one can hold prices high for long. Each round of price cuts pressures the others, and the availability of downloadable open models sets a floor that closed providers must beat on convenience or capability. This is ordinary market competition, and it works to the buyer's benefit.

What it means for builders and consumers

Cheaper tokens change what is worth building. Features that were once too expensive to run at scale — analysing every document, powering an always-on assistant, classifying millions of records — become viable. It shifts the constraint from cost to imagination and reliability. For consumers, it means more AI features bundled into products they already use, often at no extra charge, because the marginal cost of serving them has collapsed.

A note of caution: cheaper does not mean free, and heavy reasoning or long-context workloads can still add up quickly. The smart move is to match the model and the mode to the task rather than assume the lowest sticker price wins. But the broad trajectory is clear. Inference is getting cheaper, competition is keeping it that way, and that quiet economic shift may do as much to spread AI as any single capability breakthrough.