AI token cost keeps falling. The debt behind it does not.

If you buy AI by the token, the last two years have been kind to you. The same work costs a fraction of what it did. Most operators file that under good news and move on.

Paul Kedrosky does not move on. He is managing partner at SK Ventures and a fellow at MIT’s Initiative on the Digital Economy, and he spent years as a Wall Street equity analyst. On Between Two COOs he made a case that the falling AI token cost is the single most important number in the AI economy, and that almost nobody who lent money against it has done the arithmetic.

Tokens are a commodity, and a strange one

Kedrosky’s starting point is to ask what a data center actually produces. His answer is tokens. He calls data centers token factories, and he calls tokens the first major new industrial commodity in 50 years, alongside copper, iron, and uranium.

Every other commodity moves in waves. Supply tightens, prices rise, someone finds more, prices fall. Tokens do not do that. On a quality-adjusted basis, he says, their price is falling 60 to 80 percent a year, and it only goes one way. He calls them the first hyper-deflationary industrial commodity.

The growth math

Here is the part worth writing down. If the price of what you sell falls 70 percent in a year, Kedrosky says you need roughly 500 percent unit growth just to stand still. To justify the valuations the frontier model companies carry, he puts the requirement at something over 1,000 percent growth a year, for at least a decade.

His comment on whether that is realistic: “It would be the first commodity in history to do it.”

Why the price keeps falling

It is tempting to blame competition. Kedrosky says competition is the smaller part.

The bigger part is that this technology went from the lab to a billion users faster than anything before it, so the engineering has not caught up. He described a conversation with a compiler engineer at NVIDIA who used to work on Java. Back then a half percent efficiency gain earned a pizza party. Now, the engineer told him, nobody cares unless the gain is 25 percent, because gains that size are lying everywhere. Better caching and mixture of experts models do the rest.

“Competition is like the icing on the cake,” Kedrosky said. “The structural gains in token prices are baked in the cake.”

Where the debt comes in

A year ago the hyperscalers paid for the data center buildout from their own free cash flow. Kedrosky says that in the last two quarters more than 60 percent of AI capex has come from external financing: debt, private placements, and special purpose vehicles.

Outside lenders want a fixed yield for a fixed term. So the industry now has fixed obligations on one side and a product whose price falls by more than half each year on the other. As he put it, the interest rate does not change because token prices have fallen. Much of that debt comes due around 2029, and he does not expect all of it to be refinanced.

The demand everyone is counting on

The standard reply is that demand will grow fast enough to cover it. Kedrosky’s concern is who the demand is coming from today. The heaviest users of tokens are developers, and coding is expansive. A short prompt becomes thousands of lines of code that get reworked over and over. Most other knowledge work is compressive. Summarize my email. Tighten this deck. It uses far fewer tokens.

If you forecast the whole economy from how engineers behave, you will overshoot.

What this means if you run a company

Falling AI token cost is good for your budget. Keep enjoying it. Three things follow from the rest.

Do not sign long commitments with AI infrastructure vendors that carry heavy debt. Kedrosky’s line: “I plan to be around in four years, but they don’t look like they will be.”

Do not build your internal AI forecast on your engineering team’s usage. Their token consumption tells you very little about finance, operations, or sales.

And expect your vendors to look for margin somewhere else. When the core product gets cheaper every year, the companies selling it move up the stack, toward the things their customers are building.

FAQ

Why is AI token cost falling so fast? Paul Kedrosky says competition is only part of it. The technology reached a billion users before the engineering matured, so large efficiency gains are still easy to find in compilers, caching, and model architecture. He says quality-adjusted token prices are falling 60 to 80 percent a year.

What does falling AI token cost mean for AI companies? Kedrosky says a company facing a 70 percent annual price decline needs roughly 500 percent unit growth to stand still, and more than 1,000 percent a year for a decade to justify current valuations. No commodity has ever done that.

How is the AI data center buildout being financed? A year ago mostly from the hyperscalers’ own cash flow. Kedrosky says more than 60 percent of AI capex in the last two quarters came from external financing, including debt, private placements, and special purpose vehicles. Much of that debt matures around 2029.

Are developers a good guide to future AI demand? No. Kedrosky says coding is expansive, turning small prompts into large volumes of tokens, while most white-collar work is compressive and uses far fewer. Early adopters are rarely representative of the people who show up later.

What should a COO do about it? Enjoy the lower prices, avoid long commitments to heavily indebted AI vendors, do not forecast company-wide usage from engineering usage, and assume your AI suppliers will move up the stack in search of margin.