Models Keep Getting Cheaper. The AI Bill Keeps Getting Bigger.
Why AI spending is so hard to forecast, and what companies need to understand before deciding where to run their AI.
Why your AI budget never quite adds up
For two years, the headline news about AI has been that it keeps getting cheaper. Half of that is true. A small model today costs $0.05 per million input tokens — for summarizing, classifying, rewriting, that is cheap enough to round down to nothing. The other half isn't. When GPT-5 launched, it was $1.25 per million input tokens and $10 per million output. Later flagship-tier models run $5 and $30; the Pro tier sits at $30 and $180. At the top of the range, prices have not fallen. They have climbed. Across one provider's own lineup, input prices now span a factor of 600. What has genuinely become cheaper is the same capability — today a smaller model does what took a flagship a year ago. Not the best capability. Most of the time that distinction doesn't matter. It starts to matter when the invoice arrives. And almost every company that begins using AI in earnest runs into the same moment in a meeting. Finance looks at the bill and asks IT: how much did we spend this month? Then comes the second question — which departments? The first one usually gets an answer. The second often doesn't.
Cheap is real. So is expensive.
Over the same period, CFOs were hearing a different story. Reuters reported that 71% of companies experienced AI cost overruns in 2025. Meta, by some accounts, warned staff that internal AI usage could run into billions of dollars in 2026, and began setting spending caps. Both things are true at once. It isn't a contradiction — we are simply in the habit of watching unit prices, and the bill was never determined by unit price.
Price lost. Volume won.
A few years ago, enterprise AI was mostly one question, one answer: send a prompt, get a paragraph. Not anymore. More and more applications are agentic — the AI plans its own steps, retrieves data, calls tools, checks results, then produces an answer. A single task can run through several, sometimes dozens, of rounds of inference. Advisory measurements put agent loops at three to ten times the token consumption of a single prompt. And Taiwanese enterprises are moving onto that curve in numbers. iThome's 2026 CIO survey found adoption of agentic AI among large Taiwanese enterprises rose from 17% to 32% in a year — close to double. Taiwan is a useful place to watch: its economy is weighted toward manufacturing and financial services, two sectors where AI moves into regulated, operationally critical work earliest. Unit prices fell by half. Volume multiplied several times over. Multiply the two, and the bill moves in one direction only.
The trouble isn't that it's expensive. It's that it can't be estimated.
If AI were simply more expensive, that would be manageable — you budget more. What's hard is that it can't be estimated. Metered AI has an unhelpful property: the same task consumes a different number of tokens each time. Change one line of a prompt, switch models, let an agent take one more loop, and the cost changes. Chain several agents together and the variance compounds. Two further mechanisms make forecasting worse. The first is reasoning. The steps you never see — the model working through the problem — are billed as output tokens. Ask the same question twice; if the model thinks a little longer, the bill is a little higher. The second is the long-context threshold. With GPT-5.5, once a single input exceeds 272,000 tokens, the entire session is charged at double the input rate and 1.5 times the output rate. A few extra pages in the prompt, and the price tier moves. This breaks the logic of annual budgeting. IT procurement is used to "so many licenses, so many servers." AI spending behaves more like a utility bill — one whose meter you cannot read in advance. At about this point, someone usually says: then just set a cap. You can. But before you set a cap, you need to know who is using it and for what. Which is the second question from the start of this article. And this is no longer an IT problem. The FinOps Foundation surveyed 1,192 practitioners and found the share of teams responsible for managing AI spend rose from 31% to 98% in two years. A separate industry survey found 62% of organizations said an unexpected AI cost had materially changed a business decision — and in four out of ten of those cases, it had to be escalated to the board. Once a line item reaches the board, it has stopped being a technical matter.
One line is shrinking. The other is growing.
Taiwanese enterprises carry an additional squeeze. The same iThome survey shows IT budgets at large Taiwanese enterprises growing 9% in 2026, down from 13% the year before. Investment strategy has turned cautious, and resources are concentrating on projects with visible returns. Budget growth slowing, AI usage climbing, every investment asked to prove itself. Three forces at once — and the critical link is missing: if you cannot see where the money goes, you cannot prove it was worth spending.
So move it into your own data center?
As AI bills continue to rise, companies naturally consider whether they should move AI workloads back to their own data centers. That option is worth evaluating, but companies first need to understand their actual usage. Utilization is an important factor when comparing on-premises and cloud deployments. According to IDC’s assessment, workloads with sustained utilization above 80% typically have a lower three-year total cost of ownership when hosted on-premises; below 70%, public cloud is generally more cost-effective. A server purchased only to sit idle most of the time is still a waste, just in a different form. The actual cost crossover point, however, varies by industry and workload. Peak usage patterns in manufacturing and financial services, for example, can differ significantly. Without a clear picture of their own usage, companies lack a sound basis for deciding whether to move on-premises or remain in the cloud. For many applications still being validated, public cloud remains a suitable starting point. As usage patterns and requirements become clearer, companies are better placed to assess when to adjust their deployment approach.
Three things you can do now
First, answer the two questions from the opening. How much did AI cost last month, which departments used it, and for which applications? If you cannot answer, start there. Second, not every task needs the most powerful model. Summarizing meeting notes or looking up an internal policy may not require top-tier capabilities. Top-tier models can cost up to 600 times as much as entry-level models per input token. Advisory firms estimate that matching models to task difficulty, combined with caching and trimming context, can reduce costs by 30–50% while maintaining quality. Third, ensure every AI expense has an owner and a spending limit. A clearly designated owner can track usage, assess value, and decide what adjustments are needed as spending approaches the limit.
Cost is only the visible layer
This article is about money. But money is the easiest layer to see. If a company cannot see clearly what its AI costs, then what its AI did, what data it touched, what permissions it used — those are likely harder to see still. In the next article, we turn to the problem that is harder to see.