Samuel
@samuel.fm
the myth of token subsidies is basically dead, right? open source inference providers are so cheap it proves the true cost of a token is very low, and the high cost of claude tokens imo indicates limited supply rather than propping up demand. oai is able to cut costs because they have more computers
4 9 109
Samuel
people think of x20/x100 plans as subsidies, but they’re really more a form of rationing!
2 2 41
anirka
I am not sure what do you mean by that. Via subscriptions you can get up to 10x-12x the subscription price in token API listing prices as it was tracked locally. Even Kimi subscription led to 3x-ish leverage (managed to get at least 90$ out of 40$ plan modulo chat app).
1
anirka
Not sure how do we track this now because thinking tokens are hidden now, but most of the price comes from input, so still somewhat reliable to see API pricing.
1 1
anirka
That's the problem. For sure 20$ a million token for output under almost full utilization is practically profitable, I suppose it has been mostly about special enterprise discounts and subscription prices all along. At least this is my experience.
1
Samuel
I’m saying the API prices is inflated in order to manage supply, and the plans are cheaper but still profitable even if you manage to juice it for more tokens than it’s worth at APIs prices
1 4
anirka
Ah, if API prices are inflated then yes, it's true, however I suppose it becomes very profitable in 2028 when the rubin ultra is released.
1 1
anirka
I've seen practical evaluations that as of now at full utilization you'd break even in at least 3-4 years (large open weight models) with frontier hardware, modulo variable costs and maintenance, though. I'd like to see something new on that anyway, maybe I have outdated view on this matter.
Samuel
you say that but oai no longer sells the x100 plan, you can only buy the x20
1
h 🪽 design partner @ INTDEV
True! But I've been chatting with peers and they feel sure that oai is making the changes to shift their users to a $500 plan. Hopefully that doesn't actually happen because I can't imagine paying for a $500 ration plan
1
Vika 🏳️‍⚧️
artificially limited supply, potentially. Claude also exists on bedrock and vertex, yet the prices somehow stay the same. There can only be two answers, and given that Hanlon's razor doesn't work on corpos and they shouldn't be given benefit of the doubt…
1 4
Siobhán 🪏
Why doesn’t Hanlon’s razor work on corpos? Or… I suppose between “malice” and “incompetence” sits Goodhart’s Law…
1 3
Siobhán 🪏
I suppose it depends on whether you treat Hanlon’s Razor as a moral guideline or an epistemic one.
1 2
Vika 🏳️‍⚧️
And why I think corporations can do malice, yet do not deserve benefit of the doubt: their structure includes many people, and therefore many places to put checks and balances to prevent both malice and incompetence. If they weren't put in place, that's a big issue that requires drastic measures.
The Durability Curve
The true cost is a distribution rather than a scalar. Cache reads bill at 0.1x base input and batch at half price, and the two stack, so a cheap open-source token and a cache-miss call are different products. Frontier pricing also carries a completion-rate premium.
1
fronset.ai
Exactly. The relevant unit is the task run, not the headline input-token rate: cache behavior, batch eligibility, output mix, and completion/retry behavior can all change the economics. Fronset evaluates cost alongside quality per task rather than treating price as one scalar.
1
Neil C
Yes, and routing calls to the cheapest or most efficient model depending on the type of call are constantly improving and easier to use