> I don't think you can reverse this out like this because the 80% rate is before payments to "distribution partners, including Amazon". I think that payment includes the hosting cost for that those model but it's unclear.

The "hosting cost" is paid for by Anthropic and is the largest cost. The money Anthropic pay to Amazon for delivering Anthropic models via Bedrock is separate, independent of compute costs, best thought of as commission.

The forecasted / guessed / estimated 80% number is based what customers pay per token minus the projected compute costs, i.e: the people who believe that Anthropic has 80% margins on tokens believe that Anthropic spend $0.20 on inference compute for every $1 of per-token billed-via-the-api revenue.

We know that there are hundreds of thousands of fixed-price subscriptions being used to their absolute maximum, with many people bragging about how many subscriptions they run in parallel. These tokens are not included in the 80% margins, they are acknowledged to be "subsidized". People like @theo on Twitter post almost daily about how much they're milking Anthropic and OpenAI with leaderboards.

Both Anthropic and OpenAI (more so OpenAI) do "resets" where they increase the limits available to people on their fixed price plans. We know that there are people paying $1,000 per month for multiple subscriptions to generate tokens that would cost $50,000 via the API. Even if Anthropic's margins are 80% on compute for per-token billing, that's still $10,000 of cost to Anthropic generating just $1,000 in revenue. Multiply that by tens of thousands or maybe even hundreds of thousands of subscriptions.

Anthropic and OpenAI have raised over $100 billion each and continue to raise. If they're making 80% or even 50% margins on $10 billion in revenue per month they would not need to raise, they would be shouting for the roof tops about how profitable they are, they wouldn't be delaying their IPOs, yet they're only profitable by non-GAAP metrics like WeWork's classic "Community-adjusted EBITDA" or in this case "per-token-adjusted EBITDA" or whatever they will call it in their IPOs.

Yes, they're selling tokens via the API for more than they cost, they are profitable on per-token billed inference, it has positive margins, but those profits are obliterated when you account for all the inference they're paying for out of pocket on fixed price subscriptions, upon which they keep increasing limits because they desperately need to show growth further harming their profitability (consuming all of the money they make from their API).

If Anthropic and OpenAI needed to be profitable tomorrow, they could be, they could kill off all their fixed price subscription plans and charge only for usage via the API, they'd print money, but they'd lose mindshare because nobody except for enterprises can afford to pay the true cost, all the regular people would switch to cost effective good-enough models, and then within months, the enterprises would start to switch too because no longer would their employees be claude-pilled.

Anthropic and OpenAI cannot turn off subsidization, thus, their margins on per-token API billing are not important in any discussion about their long term financial wellbeing. Just look at the large scale customers like Harvey (~15 trillion tokens per month, ~$50m+ in spend) who are, sensibly, investing in building their own specialized models that are cheap to run so they can cut their spend by 90%. That's profitable revenue for Anthropic / OpenAI today, but completely gone soon.

> Do you have a source for that?

https://www.youtube.com/watch?v=7xij6SoCClI

"This week, Noah Smith and Erik Torenberg are joined by Dario Amodei, CEO and Co-founder of Anthropic. Dario talks about the economics of AI development, the comparative advantage of AI companies like Anthropic, AI safety, and his stance on California's SB 1047 bill. They also discuss the impacts of AI on global power dynamics, competition between the US and China, and inequality in an AI-powered world."

At around 12 minutes in:

"I think actually even if such a model is released one thing you know that's a this analogy to to open- Source software is that these big models they're actually very expensive to run on inference the majority of the cost is is inference not necessarily the training of the model so if you have only you know I don't know 10 20% 30% better way to do inference that can kind of negate the effect so the economics are kind of strange yes there's this giant fixed cost that you have to amortise but then there's also the per unit cost of inference and small differences in that can actually again assuming the thing is deployed widely enough make a very big difference so I don't know quite how that's going to play out"

The scales have changed since then with inference costs falling and more being spent on training but the fundamentals are the same. Inference is expensive, in part, because peak usage dictates capacity whereas capacity can dictate training. Anthropic must pay billions of dollars per month to be able to handle peak inference, hence their efforts to try and shape usage by offering discounts / flexible limits at different times of the day. They can train when capacity permits.

Garry Tan wants US open-weight AI labs to 'distill' frontier models, too