According to Synergy Research Group, Amazon, Microsoft and Google spent more than $48 billion in the second quarter, most of it on data centers.
A natural question is: what will be the return on these large AI investments?
Analysts at RBC Capital Markets came up with an early answer this week, and it's not a good one.
“Long-term software gross margins will be structurally lower as a result of GenAI,” they wrote in a research note.
According to RBC, when software moved from “on-premise” – where companies run it on their own computers – to the cloud, where it runs on rented machines in remote locations, gross margins fell from 90% to 75%.
Analysts predict that the shift from cloud computing to generative AI will drive software profit margins even lower, to around 60%.
Gross profit is a simple measure of profitability calculated by subtracting cost of goods sold from revenue.
Gross margins in the software industry have traditionally been in the 90% range, which is why the industry is so attractive to investors and why software companies command such high valuations.
Developing new software costs a lot of money up front, but once it's written, the cost of building and distributing new versions to customers is close to zero, so every time you sell that software, your profits grow bigger and bigger.
This is what tech investors mean when they constantly talk about “scale”: software businesses have traditionally been very large, meaning that higher revenues mean bigger profits.
Software scale in the GenAI era
Why might software businesses become less profitable in the coming AI era?
“It may be hard to achieve such efficiency gains with GenAI,” RBC analysts wrote, citing the companies' income statements.
Generative AI is expensive to develop, but it’s also expensive to run.
Then there's training the AI models, which involves buying very expensive GPUs from Nvidia and putting those AI chips into servers that require specialized cooling and networking in huge data centers. These facilities use a lot of power, which makes them expensive and necessitates expensive upgrades.
This doesn't even include the cost of the data needed to train the AI models: Collecting and cleaning data is still expensive, although big tech companies and startups are trying to avoid most of this cost.
Once an AI model is trained, it needs to be run. This is the inference step, where the model is shown new data or requests and infers something useful from that information. This step also requires expensive chips and incurs ongoing costs.
This is different from a traditional on-premise software business where there was nearly 100% profit on each new sale: Every time an AI customer uses a GenAI service, there are a number of costs that the provider incurs.
For example, industry analyst Dylan Patel estimated last year that ChatGPT's operating costs were costing OpenAI $700,000 per day.
Increased revenue
But RBC's analysts were not all pessimistic.
GenAI is so innovative that customers are spending heavily on new AI-powered software, and they expect their software revenues to double or even triple from current levels.
Analysts say the software market has become so big that even lower profit margins could mean more “profit money.”
Earnings dollars are the metric that management and analysts turn to when margins are declining. They are a measure of the absolute profit a company generates.
For example, if a company has revenues of $100 million and a profit margin of 10%, then its absolute dollar profits are $10 million.
Even if this hypothetical company's revenues surged to $300 million, and its profit margins fell to 8%, its income would still be $24 million, making it more profitable than before.
“While we expect GenAI to pressure margins, we believe gross margins will be accretive in a post-GenAI world over the longer term,” the RBC analysts concluded.
The big assumption here is that GenAI will deliver huge revenue accretion — hopefully RBC Capital Markets is right, but if they're not, these big AI investments could produce “pretty disastrous” financial outcomes.
