Unbelievable Upkeep
ChatGPT’s immense popularity and power make it eye-wateringly expensive to maintain, The Information reports, with OpenAI paying up to $700,000 a day to keep its beefy infrastructure running, based on figures from the research firm SemiAnalysis.
“Most of this cost is based around the expensive servers they require,” Dylan Patel, chief analyst at the firm, told the publication.
The costs could be even higher now, Patel told Insider in a follow-up interview, because these estimates were based on GPT-3, the previous model that powers the older and now free version of ChatGPT.
OpenAI’s newest model, GPT-4, would cost even more to run, according to Patel.
Athena Rises
It’s not a problem unique to ChatGPT, as AIs, especially conversational ones that double as a search engine, are incredibly costly to run, because the expensive and specialized chips behind them are incredibly power-hungry.
That’s exactly why Microsoft — which has invested billions of dollars in OpenAI — is readying its own proprietary AI chip. Internally known as “Athena,” it has reportedly been in development since 2019, and is now available to a select few Microsoft and OpenAI employees, according to The Information’s report.
In deploying the chip, Microsoft hopes to replace the current Nvidia graphics processing units it’s using in favor of something more efficient, and thereby, less expensive to run.
And the potential savings, to put it lightly, could be huge.
“Athena, if competitive, could reduce the cost per chip by a third when compared with Nvidia’s offerings,” Patel told The Information.
Though this would mark a notable first foray into AI hardware for Microsoft — it lags behind competitors Google and Amazon who both have in-house chips of their own — the company likely isn’t looking to replace Nvidia’s AI chips across the board, as both parties have recently agreed to a years-long AI collaboration.
Right On Time
Nevertheless, if Athena is all that the rumors make it out to be, it couldn’t be coming soon enough.
Last week, OpenAI CEO Sam Altman remarked that “we’re at the end of the era” of “giant AI models,” as large language models like ChatGPT seem to be approaching a point of diminishing returns from their massive size. With a reported size of over one trillion parameters, OpenAI’s newest GPT-4 model might already be approaching the limit of practical scalability, based on OpenAI’s own analysis.
While bigger size has generally meant more power and greater capabilities for an AI, all that added bloat will drive up costs, if Patel’s analysis is correct.
But given ChatGPT’s runaway success, OpenAI probably isn’t hurting for money.
Must Read – Elon Musk Lost $13 Billion on 4/20
