Clone Wars
With a silly name and an even sillier startup cost, Stanford’s Alpaca GPT clone costs only $600 to build and is a prime example of how easy software like OpenAI’s may be to replicate.
In a blurb spotted by New Atlas, Stanford’s Center for Research on Foundation Models announced last week that its researchers had “fine-tuned” Meta’s LLaMA 7B large language model (LLM) using OpenAI’s GPT API — and for a bargain basement price.
The result is the Alpaca AI, which exhibits “many behaviors similar to OpenAI’s text-davinci-003,” otherwise known as GPT-3.5, the LLM that undergirds the firm’s internet-breaking ChatGPT chatbot.
In its shockingly simple budgetary breakdown, the Stanford CRFM scientists said they spent “less than $500” on OpenAI’s API and “less than $100” on LLaMA, based on the amount of time the researcher spent training Alpaca using the proprietary models.
When evaluating Alpaca against other models, the Stanford researchers said they were “quite surprised” to find that their model and OpenAI’s “have very similar performance,” with Alpaca being ever so slightly better and generating outputs that are “typically shorter than ChatGPT.”
Leveling Out
All the same, Alpaca does, as the Stanford CRFM folks note, suffer from “several common deficiencies of language models, including hallucination, toxicity, and stereotypes,” with hallucination being of particular concern, especially when compared to OpenAI’s text-davinci-003.
As multiple machine learning enthusiasts have noted, Alpaca’s release date fell at the beginning of what AI blogger Lior Sinclair noted “might be the most eventful week AI has ever seen,” with the Stanford model being followed first by OpenAI releasing GPT-4 followed by the drop of a new version of the Midjourney image generator along with several other big news items.
While the big firms — and the up-and-coming ones like OpenAI — are making money moves, it’s admittedly intriguing to see something cheap and easy like Alpaca crop up, albeit drawing on all the human brainpower at Stanford.
