ChatGPT’s Hunger for Energy Could Trigger a GPU Revolution

The cost of making further progress in artificial intelligence is becoming as startling as a hallucination by ChatGPT. Demand for the graphics chips known as GPUs needed for large-scale AI training has driven prices of the crucial components through the roof. OpenAI has said that training the algorithm that now powers ChatGPT cost the firm over $100 million. The race to compete in AI also means that data centers are now consuming worrying amounts of energy.

The AI gold rush has a few startups hatching bold plans to create new computational shovels to sell. Nvidia’s GPUs are by far the most popular hardware for AI development, but these upstarts argue it’s time for a radical rethink of how computer chips are designed.

Normal Computing, a startup founded by veterans of Google Brain and Alphabet’s moonshot lab X, has developed a simple prototype that is a first step toward rebooting computing from first principles.

A conventional silicon chip runs computations by handling binary bits—that’s 0s and 1s—representing information. Normal Computing’s stochastic processing unit, or SPU, exploits the thermodynamic properties of electrical oscillators to perform calculations using random fluctuations that occur inside the circuits. That can generate random samples useful for computations or to solve linear algebra calculations, which are ubiquitous in science, engineering, and machine learning.

Faris Sbahi, the CEO of Normal Computing, explains that the hardware is both highly efficient and well suited to handling statistical calculations. This could someday make it useful for building AI algorithms that can handle uncertainty, perhaps addressing the tendency of large language models to “hallucinate” outputs when unsure.

Sbahi says the recent success of generative AI is impressive, but far from the technology’s final form. “It's kind of clear that there's something better out there in terms of software architectures and also hardware,” Sbahi says. He and his cofounders previously worked on quantum computing and AI at Alphabet. A lack of progress in harnessing quantum computers for machine learning spurred them to think about other ways of exploiting physics to power the computations required for AI.

Another team of ex-quantum researchers at Alphabet left to found Extropic, a company still in stealth that seems to have an even more ambitious plan for using thermodynamic computing for AI. “We're trying to do all of neural computing tightly integrated in an analog thermodynamic chip,” says Guillaume Verdon, founder and CEO of Extropic. “We are taking our learnings from quantum computing software and hardware and bringing it to the full-stack thermodynamic paradigm.” (Verdon was recently revealed as the person behind the popular meme account on X Beff Jezos, associated with the so-called effective accelerationism movement that promotes the idea of a progress toward a “technocapital singularity”.)

The idea that a broader rethink of computing is needed may be gaining momentum as the industry runs into the difficulty of maintaining Moore’s law, the long-standing prediction that the density of components on chips continues shrinking. “Even if Moore's law wasn't slowing down, you still have a massive problem, because the model sizes that OpenAI and others have been releasing are growing way faster than chip capacity,” says Peter McMahon, a professor at Cornell University who works on novel ways of computing. In other words, we might well need to exploit new ways of computing to keep the AI hype train on track.

That Normal, Extropic, and others trying to rethink the fundamentals of the computer chip are finding investors suggests that GPUs may be getting some competition relatively soon. Vaire Computing, a startup based in the UK, is developing silicon chips that work in a fundamentally different way to conventional ones, performing calculations without destroying information in the process. This approach, known as “reversible computing,” was devised decades ago and promises to make computing far more efficient but never took off. Vaire’s cofounder and CEO, Rodolfo Rosini, believes that physical limits of etching ever-smaller components into silicon means that GPUs and other conventional chips are running out of time. “We have one order of magnitude left” in chip manufacturing, Rosini says. “We could make components smaller, but the number one enemy is removing heat from the system fast enough.”

Convincing a huge industry to abandon a technology it has grown on top of for more than 50 years won’t be easy. But for the company that provides the next hardware platform, the payoff would be huge. “Every so often something comes along which will be transformative for the whole of humanity, like jet engines, transistor microchips, or quantum computers,” says Andrew Scott of 7percent Ventures, which is backing Vaire. The investors betting on Extropic’s and Normal’s own reimaginings of computing have similar hopes for their own contenders.

Even more exotic ideas—like moving away from using electricity inside computer hardware—are also gaining traction. McMahon’s own lab is looking at how to compute information using light to save energy. At a conference that he helped organize recently in Aspen, Colorado, a group of researchers from Holland showed off an idea for a mechanical cochlear implant that harnesses sound waves to power its computations.

It’s easy to dismiss chatbots, but the frenzy triggered by ChatGPT could be on track to incentivize revolutions in more than just AI software.