19 hours ago
How much energy does your AI prompt use? I went to a data center to find out.
There they were. Golden boxes, louder than a toddler on a red-eye, hotter than a campfire in a heat wave, pricier than a private Caribbean island.
Yes, real, working Nvidia GPUs.
I was under strict 'look, don't touch" orders—as if I'd, what, lick the mesh metal enclosure. Just standing there, I could hear and feel the electricity being devoured.
We've all heard about AI's insatiable energy appetite. By 2028, data centers like this one I visited in Ashburn, Va., could consume up to 12% of all U.S. electricity, according to a report from the Energy Department and Lawrence Berkeley National Lab.
And yes, we're the problem, it's us. (Insert Taylor Swift-related groan here.) Every time we ask AI to write an email, draw an anime-style George Washington or generate a video of a cat doing a back flip, we're triggering another roar in those massive halls of GPUs.
What I wanted to know was, how much power do my AI tasks actually use? The equivalent of charging a phone? A laptop? Cooking a steak on an electric grill? Powering my house?
After digging into the research, visiting a data center, bugging just about every major AI company and, yes, firing up that grill, I got some answers. But not enough. Tech companies need to tell us more about the energy they're using on our behalf.
Let's start with a recent, popular example: 'a video of a cat diving off an Olympic diving board." The moment you hit enter, that prompt gets routed to a massive data center.
When it arrives, it kicks off inference, where pretrained AI models interpret and respond to your request. In most cases, rows of powerful Nvidia graphics processing units get to work turning your weird idea into a weirder reality. Rival chips from companies like Amazon, Google or Groq are also starting to be used for inference. The model training itself happens earlier, with Nvidia chips.
The facility where I saw that 'SuperPod" of Nvidia H100 GPUs was run by Equinix, one of the world's largest operators of data centers that provide cloud infrastructure—and now, AI.
Chris Kimm, Equinix senior vice president of customer success, said that while AI training can happen just about anywhere, inference is best done geographically closer to users to deliver the best speed and efficiency.
Figuring out how much energy your individual AI prompts use would be a lot easier if the major AI companies actually shared the darn info. Google, Microsoft and Meta declined. Google and Meta pointed me to their sustainability reports.
OpenAI shared something. Chief Executive Sam Altman said that the average ChatGPT query uses about 0.34 watt-hours of energy. OpenAI wouldn't break out details on text, image or video energy usage.
Researchers have stepped in to fill the gap. Sasha Luccioni, the AI and climate lead at open-source AI platform Hugging Face, has run tests to estimate the energy required to generate different types of content. Along with other researchers, she also maintains an AI Energy Score leaderboard. Since the top AI players use their own proprietary models, she relies on open-source alternatives.
The energy required to generate content varies widely depending on the model and GPU setup. Compare Luccioni's findings with charging a typical smartphone, which uses around 10 watt-hours of energy:
• Text: A lightweight, single-GPU Llama model from Meta used about 0.17 watt-hours, while a larger Llama model running across multiple GPUs used 1.7 watt-hours.
• Images: Generating a single 1024 x 1024 image with one GPU also used 1.7 watt-hours.
• Video: This is the most intensive. Even making 6-second, standard-definition videos used anywhere between 20 and 110 watt-hours.
I wanted to better understand the stakes—literally. So I grabbed an electric grill from Home Depot, a power meter and my video producer, David Hall. About 10 minutes and 220 watt-hours later, we had a thin, medium-well steak. Translation: The energy it took to cook a decent dinner was about the same as generating two AI videos, at the high end. (Watch the video above for more steak breakdowns.)
Remember the short AI film I made using Google Veo and Runway a few weeks ago? We generated about a thousand 8-second, 720p clips for our film. Going by these estimates, we might have used roughly 110,000 watt-hours. That's nearly 500 steaks!
But, as I said, Luccioni doesn't have the power-consumption data for the commercial AI tools, and her numbers aren't a perfect match: On the one hand, our video was higher quality than the 6-second, 480p clips in Luccioni's research. On the other hand, the popular video models are likely optimized for greater efficiency, experts say.
'Until we get access to these models," Luccioni said, 'all we can do is estimate."
Her tests also use Nvidia's last-generation Hopper chips. Nvidia has seen a jump in energy efficiency with its latest Blackwell Ultra chips, according to Josh Parker, the company's head of sustainability. 'We're using 1/30th of the energy for the same inference workloads that we were just a year ago," Parker said.
That said, plenty are still using those older chips. The pod I saw at Equinix's facility? It cost over $9 million in just Nvidia hardware alone. You don't just toss that in the dumpster when new ones come out.
And I've only covered electricity. These hot GPUs also require a lot of water to stay cool, but that's a whole other story.
Data-center providers and tech companies I spoke to all said the same thing: Demand for these GPU-filled buildings keeps multiplying. Just driving through Ashburn, I saw five massive data centers going up.
The companies also stressed the improving efficiency of models and chips, and their efforts to shift to cleaner, more renewable energy sources.
No matter how efficient things get, more of us are using AI. We could all buy more efficient air conditioners, but if the planet keeps getting hotter, we're going to crank the AC more—and burn more energy.
Luccioni hopes we at least consider energy use when we use these tools, maybe think twice about generating a dozen cat videos. And it's on the companies to start sharing real numbers, so that we can make informed choices.
Back to Virginia, and those screaming GPUs. Turns out, they weren't generating Olympic kitty videos. They were owned by Bristol Myers Squibb—and they were searching for new cures to diseases.
Not all AI prompts are what you'd call a waste of energy.
