Subu
← all blog

Benchmarking Resource Consumption to Train Leading AI Models

#ai#llm#energy#sustainability#economics· 4 min read

Benchmarking Resource Consumption to Train Leading AI Models 📋

Tech benchmarks are exciting — we love seeing LLMs get smarter, faster, and more versatile with every iteration. But while we celebrate the breakthroughs, it’s important to remember the staggering amount of resources required to train these models. Imagine a city’s worth of power, water, and more going into each cycle of training. That’s the scale we’re talking about here.

Why Benchmark Resource Consumption?

As AI becomes more integral to our lives, so does its environmental impact. Training a single leading-edge AI model isn’t just a computational feat; it’s a resource-intensive process that draws on:

  1. Power🔌
  2. Water💧
  3. Carbon Emissions💨
  4. Land🖼️

Unfortunately, official data on these metrics is often scarce. Companies provide limited transparency about the resource consumption of their AI models, leaving us with gaps in understanding their environmental costs.

My Approach to Benchmarking 📋

This post aims to bridge that gap by estimating resource consumption using publicly available data, industry reports, and third-party analyses. Here’s what I’ll focus on:

  1. Power Consumption: How much electricity is consumed during the training of leading AI models? This includes GPU power consumption, the number of GPUs used, total GPU usage in hours, server power consumption, the number of servers involved, total training time, the approximate power cost ($/MWh), and the probable source of power (e.g., grid or off-grid).
  2. Water Usage: How much water is needed to cool the infrastructure? (If data is available.) This includes GPU water consumption, the number of GPUs used, total water usage in liters, and approximate water management cost ($/l).
  3. Carbon Footprint: What are the greenhouse gas emissions tied to training? (If data is available.)
  4. Land Consumption: How much physical space is required for data centers? What is the location selected for training? (If data is available.)

Why Should You Care? 🤷

The next time you marvel at an AI model’s ability to write, code, or analyze, remember the resources behind it. This isn’t just about environmental consciousness; it’s about making informed decisions as consumers and developers of AI technology. By understanding the hidden costs, we can push for more meaningful and sustainable AI practices.

In the next sections, I’ll break down the numbers, discuss trends, and explore potential solutions for reducing resource consumption in AI. Stay tuned!

Benchmarking the latest LLM models: A New Era of Efficiency 💎

For this comparison, I wasn’t able to obtain data on water and land consumption, but I plan to include it in future versions. However, I’ve researched and estimated the power consumption and carbon footprint associated with model training.

Source: https://github.com/nagusubra/model_training_resource_consumption_benchmark

Source: https://github.com/nagusubra/model_training_resource_consumption_benchmark

The race to train the most powerful AI models has often come with a hefty price tag — both in terms of computational resources and environmental impact. But amidst the giants of Meta’s LLaMA 3.1 and OpenAI’s GPT-4, there’s a shining example of efficiency: DeepSeek V3. This model stands out not only for its performance but for how economically it was trained, proving that smarter and power-efficient models are possible in the future of AI development.

DeepSeek V3 achieved groundbreaking accuracy and surpassed other models in key benchmarks — all while using just one-tenth of the GPUs required by Meta and OpenAI. To put it simply: while Meta and OpenAI poured millions of dollars into the power costs of training their models — ranging from $2 million to $4 million USD — DeepSeek’s power costs totaled just $450K USD. That’s a savings of more than 80% in their power bills!

What’s more, DeepSeek’s entire training budget came in at a remarkably low $5.576 million, a fraction of the billions spent by its larger counterparts. This level of efficiency isn’t just impressive — it’s transformative.

As we look to the future of AI, the path forward seems clear: smaller models, smarter resource use, and less environmental impact. Just as past tech revolutions have driven progress through innovation and optimization, it’s time for AI to follow suit. This is a motivating and optimistic shift toward a more sustainable and cost-effective future.

The evolution of AI doesn’t have to be fueled by ever-increasing consumption. It can — and should — be about making AI more accessible, practical, and scalable for a wide range of applications, all while reducing our reliance on excessive resources. DeepSeek V3 is proof that we can have both cutting-edge performance and environmental mindfulness in the same package.

I believe that Smaller, smarter, and more power-efficient models are the key to shaping the next wave of AI advancements. The future looks brighter — and greener — than ever before.