> If we care about saving a watt-hour, there are lots of places to look. Pointing fingers at the incredible energy consumption of internet-delivered HD video might not feel very comfortable to lots of folks.

I agree that in general, if the goal is to limit CO2 emissions and use renewable sources of energy, we ought not to focus on AI first, because it is dwarfed by many other things that we take for granted today. My canonical example I give folks is that the latte they order every day from Starbucks involves substantially more energy and water use than most uses of ChatGPT on a daily basis.

But as we move to digitize more and more of this world, and now create automated cognitive labor, we should start with the right foundations. I'd rather we not try to disentangle critical AI infrastructure from coal power plants, and I'd rather we try to limit the compute available to workloads in ways that encourage people to use the tech actually befitting of their use case rather than throw it all into the most expensive model every time.

How about the silly treadmill where we waste billions of compute to compute useless proof of work type behaviours and whenever more compute gets thrown at the problem we just make it harder to ensure there isn't better output. I believe it was called buttcoin or something silly like that.

And whose biggest claim to fame is allowing for large-scale fraud!

Oh wow, growing, drying, transporting, roasting, transporting, brewing something takes more energy and physical resources than a single query in a computer? Physical goods are amazing like that. I wonder how margins on software stuff are so high!??!

More seriously, i’m not too sure about the energy cost and IP infringed during the training and the value added to society by providing generic and mostly accurate but sometimes wildly wrong answers. Or from generating text or pretty pictures for a few milli-cents in cooling and electricity vs asking a human to do the same for a few kilo-cents.

It’s a lot of ladder kicking in the software industry these days.

Using a subpar model and having to run multiple requests may not be a better deal for climate than a sota model one-shotting the right answer.

I'm saying that you can often use a "subpar" model to one-shot an answer too, provided you work a little harder on prompting and context management.

As long as energy production and consumption has severe downstream impacts, yes, we do need to wade into this territory.

All serious, viable plans for decarbonization include a massive increase in electricity consumption, due to electrification of transportation, industrial processes, etc, along with an increase in renewable energy production. This isn't new, but AI datacenters are a very large net new single user of electricity.

If the amount of money already poured into AI had gone into the rollout of clean energy infrastructure, we wouldn't even be having this conversation, but here we are.

It makes perfect sense from a policy perspective, given that there are a small number of players in this space with more resources than most governments, to piggyback on this wave of infrastructure buildout.

It also makes plenty of financial sense. Initial capex for adding clean energy generation is high, but given both the high electricity usage of AI datacenters, and the long-term impact on the grid that someone will eventually have to pay for, companies deploying AI infrastructure would be smart to use the huge amount of capital at their disposal to generate their own electricity.

It's also, from a deployment standpoint, pretty straightforward — we're talking about massive, rectangular, warehouse-like buildings with flat roofs. We should have already mandated that all such buildings be covered in solar panels with on-site storage, at a minimum.

Sadly we’re already in the long term impact of the previous energy revolution, so we’d better get starting now instead of when we’ll feel the impact of this next compute evolution.

We should probably just do a carbon tax and not wade into that morass.

There’s a lot of focus on the carbon cost of various digital goods. I get it. Destroying the environment is a big problem. But like, maybe we also should not make a bunch of plastic crap and ship it around the world a bunch of times.

Disclaimer:Foreword Author here. I agree that there are may things one could change, however for many other services or objects you buy, you are able to estimate the env. footprint or you can change your consumer behaviour. However for the top AI-models one has no clue how much energy is used. Therefore the demands are among others for transparency from the ai companies.

Well, you almost certainly know more than me about it, since you are working in the area. From a layman’s point of view it seems like knowing that things are very carbon producing has not provoked mass behavior changes. I’d have more faith in moves that add a measurable cost. But maybe knowing how much LLMs produce could be part of motivating us to include an actual cost.

I have no idea what the carbon footprint of the coffee I drink or chair I sit in or netflix program I watch is. I can control my consumption of LLMs just as easily as those things.

> If we care about saving a watt-hour, there are lots of places to look. Pointing fingers at the incredible energy consumption of internet-delivered HD video might not feel very comfortable to lots of folks.

Air conditioning for example would be a good place to save energy, as the world wide energy consumption is a multiple of AI's consumption. But climate change will push the need (not luxury) for air conditioning up, which is the Catch-22 in this case.

The International Energy Agency (IEA) estimates that 10% of the globally generated energy is used for sir conditioning. But it would nevertheless be a good idea to require AI companies to care for renewable energy before they reach similar consumption levels.

Regarding the "morass" … we tell people how fast they can drive, or companies to limit air pollution (at least in some countries) so no problem here.

Technically, if it's all clean energy, does it matter if it's "energy-efficient"?

So it seems like the better goal is to just aim for more clean energy.

Once we've got abundant clean energy, it might not matter so much anymore, but as long as we're still burning carbon, it matters a lot. And until we get there, we should probably do both.

Net Neutrality is a really bad awkward term that constantly confuses laypeople. I get what you’re saying, but don’t lean on the term Net Neutrality in the hopes it will help people understand by building off something else they understand: People don’t understand Net Neutrality.

> 2. AI companies must disclose: a. How much electricity is used in operating their AI.

Doesn't training the model consume the most energy in most cases?

This is changing rapidly.

Google announced they are serving 500T tokens per month. State of the art models are currently trained with less than 30T tokens. Even if training tokens are more costly to run (eg, a factor of 3x for forward, backward, and weight updates, and take another factor of 2x for missing quantization), you end up in a situation where inference compute dominates training after a very short time of amortization.

This is a good point. Another point is that the better models get, the less wasted tokens there will be on unproductive token generation for answers that are wrong in some way. Better answers might lead to increased demand of course. But less waste is not a bad thing in itself. And improved quality of the answers has other economical advantages.

My view is that increased energy demand is not necessarily a bad thing in itself. First, it's by no means the dominant source of such demand, other things (transport, shipping, heating, etc.) outrank it; so a little bit of pressure from AI won't move the needle too much. Our main problem remains the same: too much CO2 being emitted. Second, meeting increased demand is typically done with renewables these days. Not because it's nice to do so but because it's cheap to do so. That's why renewables are popular in places like Texas. They don't care about the planet there. But they love cheap energy. And the more cheap, clean power we bring online, the worse expensive dirty power actually looks.

Increased demand leads to mostly new clean generation and increased pressure to deprecate dirty expensive generation. That's why coal is all but gone from most energy markets. That has nothing to do with how dirty it is and everything to do with how expensive it is. Gas based generation is heading the same direction. Any investment in such generation should be considered as very risky.

Short term of course you get some weird behavior like data centers being powered by gas turbines. Not because it's cheap but because it's easy and quick. Long term, a cost optimization would be getting rid of the gas generators. And with inference increasingly becoming the main thing in terms of energy and tokens, energy also becomes the main differentiator for profitability of AI services. Which again points at using cheap renewables to maximize profit. The winners in this market will be working on efficiency. And part of that is energy efficiency. Because that and the hardware is the main cost involved here.

Thank you!

Depends. For CoT models inference is significantly more costly (compared to regular models).

Also,

>Brent Thill of Jefferies, an analyst, estimates that [inference] accounts for 96% of the overall energy consumed in data centres used by the AI industry.

https://archive.is/GJs5n

Foreword Author here. I agree, even early estimates e.g. from Meta (2022) suggested 20% Training, 10% Experiments, 70% inference. And adoption is rising from month to month.

Those must have been about other things than LLMs though. Meta has huge inference loads for other types of models.

> Do we as a society want to wade into the morass of telling people what kinds of activities they can use energy for?

This really applies to any application which consumes high percentages of the resources available. (Compare, data centers are responsible for almost 80% of the electricity consumption in the Dublin area according to the paper.) The rational of purpose and resource demand and expected effects is secondary to this. The primary question is about (significant) quantities.

  Do we as a society want to wade into the morass of telling people what kinds of activities they can use energy for?

I mean, yeah, that's just basic civil regulation. Energy generation has massive negative externalities, and preventing waste is a worthy cause. I don't agree that AI must be singled out in that sense, but even it were, I imagine a modest push for efficiency would only help us in the long run.

  If we care about saving a watt-hour, there are lots of places to look. 

Well put, but I think it's important to bring the analysis one level up, and look at emissions. In that paradigm, meat eating and non-essential travel (yes, including vacations to Rome, business meetings, scientific conferences, and other perceived-to-be-unalienable rights) are punching way above their weight class.

For anyone who's curious on specifics re:AI emissions, the recent MIT article is the gold standard in terms of specificity, neutrality, and nuance: https://www.technologyreview.com/2025/05/20/1116327/ai-energ... .

I also did some napkin math here in 2024.12: https://bsky.app/profile/robb.doering.ai/post/3lckwra33vk2t TL;DR: Eating one less burger affords you ~300 chatbot inferences, and avoiding a flight from ATL to SFO affords you ~16,000.

I think if we do want to do this then banning bitcoin proof of work behaviours seems far more important.