First of all, it isn't wrong. The power consumption of AI training and inference is massive.
Second, that page is obviously meant to shed light at AI issues from a lot of different viewpoints, it would have been a serious omission to not mention environmental concerns.
No it is not, honestly. The argument that inference is cheaper than training does not hold up when you keep in mind that as the current model is being used for inference, another one at least as big is being trained simultaneously. Current models can't adapt to new information. So training is an ongoing thing and inference comes in addition to the training costs, not instead of.
This doesn't refute the (unmade) argument that inference is cheaper than training. Inference is cheaper than training, regardless of how much of each is taking place.
Not only that, both keep getting less expensive in various ways. small models, though still not cheap to train, are making inference vanishingly cheap.
> Alex de Vries is a Ph.D. candidate at VU Amsterdam and founder of the digital-sustainability blog Digiconomist. In a report published earlier this month in Joule, de Vries has analyzed trends in AI energy use. He predicted that current AI technology could be on track to annually consume as much electricity as the entire country of Ireland (29.3 terawatt-hours per year).
That's a lot of power, and the availability of electricity is already seen as a bottleneck in the development of AI models [2].
> He predicted that current AI technology could be on track to annually consume as much electricity as the entire country of Ireland (29.3 terawatt-hours per year).
He didn’t though, did he? If you click through and read the report, it is quite clear in saying that this is a worst case scenario if you ignore things like it being unrealistic for Nvidia to even produce the requisite number of chips. It’s also based on the efficiency of 2023 models; he assumes 3 Wh per prompt when today it is 0.34 Wh.
It also completely ignores the fact that AI use can displace greater energy use:
The carbon emissions of writing and illustrating are lower for AI than for humans
> Our findings reveal that AI systems emit between 130 and 1500 times less CO2e per page of text generated compared to human writers, while AI illustration systems emit between 310 and 2900 times less CO2e per image than their human counterparts. […] the use of AI holds the potential to carry out several major activities at much lower emission levels than can humans.
It isn’t a bad thing if AI uses x Wh if not using AI uses 1500x Wh. Looking at the absolute usage without considering displacement only gives you half the picture.
AI is on track to use more energy than Bitcoin this year. And Netflix and others are activety trying to reduce their usage, whereas all I see on the AI side right now is people scrambling to use more for bigger models in the that at some point something usable might come out of it.
Second, that page is obviously meant to shed light at AI issues from a lot of different viewpoints, it would have been a serious omission to not mention environmental concerns.