Discussion about this post

User's avatar
Chris Preist's avatar

Thanks for this - this is a really valuable contribution which I will be sharing with others, and broadly agree with your analysis. You are right that both energy-per-token and tokens-per-task are both rapidly moving downwards for both hardware and software reasons - it is a very young technology, and there are still a lot of low hanging fruit for efficiency gains. I suspect that already your estimate will be on the high side, for this reason. Things are moving so quickly.

I would partly disagree on Jevons, though - people are quick to claim Jevons whenever there are efficiency gains alongside absolute increases in energy use, and it can be for other reasons. Jevons, strictly, is that efficiency gains result in reduced use of a resource by a service (in this case electricity and hardware), which reduces the cost of the service, allowing more use of it and more innovative ways to use it.

It is certainly true that AI is being used more, and there is extensive experimentation finding new ways to use it - but it does not appear to me that this is a result of resource cost reduction but simply the natural consequence of a brand-new already cheap-to-provide technology disseminating globally at a historically exceptional rapid pace (via the global internet). The fact that the US companies charge so much more than the Chinese companies shows that energy usage is a relatively small factor in their decision about price setting. Jevons may play some role, but at the moment the main driver of increased energy use is anticipatory investment in data centre capacity for both realistic and hype-driven reasons.

Interestingly, efficiency improvements may actually result in fewer data centres being required even while Jevons results in expanded use. This scenario will happen if most of us (including coders) find that 'good enough' AI meets our needs. Already, laptops are in the pipeline which can run ChatGPT models roughly as powerful as the frontier models of 18 months ago. Some high-value uses of 'super intelligence' (eg cybersecurity) may justify frontier model development, but for most of us what we already have is nearly good enough. In this scenario, your laptop will be powerful enough to produce those axolotl games. It wont be completely guilt free though. At least initially, such laptops will move towards gamer-spec levels of power consumption when at work - so energy use will increase at home.

Bob Meyer's avatar

Thanks - fun to see a real analysis by a serious user! Since you touch on the related question toward the end: Google’s 2.7 GW data center being built N of Fort Collins CO promises to use natural gas via fuel cells, then to pump the exhaust stream (presumably pretty pure CO2) into their rock formation for permanent disposal, after using the heat to cool the project via absorption chillers. Is this do-able, and assuming it will actually happen, is there much of a reason (besides well- and delivery-methane leakage which could be drone-monitored and controlled in a sane regulatory environment) to dislike it for green reasons? I write a substack on green energy (including inherently safe small nuclear like Kairos should provide), and think Jade might have a good compromise here. -Bob Meyer

2 more comments...

No posts

Ready for more?