This post is the first in a series examining the economics and environmental benefits of AI’s demand for electricity.

AI is becoming increasingly ubiquitous. Every day, it seems there are new announcements about how LLMs and the AI-du-jour will revolutionize the way we work and live. While, depending on what you do, there are some fears that AI will take our jobs, a less-heralded (but perhaps more realistic!) concern is the worry that AI could drive global warming by increasing power demand. 

Data centers in general have become a major driver of electricity demand. When I ask Halcyon about projected electricity load growth from data centers, our software platform replies that Virginia, home to “Data center”-alley, is at the forefront of this activity: 

“The projected load growth from data centers in Virginia is significant and expected to increase in the future. Data centers, which currently represent approximately 20% of Virginia Power's electricity sales, have been a source of significant increase in demand which is expected to continue over the next decade. From 2003 to 2013, data center developers informed Dominion Virginia Power of plans to build 133 data centers in Northern Virginia, and since 2013, announcements of new data centers planned for construction have more than doubled, indicating an upsurge in construction and new loads corresponding to the data center industry's expectations of accelerated growth.

Dominion Energy Virginia announced a growth capital plan which includes spending approximately $27 billion from 2022 through 2026 to construct new generation capacity, including the CVOW [Coastal Virginia Offshore Wind] commercial project, to meet its renewable generation targets and growing electricity demand within its service territory to maintain reliability and regulatory compliance. This is intended to address both continued customer growth and increases in electricity consumption which are primarily driven by new and larger data center customers.”

Large language models (LLMs) exacerbate this electricity demand growth by consuming more electricity than traditional search engines on a per-query/per-search basis. As a company employing LLMs in service of decarbonization, this is something we need to keep in mind. For example, in order to perform our 45V comment analysis, we ingested 30,000 comments, then used a batch API process to summarize comments from ~29,500 individuals, and then ran ~20 detailed queries against each of the 400 comments submitted by companies. All in all, that’s about 40,000 queries - not exactly a light lift computationally. 

So, what can we do to minimize our impact?

If we look at the processing data from data centers that power LLM providers, the findings are fairly intuitive: most jobs (either individual chatbot responses, or, in our instance, batched queries submitted via an API) are submitted during the work day. These batches create a queue in the data center and those jobs take longer to complete. (It’s worth noting our LLM provider offers a 24-hour window to provide responses to batch queries so they can flatten demand if needed and reserve GPU capacity during peak hours.)

We initially hypothesized that submitting our batch queries after the workday was done would result in faster processing, less computational strain and thereby less energy consumption. However, in California, solar generation drives down the overall carbon intensity to zero from 10am-2pm. 

In other words, for those of us in California and other solar-heavy states like Texas and Arizona, the optimal time frame when low-carbon resources are most available via the grid is actually right when LLM api call volume is the highest, between 10am-2pm. 

(source: California Air Resources Board)

Inevitably, this comes back to business incentives: LLM providers want to utilize their expensive hardware as efficiently as possible. From their perspective, so long as they aren’t exceeding their data center’s max capacity, the cost of “normal” power vs renewable power isn’t really significant. From our perspective, even though we “get better service” via faster batch processing during evening hours, if we care about electricity utilization, the best thing we can do is align our batches with the bottom of the duck curve. 

There are a few implications to discuss. It may be more expensive to operate data centers on low-carbon grids than on higher-carbon grids. It may also be worth exploring more ways to clean up our evening energy generation so that we can use more compute power when there’s excess GPU capacity. Finally, it’s worth thinking about which businesses or industries are enabled by negative energy prices during specific and/or unpredictable times - depending on supply and demand, there could be a premium or a discount to use more GPUs during low-carbon hours. 

We hope you’re as interested in the topic as we are. Over the next several weeks, Halcyon will analyze the comments submitted in other regulations (45Y, SEC GHG emissions reporting guidelines, etc) using the approach laid out above. We’ll keep track of the tokens used, cost of our AI computation, and the time/ carbon intensity of our query runs to provide actual numbers about the impact of our work. 

Comments or questions? We’d love to hear from you - sayhi@halcyon.eco, or find us on LinkedIn and Twitter.