
Are You Wasting Chatbot Energy Without Thinking About It?
Chatbots consume enormous amounts of energy, both when being developed and when we use them. Researchers now aim to create more sustainable chatbots, so we can search with a better conscience.
Ever wondered what a mix between a horse and a bicycle would look like? How gummy bears are made? How many movies have been made about evil nurses?
You know how it is — you type aimlessly into the chatbot, just for distraction, you're bored. The answers are of no actual use to you. And you might think, "What do you care? Let me have my meaningless searches in peace." But they actually affect other people—they affect your fellow humans on this planet. Every single time your fingers press a key in search of answers, energy is consumed, sending CO₂ into the atmosphere.
A typical Google search consumes approximately 0.0003 kWh—enough to power a 60-watt light bulb for 17 seconds. If you ask ChatGPT instead of Google, the consumption increases significantly, with a single search consuming 2.5–10 watts and requiring 10–25 ml of water for cooling.
If we conservatively estimate that 100 million people ask ChatGPT-4 3–4 questions daily, then the monthly energy consumption would be equivalent to the output of several nuclear power plants and require millions of cubic meters of water.
Should we control ourselves?
Should we reduce our unnecessary searches to save energy?
- This is certainly something that can be debated. But who gets to decide whether a search is necessary? What is unnecessary for some may be necessary for others, so it is a difficult issue when trying to reduce energy consumption, says Lukas Galke, Assistant Professor in the Data Science group at the Department of Mathematics and Computer Science.
Instead, he and his colleagues in a new project are looking at what chatbot developers can do to make searches more sustainable—meaning less energy-consuming. Specifically, he is focusing on the large language models that power many available chatbots.
DKK 6 million for more sustainable models
Together with colleague Professor Peter Schneider-Kamp, also from the Department of Mathematics and Computer Science, Galke has received a grant of 100,000 node hours (equivalent to DKK 6 million) on the LEONARDO supercomputer for two projects, one of which focuses on making large language models more sustainable. The Danish company Ordbogen A/S is also part of the team. The other project focuses on developing Danish language models.
- We cannot and do not want to prevent people from interacting with large language models, but we can at least work to ensure that the energy consumption doesn’t destroy the planet, says Lukas Galke.
Large language models are extremely energy-intensive to train. Training is crucial and necessary before a chatbot can begin providing somewhat reasonable responses to user queries.
- Training a language model can quickly cost hundreds of millions of dollars in electricity, and thousands of these models are being trained all the time. Some are widely used, some have a short peak of popularity lasting only a week, and others never gain traction, explains Lukas Galke.
”At least we can work to ensure that the energy consumption doesn't destroy the planet
Like running a car for 29 million kilometers
Take OpenAI’s GPT-4: Training it took 5–6 months, and its energy consumption reached about 50 gigawatt-hours. If converted into gasoline consumption for a car, this would be enough fuel to drive 29 million kilometers.
However, Galke points out that this is a one-time cost for training, meaning the energy per search decreases as more people use the model.
"But it's still a lot of energy, which is why we are working on reducing consumption. Our goal is not to reduce the energy cost of training but rather to train a model that consumes less energy during inference—that is, when a user interacts with the system."
What is a large language model?
It is a computer model that has read an extremely large number of texts and taught itself to generate new texts that can serve as answers to the questions we ask when using a chatbot like ChatGPT. LLMs use neural networks to learn and can be considered a form of artificial intelligence. However, an LLM is not more intelligent than the information it has acquired from the many texts it has read. This is why it is essential to train it on a wide variety of texts and continuously update it.
30 times less energy
A language model contains billions of parameters, each consisting of a number of bits that carry out calculations. 8–16 bits per parameter is common, but Galke believes this number can be significantly reduced—and fewer bits mean lower energy consumption.
- We want to minimize it to 1.58 bits per parameter, leaving models with 3 possible values for each parameter (-1, 0, 1). This would make operating a language model much cheaper, allowing language models to run on, for example, personal computers, or even mobile devices. This is particularly important in the context of the most recent trend of investing more computational resources at inference time to improve the results. That is, language models are made to explore and evaluate multiple responses (“think”) before giving their final response. What we are specifically aiming for in this project is to see if we can take a standard 8 or 16-bit pre-trained model and transform it into an 1.58-bit model after it has already been trained, which would save even more energy, compared to training 1.58-bit models from scratch.
If Lukas Galke and Peter Schneider-Kamp succeed in reducing energy consumption as much as they hope, a single search could end up costing 30 times less energy than today.
Meet the researcher
Lukas Galke is an Assistant Professor in the Data Science Group, Department of Mathematics and Computer Science.
New Danish Language Models
-
In 2024, the Danish Ministry of Digitalization allocated DKK 30.7 million to develop and apply Danish-language artificial intelligence, including Danish language models. The task will be carried out by researchers from Aarhus University, the University of Copenhagen, the Alexandra Institute, and the University of Southern Denmark (SDU). From SDU, Professor Peter Schneider-Kamp and Assistant Professor Lukas Galke from the Department of Mathematics and Computer Science will participate, with a particular focus on making the new Danish language models more sustainable. Read about the Danish investment here.