What a Local LLM Costs in Electricity and Waiting Time: Measured on One RTX 5090

"How much electricity will a local LLM use?" It is one of the first questions we hear from companies starting to look at a local LLM. The hardware price is easy to look up, but the running cost and the waiting time depend on how the model is used, so there is no general figure.

So we measured it on the single-GPU server that runs the AI features of projectAI. Four models answered the same 15 prompts while we recorded GPU power and waiting time. With Qwen3.8-27B, the model we use every day, an answer cost about ¥0.01-0.03 in electricity and arrived in full in 1.4-6.1 seconds. All figures were measured in-house on 23 October 2026.

This article covers how we measured, the results by model, how to set them against cloud AI pricing, and what to decide before electricity.

The answer: about ¥0.01-0.03 an answer, a few seconds of waiting

These are the results for Qwen3.8-27B, the model we use daily. The setting that makes the model write out its reasoning before answering (reasoning mode) was off, as it is for our everyday work.

The first characters arrived after 0.2-0.3 seconds in both cases. While answering, the GPU drew about 380-440 W, which came to about 0.24 Wh for a short question and 0.84 Wh for a long output. Even 10,000 answers cost about ¥75 for short questions and about ¥260 for long outputs.

How we measured

• Hardware: one GPU with 32 GB of memory (NVIDIA GeForce RTX 5090, 575 W power limit). The inference server was vLLM 0.24 (0.31 for Muse Glimmer only), and every model was a 4-bit version (quantized)
• Prompts: 10 short questions (rewording an email, classifying an inquiry, a discount calculation and so on) and 5 long outputs (summarizing meeting notes, translating into English, writing a procedure and so on). Each prompt ran three times, one request at a time, and we used the median
• Power: the GPU board's instant reading every 0.25 seconds, summed from sending the prompt until 10 seconds after the last character. Power for the rest of the PC (CPU, power supply and so on) is not included
• Electricity price: ¥31/kWh including tax, the reference rate of Japan's home appliance fair trade council
• The server is shared with our daily work. Runs that overlapped another request were left out of the totals

Reading GPU power needs care. nvidia-smi's standard value is the average over the last second, so an answer that finishes in 2-3 seconds shows up late. We used the instant reading.

The GPU does not rest the moment an answer ends

This is one long output (9.0 seconds). The GPU drew about 460 W while answering, kept running at about 90 W for some 2 seconds after the last character, and was back to a few watts of idle within 10 seconds. This tail matters more the shorter the answer. For Qwen3.8-27B's short questions, the answer itself took 0.15 Wh, and 0.24 Wh with the tail. For Qwen3.6-35B-A3B's short questions, the tail (about 0.09 Wh) was larger than the answer itself (about 0.05 Wh). The electricity figures in this article include the tail; when requests arrive back to back, the tail overlaps the next answer and the cost comes down accordingly.

At idle, with the model loaded, the GPU drew about 2 W. Keeping a model loaded costs almost nothing in GPU electricity.

Results: the part of the model that runs for each word sets the cost

We ran the same 15 prompts on three more models from our 10-model comparison. The figures are the time until the full answer and the electricity per 1,000 answers.

The two 27B models, from different developers, were almost identical in time and electricity. Both compute the whole model (about 27 billion numbers) for every word they write, and the GPU drew about 430-440 W while answering.

Qwen3.6-35B-A3B is 35B in total but runs only 3B for each word (a mixture-of-experts design). Its long outputs took about 40% of the time and about a third of the electricity, and the GPU stayed at about 320 W while answering. On our Japanese business evaluation, however, it scored 83.3 against Qwen3.8-27B's 89.8, and we do not use it daily. The lower electricity comes at the price of answer quality.

Muse Glimmer 30B writes its reasoning before answering even at the lowest setting, so the first character took about 2 seconds and it wrote about three times as many tokens for short questions. Its electricity was about 2.2 times Qwen3.8-27B's for short questions and about 1.4 times for long outputs. Its writing speed per second was not far from Qwen3.8-27B's; the gap is almost all reasoning.

Reasoning mode multiplies time and electricity by 3 to 10

With reasoning mode on, the same Qwen3.8-27B took 4.6 seconds and ¥19.9 per 1,000 answers for short questions: about 3.3 times the time and 2.7 times the electricity. Long outputs took 62.6 seconds and ¥259.6, about ten times (medians including the runs that produced no answer), and the first character took about 58 seconds to appear.

On top of that, 2 of the 5 long prompts used up the 8,192-token limit on reasoning alone and stopped after 110 seconds without an answer, in all three runs. Reasoning helps with hard judgments, but leaving it on for everyday work inflates both waiting time and electricity. We use it only for some features, such as reviewing plans.

Setting it against cloud AI pricing

Cloud AI charges by the amount of text read and written (tokens). Applying public list prices as of 23 October 2026 to the same token counts gives:

Per million tokens, small models cost $0.10 input and $0.50 output, mid-size $2 and $10, and top models $10 and $50; several providers publish prices in each band. We converted at ¥157 to the dollar (Bank of Japan, 17:00 on 22 October). Each model counts text differently, so this assumes the same token counts and is a rough guide.

On electricity alone, a local LLM costs about the same as a small cloud model and is more than ten times cheaper than a mid-size one. But this leaves out the GPU and PC purchase and the work of maintaining and running them. Those make up most of a local LLM's cost, and the fewer the answers, the more each answer carries.

Electricity is not what decides it

The electricity per answer stayed under one yen for every model. Whether to go local is decided by three other things.

• What must not leave the company. Whether some work handles information that cannot go to a cloud service, such as client source code or meeting notes
• Whether the wait is acceptable. Without reasoning it is a few seconds; with reasoning it can exceed a minute. It depends on whether a person waits at the screen or the work runs in the background
• Whether hardware and operations pay off over the number of answers. Estimate it together with which models each hardware tier can run

We also weigh these three per feature and use cloud AI and a local LLM side by side (local LLM case study).

FAQ

How much does a local LLM cost in electricity per month?

It depends on the number of answers. In our measurement, GPU electricity came to about ¥75-260 per 10,000 answers (Qwen3.8-27B, reasoning off). Power for the rest of the PC and for idle time comes on top.

Does a GPU cost electricity just by sitting there?

In this measurement, the GPU drew about 2 W at idle with the model loaded. We did not measure the whole PC, and the CPU and power supply add to that.

Is it cheaper than cloud AI?

On electricity alone, it is about the same as a small cloud model and more than ten times cheaper than a mid-size one. Once hardware and operations are included, the cloud can be cheaper while the number of answers is small.

How long is the wait?

With Qwen3.8-27B and reasoning off, the first character took 0.2-0.3 seconds and the full answer 1.4 seconds for short questions and 6.1 seconds for long outputs. With reasoning on, long outputs took over a minute.

Summary

Measured on one RTX 5090, a local LLM used about ¥0.01-0.03 of electricity per answer, with a wait of a few seconds. That is about the same as a small cloud model, so electricity does not decide adoption. What decides it is whether some information must stay in-house, whether the wait is acceptable, and whether hardware and operations pay off over the number of answers.

If you want to know whether a local LLM or cloud AI suits your work, talk to one of our project managers in a free estimate consultation. We will work out the hardware and cost outlook from your expected number of answers.

Related articles and pages

• Local LLM build and implementation: hardware to operations
• 10 local LLMs compared on Japanese business work: how to choose and delegate
• Muse Glimmer 30B vs Qwen3.8-27B on the same agent evaluation
• How far can a harness improve a local LLM agent?
• Local LLM hardware requirements and recommended models
• Local LLM case study: cloud AI or local, feature by feature

Sources

• Japan Fair Trade Council of Home Electric Appliances, "FAQ" (reference electricity rate ¥31/kWh, revised 22 July 2022; in Japanese)
• Bank of Japan, "Foreign Exchange Rates (daily)", 22 October 2026 (in Japanese)
• Anthropic, "Pricing" (read on 23 October 2026)
• OpenAI, "Pricing" (read on 23 October 2026)
• NVIDIA, "nvidia-smi documentation"
• Qwen, "Qwen3.8-27B" (Hugging Face)
• cyankiwi, "Qwen3.8-27B-AWQ-INT4" (the version measured)
• Qwen, "Qwen3.6-35B-A3B" (Hugging Face)
• QuantTrio, "Qwen3.6-35B-A3B-AWQ" (the version measured)
• Google, "gemma-3-27b-it" (Hugging Face)
• gaunernst, "gemma-3-27b-it-qat-autoawq" (the version measured)
• NVIDIA, "Muse-Glimmer-30B-NVFP4" (the version measured)
• vLLM, "vLLM" (GitHub)