Article Categories

All Articles

Soft Skills

Organizational Culture

Technology

BLOG

10 Local LLMs Compared on Japanese Business Work: How to Choose and Delegate

October 20, 2026

Technology

"Which local LLM should we use for Japanese business work?" It is one of the questions we hear most often in local LLM consultations.

We run the AI features of projectAI, which handles clients' source code and meeting notes, on our own GPU server instead of sending data to a cloud AI. From July to October 2026 we evaluated 10 models that run on a single GPU, all with the same tasks modelled on our own work.

As of October 2026 the model in daily use is Alibaba's Qwen3.8-27B (89.8 points). Picking by score alone does not work, though. Japanese tuning, recency and speed each failed to decide the choice, and the weak spots every model keeps are covered by checks. All figures were measured in our own environment.

The three gates between 10 models and Qwen3.8-27B, the model in daily use (fits the GPU, follows instructions, the key task), with the examples that failed at each gate

The question is not "can it solve this?" but "can we hand this over?"

Public benchmarks help narrow down candidates, but most use English tasks and measure whether a model can solve a problem. For business work we need to know other things: whether it writes polite Japanese fit for clients, returns the agreed format (JSON) and choices, avoids inventing what the text does not say, and behaves the same way every time.

So we built 10 tasks from work projectAI already gives to AI: a polite email, translation that keeps set terms and placeholders, classifying and extracting from chat messages, summarising, strict formats such as "exactly three lines", rewriting a rude message politely, handling a false premise, and a multi-step calculation.

How we evaluated

The hardware was one GPU with 32 GB of memory (NVIDIA GeForce RTX 5090). The inference server was vLLM, and the models were 4-bit versions (quantised to make them lighter). The 8-bit version of Qwen2.5-Coder-32B did not fit in GPU memory, spilled into main memory and ran about 28 times slower. To run a model of around 30B on a 32 GB GPU, 4-bit is a given.

Scoring: 100 points from accuracy 30, natural Japanese 25, following instructions 15, format 15, stability 10 and efficiency 5. An invented fact capped accuracy at 3 out of 10
Repetition: the four tasks whose results varied most were run five times each
Other evaluations: the ability to call system functions (tools) on the public BFCL benchmark, and agent ability on Terminal-Bench and our own tasks

Result: Qwen3.8-27B, at 89.8 points, is in daily use

Scores on the Japanese business test. Qwen3.8-27B is highest at 89.8, Qwen3.6-35B-A3B scores 83.3, Qwen3.6-27B 79.4, and the Japanese-tuned Qwen3 Swallow 32B 63.0

The tenth model, Meta's Muse Glimmer 30B (evaluated in October), was compared on the agent evaluation only. It passed the same number of tasks as Qwen3.8-27B and took about 2.2 times as long per task on coding and about 1.2 times on general tasks (comparison article). Results vary from run to run, so a gap of a few points is within the margin of error.

Japanese tuning does not guarantee strength on Japanese work

Qwen3 Swallow 32B is Qwen3-32B with additional Japanese training. It has the same size and the same quantisation, yet it scored 63.0 against 74.0.

It scored 95.2 on classifying chat messages, among the best of all models
On translation into English, it returned the Japanese text with only the set terms swapped
On the false-premise task, it invented a fact: that Hirobumi Ito founded NTT

The Japanese-developed LLM-jp-4, with reasoning turned on, was the best of July on the polite rewrite and the calculation, yet it also returned Japanese where English was asked for. Being strong in Japanese and keeping to the requested language and format are separate abilities.

A newer version can lose to an older one, depending on the task

In July the top model was the previous generation, Qwen2.5-32B (76.6). The newer Qwen3-32B (74.0) classified a "production outage" message as a "request" and filled a due date with a date that was not in the text. It failed the same way on all five runs.

Qwen2.5-32B, on the other hand, named Shigeru Yoshida as Japan's first prime minister on all five runs (the answer is Hirobumi Ito). Which model is better switches with the task you give it. Failures are a trait of the model, not chance, so a single try misleads.

A higher score and more speed don't win if the key task gets worse

Qwen3.6-35B-A3B scored 83.3, the best at the time. It is 35B in total but runs only 3B per step (a design called MoE), and it answered in 0.78 seconds on average, less than half the time of Qwen3.6-27B, our model in daily use then.

We still did not switch. On BFCL's multi-turn category it scored 50.9% against the 27B's 56.0%. Mr.AI, projectAI's AI agent, calls system functions many times as it works. What decides a switch is the score on the work that matters most to you, not the overall score. Decide that before you compare.

The next generation fixed the weak tasks

Qwen3.8-27B, released in August, succeeds Qwen3.6-27B. Under the same conditions its score rose from 79.4 to 89.8. The gains came on the tasks the older model was weak at.

Task	Qwen3.6-27B	Qwen3.8-27B
Rewriting a rude message	62	80
Handling a false premise	58	90
Multi-step calculation	42	88

BFCL multi-turn also rose from 56.0% to 59.0%, and speed stayed about the same. We switched our daily model the day we evaluated it, with a rollback procedure ready. The service was down for about three and a half minutes.

Reasoning mode is not a cure-all

We also tried the setting that writes out its thinking before answering (reasoning mode) on Qwen3.6-27B. On 40 hard questions it answered 35 correctly without reasoning and 36 with it, and took about five times as long per question (6.1 and 29.3 seconds). With reasoning on, it named a magazine that does not exist as the one that serialised "I Am a Cat", a question it got right without reasoning. We run daily work without reasoning.

Checks cover the weak spots that remain

Some weak spots tend to remain whichever model you use.

Weak spot	How we cover it
Missing an exact character count	Count the output in code and retry if it fails
Inventing unfamiliar names	Answer from supplied documents and check the source
Other languages mixed into Japanese	Check the output before users see it
Calling a function it should not	Decide by rule whether an action may run

Character counts are hard because models handle text in units called tokens, not characters.

Agents hit a ceiling too. On Terminal-Bench, Qwen3.6-27B in our own agent stayed at 46.2–48.7% after repeated improvements, short of the standard agent's 51.3%, and most remaining failures came from the model's ability. With Qwen3.8-27B it now passes all 25 of our own tasks. On past development requests it fixed small changes on the same lines, in the same way, as the cloud AI (Claude), but on a larger bug it stopped at the smallest change that made the symptom disappear.

How we delegate (as of October 2026)

Hand to the local LLM: translation, summaries, classifying messages, extraction with evidence, single function calls, small well-scoped fixes
Hand over with checks: due-date extraction (matched against the text), polite emails (stating who does what), formats such as character counts (verified in code)
Keep a person in the loop: unfamiliar names and legal details, larger development changes

How we decide what to delegate is in Delegating work to AI, and how we switch between cloud and local per feature is in our local LLM case study.

Limits of this evaluation

The task set is small, and every model was a 4-bit version. LLM-jp-4 ran on CPU (llama.cpp) because the GPU was in use, so its speed was not compared
Each result is for the version available at the time. For LLM-jp-4, the successor LLM-jp-4.1 was released in September 2026
The results reflect our work and our environment, not the overall merit of each model

FAQ

Which local LLM suits Japanese business work?

In our evaluation, Qwen3.8-27B scored highest as of October 2026 at 89.8, and it runs on one 32 GB GPU. The right model depends on the work, so test candidates on tasks from your own business first.

Is a Japanese-tuned model the safe choice?

Not necessarily. Such models were strong at classification and polite Japanese but sometimes returned Japanese where English was asked for. For translation and work that feeds other systems, check that the model keeps to the requested language and format.

Should reasoning mode be on?

It depends on the work. On our 40 hard questions it added one correct answer and took about five times as long. We run daily work with it off.

Should we switch every time a new model comes out?

Switch only when the new model beats your current one on the same evaluation. We evaluated 10 models and switched once, in August.

Where do we start with our own evaluation?

Write about 10 tasks from work you want to hand to AI and decide the correct answer for each. Run unstable tasks five times, and penalise invented facts most heavily.

Summary

Comparing 10 models on the same tasks showed that you should choose by the key task and by whether a model keeps to the requested language and format, not by Japanese tuning, recency or speed. If checks in code cover the weak spots every model keeps, a single GPU can take on a great deal of work.

We build evaluation tasks from your work and compare candidate models and hardware before you buy anything (how adoption works). If you want to know which model fits your work, ask a project manager through our free estimate consultation. Questions as early as "should we switch our current model?" are welcome.

Development Consultations and Estimates

As your co-creation partner, we drive business development and DX.

We start with a call with your project manager to confirm what to build and the expected cost range.