Case Studies by Field
All Case Studies31
Cloud Services9
Core & Business Systems4
Construction DX2
E-commerce Development3
Inventory & Order Management4
AI Adoption11
DX Promotion8
New Ventures12
CASE STUDY
Running AI Without Data Leaving the Company: How We Choose Cloud AI Services or a Local LLM Feature by Feature

Introduction
We run the AI features of our business tool, projectAI, by routing each feature either to "cloud AI services such as ChatGPT and Claude" or to "a local LLM running in an environment we manage ourselves." This case study covers why we built the system, how we built it, how we choose between the two, and what we learned from running it. There are three key points.
- Where processing happens can be chosen for each AI feature, and a switch takes effect across the whole system in about 30 seconds
- When the local LLM is unavailable, work switches to a cloud AI service, and when a cloud AI service is unavailable, it switches to the local LLM, automatically
- Before handing a feature to local, we run it in parallel with a cloud AI service on the same input, and a person compares the results and makes the call
In this case study, "an environment we manage ourselves" means our own equipment and the cloud environments we contract for and manage. "Local LLM" refers to the AI running inside them. We use these terms in the sense of not sending data to third-party AI vendors.
projectAI's AI features as a whole are covered in our case study on running projectAI company-wide, and our support for building local LLMs in your environment on the Local LLM Build and Implementation Support page.
Why We Needed to Run AI in an Environment We Manage Ourselves
The Biggest Reason Was Cost
projectAI calls AI in many features, such as translation, sorting chat messages and organizing meeting minutes. Cloud AI services charge for every call. The more work is given to AI, the more the cost grows with it.
This cost becomes large for work that is repeated in high volume every day, such as translation. A local LLM costs money for equipment, power and maintenance, but there is no charge per call. Moving high-volume processing to local keeps costs from ballooning even as we use AI in more places.
Next Came Confidentiality: What the AI Reads Is Information Entrusted to Us by Clients
In projectAI, AI does its work by reading chats with clients, meeting minutes, incident reports, documents entrusted to us by clients, specifications and tasks. All of it is information our clients have entrusted to us.
Many cloud AI services for businesses offer contracts under which input is not used for training. We are not avoiding cloud AI services because they are dangerous. But information that contracts with clients or internal rules say must "not be sent to outside AI vendors" cannot be given to a cloud AI service, however convenient it would be. For work that handles such information, we had no choice but to give up on using AI at all.
This concern is not ours alone. In the Ministry of Internal Affairs and Communications' 2026 White Paper on Information and Communications in Japan, "security risks such as leaks of internal information" was the most commonly cited risk that Japanese companies are concerned about in using generative AI, at 46.4%.
We Cannot Propose What We Do Not Run Ourselves
The other reason is responsibility for what we propose. If we tell clients "we can also build a setup that keeps costs down and keeps data inside the company," we need to use that setup every day ourselves and know both its strengths and its difficulties. So we started by using a local LLM for real work in our own business tool.
How We Built It: Three Parts
The system is made up of three parts: the routing settings, the in-house processing machine, and a record of every call.

1. Routing Settings for Each Feature
For each AI feature, we choose one of the following four processing methods.
| Setting | Where it is processed | When it is used |
|---|---|---|
| Cloud | A cloud AI service | Processing that contains nothing confidential. Newly added features always start with this setting |
| Parallel | A cloud AI service returns the response, and the same input is also processed locally so the results can be compared | Features being considered for moving to local |
| Local | Local. When it is unavailable, work switches to a cloud AI service, and the reason is recorded | Features we have decided to move |
| Local only | Local only. Even when it is unavailable, nothing is sent to a cloud AI service | Features that handle information that cannot be given to a cloud AI service |
Only administrators can change the settings, and a history of changes is kept. Changes take effect across the whole system in about 30 seconds, with no need to stop the system. There is also a switch that sends everything back to cloud AI services at once.
2. An In-House Processing Machine with No Way In from Outside
The in-house processing machine does not wait to be called from outside; it goes and fetches the work itself.
The business tool simply places requests to be processed locally in a queue. The in-house processing machine connects outward, picks up requests from the queue, processes them, and writes the results back. We have built no entry point through which anything can reach the processing machine from outside.
The processing machine regularly sends a signal saying "I'm running." If that signal stops, or processing does not finish in time, work switches to a cloud AI service automatically. In the other direction, when a cloud AI service is unavailable because of an outage, work can switch to the local LLM automatically. Only features set to local only stop instead of being sent to a cloud AI service.
3. A Record of Every Call
Every time AI is called, we record where it was processed, whether it switched and why, the cost, and which project the call was for. The admin screen shows how many switches have happened recently.
Costs are shown in yen for each feature. Calls processed locally are recorded as 0 yen, since there is no per-call charge. Equipment, power and maintenance still cost money, however. We think about those fixed costs separately from the record.
How We Decide Between Cloud AI Services and a Local LLM, Feature by Feature
Four Criteria
- The confidentiality of the information given to the AI. Features that handle information that cannot leave the company move to local regardless of cost. We divide the information the AI reads into three levels and move the most confidential first.
- Call volume and cost. The more a feature is called each day, the more the cost falls when it moves to local. Because per-feature charges are shown in yen, we can decide where to start with numbers.
- The size of the input and the length of the output. Processing that reads long documents at once, or writes long text, takes time locally. We move such features later.
- How easy it is to check quality, and the impact of mistakes. The first thing we moved was translation. It is called often, its quality is easy to check by comparing it with the original, and mistakes are easy to fix.
| Confidentiality | Examples of information the AI reads | Policy |
|---|---|---|
| Highest | Chats with clients, meeting minutes, incident reports, documents entrusted to us | Move to local as a priority |
| High | Specifications, tasks | Move in turn while checking quality |
| Low | Supplementary information attached to work records | Can stay on cloud AI services |
Features that may switch to a cloud AI service when local is unavailable are set to "Local"; features that must not switch are set to "Local only."
How We Divide the Work Today
- Processed locally: translation, and linking task references in text to the actual tasks
- Compared in parallel runs: extracting decisions from meeting minutes
- Split by purpose: for the program-fixing work done by our AI agent "Mr.AI," we offer the option of fixing small bugs locally on projects whose code cannot leave the company. In our comparison, small fixes reached the same result as cloud AI services, but fixes spanning several features showed a tendency to skip tests, so adding features stays with cloud AI services
The local LLM is not a replacement for cloud AI services; each is a backup for the other. If one becomes unavailable, work switches automatically and keeps going.
Check in Parallel Runs Before Handing Over
Whether to hand a feature to local is decided by the results of parallel runs. The same input is processed by both a cloud AI service and the local LLM, and the two outputs are saved as a pair (deleted automatically after 7 days). On the admin screen, the two are shown side by side, a person judges them as "OK for local," "equivalent" or "worse," and the results are tallied per feature. Cases where the local model failed to produce an answer are counted separately so that failures are not buried in the totals.
In the white paper mentioned above, 29.1% of Japanese companies were regularly evaluating and verifying generative AI after introducing it. Not only for local LLMs, AI quality cannot be known without "a way to measure it after it is in place." Parallel runs build that into daily operations.
Strengths and Difficulties We Found in Operation
Strengths
- The cost of high-volume processing goes down. Moving work repeated every day, such as translation, to local removed the per-call charges for it.
- You do not have to move everything at once. Because features can be switched one at a time, without stopping anything, in about 30 seconds, we can move them little by little while checking. If something does not work, we can move it back with the same effort.
- Work does not stop when one side does. Because the local LLM and cloud AI services switch automatically, an outage on one side is less likely to stop our work.
- The records let us notice problems. Because the number of switches and their reasons are kept, we can check against the records any assumption that "it was supposed to be processed locally."
Difficulties
- Designing waiting times for long processing is hard. Translating long documents sometimes exceeded the waiting-time limit. And because work switches to a cloud AI service when the local wait exceeds the limit, there were cases where the same work was processed on both sides and charged twice. We are reviewing the waiting time and the amount handed over at once for each feature.
- Work was being processed by a cloud AI service without our noticing. A list handed over in one go exceeded the size limit, and it kept being processed by a cloud AI service instead of locally. We noticed because it was in the records, but without the habit of checking them, we would have missed it.
- New features tend to slip outside the routing. As we added features, some processing occasionally got in without going through the routing system. Each time we add a new feature, we check that it is covered by the routing.
- Keeping internal information out of view. Internal information about where something was processed was once shown on the client's screen, and we fixed it so that it appears only on internal screens.
- Owning equipment makes running it our job too. Stopping, updating and watching over the processing machine is our work. We can keep it going because there is a system that switches automatically when it stops.
Lessons for Companies Starting Out
- You do not need to move everything to local. Take stock of the information given to AI by confidentiality and of the calls by volume and cost, and move the features where the effect is largest first.
- Compare in parallel runs before moving. A person makes the judgment. Based on those judgments, switch the features that can safely be handed over.
- Decide in advance how to switch when one side is unavailable. Is it acceptable to switch to a cloud AI service, or should work stop and go to a person? Decide this for each feature.
- Record where every call was processed and why. Without records, you cannot confirm that costs really fell or that data is really being processed inside the company.
- Decide who will run the equipment. It is only ready for business use once you have decided how it behaves when it stops or is updated.
Through our Local LLM Build and Implementation Support, we design this approach to fit your work and how you handle information. You can start with a single feature.
Frequently Asked Questions
Is a local LLM less accurate than a cloud AI service? It depends on the feature. Processing that outputs in a fixed format, or that has short input, could be handed over, while long documents and decisions that span several features stay with cloud AI services. That is exactly why we check in parallel runs before switching.
Does no data at all ever leave the company? Processing for features set to "Local only" is never sent to third-party AI vendors. Features set to "Local" may switch to a cloud AI service when they cannot be processed locally, and in that case the reason is recorded. Which setting to use is decided feature by feature.
Does a local LLM cost nothing? There is no per-call charge, but equipment, power and maintenance cost money. The more a feature is called, the bigger the effect of moving it. How we think about costs is summarized on the Local LLM Build and Implementation Support page.
Is it better not to use cloud AI services such as ChatGPT and Claude? No. We keep using cloud AI services too. There are many situations where they are better suited, such as processing that contains nothing confidential or that handles long documents, and they also act as a backup for the local LLM. What matters is choosing feature by feature.
Can a small company, or a single department, get started? Yes. The basic approach is to start with one feature that is called often or handles highly confidential information, checking it in parallel runs as you go.
Related Case Studies and Services
- Local LLM Build and Implementation Support — Using generative AI without sending confidential data outside the company
- Case study: running the AI agent Mr.AI in-house — Taking change requests from specification to production
- Case study: our in-house tool cockpit — Running many projects and AI agents from one screen and a phone
- Case study: projectAI in company-wide use — 45 AI features, and how data and costs are handled
- Eight AI agent and generative AI case studies — Field page listing the AI built into daily business
- AI Implementation Support
Source: Ministry of Internal Affairs and Communications, "2026 White Paper on Information and Communications in Japan: Risks of Concern and the Status of Efforts to Address Them" (in Japanese), Figure I-2-1-8 "Risks of concern in using generative AI (by country)" and Figure I-2-1-9 "Status of efforts to address risks (by country, and by company size in Japan)"
FREE ESTIMATE
Free Estimates: Learn What to Build and the Cost Range on the First Call
Your requirements do not need to be settled. Tell us in a sentence what you want to build, and your project manager will sort it out with you.