AI Agent Case Study: Sorting Sales Pitches Out of Inquiries, So Only 3% Reach a Person

Most of what arrives through a company website's contact form is sales. Recruitment agencies, SEO offers, tool vendors. Because a genuine inquiry must never be missed, every message had to be opened, read and closed once it turned out to be a pitch. That check was eating into our CEO's time every day.

Since the summer of 2026 our own site has run a system in which AI judges each inquiry as genuine or sales and notifies the CEO only about the genuine ones. This article is that AI agent case study. It is not a roundup of other companies' cases but one system we built ourselves and that stuck, told from the starting point through the mechanism, the design decisions, the results and how a small company can do the same.

What an AI agent is, and how it differs from a chatbot

An AI agent works through several steps by itself within a defined role: looking things up, judging, acting. Unlike a chatbot that answers one question and stops, or a generative AI that returns text when asked, it is connected to business data and tools, looks at the result and chooses the next step.

Our inquiry-handling AI agent reads each message, judges it, and chooses by itself whether to notify the CEO, ask the sender a follow-up question, or just file the record. The role is small, but judging and acting inside the flow of work is exactly what makes it an agent.

The starting point: checking sales mail took time

Our contact form receives submissions every day. Almost all of them are recruitment and engineer placement offers, SEO and marketing proposals, tool and system pitches, and subsidy consultants.

The problem was not the volume itself but the fact that a genuine inquiry could not be missed. A development request, an interview request or a job application arrives through the same form with the same notification as a sales pitch. So each one had to be opened and read, and it took a few minutes per message to be sure it was sales. The notifications could not be switched off, and most of them were sales. We considered keyword filters, but every pitch is worded differently and the same words appear in genuine inquiries. The text had to be read for intent, and that is work suited to AI.

How it works: the inquiry AI agent only judges, people write the replies

What we built is a judge that reads the text of an inquiry and sorts it into four labels. The judging runs on the AI platform of our project management tool, projectAI; the website never calls an AI directly.

• Genuine. Development or consulting requests, estimate requests, interview and speaking requests, and applications from job seekers themselves.
• Sales. Anything offering, proposing or pitching something to us: recruitment agencies, SEO and advertising proposals, tool pitches, subsidy consultants. "Are you looking for engineers?" or "We can rank your site higher" count as sales even when they look like hiring or consulting.
• Spam. Meaningless strings, obvious spam, phishing. Bot submissions whose name and body are random letters get this label without calling the AI at all.
• Unclear. A greeting alone, no stated purpose, anything where genuine or sales cannot be decided.

What happens after the verdict is a fixed procedure decided by people. Genuine and unclear inquiries notify the CEO and the sender receives an auto-reply confirming receipt. Sales and spam are filed with no notification and no auto-reply. In the chat, an unclear message gets up to two follow-up questions from the AI, and if the purpose is still unclear the conversation is handed to the CEO. Every reply is a template written by a person. The AI never writes and sends text.

The design priority was never to drop a genuine inquiry. If the AI's verdict does not arrive within 15 seconds, if the call fails, or if the verdict is unclear, the inquiry is treated as genuine and notified. Job applications are never judged by AI and always go to a person. Each verdict is recorded with a one-line reason, and inquiries judged as sales stay in the list, so a person can review them later and reopen one. The criteria themselves are written in plain Japanese, so when the pattern of pitches shifts, editing that text is all it takes.

Only the first 1,500 characters are handed to the AI. The model can be switched between a cloud AI (Claude) and a local LLM running on our own server from projectAI's settings screen. Either way, the cost and the result of every call are logged.

The results: only 3% of inquiries reach a person

Here are the verdicts for the inquiries received over about two and a half months (mid August to late October 2026).

97% were sales or spam and were never opened by a person. 3% reached the CEO. It is often said that 80 to 90% of contact-form submissions are sales or spam, and our share sat in that range. No submission in this period was judged genuine; the genuine inquiries arrive through other doors, such as the free estimate consultation and the chat.

Notifications fell by the same share. Each check takes only a few minutes, but it was time taken every day, as an interruption, every time a notification rang. The larger change is that a notification now most likely means a real inquiry.

No misjudgement was found in this period. The 2% that timed out fell on the notify side, so nothing was lost.

Why it stuck: what matters when introducing an AI agent

The system has kept running without a pause for reasons that lie in how it was built.

• It sits inside the flow of work. The verdict runs right after a form or chat message is sent, and people only receive the result. Nobody opens an AI screen and pastes anything in.
• The AI's job is limited to judging. It returns one of four labels and a reason, and never writes a reply. If it is wrong, what reaches the sender is still a template written by a person.
• Doubt goes to a person. Unclear, timed out and failed all land on the notify side. Because the design does not over-trust the AI, we could cut notifications with confidence.
• Records are kept. Verdicts and reasons stay in the list, inquiries judged as sales can be reviewed later, and the cost of every call is logged.
• Where the data goes is decided. Judging runs on our own AI platform and can be switched to a local LLM; only 1,500 characters are handed over.

Our other uses of AI follow the same thinking: an assistant that answers stock questions in a core system, reading and sorting of bank passbooks and site photos, grading of personal statements, and the AI agent "Mr.AI" that implements change requests. They are collected in our AI agent and generative AI case studies.

How a small company can introduce an AI agent

Automatic sorting of inquiries is a good first step for a small company bringing an AI agent into daily work. It takes time, the right answer is easy to check, and an escape route for mistakes is easy to build. There are four steps.

1. Put the definition of "genuine" into words. Write down, with examples, which messages a person should read. That text becomes the judging criteria as it is.
2. Send doubt to a person. Keep unclear and failed on the notify side, and never build a path where the AI silently discards anything.
3. Let people write the replies. Make auto-replies and refusals templates, and leave only the judging to the AI.
4. Record the numbers. Keep the breakdown of verdicts and the cost, and review them once a month.

The same shape fits an email inbox, a contact form or a chat, and it can be added to an existing business system later. On our own site, the chat verdict went live first and the form followed a month later.

FAQ

How does an AI agent differ from generative AI, RPA and a chatbot?

Generative AI produces text or images, RPA repeats a fixed procedure, and a chatbot answers questions. An AI agent judges within its role and chooses the next step itself. In this case, that is the part that decides whether to notify a person or ask a follow-up question.

Could the AI throw away a genuine inquiry by mistake?

There is no path where the AI throws anything away. Inquiries judged as sales stay on record and can be reviewed later. Unclear, timed out and failed are notified as genuine.

Is it rude to the salespeople?

Messages judged as sales receive no auto-reply and are simply filed. In the chat, a message written by a person explains that we do not take individual proposals and asks that development requests come through the contact form.

Is it safe to hand inquiry text to an AI?

Judging runs on our own AI platform and only the first 1,500 characters are passed on. If data must not leave the company, the model can be switched to a local LLM running on your own server.

What does it cost?

Each verdict reads one short text once, so the AI usage fee is tiny. The development cost depends on how many entry points are covered and how notifications work, so we give a figure after hearing about your work in a free estimate consultation.

Can a small company introduce this, and how long does it take?

Yes. On our own site the chat verdict went live first and the form followed a month later. With one entry point, going from writing the criteria to running in production takes a few weeks as a guide.

Can it be used on our own inbox?

Yes. Incoming email can be judged the same way as forms and chat, notifying the person in charge only about genuine messages. The criteria are written in words to fit your business.

Summary

In this AI agent case study, handing the sorting of inquiries to AI brought the share a person reads down to 3%. What made it work was the design: give the AI only the judging, send doubt to a person, and keep records. AI takes root not through a large system but through one case that removes a small daily interruption.

We take on everything from mapping where AI should work, through trials and integration into your business systems, to operation (AI implementation support). Even if you are still asking which of your tasks to start with, talk to one of our project managers in a free estimate consultation. You can also reach us through the contact form.

Related articles and pages

• AI agent and generative AI case studies: our 8 cases by industry, AI and key point
• AI implementation support: from trial to production and operation
• Local LLM deployment support: generative AI that keeps confidential data in-house
• What it means to delegate work to AI: what we learned running Mr.AI in-house
• Common AI adoption failures and the consultations we actually receive
• projectAI company-wide case study: an internal business platform with AI built in