The bot captures, the MCP consumes
How the store finds out what is happening in its chats. There is no dashboard: there is a server it queries from its own Claude, and a report every morning. And a rule about what is the bot's job and what is not.
One question is left, and it is the one the owner cares about most: how they find out what is happening in their chats. The bot handles around six thousand conversations a month, and each one says something about what people want. Nobody has time to read them. The short answer is that I did not build a dashboard: the store asks its own Claude. The same server that runs the bot exposes an MCP, and Claude queries it on the store’s behalf.
Behind that sits a split of work I only understood after tripping twice: the bot captures, the MCP consumes.
A question from the store’s Claude
The owner opens Claude, the same one they use for everything else, and types: “what did people ask for yesterday that we did not have?”. What comes back looks like this, reconstructed and with made-up data:
Yesterday, requests with no stock: nine.
Four for the same model in 256 GB, which is one you restock. Two left a phone number for the waiting list.
Three for a model you do not carry.
Two for a color that did not come in.
To do today: call the two on the list and restock the 256.
There is no screen to learn and no filter to configure. If the answer raises another question, “and which ad did they come from?”, it gets answered in the same chat.
Underneath, Claude called tools on the MCP. Today there are seventeen, and sixteen of them only read. They count and list conversations, fetch the messages of a chat, show what the bot did on each turn, and read the release notes I covered in the previous post. The only one that writes files a report when the bot got something wrong.
The report every morning
One-off questions are useful, but the store wants the same thing every morning. For that there is a skill installed in its Claude. Its only source is the MCP: it cannot see the code or the database. The report brings four things:
- Demand the store could not meet. What people asked for and was not there, split by whether it can be restocked or not.
- Waiting list. Who left their details to get a call when the device comes in.
- Reservation funnel. How many reached the terms, how many reached the form, and how many were completed.
- Source. How many conversations each ad, story or post brought in.
Reading every chat of the day to build that would cost a fortune, so the report reads few. First it asks for two indexes and crosses them. One says what the bot did in each chat. The other says which chats contain certain phrases. Only the chats that either one points at get read in full.
The last stretch repeats one of the bot’s decisions. The model reads the chats that were pointed at and notes each case. A script computes the numbers from those notes, and always fills the same template. If a number in the report does not come from a noted case, it does not exist. It is the rule from the map of the series applied again: no figure travels through the model.
The phone numbers on the waiting list are personal data. That is why the report is a private file that belongs to the store, and sharing it is their call.
What went wrong: guessing what the bot already knew
The first version of the report had a single index, the one with the phrases. It asked Chatwoot’s search for the chats where the bot had said “we do not have it” and its variants, and then read them.
The search looks for loose words and drops the most common ones. The Spanish for “we do not have it” ended up as a search for “have”. I measured it on September 7: of 203 results, only 115 contained the phrase.
The underlying mistake was something else. I was guessing, from the text, something the bot already knew exactly. When the bot says “we do not have it”, it is because a moment earlier it searched the catalog and the search came back empty. That is recorded in the turn’s trace.
The other index shipped that same day: a tool that shows those traces as events. For each conversation it says which tools the bot called, how many searches came back empty, why the chat was handed to a person, and what stage the reservation reached. It calls no model and summarizes nothing. Reading one day of chats was around seven hundred thousand tokens. The events for the same day are between ten and twenty thousand.
The second stumble: still searching
Events do not see everything. A customer who arrives saying they saw a different price posted on Instagram is customer text, and no tool of the bot records it. They also miss what happens after the handoff, when a salesperson closes the reservation by hand. For that I kept phrase search alive, as a complement.
Two weeks later I looked at where the time was going. In the fourteen days ending September 22, that search alone took 53% of the time of the entire MCP. The slowest call took 47 seconds. Nothing was badly written. The search returns fifteen results per page, each page takes about two seconds, and the report was searching for more than twenty phrases.
I had fixed the data and kept the habit. The fix was to stop searching. Now the server fetches the text of each of the day’s chats and checks which phrases appear, exactly as written. A full business day takes about eleven seconds, and a phrase no longer gets mistaken for a loose word.
The boundary
Those two stumbles are where the rule in the title came from. The bot captures: while it serves customers, it leaves ordered data about what it did. The MCP consumes: anything that is a report, a count or a question gets built afterwards, by reading.
The rule gets tested when a request comes in. The store wanted the bot to label every conversation with the stage of the flow it was in. I said no. It meant putting one client’s classification into the prompt, and handing the model an extra task it only completes some of the time. The same information comes out of the events, complete and without touching the bot.
It has a limit, and it is worth saying. The MCP can only count what somebody captured, and the sale is not in any data. The report knows a reservation was closed, not that the device was paid for, so it does not report conversion to sale. I do not know yet whether the line is in the right place. If the traces ever stop being enough, the fix will be for the bot to capture better, not for the report to guess more.
Seven posts later, the whole system fits in one sentence: the model writes, the code decides, and the store can ask about everything that got recorded.