Jev for Brand Mention Tracking: How We Found Our AI Visibility Numbers Were 6% Low

Counting how often AI engines name a brand has become the main number behind any AI visibility report, and we use Jev - a small AI model from TypeSafe - to decide whether a given answer names a brand or not. When we tested it on 3,566 answers across ten brands it was right every single time it said yes. However our counts were still missing about 6% of real mentions, and the cause turned out to be the question we were asking it - a gap that any AI visibility tool can have.

Key takeaways

• Jev was right on 140 of 140 sampled "named" verdicts, checked by a reader who could not see its answers (October 2026, ten brands, five AI engines).
• About 6% of real mentions were recorded as absent. Nearly all of them were answers that used a short form of a name - "Acme" for a brand stored as "Acme Retail".
• One brand's own name was hit hardest. In 117 answers that used its short name our tool counted 38.
• Rewording one question recovered all 74 missed mentions in the sample, with two new errors.
• You can check your own tool in 15 minutes with 20 AI answers and a list of the ways your name gets written.

About the data: all 3,566 answers in this test come from client and prospect accounts in our AI visibility product. They are anonymised - no brand is named, and the one passage quoted has its brand, product and industry changed. Other posts in this series also use queries from our own site, tenpointlabs.com.

What is AI brand mention tracking?

AI brand mention tracking counts how often AI engines such as ChatGPT, Perplexity and Google's AI answers name your company when they are asked the questions that your buyers ask. It is the main number behind any AI visibility report and getting it right is a good deal harder than it sounds.

A simple string search fails in two directions at once. It will count "a sunset clause" as a mention of a company called Sunset, and it will miss your brand whenever an engine writes the name differently from the way you stored it. For that reason most tools - ours included - put a model between the search and the count. We have covered the difference between being cited and being named in what AI citations and mentions measure and the wider method in how to measure AI share of voice.

What did we find?

Our product asks Jev one yes-or-no question for every tracked name that code finds in an answer. Jev returns a probability and we count 0.8 or higher as named and 0.2 or lower as absent, with anything that falls in between going to a larger model for a second opinion.

For this test we re-ran all 3,566 answers that our product holds for ten brands, which came to 2,925 name checks, and had 300 of those labelled by a reader who saw only the passage and the name. I then checked 30 of them myself and agreed with the reader on 26.

Table 1. Jev's verdicts on 300 hand-labelled name checks, October 2026

The pattern in the errors only became clear once we sorted the rows by how the name had appeared in the passage.

Table 2. All 2,925 name checks, by the form of the name in the answer

When an answer used the stored name Jev was almost never wrong, but when it used a short form Jev said no more often than it said yes.

Why did a correct model give wrong counts?

The search step in our product knows about aliases, which is how it finds "Acme" when the brand is stored as "Acme Retail". The question that we then sent to Jev did not carry that knowledge with it, because it asked whether "Acme Retail" was used as the company's name in the passage. Those exact words were not there and Jev answered the question it had been given.

According to TypeSafe's own documentation the model "answers the question you wrote, not the one you meant", and the effect is easy to see on an example. The passage below is a real one in which the brand, product and industry have been swapped so that the client can't be identified, and both scores come from running the altered text through Jev.

"Ledgerline can help coordinate the month-end close and the required reconciliations, but it is not a CPA or audit firm…"

The second opinion did not save us either. A confident no never reaches the larger model at all, and on the cases that Jev was unsure about the larger model sided with "absent" on 24 of the 59 real mentions.

What does this mean for your AI visibility numbers?

If your tool stores your brand as "Acme Retail" and the engines simply call you "Acme" then your reported visibility may well be lower than the truth. The same gap applies to your competitors, which means that a rival the engines refer to by a short name may look weaker against you than it really is.

Three kinds of brand are the most exposed to this:

• Names with a suffix such as AI, Labs, HQ, Health or Software, which engines often drop.
• Names that are also ordinary words, where the tool has to be strict and can become too strict.
• Names shared with another company, where a loose rule counts the wrong business.

Jev handled the last two of those well in our test. It correctly rejected an unrelated studio and a consumer product that shared one brand's name, along with an ordinary word that happened to match another.

How can you check your own tool in 15 minutes?

1. List every way your name gets written. Include the short form, the form without its suffix, common misspellings and your main product's name.
2. Collect 20 AI answers to questions your buyers ask, from the engines your tool tracks.
3. Search each answer by hand for every form on your list, and note which answers name you.
4. Compare your count with what your tool reports for the same questions.
5. Ask your vendor two questions: how do you match short forms of a name, and how do you tell my company from another with the same name?

If the count you reach by hand is the higher of the two then the gap is most likely in the way your tool handles name forms.

How did we fix it?

We changed the question so that it now asks about the form found in the text and gives Jev the stored name as context. On the same 300 labelled checks:

• All 74 missed mentions came back as named.
• 132 of the 140 correct yeses stayed yes, and 8 moved to unsure, where the larger model picks them up.
• Two passages about a different company with the same short name were wrongly counted, which the old wording had rejected.

That is a better trade than the one we had before but it is still a trade. Loosening the question recovered the real mentions and let two wrong ones through, and so the second opinion and a human spot-check are likely to remain part of the system for some time. The wider lessons on wording are in how to write Jev questions for SEO.

How we tested it

• Data: every answer our product held for ten brands on 9 October 2026, from five AI engines. The brands are clients and prospects in software, consumer goods, health and professional services, and none is named here.
• Model: jev-1.13.0, one request per answer, reading only the paragraphs that contain a candidate name.
• Labels: 300 name checks labelled by a reader that saw the passage and the name and nothing else. That reader was a larger AI model working blind. I labelled 30 myself and agreed on 26.
• Sampling: the 300 over-represent Jev's "absent" and "unsure" verdicts, so the tables above are counts within each row and should not be averaged.
• Stability: re-sending 300 requests changed 4 of 598 verdicts, all between "named" and "unsure". Jev does not return identical probabilities for identical requests.
• Speed: median response 408 ms, with no failed requests.

Two smaller findings. In 60 answers where the search found no tracked name, 2 did name a tracked company. One name contained a dot and the search missed it, and one was written in a short form that was missing from its alias list.

Limits. One reader labelled the sample and it is small for any single brand. The 6% figure is an estimate scaled from the sample and it leaves out a group of cases where an answer used a product's former name.

FAQs

See your own numbers

Our Free AI Visibility Audit asks the major AI engines your buyers' questions and shows how often they name you and who they name instead. As more buyers start their research inside an AI answer this count is likely to become one of the numbers a marketing team watches most closely.

This post is part of a series: Jev for SEO: what worked, what failed, and what we measured.