Jev for Query Fan-Out: Why You Can't Tell AI Queries From Human Ones in Search Console

Query fan-out is what happens when an AI assistant sends its own searches to Google while it researches an answer for someone. Those searches now sit in Search Console next to the ones that people typed, and we tried a regex and two different Jev designs on about 22,000 queries to pull the two apart. Each method flagged somewhere between 10% and 29% of queries, and two careful readers then disagreed on most of the rows they checked - which means that none of the methods can honestly be called accurate.

Key takeaways

• Depending on the method, 10% to 29% of queries on five B2B sites were flagged as machine-sent (14,902 queries, April to September 2026).
• By site, a regex flagged between 17% and 37%. The share varies a great deal from one site to the next.
• Two readers disagreed on 24 of 30 queries. One read a long, well-formed question as an assistant's search and the other read it as a person typing into AI Mode.
• Asking Jev "did an AI write this?" failed. Only 5 of 7,157 queries got a confident yes.
• Asking Jev about things visible in the text worked and gave a description of each query that you can filter on.

About the data: this test covers about 25,000 Search Console queries. Our own site, tenpointlabs.com, supplies 2,944 of them and every query quoted here. These are the queries Search Console reports for web search between 9 June 2025 and 9 October 2026. They are a sample of our traffic and not all of it, because Search Console leaves out anonymised queries, and no GA4 data is used. The other 22,059 come from six client sites, which are anonymised - described by industry, never named and never quoted.

What is query fan-out?

Query fan-out is the set of searches that an AI system runs on your behalf. When someone asks ChatGPT, Perplexity or Google's AI Mode a question the system breaks that question into several searches, reads what comes back and writes a single answer. Those searches can then show up in your Search Console data right alongside the ones that people typed themselves.

The reason they matter is that nobody ever saw a results page. A fan-out query can add impressions to a report - and even a high average position - with no person behind it and no chance of a click. We first wrote about these queries in August, in what conversational search queries in Search Console tell us.

What did we find?

To see how far the methods differ we ran three of them over a random sample of up to 3,000 queries from each of five B2B software sites.

Table 1. Share of queries each method flagged as machine-sent, five B2B sites, April to September 2026 (14,902 queries)

The regex and the last method in the table agreed on 91.6% of queries. Our hope had been that where the two differed a blind reader would be able to tell us which one was right.

Why can't anyone say which method is right?

The simple answer is that the readers could not agree with each other. A larger AI model labelled 500 queries without seeing any method's answer and was unsure on 35 to 56 of every 100. I then labelled 30 of the same queries myself and we matched on just 6.

Nearly all of that gap came from a single kind of query. I marked 18 of the 30 as typed by a person, and the other reader had called 15 of those machine-sent. They were full questions of the sort that a person now types into AI Mode and that an assistant also writes when it restates a request. These two both reached our own site:

• "how can brands earn more citations in chatgpt and perplexity?"
• "what makes content more likely to be cited by ai tools"

There is nothing in either one that tells you who sent it. Some queries do give themselves away however, and this one reached our site as well:

"context: location: united states (not for language). do not include location references in your response. question: best citation analysis tools for b2b brands"

No person types that into a search box. But between a query like that and a two-word keyword there is a very wide band in which the text carries no proof either way.

What does this mean for your Search Console numbers?

If a quarter of your queries may be machine-sent and there is no way of telling which quarter then three habits are worth adopting.

• Stop reading impression growth as demand until you have looked at what kind of queries actually grew.
• Report a range. "Between 10% and 29% of our queries look machine-sent, depending on how strictly we count" is a true sentence, where any single percentage is a guess.
• Judge content on the queries that earn clicks. A page that ranks for forty long questions and gets no clicks has told you very little about your buyers.

Anyone who quotes you an exact share of AI queries in Search Console is really describing the rule they used, and that rule is a choice.

How can you check your own site in 15 minutes?

1. Open Search Console's performance report and set the range to the past six months.
2. Filter queries with this custom regex to see those of ten or more words: ^(\S+\s+){9,}\S+$
3. Note their share of impressions and of clicks. A large share of impressions with almost no clicks is the pattern to look for.
4. Read the top 50 by impressions. Count how many no person would type.
5. Repeat with ^best |^top and look for queries in the top ten positions with no clicks at all.

On our own site 693 of 2,944 queries had ten or more words, which is 24% of the queries Search Console reports for us.

How did Jev do?

Jev did badly when it was asked the obvious question and proved useful once it was asked a better one.

The first design asked whether an AI assistant had written the query. Authorship is simply absent from the text and so Jev's answers sat near the middle. On the first site only 5 of 7,157 queries reached 0.8 and Jev flagged just 16% of what the regex flagged.

The second design asked eight separate yes-or-no questions about things that anyone can see in the query:

1. Is it a complete, grammatical sentence or question?
2. Does it tell the reader to research or assess something?
3. Does it state three or more separate conditions?
4. Does it describe the asker's own situation?
5. Does it read like a filled-in template?
6. Does it contain search-engine syntax?
7. Is it written in clipped keyword style?
8. Does it ask for the best or top option in a category?

Code then weighs the eight answers and is able to add the facts that Jev never sees - whether the query earned a click and where it ranked. Here is what those answers look like on three queries from our own site.

Table 2. Jev's answers on three queries from tenpointlabs.com (probability of yes)

This is the use of Jev that we would recommend. It gives you a description of every query that you can filter and count, and it leaves open the question of who sent it - which is exactly where the evidence leaves that question too. The eight questions and how they were written are covered in how to write Jev questions for SEO.

How we tested it

• Data: 7,157 queries from one legal-services software site, then a random 3,000 from each of five other B2B sites (2,902 from the smallest). Our own site's 2,944 queries supply the printed examples.
• Model: jev-1.13.0. The designs were fixed on the first site and run unchanged on the five others.
• The regex flags queries that are quoted, phrased as instructions, twelve or more words long, or "best"-style queries shown with no clicks. Any query that earned a click counts as human.
• Labels: 500 queries labelled blind by a larger AI model, 100 per site, and 30 labelled by me. We agreed on 6 of 30, so we report no accuracy figure for any method.
• Stability: re-sending 300 queries changed 1 to 3 final decisions, all on rows close to the cut-off.
• Cost: the eight-question design used about 2.4 times the tokens of the one-question design.

FAQs

See how AI engines answer your buyers

Fan-out queries are the trail that AI engines leave while writing answers. Our Free AI Visibility Audit shows the answers themselves - which companies the major AI engines name when your buyers ask. How search engines choose to report these queries will be an interesting story to keep an eye on.

This post is part of a series: Jev for SEO: what worked, what failed, and what we measured.