Five Instruments Measured AI Search Traffic. None of Them Agree

Five instruments, five answers, one week#

In seven days, four separate attempts to measure ai search traffic landed in public, and not one of them lines up with the others. A practitioner's nine-month GA4 study says roughly a fifth of AI Overview arrivals are being filed as Direct. A vendor's telemetry says Reddit fell off a citation cliff inside ChatGPT. A preregistered field experiment says AI Mode cuts click-through by 18.8 percentage points. Google says total click volume is broadly stable and calls the outside methods flawed. Four instruments, four verdicts, one week.

Add the two older instruments everyone quotes and the fog thickens rather than lifts. The Reuters Institute's Digital News Report 2026 asked people what they do: 42% of those using AI chatbots for news say they always or often click through to the original source. Pew Research Center watched what people actually did, and its methods paper records clicks on links inside an AI summary on about 1% of visits. Different surfaces, different populations, so not a like-for-like contradiction, but roughly a fortyfold gap between the stated and the seen. That gap is the whole ai search traffic story, and it is structural rather than accidental.

Read those bars slowly. Pew is not asking anyone anything; it is watching browsers, which makes it the closest thing in this pack to a plain count. Even so, it covers one month of one country's browsing in early 2025, on Google alone, because the researchers could not reliably identify AI summaries on other engines. That is not a flaw buried in a footnote. It is a scope statement, printed by the authors, and it is exactly the sentence that evaporates when a figure gets lifted into a LinkedIn post about ai search traffic.

The panic arrives in plainer language than the studies do. People Also Ask boxes carry it unpunctuated and lowercase: is ai killing website traffic, and how is ai affecting google search. Neither question has a single number for an answer, and that is not a failure of the people answering. Each instrument here measures a different thing, on a different surface, with a different denominator, over a different window, and most are run by someone who sells into the result. The disagreement is a design property, not a scandal in the undergrowth.

Here is the roll call. One brand's GA4 property, tagged by a URL fragment. One GEO vendor's scraper, pointed at citation lists. One randomised field experiment carrying a preregistration. One opt-in panel linking chats to visits. One browsing panel of 900 American adults. Five instruments, five answers about the same ai search traffic, and the only design built to establish cause is the one nobody quoted this week. If you want the discipline rather than the drama, that is what the SEO and GEO work at folkfox is built around: choose an instrument, then live openly with what it cannot see.

What each instrument is actually counting as ai search traffic#

Only one line moved sharply, and that is the point of the picture. Promptwatch, a GEO tracker in Amsterdam, reports Reddit's share of ChatGPT Search citations falling from a 3.83% average between 18 July and 7 August to a 0.52% average across four days, 14 to 17 August, which it calls an 86.4% relative drop. On the same page, a seven-day-versus-seven-day comparison of the same phenomenon reads minus 54.4%. Two windows, two headlines, one dataset, and both get quoted as ai search traffic evidence. The larger number travelled.

Promptwatch is admirably blunt about the risk its own ai search traffic numbers carry. Its page says the chart shows when each change happened and not why, that a data-collection issue cannot be ruled out, and that readers should check citation collection volume around 14 August before treating the fall as a change in ChatGPT's source selection. Search Engine Land's coverage carried that caveat forward, to its credit. A scraper breaking looks exactly like a cliff. Meanwhile Google's AI Overviews showed no equivalent break, drifting from 2.37% to 2.10%, while AI Mode slid from 2.22% to 1.54%. The contrast between surfaces is the finding, not the size of the drop.

Denominators are where most ai search traffic reporting quietly goes wrong. Scrunch's opt-in panel study of news reading found that only about 1.1% of the news visits observed after an AI conversation arrive carrying an AI referral, with direct navigation accounting for about three quarters and traditional search about 9%. That is not, and cannot be flipped into, the share of visits caused by AI; the authors state plainly that it should not be inverted, and they are right to.

The same ai search traffic study cuts against its own headline. Its raw click gap, roughly 20% of news searches converting to a publisher click with an AI Overview against roughly 30% without, shrinks to about two percentage points once the query is held fixed, which the authors call too small to tell apart from zero. Their reading is selection rather than suppression: the overlay tends to appear on searches that never sent many clicks anyway. That is a very different trail from the one the raw numbers suggest.

Stated, observed, and the space between#

The Digital News Report 2026 runs on a total sample of 97,520 across 48 markets, about 2,000 per market, and it flags its own hazard: this may be an area where what people say and what people do differ. It also warns that click-through rates from chatbots, social and search rest on quite different user bases and should be compared with caution. Pew's 900 adults were asked nothing at all; a tracking app watched them. Self-report measures intention, observation measures behaviour, and neither substitutes for the other.

The same ai search traffic contradiction turns up in ordinary dashboards, with no vendor involved at all.

Impressions and clicks doubling while sessions and users halve is no paradox once you notice the two systems count different events at different moments with different rules about what a visit even is. Search Console counts on Google's side of the door, analytics counts on yours, and everything between them, consent prompts, redirects, referrer policies, is a burrow the signal can disappear into. It is the denominator problem again, arriving one practitioner at a time.

Line them up and the pattern is plain: nobody here is measuring ai overview traffic itself. They are measuring fragments, citations, panels, extensions and self-descriptions, each a proxy standing in for a quantity only Google can count directly. That is not a reason to give up. It is a reason to name your proxy out loud every time, because a number without its instrument attached is a rumour with a decimal point.

The fragment problem under the tidiest number#

The most quotable figure of the week comes from a single brand's analytics. Writing in Search Engine Land, Alex Galinos of Elife Group describes tracking AI Overview arrivals for a transportation brand from September 2025 to June 2026, logging 51,200 tracked events across 1,661 cited snippets, and finding an average misattribution rate of 22.4%. Read the scope before the number. This is one transportation brand's GA4 property, tagged by one method, with no control site and no outside replication: 22.4% of that property's fragment-tagged events landed in Direct rather than Organic. It is not an industry rate for ai search traffic, and it was never offered as one.

The method behind that ai search traffic figure is where the thicket thickens, and this needs stating carefully rather than gleefully. The dimension is described as firing whenever a session lands carrying a #:~:text= fragment. Mozilla's MDN documentation says the fragment directive is stripped from the URL during loading so that author scripts cannot directly interact with it. Google's own web.dev documentation says the same thing in almost the same words. A GA4 tag is an author script. Either the implementation reaches for document.fragmentDirective, which the article never mentions, or the dimension is firing on something else. The study does not address the tension, and it deserves an answer rather than a verdict.

The author is not hiding the softer edges of his ai search traffic method either. He notes that Featured Snippets and People Also Ask use the same fragment, so some share of what he captures may bleed in from those formats. His check on that covers Featured Snippets only, at one unnamed moment, through a third-party tool; People Also Ask, which appears on far more result pages, is raised as a contaminant and then never quantified. He also writes that Google only sometimes appends the fragment. So the error runs in both directions at once, over-counting and under-counting, and neither direction is sized.

The ai search traffic share figure needs the same care. The article reports that 7.53% of organic sessions came from AI Overviews, then explains a few hundred words later that the custom dimension is event-scoped rather than session-scoped, so the percentage compares events against sessions, which the author says isn't an ideal comparison. A ratio with events on top and sessions underneath is not a share: one session can fire the event more than once, so the figure is biased upward by an unmeasured amount. Treat the monthly spread the same way, 29.3% in May 2026 against 16.8% in April, two consecutive months, with the rest of the series undisclosed.

None of this makes the work worthless. It is first-party evidence, published with its own caveats attached, which is more than several better-funded studies manage. It does mean the honest sentence runs longer than the headline: one transportation brand's GA4 showed about a fifth of its fragment-tagged sessions arriving as Direct across nine months of tracking. Anyone quoting the short version as an ai search traffic constant is selling something. We made the same case in a different key when a single reading turned out to be a rumour, and the principle holds in this hedgerow too.

Who is selling what, Google included#

Follow the money and the picture sharpens without curdling into cynicism. Search Engine Land, which published the GA4 study, is owned by Semrush, which sells AI-visibility tracking; the author is a practising SEO and GEO consultant, and the finding that organic traffic is under-reported is the standard sales argument for a GEO retainer. Promptwatch sells GEO monitoring and its page ends in a demo booking. Scrunch sells AI brand-presence monitoring, and its conclusion, that referral traffic is no longer the metric to watch, is also its product description. Ahrefs sells the rank tracking its own 300,000-keyword CTR study creates demand for. None of that makes any of them wrong about ai search traffic.

Google has an ai search traffic interest too, and a thinner evidence base than any of them. Its position, published by Liz Reid, is that total organic click volume from Google Search to websites has been relatively stable year over year, and that third-party reports of dramatic declines rest on flawed methods. No sample, no method and no definition of quality travels with the claim. The fairest reading is that Google describes an aggregate across all queries while its critics measure the queries where an AI feature actually appeared. Both statements can be true at once and still sound like a fight.

The ai search traffic counter-evidence deserves more room than it usually gets. A natural experiment by Zhang, Cui and Zhang used Google's own moderation rule, which permits safe-for-work subreddits into AI Overviews and bars adult ones, as a control group. It found AI Overviews increased engagement: daily comments rose 12.0% and commenting users 12.4% against the excluded communities. The same paper reports that AI Mode largely eliminates those gains, which is a warning about interfaces rather than a rescue. Set that beside Scrunch's within-query null and the tidy story that AI summaries suppress every click starts to look like an average hiding two opposite effects.

One instrument in this pack was built to establish cause, and it drew the least attention. Wang and colleagues at Northeastern and Pennsylvania ran a preregistered randomised field experiment, enrolling 1,444 participants and analysing the 1,100 who ran at least one search through a browser extension in March 2026. Assignment to AI Mode reduced click-through by 18.8 percentage points, with a 95% confidence interval running from minus 22.2 to minus 15.3. Exposure to a search page stripped of AI features raised click-through by 8.8 points. It is a preprint, its participants were recruited rather than sampled probabilistically, and it is still the strongest design on the table.

The lesson is not that vendors lie about ai search traffic. It is that a vendor's instrument is built to see the quarry the vendor sells, and it will see that quarry sharply while the rest of the field stays a blur. Read the design before the headline, every time. Positioning work runs on the same discipline, which is why the brand strategy and paid search practices at folkfox start from what a client can genuinely measure rather than from whatever a dashboard is willing to display.

Choose an instrument, then state its limits#

Pick the instrument that matches the decision, never the one with the biggest number. If the question is whether AI answers are eating clicks on queries you already win, the randomised design is your reference point and your own before-and-after is supporting evidence. If the question is whether you appear in answers at all, a citation tracker is fine, provided its window and its sample get stated every single time you quote it. If the question is where ai search traffic lands inside your own reports, the fragment method is a hypothesis to test on your property, not a finding to inherit from someone else's.

That brings you to what geo analytics should actually track. Google's generative AI performance report covers AI Overviews and AI Mode and carries impressions only; the help page never uses the word click, and the launch post lists impressions, pages, countries, devices and dates as the available dimensions, with further metrics promised over time. It is still rolling out to a subset of sites. Google separately confirms that traffic from AI features is folded into the Performance report under the Web search type, so those clicks do sit in your data, unlabelled and unbrushed out.

Triangulate with something that does not lean on your analytics at all. Cloudflare's crawl-to-refer ratio, published with its method in full, ran from Anthropic's 70,900 to 1 down to Mistral's 0.1 to 1 across one week of June 2025, and it is computed from the Referer header, precisely the signal every study here agrees is degrading. SparkToro and Datos made the same point earlier from a clickstream panel: much of the web now sends dark traffic with no referral string attached. Their 2024 figures are stale; the mechanism they described is not.

So state your ai search traffic limits in writing, on the slide, beside the number. Which instrument, which window, which denominator, which surface, and what it cannot see. That single habit turns a marketing claim into a measurement, and it is the difference between a board meeting that ends in a decision and one that ends in a mood. When the readings do move for real, as they did in Shopify's structured-data numbers and again when the reporting finally caught up with the drop, you will know it because your instrument held still while the reading changed.

Paid media has already moved to this footing: ChatGPT ads landed across 31 markets this morning, which means measurement inside assistants is arriving on the paid side before the organic side has settled on a ruler. If your team wants one instrument, one denominator and one honest slide instead of five contradictory ones, talk to folkfox, or start with the content marketing services that put the primary document in front of the trade press version of it.

Frequently asked questions#