Key takeaways
- A user-agent string proves nothing. Anyone can send a request labeled GPTBot, so real crawler analytics has to check the source IP against the operator's published ranges.
- Crawler analytics is rare inside GEO platforms. In Prefer's September 2026 comparison it appears as a mid or top tier feature at Otterly, Promptwatch, Qwairy and Profound, and most trackers skip it.
- I could confirm IP-verified crawler logs for Promptwatch from its own published data. I did not verify the other tools one by one, so this guide gives you a test to run instead of a fake 22-row scorecard.
- Ask every vendor five questions (listed below). If they can't answer the IP-range one, you are looking at user-agent counting.
Why I'm not giving you a 22-row checkmark table
The title promises a check across 22 tools. Here is the honest version: vendor pages mostly say "crawler analytics" or "AI bot tracking" and rarely say how a request gets verified. Filling a table with green checks from marketing copy would be exactly the sloppiness this feature exists to prevent.
What I can do is explain what verification means, show which claims are documented, and hand you a checklist that takes about 20 minutes per vendor. That is more useful than a grid you can't trust.
What IP verification actually means
AI crawlers read raw HTML and don't run JavaScript, so Google Analytics and tag-based tools never see them. The only places they show up are server logs or the edge (CDN or host). Once you have those logs, you still have a problem: the User-Agent header is self-declared.
Digital Applied's August 2026 bot verification playbook puts it bluntly: user-agent strings prove nothing, and most major AI vendors now publish IP-range JSON files, reverse-DNS suffixes, or both precisely because spoofing is trivial.
There are two accepted verification methods:
- IP-range matching. Compare the request's source IP with the JSON ranges the operator publishes (OpenAI, Anthropic, Perplexity and others do this).
- Reverse DNS then forward DNS. Google documents this approach. A request from 66.249.66.1 resolves to crawl-66-249-66-1.googlebot.com, and that hostname must resolve back to the same IP. Googlebot hostnames follow crawl-.googlebot.com, geo-crawl-.geo.googlebot.com and rate-limited-proxy-*.google.com.
host 66.249.66.1
1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com.
host crawl-66-249-66-1.googlebot.com
crawl-66-249-66-1.googlebot.com has address 66.249.66.1
A platform that does either one at ingestion is doing verification. A platform that groups hits by user-agent string is counting claims.
Why it matters for GEO decisions
Crawler data answers questions that prompt tracking can't. Did GPTBot or OAI-SearchBot ever fetch the page you just published? Did it hit a 403? Is Claude's citation crawler reaching you at all? If spoofed traffic is mixed in, every one of those answers is polluted.
The bots also do different jobs. GPTBot and ClaudeBot collect training data. OAI-SearchBot, Claude-SearchBot and PerplexityBot fetch pages for live answers. A tool that blends them into one number hides the half that produces citations. And user-triggered fetchers like ChatGPT-User and Perplexity-User are documented by their vendors as not reliably honoring robots.txt, because they fire when a person asks a question.
The mix also moves fast. Promptwatch Data's AI crawler traffic report shows OpenAI at 79.8% of verified AI crawler requests in Aug 31 to Sep 6, 2026, down from 94.8% in Jun 8 to 14, 2026. Meanwhile Meta-WebIndexer grew from about 2.2% to 37.8% of tracked requests between mid-July and Aug 9, 2026. Note the caveat in that second report: its Meta share is identified by user agent, while the headline crawler report is the IP-verified one. Even a vendor with verification can mix methods across datasets, so read the method note.
What the market looks like
Prefer's comparison of AI crawler analytics tools (checked Sep 26, 2026) lists these as crawler analytics sold as a tier inside visibility platforms: Otterly at $189, PromptWatch at $245, Qwairy at $519, and Profound on custom pricing. Prefer publishes that list and ranks itself first, so treat its rankings as a vendor claim. The pricing snapshot is still a handy reference point.
The same comparison names standalone and edge options outside the GEO category:
| Option | Type | Notes from the comparison |
|---|---|---|
| Cloudflare AI Crawl Control | Edge | Free on every plan, needs a Cloudflare zone |
| Known Agents (formerly Dark Visitors) | Standalone | Free plan, 2,000+ named agents, AI chat referral tracking, any host |
| Prefer Agent Analytics | Log-based | $99/mo, ties crawls to citations |
| Vercel bot data | Edge | Only for Vercel deployments |
| Otterly, Promptwatch, Qwairy, Profound | Inside a GEO platform | $189, $245, $519, custom |
The split matters. Edge tools only see sites they serve. Log-based tools work on any host.
Platforms with crawler data, and what I could confirm
Promptwatch
Promptwatch is the one I can document properly, because it publishes its method. Its crawler analytics counts requests from raw server and CDN logs and only includes requests whose source IP matches the operator's published IP ranges, so spoofed user agents are excluded. It works with Cloudflare, Vercel, Fastly and raw server logs, and the official WordPress plugin, and it separates bots by job: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Perplexity-User, Meta's crawlers, Mistral, xAI, Cohere and DeepSeek among them.
What I like is what happens after the log line. Pages get a crawl-to-citation path with a citation rate, errors come with plain-language explanations, and the fix loop (content, publishing, re-checking whether it got crawled and cited) lives in the same product. The 2026 comparison of 22 platforms on Promptwatch's own site has it as the only one with the full stack, which is a vendor claim, but it matches what I could verify on crawler logs specifically. Pricing starts with crawler logs on Professional at $245/mo (25M crawler logs); confirm current tiers at promptwatch.com/pricing.
Profound, Qwairy and Otterly
Prefer lists all three as selling crawler analytics as a tier. That tells you the feature exists. It doesn't tell you whether requests are IP-verified, and I did not check their docs. Treat them as candidates and run the checklist below.
Monitoring-only trackers
Many tools in this category track prompts and citations but have no crawler layer described in the sources I reviewed. Examples in our catalog include Peec AI, AthenaHQ, LLM Pulse and Rankscale. If crawler visibility matters to you, confirm with the vendor rather than assuming.
Technical SEO crawlers and log analyzers
Some older tools in the catalog, such as Botify, Oncrawl and FandangoSEO, are built around log analysis for classic search bots. They can be useful if you want server-log depth, but check whether they classify AI bots and verify them against operator IP ranges.
The five-question vendor test
Send these to any vendor, or look for the answers in their docs.
- How do you verify that a request comes from the claimed operator? You want IP-range matching or reverse DNS. "We parse the user agent" is a fail.
- Do you filter or flag spoofed bots, and can I see the excluded count? A good answer shows spoofed traffic as a separate bucket.
- Do you separate training, search and user-triggered bots? OAI-SearchBot and GPTBot should never be one line.
- Where does the data come from? Cloudflare, Vercel, Fastly, CDN logs, a plugin, or raw server logs. Match this to your stack.
- Can you connect a crawl to a citation on a specific page? Without that, the logs are interesting but not actionable.
Also check what is mentioned in the method notes. Promptwatch's report cautions that shares are of all tracked logs for a week, so a bigger share doesn't mean more requests in absolute terms. A vendor that documents caveats like that probably takes its methodology seriously.
A quick DIY sanity check
Before paying anyone, spend ten minutes on your own logs. Pull a day of requests with GPTBot or ClaudeBot in the user agent. Run reverse DNS or an IP-range match on a sample. If a chunk fails, you have spoofed traffic, and any tool that counts by user agent would have counted it as real.
Then compare your mix with the public benchmark. The advice from Promptwatch's crawler report is sound: if a provider is a large share of the benchmark but near zero in your logs, you are probably blocked or unreachable for that provider. Check your robots.txt too. Cloudflare's managed robots.txt block covers GPTBot, ClaudeBot and Google-Extended but not OAI-SearchBot or PerplexityBot, while a plain User-agent: * disallow blocks all of them. Check the rule, not the intention.
Which approach fits which team
| Situation | What to do |
|---|---|
| You want crawls, citations, traffic and content fixes in one product | A GEO platform with verified logs, such as Promptwatch |
| You're on Cloudflare and only need bot visibility | Cloudflare AI Crawl Control, free |
| You need logs on any host, no GEO suite | A standalone like Known Agents |
| You only track mentions and prompts | A monitoring tracker, but accept that you can't see crawler behavior |
| Enterprise with a technical SEO team | Log analyzers like Botify plus a GEO platform |
If you're still building a shortlist, the GEO software directory at bestgeosoftware.com lists more options by category.
What I'd do this week
Pick three vendors, send the five questions, and ask for a sample export of crawler rows with the verification column. The ones that answer clearly in a day are the ones that built the feature properly. Everyone else is selling a dashboard of user-agent strings.
One limit of this guide stands: I confirmed verification method only for Promptwatch. For the other tools, the table above tells you where crawler analytics exists, and the test tells you whether it holds up.