We've added agent analytics, a view that separates AI crawler traffic - GPTBot, ClaudeBot, PerplexityBot and other named agents - from human visits. You get a clear read on which AI engines are fetching your content, which pages they pull and how that activity trends over time. It is built for growth and search-marketing leaders who need evidence that their content is actually reachable by the engines they want citations from.

Most analytics tools were designed to count people. That leaves a blind spot: if an AI engine never crawls a page, it cannot cite it, and no amount of on-page optimisation will change that. Agent analytics closes the loop between "we published it" and "the engines fetched it".

What is agent analytics?

Agent analytics is a traffic-analysis view inside WriteWorks that identifies visits from AI crawlers and reports them separately from human traffic. Instead of one blended visitor count, you see bot activity broken out by named agent, so GPTBot activity is distinguishable from ClaudeBot, PerplexityBot and the rest.

That distinction matters for two reasons. It stops AI crawler volume from quietly inflating your human traffic numbers, and it turns crawler behaviour into something you can monitor deliberately rather than discover by accident in raw server logs.

How agent analytics works

Agent analytics classifies incoming requests by user agent, matches them against a maintained list of known AI crawlers, and groups the results by engine and over time. Crawlers self-identify in the user-agent string - the same mechanism that underpins the Robots Exclusion Protocol standard and the crawler directives in your robots.txt file.

From there, WriteWorks presents:

  • Bot versus human split - how much of the measured traffic comes from AI agents rather than people.
  • Breakdown by agent - which named crawlers are visiting, and their relative share of bot activity.
  • Trend over time - whether crawler activity is rising, flat or dropping across the selected period.
  • Page-level detail - which URLs agents are fetching, so you can see whether the pages you care about are being reached.

Two honest limits. First, a crawl is not a citation: agent analytics tells you an engine fetched a page, not that it referenced you in an answer - that side of the picture sits with citation tracking across AI engines. Second, user-agent strings can be spoofed or omitted, so treat the figures as a strong directional signal rather than a forensic audit.

Who agent analytics is for

It is especially useful if you:

  • Publish frequently and want to confirm that new or updated pages are being crawled by AI engines, not just by traditional search bots.
  • Have made technical changes - robots.txt rules, a CDN bot-management policy, a migration - and need to check whether AI crawler access changed as a result.
  • Report to a leadership team that asks whether AI search is a real channel, and need engine-level evidence rather than anecdote.
  • Manage several brands or domains and want a consistent way to compare crawler coverage across them.
  • Are investigating why a high-quality page is not appearing in AI answers, and need to rule out access problems before rewriting anything.

Experienced technical SEOs who already parse log files benefit too: the value here is a maintained agent list, engine grouping and trendlines without maintaining a log-parsing pipeline.

What agent analytics changes

It gives you a diagnostic step that most AI-visibility workflows are missing. When visibility for a topic is flat, the first question becomes answerable: is this a crawling problem, a content problem, or a competitive one?

It also changes how you read your own traffic reports. Once bot visits are separated, human engagement figures get cleaner - a useful complement if you already connect Google Analytics 4 to WriteWorks to compare AI-driven referrals with organic search.

And it makes technical decisions measurable. If a bot-blocking rule is added at the edge, crawler activity for the affected agents should visibly change; if it does not, the rule may not be doing what was intended.

How to use agent analytics

1. Open your workspace and select Analytics in the main navigation.

2. Choose the Agent analytics view to see AI crawler activity separated from human visits.

3. Set the date range you want to assess - a longer window makes trends easier to read than a single week.

4. Review the breakdown by agent to see which engines are crawling, and which are absent.

5. Drill into page-level detail to check whether your priority URLs are being fetched.

6. Compare against your citation and visibility data to work out whether the gap is access or relevance.

Tips for a better result

  • Check your robots.txt before drawing conclusions: an agent that never appears may simply be disallowed.
  • Look at the direction of travel, not single-day spikes. Crawl activity is naturally uneven.
  • Pair crawler data with per-URL performance using AI search performance per URL to see which fetched pages actually earn citations.
  • Re-check crawler activity a few weeks after any infrastructure or CDN change.

Frequently asked questions

Which AI crawlers does agent analytics track?

Agent analytics identifies named AI crawlers by their user-agent strings, including GPTBot, ClaudeBot and PerplexityBot, alongside other recognised AI agents. The list is maintained as engines publish new or renamed agents, so newly announced crawlers can be reflected without changes to your site or tracking setup.

Does an AI crawler visit mean my page will be cited?

No. A crawl confirms an engine fetched the page, which is a prerequisite for citation but not a guarantee of it. Whether the page is then referenced in an answer depends on relevance, competing sources and the prompt. Use agent analytics for access, and citation tracking for outcomes.

How is this different from bot traffic in standard web analytics?

Most analytics tools filter or bundle bot traffic to keep human metrics clean, which hides which specific AI agents visited. Agent analytics treats those visits as the subject rather than noise, grouping them by named crawler and engine and trending the activity so it becomes a measurable signal.

Can agent analytics show whether my site is blocking an AI crawler?

Indirectly, yes. If an agent shows little or no activity while others crawl normally, that pattern points to a robots.txt directive, firewall rule or CDN bot policy worth checking. Agent analytics reports observed behaviour; the block itself is confirmed in your own site configuration.

The bottom line

Agent analytics turns AI crawler behaviour from an invisible technical detail into a measurable input, so you can tell the difference between content that engines never see and content they see but do not cite.

If you want that visibility across your own domains, explore AI agent analytics or see how it fits the wider workflow for growth leaders working in AI search.