Based on what buyers actually ask
Auto-generated tagsFeatureAI-powered
TTFB and page size checker
A TTFB checker and page size checker built for AI crawlers
AI assistants fetch pages live while they compose an answer, and they don't wait long. WriteWorks times the server response on every page it crawls, once as its own crawler and once with an AI crawler's user agent, and measures the size of the HTML each page sends. Slow responses and pages over the 2 MB that several AI crawlers read are flagged, page by page.
7-day trial · Cancel anytime
Google Search Console
Google Analytics 4
Google Ads
Ahrefs
Semrush
DataForSEO
Firecrawl
Tavily
Exa
Parallel
G2
WordPress
Webflow
Contentful
Sanity
Payload
BaseHub
Framer
Notion
Gamma
v0
Slack
Linear
Atlassian
Monday.com
Granola
Klaviyo
Customer.io
OpenAI Ads
Claude and MCP clients The platform
See WriteWorks in Action
This is the working product, not a mockup: the same visibility scores, briefs and published content your team would be looking at. Start a free trial and point it at your own domain, or take the tour of what sits behind the screen.
Home
Ask AI
Opportunities128
Research
Create
Measure
AI visibility
AI discoverabilityNew
AI crawlers
Reports
Context
Support
Settings
Sign out
SCSam Cartersam@example.com
WRITEWORKS
Measure/AI discoverability
Search brands, prompts, competitors…⌘K
New1.2M (62%) leftAsk AI
The challenge
Why speed for AI crawlers is its own problem
Live fetches, bot user agents and pages that are simply too big.
AI answers don't wait for slow servers
When an assistant fetches sources mid-answer, a page that takes two seconds to start responding is easy to drop in favour of one that answers in half a second.
Speed tests time a browser, not a bot
Lab tools measure how a page paints for a person. They don't tell you how quickly your server answers a request with GPTBot or ClaudeBot in the user agent, which can be very different behind a CDN or firewall.
Oversized HTML hides the content
Inline scripts, SVGs and JSON state can push a page past 2 MB. Several AI crawlers stop reading at about that point, and the main content is often below it.
One URL at a time
A TTFB checker that tests a single URL won't find the 30 product pages that time out because they skip the cache.
The solution
How the AI crawler speed test works
Time it as a bot, apply the thresholds, check the size, track it.
Every page is fetched by the WriteWorks crawler and probed once with an AI crawler's user agent, rotating through GPTBot, OAI-SearchBot, PerplexityBot and ClaudeBot across the crawl. The slower of the two timings is the one that counts.
Under 0.8 seconds to start responding is the target. Between 0.8 and 1.8 seconds is a notice that costs 8 points of bot access; over 1.8 seconds is a warning that costs 20.
The uncompressed HTML of each page is measured. Over 2 MB is an error that costs 30 points of bot access, with the fix: move inline scripts, styles, SVGs and data payloads out, and paginate very long lists.
The Pages view shows TTFB and HTML size for every page and sorts by slowest TTFB. Each page's detail charts its response time and HTML size across crawls, so you can see a fix land.
Success story
What real brands fixed with WriteWorks
What each brand fixed, measured before and after with the same audit, in their own words.
“Turning caching on made our pages 2.5 times faster for crawlers. While we were in there, we fixed the headings too: guides with a clean heading structure went from 32% to 100%.”
S4
Capabilities
Server response time and page size, checked
Two of the bot access checks in AI discoverability.
Server response time
Time to first byte for every crawled page.
AI user agent probe
Timed as GPTBot, OAI-SearchBot, PerplexityBot or ClaudeBot.
Page size checker
Uncompressed HTML size, flagged over 2 MB.
Slowest TTFB sort
Find the slowest pages across the whole crawl in one click.
Refused AI user agents
Pages that error for the AI user agent but load for the crawler are reported.
Trend per page
Response time and HTML size charted crawl by crawl.
Time savings
What changes
From one-off URL tests to every page, every crawl.
TaskBeforeWith WriteWorksTime saved
Find slow pages for AI crawlersTest URLs one at a timeEvery page timed, sorted by slowestThe sampling
Check the bot path, not the browserA lab score for a personTimed with an AI crawler's user agentFalse comfort
Catch oversized pagesNotice when content goes missingHTML over 2 MB flaggedLost content
Confirm a caching fixRe-test by handResponse time trend per pageThe re-test
Included
What's included
Every speed and size check in the crawl.
Time to first byte measured on every crawled page
One probe per page with an AI crawler's user agent
Probes rotate through GPTBot, OAI-SearchBot, PerplexityBot and ClaudeBot
The slower timing counts
Notice from 0.8 seconds, warning over 1.8 seconds
Uncompressed HTML size measured per page
Error for pages over 2 MB
TTFB and HTML size columns in the Pages view
Sort by slowest TTFB
Per-page trends for response time and HTML size
Errors returned only to the AI user agent reported
Part of the bot access pillar, worth 35% of the score
Built for
Who uses it
The people who own the servers and templates.
Technical SEO
Server response time and page size across the whole site, as AI crawlers meet them.
Developers and DevOps
The slow templates and oversized payloads, with the fix for each.
Ecommerce teams
Large catalogue and listing pages that drift over the size limit.
Agencies
A quick read on whether a client's hosting is costing them AI answers.
Featured story
Fhresh
“We're a booking marketplace, so listings are everything. Since the fixes, pages fully readable without JavaScript went from 84% to 98%, our sitemap dates finally mean something, and our full llms.txt file is real. Over a third of our audited category pages now list their businesses as structured data, with the remaining markets up next.”
Fhresh team
Read the full story72% → 98%
Pages answering crawlers in under 0.8 s
84% → 98%
Pages fully readable without JavaScript
29% → 58%
Key pages with the schema AI expects
FAQ
Frequently asked questions
Everything teams typically ask before getting started.
What is a TTFB checker?+
A TTFB checker measures time to first byte: how long a server takes to start responding to a request. WriteWorks checks it for every page in a crawl, including once with an AI crawler's user agent, because that is the request that decides whether a page makes it into a live AI answer.
What server response time should I aim for?+
Under 0.8 seconds. WriteWorks flags 0.8 to 1.8 seconds as a notice, because faster sources are preferred when an assistant fetches pages live, and over 1.8 seconds as a warning. The usual fixes are serving the page from a CDN cache for bot traffic and removing slow server-side work from the request path.
Why does page size matter for AI crawlers?+
Several AI crawlers stop reading a page at about 2 MB of HTML. Anything below that point, often the main content, is never seen. The page size checker measures the uncompressed HTML and flags any page over 2 MB as an error.
Is this a page speed test?+
Not in the Core Web Vitals sense. It doesn't render the page in a browser or score how it paints for a person. It times how quickly your server answers a crawler, and how much HTML it sends, which is what matters to an AI crawler that reads the raw page.
Which AI user agents does it test with?+
Each page is probed once, with one of GPTBot, OAI-SearchBot, PerplexityBot or ClaudeBot, rotating across the crawl so a 1,000-page crawl doesn't send thousands of extra requests. The request comes from WriteWorks, not from the AI vendor's own network, so a firewall that allows verified bots by IP may treat it differently.
How is this different from AI crawl budget?+
AI crawl budget uses your real crawler logs to show where AI bots spend their visits. The speed test checks every page's response time and size from the crawl, whether or not a bot has visited yet. Together they show where bots go and whether each page answers fast enough when they arrive.
Thresholds
What the speed test flags
Server response over 0.8 s
A notice, costing 8 points of bot access. Within crawl limits, but faster sources are preferred.
Server response over 1.8 s
A warning, costing 20 points. Assistants fetching live tend to drop slow sources.
HTML over 2 MB
An error, costing 30 points. Content past the limit is never read.
Refused AI user agents
Pages that load for the crawler but error for the AI user agent, often a CDN or firewall rule.
Audit AI bot access Explore more
Keep exploring
Related solutions, adjacent use cases, and platform features.
Time every page the way AI crawlers fetch it
Server response time with an AI user agent and the 2 MB page size check, on every page, inside the AI discoverability audit.