Weekly site audits for AI readability: the 21 checks that matter and how to fix each (2026)

Key takeaways

• AI readability fails for mundane reasons first: blocked crawlers, content that only exists after JavaScript runs, error responses, and stale pages. Fix those before you touch schema or llms.txt.
• Run the audit weekly, not quarterly. A WAF rule change or a deploy can cut off a whole AI provider overnight, and you want to catch that in days.
• The 21 checks below fall into five groups: access, rendering, discovery, content structure, and freshness and trust. Each has a quick test and a fix.
• Compare your own crawler logs with Promptwatch Data's AI crawler traffic report. If a provider is large there and near zero in your logs, you are probably blocked.
• Treat schema and llms.txt as clarity signals. Nobody has shown they move citations, so they go last on the list.

Why a weekly audit, and what AI readability means

An SEO audit asks whether your pages rank. An AI readability audit asks something narrower: can an AI system reach your page, read the content, pull out a clean statement, and trust it enough to cite? Google says established SEO practices still apply to AI Overviews and AI Mode, so most of this is classic technical hygiene with a few new failure modes.

The weekly part matters more than people think. Most AI visibility losses I see traced back to a change somebody made without realizing: a new bot-protection rule, a template rewrite that moved content into client-side rendering, a migration that left redirect chains. Quarterly audits find these after the damage. A weekly pass with a diff of what changed since last time finds them while they are cheap.

The set below is a framework I would use, not an official standard. Promptwatch's own site audit runs 21 checks weekly with a health score and a what-changed view, and the groupings here cover similar ground, but treat my list as a working checklist you can adapt.

How to run the routine

You need three data sources: a crawler, your server or CDN logs, and Search Console. Pick a fixed set of 20 to 50 priority URLs (product pages, landing pages, docs, pricing, your top comparison pages) and score them every week as Pass, Fix, or Investigate.

For crawling, Screaming Frog SEO Spider is the standard desktop option and handles most of checks 4 to 18.

Google Search Console gives you indexing status and the Google side of crawling.

For the AI-specific layer (which bots visit, what errors they hit, which pages get cited), a platform like Promptwatch runs weekly audits and reads IP-verified crawler logs from Cloudflare, Vercel, Fastly, and raw server logs, so you are not guessing.

Bigger sites with log-analysis needs sometimes use enterprise crawlers such as Botify or Lumar. Browse more options in the GEO software directory at bestgeosoftware.com.

The 21 checks at a glance

Access checks (1 to 5)

1. robots.txt per AI bot

Test: open /robots.txt and read it bot by bot. Look for rules on GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Google-Extended and similar. A blanket Disallow or an old rule copied from a template is the usual culprit.

Fix: decide intentionally. Training crawlers and answer-time fetchers are different things. Blocking a training bot is a policy choice. Blocking the bot that fetches pages when a user asks a question removes you from answers. Allow the second group on any page you want cited.

2. CDN and WAF rules

Test: robots.txt can say yes while your firewall says no. Fetch a key page with the user agents of major AI bots from an outside IP and check you get a 200, not a 403 or a challenge page.

Fix: allowlist verified bot IP ranges rather than user agent strings, since user agents are easy to spoof. Promptwatch's AI crawler URL inspector and Agentic Readiness Check are free ways to see which crawlers can read a page.

3. Crawler logs by provider

Test: pull a week of verified AI bot requests and group by provider. Promptwatch Data's crawler report found OpenAI made up 79.8% of verified AI crawler requests in the week of Aug 31 to Sep 6, 2026, down from 94.8% in early June, so the mix is shifting toward Anthropic, Google, Perplexity and Mistral.

Fix: if a provider is large in that report but near zero in your logs, you are likely blocked or unreachable. Work backward through checks 1 and 2. Also check which bot does the work, because training fetches and answer-time fetches mean different things.

4. HTTP status codes

Test: crawl the priority URLs. Every one should return 200. Watch for soft 404s, 5xx errors that only appear under load, and 403s served to bots.

Fix: repair or redirect errors, and make sure error pages return real error codes instead of a 200 with a friendly message.

5. Redirect chains

Test: any URL that takes more than one hop to resolve. Fetchers with tight time budgets may give up.

Fix: point internal links and sitemap entries straight at the final URL, and collapse chains to a single 301.

Rendering checks (6 to 10)

6. Server-rendered content

Test: view source, or fetch the page with JavaScript off. If your product details, pricing, or body copy are not in the raw HTML, many AI fetchers never see them, because they do not execute scripts.

Fix: server-side render or statically generate the content that matters. If a rebuild is not realistic, a prerendering service such as Prerender.io serves rendered HTML to bots.

7. Response speed and timeouts

Test: time to first byte for bot requests, which can differ from what a human sees if bots get routed differently. Check logs for timeouts.

Fix: cache aggressively for bot traffic and fix slow database calls on templates, since a slow template affects every page built on it.

8. Canonical tags

Test: each priority page should have a self-referencing canonical, and canonicals should not point to redirected, blocked, or noindexed URLs.

Fix: correct template logic first, then individual exceptions. Conflicting canonicals make it unclear which version should be cited.

9. Stray noindex and X-Robots-Tag

Test: check both the meta robots tag and the HTTP header. Staging settings leaking into production is the classic cause, and the header version hides from a casual view-source check.

Fix: remove it, then add a deploy check so a noindex on a priority URL pages somebody.

10. Login, paywall and consent walls

Test: load the page in a clean session with no cookies. If a cookie banner, interstitial, or login gate covers the content, a fetcher may only see the gate.

Fix: make sure the primary content is in the initial HTML, and keep documentation and pricing public. Promptwatch's citation data shows documentation is cited consistently, and gated docs can't be.

Discovery checks (11 to 14)

11. XML sitemap accuracy

Test: the sitemap should hold only canonical, indexable, 200-status URLs, and include your priority pages. Compare it against the crawl.

Fix: generate it from your CMS rather than by hand, and drop redirects and blocked pages.

12. Orphan pages

Test: pages in the sitemap or analytics with zero internal links pointing at them.

Fix: link them from relevant hubs and category pages. An orphan page with great content is still a page nobody finds.

13. Broken internal links

Test: crawl for internal 404s and 5xx targets.

Fix: update links in the source content, not with a pile of redirects.

14. Duplicate and near-duplicate pages

Test: duplicate titles, near-identical body text, parameter URLs and printer versions.

Fix: consolidate with a 301 or a canonical. When several near-identical pages compete, none of them is clearly the one to cite.

Content checks (15 to 18)

15. Title tags

Test: missing, duplicated, or vague titles on priority pages.

Fix: write titles that state what the page is and who it is for. Titles are often the first thing a retrieval system reads.

16. Heading structure

Test: one H1, logical H2 and H3 nesting, and headings that read like the questions people ask.

Fix: rewrite vague headings (Overview, More info) into specific ones (How much does X cost, Who is X for).

17. Answer-first extractability

Test: read the first 100 words under each heading. Does the page state the answer, or does it warm up? Watch for vague pronouns like this approach, which only make sense inside the page, and for facts buried in long paragraphs.

Fix: lead with the definition or answer, then support it with steps, examples, or a table. Make each section stand alone. Add a short plain-language FAQ for your brand. Promptwatch Data's content type report (Sep 28 to Oct 4, 2026) found product pages were 22.4% of ChatGPT citations, landing pages 16.8%, and FAQ pages 3.3%, so explicit pricing, specs, and who-it's-for on product pages deserve more attention than a blog sidebar.

18. Thin or bloated pages

Test: pages under a couple hundred words that are meant to be substantive, and 5,000-word pages where the answer is buried.

Fix: expand thin pages with facts, or merge them into a stronger page. Split bloated pages by topic.

Trust and freshness checks (19 to 21)

19. Structured data validity

Test: validate JSON-LD for Article, Organization, Product, FAQPage and BreadcrumbList on priority pages. Marked-up content must be visible on the page, and names, prices, and dates must match what users see.

Fix: correct errors and remove markup that contradicts the page. A tool like Schema App handles this at scale, and CMS plugins such as Yoast SEO cover the basics.

I would be honest about the limits here: we found no solid evidence that schema or llms.txt directly lifts AI citations. Treat them as clarity signals and do them after checks 1 to 10.

20. Freshness and visible dates

Test: dateModified and the visible updated date on priority pages, and whether prices, specs, and availability are current.

Fix: update on a schedule. Promptwatch Data's half-life report tracked every page ChatGPT cited on July 6, 2026 for 90 days. Landing pages (78%) and docs (76%) were still cited after 30 days, while news lost half its pages within 36 days. So keep pricing, product, and docs pages accurate, and use news for short bursts only.

21. Author, entity and fact consistency

Test: do authors, company details, contact information, and key facts (pricing, features, locations) match across your site and your major third-party profiles?

Fix: pick one source of truth, update it everywhere, and name authors on content. If your site says one thing and an AI assistant says another, start by fixing your own pages.

Prioritizing and tracking

Do not try to fix 21 things at once. Work in this order:

1. Anything that blocks access (checks 1 to 3, 6, 9). These are all-or-nothing.
2. Errors and wasted fetches (4, 5, 7, 8, 10).
3. Discovery and duplication (11 to 14).
4. Content structure (15 to 18).
5. Freshness, trust, and markup (19 to 21).

Keep a simple sheet with one row per priority URL and one column per check. Record the date, the result, and the change since last week. The what-changed column is the useful part, because it ties a drop to a deploy.

Then measure the effect. Weekly crawls tell you the site is readable. They do not tell you if AI is citing you. For that, track a fixed set of prompts across models and watch citations and AI-referred traffic over time. Promptwatch combines crawler logs, citation tracking, and visitor analytics so you can see the path from a bot visit to a citation to a click. Other trackers are listed at ai-rank-tools.com if you want to compare.

A 30-minute weekly version

If you only have half an hour:

• Check AI crawler volume by provider against last week (check 3).
• Run your crawler on priority URLs and review only new errors (4, 5, 9, 13).
• Spot-check two recently published pages with JavaScript off (6).
• Review any page updated this week for visible dates and answer-first structure (17, 20).
• Log what changed and who owns the fix.

The discipline is more valuable than the depth. A shallow audit that runs every week catches the changes that quietly remove you from AI answers.