PlugPress

Saddle Rank

Choose Which AI Crawlers May Read Your Site

Two switches decide whether AI answers and AI training may use your site. See what each does and how to track visits.

Saddle Rank gives you two switches: one for AI answers and one for AI training. It turns your choice into robots.txt rules for the AI crawlers it knows, so you do not have to write them by hand. Open Saddle → Rank → AI visibility → AI crawlers to reach the AI access page.

What do the two switches do?

Switch When it is on When it is off
Show my site in AI answers ChatGPT search, Claude, Perplexity, Meta AI, Alexa and others may fetch your pages and cite them. Those crawlers are blocked, so those AI answers cannot cite your pages.
Let AI companies train on my content GPTBot, ClaudeBot, CCBot and others may collect your text to train AI models. Those crawlers are blocked, and the shared AI training opt-out is added.

The two are separate choices. You can stay in AI answers and still keep your text out of training.

Until you choose, Saddle Rank changes nothing. Your robots.txt stays as it was.

What about Google and Bing?

Google’s AI Overviews and Bing’s Copilot read your pages through Googlebot and bingbot. These switches leave both alone.

One thing to know before you turn training off: it also blocks Google-Extended. That is Google’s rule, and it removes your site from Gemini app answers too.

Can I choose crawler by crawler?

Yes. Open Each crawler on the same page. The crawlers sit in three groups:

  • AI answer crawlers fetch your pages for AI answers and AI search, so the answer can cite you. Examples: OAI-SearchBot, ChatGPT-User, Claude-SearchBot, PerplexityBot.
  • Training crawlers collect text to train AI models. Examples: GPTBot, ClaudeBot, CCBot, Google-Extended.
  • Other AI crawlers do neither. They run ad checks and research crawls. Neither switch changes them.

The page also shows The robots.txt crawlers receive, so you can read the exact file before you press Save.

How does this affect my readiness score?

Only the AI answer crawlers count. They are worth 22 points, and you earn a share for each one you let in. Blocking training crawlers costs no points.

If answer crawlers are blocked, the list of fixes shows it. Press Fix, read what will change, then press Approve and apply. Training crawlers you block stay blocked.

Why are my choices not on the live site?

The page warns you about two cases:

  • “Discourage search engines” is on in Settings → Reading. These rules do not reach the live site until you turn it off.
  • A robots.txt file sits on your server. A real file in your site’s root folder overrides these choices. Delete it to manage access here.

Can I add my own robots.txt rules?

Yes, but most sites never need to. Turn on Custom crawler rules (robots.txt) in Settings → Advanced → Search engines. It is off by default, because a wrong rule can block crawling across the whole site. Then add your lines under Your own robots.txt lines on the AI access page.

How do I see which AI crawlers visit?

Saddle Rank logs each visit from a known AI crawler and keeps the log for 90 days. You can see it in three places:

  • Saddle Analytics. If you run it, its AI crawlers card shows the visits.
  • Your AI agent. An agent connected through Saddle can read the crawler visits.
  • The email digest. It reports how often AI crawlers fetched your site and which pages they read most.

Saddle Rank has no page of its own for the log. To stop logging, turn off AI bot tracking in Settings → Advanced → AI visibility.

Why has a crawler I allow never visited?

Once the log is 30 days old, the readiness check looks for AI search crawlers that your robots.txt lets in but that never arrive. If it finds any, something in front of WordPress may be turning them away:

  • Your host’s firewall. Ask your host whether it blocks AI bots.
  • Cloudflare. Open Security, then Settings, filter by Bot traffic, and check that “AI Crawler and Agent access policies” does not block the Search category.
  • A page cache. A cache that answers before WordPress runs can hide visits from Saddle Rank. The crawler came, but the visit was not logged.

Do AI crawlers read special files, or my normal pages?

Your normal pages. The big AI crawlers work like Google: they read robots.txt, follow your sitemap and fetch the pages a visitor sees. So the most useful thing you can do is let them in and keep your pages clear and current.

Saddle Rank never serves a crawler something different from what a visitor gets. Showing crawlers different content is called cloaking, and search engines penalize it.

What to do next