Menu
SEO and addresses
Search engines and AI crawlers
Explains the three search visibility settings and when a change takes effect.
One setting controls who may read your whole site: search engines, AI crawlers, both or neither. Find it in the SEO view under Indexing.
The three options
| Option | What it does |
|---|---|
| Public | Search engines and AI can access your content. This is the default. |
| Block AI crawlers | Asks AI crawlers not to read your site, which blocks AI training and reduces how often your site appears in AI-generated answers. Search engines are unaffected. |
| Hidden from search | Asks all crawlers to stay away and tells search engines not to list any page. Your site won't appear in search results. |
These settings work by publishing instructions that well-behaved crawlers follow: rules in your site's robots.txt file and, for Hidden from search, a "don't index" tag on every page. A crawler that ignores the rules can still read a public website. To keep a site completely private, don't publish it.
When a change takes effect
The setting saves as soon as you pick an option, but your live site changes only when you publish again. Until then, the Publish panel shows Indexability changes pending — republish to apply.
You can't change this setting while a publish is running.
Hiding a single page
To keep one page out of search results while the rest of the site stays public, open that page under Page-level SEO and turn off Search Engines. See Page SEO. Hidden from search overrides every page's own setting.
Hidden pages are also left out of your sitemap. See What Starjay generates for you.