How to check whether AI search crawlers can read your clinic website

1. Choose the search service to check

ChatGPT web search, Google AI features, and Bing’s AI answers are separate services. Decide which service and pages to check, then read that service’s crawler documentation. OpenAI documents OAI-SearchBot for search separately from GPTBot, which is used for model training.

To check rules for OpenAI’s search crawler, review rules for OAI-SearchBot and site-wide rules in robots.txt. You can manage training preferences through separate rules for GPTBot.

2. Review three places together

First, open the public URL without signing in. Second, review the rules in the site’s robots.txt that apply to that path. Third, check page-level search directives and the security settings of your CDN or hosting service. A page visible to a person may still be inaccessible to some crawlers.

If the source of a block is unclear, give your developer or hosting provider the inspection time and exact URL. Review the path you plan to make public and the likely effects before changing access policies or security features wholesale.

3. Record access and answer observations separately

Search crawler access is the starting point for a visibility check. Keep separate notes on whether the public URL opened, whether you verified indexing or crawling in the search service, and whether the AI answer actually cited the page. That makes the next check clear.