Site Grounding: When to Opt In or Out of AI Search

Controlling site grounding means deciding which pages generative search systems should be able to retrieve and use in their answers. For SEO teams and multi-site operators, the practical choice is usually not a single AI opt-in switch: it is a set of decisions about public access, crawling, indexing, and search-result eligibility.

Allowing access can make valuable content eligible for discovery in AI-powered search, while restricting access can protect material that should not be publicly retrievable. The right policy separates those goals by page type and platform, preserves conventional search visibility where appropriate, and measures outcomes without confusing eligibility with guaranteed inclusion.

What site grounding means,and what you actually control

Grounding connects an AI-generated response to information retrieved from external sources. Google describes grounding in Search as retrieval-augmented generation, or RAG: its systems retrieve relevant, up-to-date pages from the search index to support generative responses.

To opt into site grounding in practice, make the intended pages publicly accessible and eligible for the relevant search system to crawl and retrieve. To opt out, use the platform’s supported exclusion controls or restrict access. Neither approach is a universal switch across every AI service.

That distinction matters because a website can be accessible without being selected for a particular answer. Google says its core Search ranking systems help retrieve relevant pages. Permission to retrieve content establishes an opportunity; it does not require the system to use your page.

Separate the decisions behind “AI visibility”

Access: Can the relevant crawler reach the page, including through your hosting and CDN configuration?
Indexing and eligibility: Is the page permitted to participate in the relevant search experience?
Selection: Does the system consider the page useful for the question being answered?
Business value: Does that visibility support discovery, customer education, qualified visits, or another defined objective?

The first two areas are where technical controls matter most. The third depends on retrieval and relevance systems that site owners do not directly operate. The fourth requires your own evaluation rather than an assumption that every AI appearance is beneficial.

Start by defining the outcome you want. A public documentation library may benefit from being retrieved for troubleshooting questions. A private customer workspace has a different purpose and should not become public merely to pursue search visibility. Treating both as one site-wide AI decision creates unnecessary conflict between acquisition, content, and security teams.

When opting into generative search supports your goals

Opting in makes sense when you want public content to help people discover, evaluate, or use your organization’s expertise. Google’s guidance ties generative search visibility to crawlable public pages and emphasizes that established SEO best practices remain foundational.

The strongest starting point is content you already want prospective customers or users to find through search. Rather than exposing additional material simply because generative search exists, review whether your existing public content is both accessible and worth retrieving.

Prioritize pages with a clear public purpose

Product and service explanations: Pages that clearly explain what you offer, who it serves, and where its limits apply.
Public support documentation: Instructions that help users solve problems without revealing private account information.
Original research and analysis: Material whose methods, findings, and context can be evaluated by readers.
Expert educational content: Pages that answer specific questions with useful detail rather than repeating generic summaries.

These are prioritization examples, not categories that search providers promise to favor. Their practical advantage is that each has a recognizable audience need and a reason to exist independently of AI visibility.

Choose the business objective before expanding access

For a software company, the objective might be making public implementation guidance easier to discover. For an agency, it might be exposing original expertise to potential clients. For a multi-location business, it might be helping people retrieve accurate service information.

Write that objective beside the page group in your content inventory. This keeps the decision concrete: you are enabling retrieval of a particular resource for a particular purpose, not agreeing that every page should be available to every automated system.

Also consider the trade-off between answer visibility and visits to your website. An AI-generated response may address part of the reader’s question before they visit. Evaluate whether the content’s purpose is served by wider access to the information, by an on-site action, or by both; do not assume those outcomes are interchangeable.

A selective approach is often more practical than a blanket expansion. Begin with public pages that have a clear owner, current information, and an established role in your acquisition or customer-support strategy.

When opting out is the more appropriate decision

Opting out is appropriate when public machine retrieval conflicts with the content’s intended audience, commercial purpose, or confidentiality requirements. Search visibility is not a reason to expose material that should remain restricted.

Google’s grounding guidance centers on publicly accessible content rather than private or hidden site sections. That makes the first question straightforward: should an unauthenticated member of the public be able to read this page at all?

Distinguish private content from public content you do not want surfaced

Private customer records, internal operating documents, and restricted workspaces belong behind appropriate access controls. A crawler preference is not a substitute for deciding who is authorized to access sensitive information.

A different situation arises when content is public but you want to limit its presence in search listings or AI formats. Google says site owners have controls over what appears in those experiences, including indexing and snippet-style eligibility. The appropriate choice depends on the outcome you want and the controls supported by that platform.

Confidential content: Keep it restricted to its authorized audience rather than relying on search exclusion alone.
Public content with limited search value: Assess whether it should remain eligible for search inclusion at all.
Public content with presentation concerns: Review supported listing and snippet controls before choosing a broader exclusion.
Content with mixed purposes: Consider separating a useful public explanation from the restricted material it describes.

That last option can resolve a common tension. A business may want people to discover its research program without making every underlying document public. It can publish an accurate public overview while keeping restricted material in an appropriately controlled environment.

Account for the visibility you may give up

Controls that affect crawling, indexing, or listing eligibility can reach beyond the specific AI appearance you are trying to prevent. Before applying them broadly, determine whether conventional search discovery for the same pages remains important.

If the organization wants ordinary search traffic but not generative use, do not assume one universally supported directive will deliver that exact combination. Check the current platform documentation and evaluate the available controls against that specific requirement. Where the required separation is unavailable or unclear, record the limitation before making the decision.

How Google, Bing, and OpenAI approach grounding controls

The platforms share a reliance on accessible information, but their controls and reporting should not be treated as interchangeable. Maintain separate policies for Google Search, Bing’s AI search experiences, and ChatGPT Search rather than copying one configuration across all three.

Google: manage Search access and eligibility

Google describes generative grounding as dependent on public, crawlable content retrieved through its Search systems. Its Search Central guidance also says site owners can control what appears in Search listings, including AI formats.

For an SEO team, that points toward familiar areas of work: confirm that the intended pages are accessible, review indexing and snippet-related settings, and maintain useful content. Google’s guidance does not frame this as a separate universal AI-only enrollment process.

Google has also introduced dedicated Search Generative AI performance reporting in Search Console, with coverage for Search and Discover. Use the available reporting to investigate visibility rather than relying only on occasional manual searches.

Preferred sources is a separate development. Google is extending that feature to AI Overviews and AI Mode, indicating that source selection also involves controls within the search product. It should not be mistaken for a publisher-operated switch that guarantees grounding.

OpenAI: distinguish OAI-Searchbot from Google’s crawlers

OpenAI Help explicitly connects ChatGPT Search eligibility to allowing OAI-Searchbot to crawl a website and permitting traffic from OpenAI’s published searchbot IPs through the host or CDN. OAI-Searchbot is an OpenAI search crawler, not a Google crawler.

This distinction prevents a consequential implementation mistake: allowing or blocking OAI-Searchbot does not, by itself, establish a Google Search grounding policy. For ChatGPT Search, examine both crawler permissions and network access rather than looking for a content submission form.

Bing: use grounding transparency without treating feedback as exclusion

Microsoft’s Bing Webmaster Tools update introduced Topics Portfolio in beta within AI Performance. It groups grounding queries into thematic clusters, helping site owners understand the subject areas in which their sites are used in AI Search.

The update also adds thumbs-up and thumbs-down feedback on topic rows and opportunity queries. That provides a relevance-feedback mechanism, but it is not the same as a direct site-level grounding toggle.

Microsoft’s Web IQ announcement similarly emphasizes retrieval efficiency and grounding quality. For site owners, the useful implication is to assess how well content answers the relevant topics, not to infer an unpublished permission control from the system’s performance goals.

Turn your grounding policy into a page-level implementation plan

A policy becomes useful when it identifies the pages involved, the intended outcome, and the configuration responsible for that outcome. Avoid beginning with a site-wide crawler change before you know which content needs different treatment.

Inventory meaningful page groups. Separate public marketing pages, documentation, editorial resources, account areas, and restricted content. Group pages by purpose rather than treating every URL as an unrelated decision.
Record the desired audience. State whether each group is public, customer-only, employee-only, or otherwise restricted. Resolve access questions before optimizing retrieval.
Specify the platform outcome. Note whether you want Google Search eligibility, ChatGPT Search eligibility, Bing AI visibility, or a restriction on a particular experience.
Review the relevant technical layer. Examine crawl permissions, indexing and listing controls, and host or CDN behavior as applicable. For OpenAI search access, include its published searchbot IP requirements.
Validate representative pages. Check actual page access and configuration instead of assuming that an intended policy has been deployed consistently.
Assign ownership and a review trigger. Identify who approves the policy and when changes to templates, infrastructure, or content purpose require another review.

Check permissions and delivery separately

A policy allowing a crawler is not useful if the infrastructure prevents it from accessing the intended content. OpenAI’s guidance makes this explicit by addressing both OAI-Searchbot crawling and host or CDN permissioning.

For a multi-site operation, inspect these layers separately in your audit. One team may manage crawler rules while another manages network security. A shared dashboard can track the intended status, the observed access result, and the person responsible for resolving a mismatch.

Avoid assuming every exclusion mechanism has the same effect

Crawl access, index participation, and presentation controls operate on different parts of the discovery process. Choose the control that corresponds to the desired outcome and verify its current documented behavior for the relevant service.

For example, a requirement to protect private information is an access-control requirement. A requirement to change whether a public page appears in search is a search-eligibility question. Treating those requirements as identical can lead to either unnecessary loss of visibility or insufficient protection.

Document the reason for each exception. A future administrator should be able to tell whether a restriction protects confidential content, implements a publishing decision, or reflects an outdated configuration that needs review.

Keep search grounding separate from model-training decisions

Retrieving a public page to support a search answer and using information to train a model are different activities. A decision about one should not be assumed to settle the other.

The OpenAI search guidance discussed here concerns OAI-Searchbot and eligibility for inclusion in ChatGPT Search. OpenAI’s enterprise privacy and data-control materials address a different context: enterprise customer data is not used for training unless customers explicitly opt in, and account owners can later opt out again for shared traffic settings.

A search-crawler permission answers whether a service can retrieve your public content for search. It should not be treated as a complete statement of your organization’s training, privacy, or data-sharing policy.

The distinction is especially important for companies that both publish websites and use enterprise AI products. Their public publishing configuration and their enterprise account settings belong in separate, coordinated reviews.

Ask the right question of each control

For public search: Which service can retrieve which public pages, and what makes those pages eligible for its search experience?
For enterprise AI usage: Which account settings and contractual terms govern the data employees submit?
For confidential information: Which access controls keep the material restricted to authorized users?
For governance: Who verifies that these decisions remain aligned as products and policies change?

This separation also improves internal communication. A request to “opt out of AI” is too broad to implement reliably without clarification. Ask whether the requester means public search retrieval, generative presentation, model training, employee data sharing, or all of those areas.

Once the scope is explicit, route each requirement to the appropriate owner. SEO can manage search visibility decisions, infrastructure teams can validate access, and privacy or legal teams can review the relevant data-use requirements. No single search setting should be presented as resolving all of them.

Improve grounding eligibility without chasing AI-specific hacks

Once you decide that a page should be available for retrieval, the next task is to make it worth finding. Google’s guidance emphasizes valuable, unique, non-commodity content and explicitly challenges overstated AEO and GEO tactics.

That supports a practical priority order: resolve access problems, establish a clear purpose for the page, and improve the substance of the answer. Do not divert resources into an alleged grounding shortcut without documented evidence that it addresses an actual requirement.

Make the information useful in its own right

Answer a defined question. State what the page helps the reader understand or do, and keep the scope coherent.
Contribute original value. Use your own product knowledge, documented methods, research, or subject-matter expertise where available.
Explain conditions and limits. Make clear when the guidance applies and when the reader needs a different approach.
Keep public information accurate. Review pages when the underlying service, process, or evidence changes.
Use readable structure. Descriptive ings and focused paragraphs help readers locate the information they need; they are not a guarantee of AI selection.

Consider a public implementation guide. A generic introduction to the topic may add little if many other pages say the same thing. A precise explanation of your product’s supported workflow, prerequisites, and limitations gives the page a distinct purpose.

Apply the same discipline to content expansion. If an AI visibility report reveals a topic where you want stronger coverage, first decide whether you possess useful information to contribute. A missing appearance is not, by itself, a reason to publish another interchangeable article.

For agencies, this creates a defensible client recommendation: improve the page’s usefulness and accessibility because those are controllable, documented priorities. Avoid promising that a formatting change, special phrase, or new label will secure a place in a generated answer.

Measure AI search visibility without confusing signals

Measurement should show whether your access policy is working and whether the resulting visibility matters. Keep those questions separate so a lack of appearances does not automatically trigger unnecessary technical changes.

Start with access and policy compliance

For each important page group, record the intended public-access status and relevant search permissions. Check whether the observed configuration matches that intent, including infrastructure restrictions where applicable.

This is a technical compliance check, not a performance judgment. A page can be correctly accessible without appearing in the answers you inspect. Conversely, a reported appearance should not replace a review of whether the underlying content is appropriate to expose.

Use platform reporting for visibility and topic relevance

Google’s dedicated generative AI performance reporting in Search Console provides a platform-level view of visibility. Bing’s Topics Portfolio adds a thematic view of grounding queries, helping teams examine which subject areas connect their content to AI Search.

Use the reports according to the information they actually expose. Do not assume every platform offers identical metrics, attribution, or topic detail. Where reporting is incomplete for your business question, describe that gap rather than filling it with an estimate presented as fact.

Group observations by content purpose. Compare documentation with documentation and educational content with similar resources, rather than mixing every page into one conclusion.
Review topic fit. Assess whether the reported themes match the audience and expertise the site is intended to serve.
Connect visibility to business evidence. Examine relevant visits, inquiries, or support outcomes where your own measurement supports that connection.
Record changes alongside results. Keep a log of access changes, content updates, and reporting observations so future reviews have context.

Manual searches can help investigate an example, but they should not become the entire reporting system. A team needs a repeatable view of its policy, observed visibility, and business outcomes, not a collection of isolated screenshots.

For multi-site reporting, centralize the decision record as well as the performance data. Otherwise, a site deliberately restricted from retrieval may be incorrectly flagged as an optimization failure, while an accessible site with weak content may be mistaken for a permissions problem.

Build a repeatable opt-in and opt-out review

The best grounding policy is specific enough to implement and simple enough to maintain. Review content groups when their purpose, access requirements, platform controls, or business priorities change rather than treating the initial setting as permanent.

Use a short decision sequence

Should the content be public? If not, prioritize appropriate access restrictions.
Should it be discoverable through the relevant search service? If yes, preserve the required crawl and eligibility conditions.
Is there a narrower presentation concern? Review the service’s supported listing or snippet controls before making a broader change.
Does the content offer enough value to maintain? Improve, consolidate, or reconsider it according to its purpose, not merely its AI visibility.
Can the outcome be verified? Assign a technical check and a reporting review appropriate to the platform.

Assign responsibilities before a conflict arises. SEO should explain discovery trade-offs; content owners should define publishing intent; infrastructure teams should validate delivery; and privacy or security stakeholders should determine requirements for restricted information.

For an agency managing multiple clients, keep approvals client-specific. One organization may prioritize broad public distribution of its educational content, while another may have a narrower publishing model. A shared operational checklist can support both without imposing identical access decisions.

The same principle applies within an enterprise. A marketing resource center, a public developer portal, and an authenticated customer workspace need different treatment even when they share a brand. Centralized oversight should make those differences visible, not erase them.

Finally, establish what would justify reversing the decision. A change in confidentiality requirements, content ownership, or commercial purpose may warrant restriction. A previously restricted resource that becomes a maintained public guide may warrant a fresh eligibility review. This makes opting in or out an accountable publishing decision rather than a reaction to AI-search lines.

Control site grounding by separating public access, search eligibility, source selection, and business value. Allow retrieval where it serves a clear purpose, restrict content that should not be public, and use each platform’s documented controls without assuming that search settings also govern training or private data use.

Start with an inventory of your highest-value public pages and your most sensitive restricted areas. Record the intended policy, verify the technical implementation, and monitor the available platform reports from a shared dashboard so your team can act on real mismatches rather than chase an imaginary universal grounding switch.

When to opt in or out: controlling site grounding in generative search