GEO and AI Search: A Practical, Measurable Guide

The short version: generative engine optimization (GEO) is good technical SEO plus original, attributable information and honest measurement. There is no special schema, magic text file, or reliable prompt pattern that guarantees a citation. Make the page indexable, publish information worth citing, keep it current, and measure citations and conversions where the platforms expose them.

Google now says this directly: its existing search quality and technical requirements also apply to AI Overviews and AI Mode. It specifically advises site owners not to overfocus on artificial content chunking, unnecessary AI text files, or special markup. See Google’s guide to generative search visibility.

What GEO means in practice

Generative engine optimization is the work of making a source eligible, useful, and trustworthy enough to be referenced in an AI-generated answer. It does not replace SEO because most retrieval systems still depend on a search index, a crawler, or both.

The useful distinction is about measurement:

Do not reduce GEO to a single “citation probability” score. Different products retrieve, summarize, cite, and report sources differently. A page can be crawled without being cited, cited without receiving a click, or visited without a usable referrer.

Step 1: separate search, training, and user-directed crawlers

Crawler names encode different purposes. Treating every AI-related user agent as one bot leads to bad policy and bad analytics.

Sources: OpenAI publisher guidance, Anthropic crawler guidance, Perplexity crawler guidance, and Google’s AI feature controls.

If the goal is discovery in AI search, allow the search crawlers you want. Training is a separate policy decision. For example:

# Search discovery
User-agent: OAI-SearchBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

# Separate training preference
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

Crawler groups do not merge under the Robots Exclusion Protocol. If you create a bot-specific group, repeat every rule that bot needs instead of assuming it inherits User-agent: *.

Test the whole request path

A permissive robots.txt is not enough when a CDN, WAF, rate limiter, or JavaScript challenge returns a 403 or 429. Test representative public pages with the actual search-crawler user agent and verify a usable 200 response. When a crawler publishes IP ranges, verify both the claimed user agent and network source before creating an allow rule.

WebDecoy’s crawler verification tools and Agent Identity product separate a claimed name from a proven identity. That distinction matters: a copied user-agent string is not an authentication mechanism.

Step 2: make the source worth citing

The strongest citation candidates are not generic summaries. They contribute something a generated answer cannot obtain from every competing page.

Prioritize:

• Original measurements with a stated window and sample size.
• Stable definitions with explicit boundaries and counterexamples.
• Reproducible methods, code, and configuration.
• Versioned compatibility and installation information.
• Public datasets with a revision date, methodology, and license.
• Comparisons that cite primary sources and disclose what was not tested.

For a technical page, put the direct answer near the top, then show the evidence. A useful opening often contains:

1. The answer or definition.
2. The scope and applicable version.
3. The most important limitation.
4. A link or path to reproduce the result.

That is good documentation design for people. It also gives retrieval systems a self-contained passage without manufacturing dozens of near-duplicate pages.

Prefer attributable claims over unsupported precision

Weak:

Our advanced system is highly accurate and better than legacy bot protection.

Useful:

In WebDecoy’s 27 July–26 August 2026 dogfood snapshot, the site recorded 13,828 attributable AI crawler fetches and five referrals carrying a known AI-platform referrer. Because platforms can omit the referrer, five is a floor rather than a complete count.

The useful version states the source, window, measurement, and limitation. It can be cited without turning a local observation into a universal claim.

Step 3: publish honest metadata

Use ordinary web metadata well:

• A unique title and description that match the visible page.
• A self-referencing canonical URL.
• Visible publication and modification dates when freshness matters.
• A named author or accountable organization.
• Structured data that describes content users can actually see.
• A complete sitemap with accurate lastmod values, not the build time stamped onto every URL.

There is no special GEO schema. Article, TechArticle, SoftwareApplication, Dataset, and BreadcrumbList can clarify what a page contains, but they do not create authority. Never publish invented reviews, prices, performance numbers, or dates in JSON-LD. Structured data must agree with the page.

What about llms.txt?

llms.txt is an experimental convention used by some developer-documentation tools. It can be a convenient, generated index for an agent, but major search visibility does not depend on it. Google explicitly says no AI text file is required for AI Overviews or AI Mode.

If you publish one, generate it from the same navigation or sitemap as the site so it cannot become a second, stale information architecture. Do not treat it as a ranking lever.

There is likewise no established /.well-known/ai-instructions.txt standard that makes answer engines follow brand wording or attribution preferences. Put important identity, licensing, pricing, and citation information in visible pages and standard metadata.

Step 4: measure citations, referrals, and conversions separately

No single metric describes AI visibility.

Citation reporting

Bing Webmaster Tools AI Performance reports when URLs are cited across Microsoft AI experiences. Google’s Generative AI performance report is rolling out in Search Console; when available, it separates visibility in generative features while retaining ordinary Search reporting.

Record, by canonical URL:

• Citation or referenced-URL count.
• Answer surface and query/theme where exposed.
• Search impressions and clicks.
• Publication and modification date.
• Meaningful conversion after arrival.

Referral reporting

OpenAI says ChatGPT citation links include utm_source=chatgpt.com. Preserve that parameter in analytics and group known AI referrers without rewriting canonical URLs.

Referrals are still incomplete. Some products omit the HTTP Referer, open links through an intermediary, or send a user-directed fetch that is not a human visit. WebDecoy’s AI Traffic report therefore treats identified referrals as a floor and its crawl-to-referral ratio as an upper bound.

Crawl reporting

Crawler activity answers which automated clients fetched which pages. It does not prove that a model trained on a page, that an answer cited it, or that a crawl caused a later visit.

Use crawl activity as a diagnostic:

• No crawl: inspect discovery, internal links, robots, status codes, and WAF events.
• Crawled but absent from citation reports: improve the source or reassess query relevance.
• Cited but no qualified traffic: evaluate the citation context and landing-page promise.
• Qualified traffic but no conversion: fix the landing path, not the crawler policy.

Step 5: automate evidence collection, not mass rewriting

Automation is useful for finding stale facts, missing dates, broken citations, schema mismatches, and pages whose measured intent conflicts with their copy. It is risky when it generates large numbers of query permutations, FAQs, ratings, or unsourced claims.

A safe review pipeline should:

1. Read search, citation, crawl, referral, and conversion data.
2. Select a small set of pages with a clear user or freshness problem.
3. Retrieve current primary sources.
4. Propose changes with every factual claim linked to evidence.
5. Require human review.
6. Validate rendered metadata, links, structured data, and sitemap output.
7. Record the baseline and wait for a comparable measurement window.

Do not infer “not cited” from zero referrals. Do not infer “cited” from a crawler hit. Do not publish generated FAQ or schema content that is absent from the visible page.

A practical 30-day plan

Week 1: access and baselines

• Verify the marketing and documentation hosts in Google Search Console and Bing Webmaster Tools.
• Submit and inspect both sitemap indexes.
• Confirm OAI-SearchBot, Claude-SearchBot, PerplexityBot, and Googlebot can fetch representative pages.
• Export the first URL-level search and citation baselines.

Week 2: source quality

• Pick the five pages already earning relevant impressions or citations.
• Add missing methodology, applicable versions, visible update dates, and primary-source citations.
• Consolidate overlapping pages instead of creating new query variations.

Week 3: citable assets

• Publish one original dataset or reproducible tool at a stable URL.
• Link it from the relevant research article, product page, and documentation.
• Offer JSON or CSV when the underlying information is naturally tabular.

Week 4: conversion and review

• Measure AI referral to signup or another meaningful action.
• Compare URL-level citation and search visibility without changing the measurement window.
• Keep changes that improve relevant visibility or conversion; revise pages that attract the wrong intent.

The bottom line

GEO is not a collection of secret tags. It is the discipline of making public information accessible, distinct, attributable, current, and measurable.

WebDecoy contributes the request-side evidence: which crawlers arrived, which identity claims held up, what they fetched, and which identifiable AI referrals returned. Search and answer platforms contribute citation and impression data. Analytics contributes the conversion outcome. Keep those observations separate, then join them by canonical URL and time window.

Measure AI crawler activity and referrals with WebDecoy →