The SEO and AEO Standard
Search now has two front doors: the ranked results people scroll, and the answers AI systems write and cite. Both open the same way, for a page that can be fetched, read and trusted. This standard is the floor every site we build ships at.
Version 1.1. Updated November 12, 2026. By Kisora Thomas, Apotome Labs.
Summary
Every public page we ship is complete HTML before any script runs, lives at exactly one address, describes itself in structured data that matches what a person sees, opens with a plain answer, and is measured from launch day. Those five commitments cover Google and Bing, and the answer engines inside ChatGPT, Claude, Perplexity, Copilot and Siri.
Each rule carries one of three levels:
- Must. A launch blocker. The site does not go live without it.
- Should. Expected on every site, scheduled within 30 days of launch when missing.
- Could. Worth doing when a site's goals justify the effort.
The standard is reviewed every quarter against the search engines' own documentation, and sooner when one of them changes how it crawls, ranks or cites.
What changed in search
Five facts shape everything below. Each comes from the company that runs the system.
- There is no special markup for AI answers. Google states that a page needs only to be indexed and eligible for a snippet to appear in AI Overviews and AI Mode, and that no special schema is required.1 Its 2026 guide calls optimizing for these features "still SEO".2
- Most AI crawlers do not run JavaScript. A study of crawler traffic found that the crawlers from OpenAI, Anthropic, Meta, ByteDance and Perplexity fetch scripts but do not execute them.3 A site that builds its content in the browser is an empty page to them.
- Answering and training are now separate crawlers. OpenAI, Anthropic and Perplexity each publish one user agent for their search index and another for model training.4, 5, 6 A site can welcome one and refuse the other.
- Some rich results are gone. FAQ rich results stopped appearing in Google Search on May 7, 2026, and HowTo results were removed in 2023.7
- AI visibility is now measurable. Search Console reports impressions in AI Overviews and AI Mode for every site since August 31, 2026,8 and Bing Webmaster Tools reports citations in Copilot.9
Principles
Seven ideas produce every rule in this standard. When a case is not covered, we decide it from these.
- The HTML is the product. Engines read the first response from the server. Title, description, canonical, structured data and the page's real text are all in it.
- One URL per thing. Every page, product, event and article has exactly one address. Every variant redirects to it or names it.
- Structured data is a contract. It states the same facts the page shows, generated from the same source, so the two cannot drift.
- Write for the answer. Each page opens with the plain answer to the question it exists for. Detail follows.
- Be an entity, not a page. A business is named the same way on its site, its Google Business Profile, its schema and its social profiles, and they link to each other.
- Private stays private. Admin, checkout, order and account pages are marked noindex in their HTML and kept out of the sitemap.
- Measured, or it did not happen. A rule that cannot be checked is not in this standard.
The page a crawler reads
Must: every public page is prerendered. At build time each public route is written to its own HTML file with its full head and its readable body. The app then takes over in the browser. Google renders JavaScript but queues it, and recommends server-side or static rendering because not every bot can run scripts.17 Prerendering serves every crawler the same complete page, which also rules out the deprecated practice of showing bots a different version.
Must: one canonical address. One host, HTTPS only, no trailing slash. The canonical host is whichever one answers with a 200; when the apex redirects to www, the canonical is www. A canonical pointing at a redirect sends every crawler through an extra hop.18
Must: a complete head on every page. Titles run 50 to 60 characters as "Page · Brand", and drop the brand when it would push past 65, so the subject is never cut off. Descriptions are unique, 120 to 160 characters, and answer what the page is for. Public pages allow full-size image previews and unlimited snippets; nosnippet would remove a page from AI Overviews entirely.19
<title>The five-minute findability test · Apotome Labs</title>
<meta name="description" content="Before you spend a dollar on marketing, run this quick audit to see whether the internet can find you at all.">
<meta name="robots" content="index, follow, max-image-preview:large, max-snippet:-1, max-video-preview:-1">
<link rel="canonical" href="https://www.apotomelabs.com/journal/five-minute-findability-test">
<meta property="og:type" content="article">
<meta property="og:image" content="https://www.apotomelabs.com/og/default.png">
<meta property="og:image:width" content="1200">
<meta property="og:image:height" content="630">
<meta name="twitter:card" content="summary_large_image">
<script type="application/ld+json">{ "@context": "https://schema.org", "@graph": [ ... ] }</script>Must: real status codes. An unknown address returns a 404 with a useful page, never a 200 showing "not found" and never a silent redirect to the home page. Moved pages return a 301 to their new address, kept for good.
Must: private pages say so in the HTML. Admin, account, checkout and order pages carry noindex in the document itself and in an X-Robots-Tag header. A crawler reads that before any script runs.
Sitemaps and robots.txt
Must: both files are generated, never hand-written. They come from the same route list the site's router uses, so a page cannot exist without being listed, and a private page cannot be listed by mistake.
The sitemap lists every indexable page with an absolute URL. It carries a last-modified date only when the date is real, taken from the content record. Google uses lastmod only when it is consistently accurate and ignores priority and changefreq altogether,10 so a build timestamp on every entry teaches it to distrust the file. We leave those fields out.
User-agent: *
Allow: /
Disallow: /admin
Disallow: /account
Disallow: /checkout
Disallow: /order
# Answer engines: always allowed
User-agent: OAI-SearchBot
User-agent: Claude-SearchBot
User-agent: PerplexityBot
User-agent: Applebot
User-agent: bingbot
User-agent: Googlebot
Allow: /
# Training-only crawlers: the owner's choice
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
User-agent: Google-Extended
Allow: /
Sitemap: https://www.example-bakery.com/sitemap.xmlShould: IndexNow. One request tells Bing, Yandex, Naver, Seznam and Amazon that a page changed.11 Bing recommends it so Copilot cites the current version of a page.9 Google does not take part, so the sitemap still does that work.
Structured data
Must: one JSON-LD graph per page, generated from content. Each node has a stable @id so nodes reference each other instead of repeating. Every fact in it is visible on the page. The build refuses to ship a placeholder phone number, a sample address or an empty required field, because a false claim in structured data is worse than none.
Ratings are only marked up for reviews a site collects itself. Google does not show stars from a business marking up its own reviews of itself.12
| Site | Home page | Other pages |
|---|---|---|
| Local business | LocalBusiness or its subtype (Restaurant, Bakery, MedicalBusiness, ProfessionalService) with address, geo, hours and phone | Product, Service, Event, Menu |
| Organization or nonprofit | Organization or NonprofitOrganization | Event, Article, ProfilePage for staff |
| Person: artist, speaker, author | ProfilePage with a Person as its main entity | VisualArtwork, Book, CreativeWork |
| Store | Store with return and shipping policies | Product with an Offer |
| Every site | WebSite | WebPage and BreadcrumbList |
{
"@context": "https://schema.org",
"@graph": [
{
"@type": "Bakery",
"@id": "https://www.example-bakery.com/#organization",
"name": "Juniper Street Bakery",
"url": "https://www.example-bakery.com/",
"address": {
"@type": "PostalAddress",
"addressLocality": "Raton",
"addressRegion": "NM",
"addressCountry": "US"
},
"openingHoursSpecification": [{
"@type": "OpeningHoursSpecification",
"dayOfWeek": ["Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"],
"opens": "07:00",
"closes": "18:00"
}],
"sameAs": ["https://www.facebook.com/juniperstreetbakery"]
},
{
"@type": "WebSite",
"@id": "https://www.example-bakery.com/#website",
"url": "https://www.example-bakery.com/",
"publisher": { "@id": "https://www.example-bakery.com/#organization" }
}
]
}We validate every page type with Google's Rich Results Test and the Schema Markup Validator before launch. We do not add FAQ markup to pages for its own sake now that the rich result is gone, but we keep it on a real FAQ, where it is a clean question and answer format for assistants to read.
Answer engines
An assistant quotes passages, not pages. It cites a page it can fetch as plain HTML, that ranks for the questions behind the answer, that states the answer plainly, and that belongs to a business it can verify elsewhere. Sections 04 to 06 cover the first two. This section covers the rest.
Must: a crawler policy, decided once. Answer crawlers are always allowed, because blocking one removes the site from that assistant's answers and gains nothing.4, 5, 6, 21 Training crawlers are the owner's call: blocking them affects no documented search or answer product. Our default is to allow them for businesses and organizations, whose aim is to be known, and to block them for artists, photographers and authors, whose work is the product.
| Purpose | User agents | Default |
|---|---|---|
| Search and answers | Googlebot, bingbot, OAI-SearchBot, Claude-SearchBot, PerplexityBot, Applebot | Always allowed |
| Model training | GPTBot, ClaudeBot, CCBot, Google-Extended, Applebot-Extended, meta-externalagent, Amazonbot, Bytespider | Allowed for businesses, blocked for creative work |
Must: answer first. The first paragraph under the heading states who, what, where, and the one fact that matters, in sentences that stand on their own. Google's guidance is that there is no need to cut content into small pieces for AI; one clear opening does the work.2
Welcome to our website! We are passionate about bringing quality baked goods to our wonderful community, and we can’t wait to see you.
Juniper Street Bakery bakes sourdough, pastries and custom cakes in Raton, New Mexico. We are open Tuesday to Saturday, 7 a.m. to 6 p.m., and take cake orders two days ahead.
- Specific facts. "Serving the area for years" cannot be cited. "Serving Colfax County since 2019" can.
- Real tables. Hours, prices and comparisons go in HTML tables or lists, never in images.
- Real authors. Anything written in the first person names its author and links to who they are. Google counts invented author profiles as deception.13
- Checked copy. Text drafted with AI is fact-checked by a person before it ships, including titles, descriptions, alt text and structured data.14
Must: the Search generative AI control stays on. Search Console lets a site exclude itself from AI Overviews and AI Mode.23 We confirm it is set to include at launch, and change it only at an owner's written request.
Could: llms.txt. A short Markdown index of a site for language models, proposed in 2024.20 Google says it ignores such files,2 and no major assistant documents reading them. We publish one because it is generated from the same route list and costs nothing, and we do not count on it.
Local search
Most of the businesses we work with serve a town or a county, so the map and "near me" answers matter more than national rankings. The website's job is to be the source every directory and assistant can check against.
- Must: the owner owns the Google Business Profile. We are added as a manager. Its category matches the site's schema type.
- Must: name, address and phone match everywhere, character for character: the footer, the contact page, the structured data, Google, Apple Business Connect, Bing Places and the social profiles.
- Must: the town is on the page. In the home page title or opening line, and on a contact page with the address in text, hours in a table and directions in words.
- Should: reviews live on Google. The site links to the review form. Google's policies forbid incentives and asking only happy customers.15
- Could: Apple and Bing listings. Apple Business Connect and Bing Places are free, take under an hour, and feed Apple Maps, Siri, Bing and Copilot.
Performance
Core Web Vitals are measured on real visits and used in ranking.22 The thresholds have not changed since Interaction to Next Paint replaced First Input Delay in 2024.16 We aim tighter than Google's "good" so that pages still pass on slow rural connections.
| Metric | Google: good | Our target (mobile, lab) |
|---|---|---|
| Largest Contentful Paint | 2.5 s or less | 2.0 s or less |
| Interaction to Next Paint | 200 ms or less | 150 ms or less |
| Cumulative Layout Shift | 0.10 or less | 0.05 or less |
- Must: images reserve their space. Every image has a width and height, so nothing jumps as it loads. Content images ship as AVIF or WebP, lazy below the fold. The largest image above the fold loads first.
- Must: fonts are self-hosted. Two families at most, WOFF2 only, with a swap or optional display rule.
- Must: heavy features load late. 3D scenes, maps, booking widgets and video load when they are needed, not with the page.
- Must: Lighthouse mobile Performance of 90 or more on the home page and each page template, or 80 for sites built around 3D or video.
How we check
The build itself refuses to ship a site that breaks the basics: an empty or duplicate title, a missing description, a share image that does not exist, a placeholder fact in the structured data, or a page with no language set. Then, before launch:
| Check | How | Passes when |
|---|---|---|
| What crawlers see | Fetch each page type as an AI crawler, with no JavaScript | Title, description, canonical, structured data and body text are all present |
| Structured data | Rich Results Test and Schema Markup Validator | No errors on the home page and each page type |
| Addresses | Request the other host, http, a trailing slash and a nonsense path | 301, 301, 301 and 404 |
| Files | Request robots.txt, sitemap.xml and llms.txt | Each returns its own file, not the home page |
| Previews | Facebook, LinkedIn and X card validators | A 1200 by 630 image with the right title |
| Speed and access | Lighthouse mobile and axe | The targets in section 09, accessibility 90 or more |
At launch every site is registered in Google Search Console and Bing Webmaster Tools, its sitemap submitted to both. We review the reports monthly for the first quarter and quarterly after that, including the AI performance reports in each.8, 9 Every site carries cookieless analytics that store nothing on the visitor's device, so visits arriving from assistants are counted without a consent banner.
What we do not do
- No "AI optimization" tools. Google states that no third-party tool has access to its ranking or AI systems.2
- No manufactured mentions. Paid listicle placements and traded citations are not helpful by Google's own account, and its spam policies now apply to AI answers.2, 7
- No separate version for bots. Everyone, person or crawler, gets the same page.
- No keyword lists. Not in titles, not in business names, not in hidden text.
- No review tricks. No incentives, no filtering who gets asked, no marking up reviews a business wrote about itself.
- No thin location pages. A page per town exists only when there is something true and specific to say about that town.
References
- 1AI features and your websiteGoogle Search Central
- 2Optimizing your website for generative AI features on Google SearchGoogle Search Central
- 3The rise of the AI crawlerVercel
- 4Overview of OpenAI crawlersOpenAI
- 5Does Anthropic crawl data from the web, and how can site owners block the crawler?Anthropic
- 6Perplexity crawlersPerplexity
- 7Latest documentation updatesGoogle Search Central
- 8Generative AI performance reports in Search ConsoleGoogle Search Central Blog
- 9Introducing AI Performance in Bing Webmaster ToolsBing Webmaster Blog
- 10Build and submit a sitemapGoogle Search Central
- 11IndexNow: frequently asked questionsIndexNow
- 12Local business structured dataGoogle Search Central
- 13Creating helpful, reliable, people-first contentGoogle Search Central
- 14Using generative AI content on your websiteGoogle Search Central
- 15Maps user contributed content policyGoogle
- 16Web Vitalsweb.dev
- 17Understand JavaScript SEO basicsGoogle Search Central
- 18How to specify a canonical URLGoogle Search Central
- 19Robots meta tag, data-nosnippet, and X-Robots-Tag specificationsGoogle Search Central
- 20The /llms.txt filellmstxt.org
- 21About ApplebotApple
- 22Understanding page experience in Google Search resultsGoogle Search Central
- 23Search generative AI controlSearch Console Help