AI SEO Workplace
Machine-Readable Infrastructure

Meta Robots Patterns

Last reviewed:

Add to the <head> of HTML pages. For HTTP-header equivalents (needed for non-HTML resources), see X-Robots-Tag.

Core directives

<!-- Default: index this page and follow its links (implicit; no tag needed) -->
<meta name="robots" content="index, follow">

<!-- Do not index this page; still follow links -->
<meta name="robots" content="noindex, follow">

<!-- Index this page; do not follow links -->
<meta name="robots" content="index, nofollow">

<!-- Do not index and do not follow links -->
<meta name="robots" content="noindex, nofollow">

<!-- Do not show a cached version of this page -->
<meta name="robots" content="noarchive">

<!-- Do not show a snippet for this page in search results -->
<meta name="robots" content="nosnippet">

Snippet and preview controls

<!-- Limit text snippet to 160 characters -->
<meta name="robots" content="max-snippet:160">

<!-- Disable text snippet entirely (same as nosnippet) -->
<meta name="robots" content="max-snippet:0">

<!-- Allow unlimited snippet (default) -->
<meta name="robots" content="max-snippet:-1">

<!-- Control image preview size: none | standard | large -->
<meta name="robots" content="max-image-preview:large">

<!-- Limit video preview to 30 seconds -->
<meta name="robots" content="max-video-preview:30">

<!-- Combine multiple directives -->
<meta name="robots" content="max-snippet:200, max-image-preview:large">

AI and training controls

<!-- Google: opt out of AI Overviews and AI Mode previews (controls snippet use) -->
<meta name="robots" content="nosnippet">

<!-- Google: limit AI feature snippet length -->
<meta name="robots" content="max-snippet:0">

<!-- Generic: some crawlers honour these non-standard tags -->
<meta name="robots" content="noai">
<meta name="robots" content="noimageai">

noai and noimageai are not part of any formal standard; Google does not document support for them. AI training controls are primarily handled via robots.txt user-agent blocks, not meta tags.

Bot-specific targeting

<!-- Target Googlebot only -->
<meta name="googlebot" content="noindex">

<!-- Target Googlebot-News (Google News) -->
<meta name="googlebot-news" content="nosnippet">

<!-- Target Bingbot only -->
<meta name="bingbot" content="noarchive">

Inline data-nosnippet

<p>This paragraph appears in snippets normally.</p>

<p data-nosnippet>This paragraph is excluded from snippets.</p>

Supported on <div>, <section>, and <span>. Useful for paywalled content where the page should stay indexed but preview text should be limited.

Field notes

  • noindex must be discoverable by the crawler — don’t block the page in robots.txt and also set noindex; the crawler can’t see the tag if it can’t fetch the page.
  • Bot-specific directives override the generic robots tag for that bot, and the more restrictive interpretation wins when they conflict.
  • When both the X-Robots-Tag HTTP header and HTML meta robots are present, the most restrictive combination applies.
  • nosnippet is Google’s primary mechanism for opting out of AI Overview previews, not just traditional snippets.
  • unavailable_after: 2025-12-31 tells Google to stop showing a page after a date — useful for time-limited promotions or events.