Cloudflare Launches Redirects for AI Training to Force AI Crawlers to Follow Canonical URLs

<h2>Breaking: Cloudflare Offers New Tool to Combat AI Training on Outdated Content</h2> <p>Cloudflare today announced <strong>Redirects for AI Training</strong>, a feature that converts existing canonical tags into HTTP 301 redirects for verified AI training crawlers. The solution turns advisory signals into enforceable commands, ensuring AI models train only on the most current version of web content.</p><figure style="margin:20px 0"><img src="https://cf-assets.www.cloudflare.com/zkvhlag99gkb/65vHEPjmVJ1o4Bz3t4ilEv/9d4797939a26b96ddc1085abcaf9e9c4/OG_Share_2024-2025-2026__30_.png" alt="Cloudflare Launches Redirects for AI Training to Force AI Crawlers to Follow Canonical URLs" style="width:100%;height:auto;border-radius:8px" loading="lazy"><figcaption style="font-size:12px;color:#666;margin-top:5px">Source: blog.cloudflare.com</figcaption></figure> <p>“Status codes are the language the web uses to communicate policy to crawlers. With one toggle, canonical tags become redirects for AI training crawlers,” said a Cloudflare spokesperson. The feature is available immediately on all paid Cloudflare plans.</p> <h2>Background</h2> <p>AI training crawlers have been consuming deprecated content at alarming rates. On Cloudflare’s own developer documentation site, AI crawlers visited 4.8 million times in the last 30 days, ingesting outdated pages as often as current ones <a href="#problem">despite clear signals</a>.</p> <p>Standard web signals like <code>noindex</code> meta tags, deprecation banners, and canonical tags failed to deter them. “Every advisory signal says this content is outdated, look elsewhere, but AI crawlers don’t reliably honor those signals,” Cloudflare reported.</p> <h2>How It Works</h2> <p>The <code>&lt;link rel="canonical"&gt;</code> tag, defined in RFC 6596 and already present on 65–69% of web pages, tells automated systems which URL is authoritative. Redirects for AI Training enforces that directive by issuing a 301 redirect to verified AI training crawlers.</p> <p>This closes a critical gap: previously, no inline directive could say “don’t train on this.” Banners work for humans but AI crawlers read entire page text, including the warning, and often return thousands of times.</p> <blockquote> <p>“Keeping a deprecated page live with a warning may work for humans, who read the notice and navigate on, but AI training crawlers ingest the full text and risk treating the banner as just one more paragraph.” — Cloudflare</p><figure style="margin:20px 0"><img src="https://blog.cloudflare.com/cdn-cgi/image/format=auto,dpr=3,width=64,height=64,gravity=face,fit=crop,zoom=0.5/https://cf-assets.www.cloudflare.com/zkvhlag99gkb/2G12Bcuv6e6Rz3pcl5oUpL/4b8d1bf1a33556b5d98446090d3d03d5/Cam_Whiteside.png" alt="Cloudflare Launches Redirects for AI Training to Force AI Crawlers to Follow Canonical URLs" style="width:100%;height:auto;border-radius:8px" loading="lazy"><figcaption style="font-size:12px;color:#666;margin-top:5px">Source: blog.cloudflare.com</figcaption></figure> </blockquote> <h2>What This Means</h2> <p>Redirects for AI Training turns passive signals into active enforcement. Site owners no longer need to manually block crawlers per path or per update. Instead, existing canonical infrastructure does the work automatically.</p> <p>The effect is cumulative: because AI agents don’t always fetch content live but rely on trained models, feeding them current URLs prevents propagation of outdated information. For the broader web, Cloudflare’s <a href="https://radar.cloudflare.com/ai">Radar AI Insights</a> now includes Response status code analysis, showing the distribution of 2xx, 3xx, 4xx, and 5xx codes AI crawlers receive across all Cloudflare traffic.</p> <p>“What crawlers need is specific direction: ‘Here is where the current content lives,’” the company emphasized. The new feature delivers exactly that, turning the web’s existing canonical tag infrastructure into a reliable guide for AI training.</p> <h3>Key Benefits</h3> <ul> <li>One-click toggle on all paid Cloudflare plans</li> <li>No new HTML tags required – works with existing canonical URLs</li> <li>Works with platforms like WordPress, Contentful, and EmDash that generate canonical tags automatically</li> <li>Provides a clear, enforceable alternative to robots.txt or blocking, which creates a signal void</li> </ul> <h2>Availability</h2> <p>Redirects for AI Training is rolling out now to all paid Cloudflare customers. For more details, visit the <a href="https://blog.cloudflare.com/redirects-for-ai-training">official announcement</a>.</p>