Cloudflare Bot Preference Sync will write your robots.txt from 21 August 2026 - and it ignores your custom rules
Cloudflare's Bot Preference Sync, announced 21 August 2026, writes robots.txt from your Search, Agent and Training settings. It is category-wide only and its documentation page returns a 404.
