How to block AI Crawler Bots using robots.txt file
Open link in next tab
Just a moment...
https://www.cyberciti.biz/web-developer/block-openai-bard-bing-ai-crawler-bots-using-robots-txt-file/
https://www.cyberciti.biz/web-developer/block-openai-bard-bing-ai-crawler-bots-using-robots-txt-file/
Lies, as in that it's not really “blocking” but a mere unenforceable request? If you meant something else could you please point it out?
That is what they meant, yes. The title promises a block, completely preventing crawlers from accessing the site. That is not what is delivered.
Lie. Or at best, dangerously wrong. Like saying "Crosswalks make cars incapable of harming pedestrians who stay within them."
It's better than saying something like "there's no point in robots.txt because bots can disobey is" though.
Assuring someone that they have control of something and the safety that comes with it, when in fact they do not, is well outside the realm of a simplification. It's just plain false. It can even be dangerous.