Cynicus Rex
- 3 Posts
- 6 Comments
Joined 8 months ago
Cake day: January 21st, 2024
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
2·1 month agoWhat does the acronym SO stand for?
67·1 month agoLies, as in that it’s not really “blocking” but a mere unenforceable request? If you meant something else could you please point it out?
Unfortunate indeed.
“Can AI bots ignore my robots.txt file? Well-established companies such as Google and OpenAI typically adhere to robots.txt protocols. But some poorly designed AI bots will ignore your robots.txt.”
#TL;DR:
User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: Google-Extended Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Amazonbot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Omgilibot Disallow: / User-Agent: FacebookBot Disallow: / User-Agent: Applebot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Bytespider Disallow: / User-agent: Claude-Web Disallow: / User-agent: Diffbot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: YouBot Disallow: /
7·1 month agoThanks for the share, had no idea. It felt so nice that I already added two contributions to OpenStreetMap.
What if we could avoid typing “+1” with upvotes? ;)