Login

Recent searches

No recent searches

Search options

Only available when logged in.
mastodon.ar.al is one of the many independent Mastodon servers you can use to participate in the fediverse.
This is my personal fediverse server.

Administered by:

Server stats:

1
active users

mastodon.ar.al: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.7

Public
Liam @ GamingOnLinux 🐧🎮

People and SEO Spam like this are ruining the internet. Fuck this shite.

Aral Balkan

@gamingonlinux What an absolute asshat! And he’s proud of it too. Un-fucking-believable!

Nov 27, 2023, 04:40 PM·Public·Mona for iPhone
0boosts·2favorites
Public
Deus

@aral @gamingonlinux n00b question. I have blocked Google using a robots.txt rule for my instance and have also done the following. Can they still scrape off content from my site? Granted there are tools available to do that but wanted to know still.

  1. List of “AI” bots to block from crawling your website: https://neil-clarke.com/block-the-bots-that-feed-ai-models-by-scraping-your-website/
  2. From OpenAI. This will take a few minutes. Disallow GPTBot and then customize the access to GPTBot in robots.txt: https://platform.openai.com/docs/gptbot
neil-clarke.comBlock the Bots that Feed “AI” Models by Scraping Your Website – Neil Clarke
ExploreLive feeds
About