8Petros [$ rm -rv /capitalism/*]

8Petros [$ rm -rv /capitalism/*] 1 miesiąc temu friendica (DFRN)

#marginalia #search #splash screen shows some cryptic messages every time I open it. There is even an accordion "Why am I seeing it?", but I never managed to open it - too fast transition to actual search page.

Anyone explain it to me, please?

#search #marginalia #splash

Internet

noodlejetski 1 miesiąc temu • mastodon

LLM scraper protection github.com/TecharoHQ/anubis

Why am I seeing this?

You are seeing this because the administrator of
this website has set up Anubis to protect the
server against the scourge of AI companies
aggressively scraping websites. This can and
does cause downtime for the websites, which
makes their resources inaccessible for everyone.

Anubis is a compromise. Anubis uses a Proof-of-
Work scheme in the vein of Hashcash, a
proposed proof-of-work scheme for reducing
email spam. The idea is that at individual scales
the additional load is ignorable, but at mass
scraper levels it adds up and makes scraping
much more expensive.

Ultimately, this is a hack whose real purpose is to
give a "good enough" placeholder solution so that
more time can be spent on fingerprinting and
identifying headless browsers (EG: via how they
do font rendering) so that the challenge proof of
work page doesn't need to be presented to users
that are much more likely to be legitimate.

Please note that Anubis requires the use of
modern JavaScript features that plugins like
JShelter will disable. Please disable JShelter or
other such plugins for this domain.

GitHub - TecharoHQ/anubis: Weighs the soul of incoming HTTP requests to stop AI crawlers

Weighs the soul of incoming HTTP requests to stop AI crawlers - TecharoHQ/anubis

^GitHub

Ten wpis został zedytowany (1 miesiąc temu)

8Petros [$ rm -rv /capitalism/*] 1 miesiąc temu friendica (DFRN)

Would make sense to mine some XMR at the same time, to support the website.

8Petros [$ rm -rv /capitalism/*] 1 miesiąc temu friendica (DFRN)

@timorl - should we deploy it?

@timorl

timorl 1 miesiąc temu • akkoma

Probably not worth it if we don’t currently have resource trouble. We should probably get a bit better monitoring though. <_<”

Oh and fyi this is likely not a long-term solution, if it gets too widespread then working around this specific countermeasure shouldn’t take more than a couple of days for a single dev.

Marginalia 1 miesiąc temu • mastodon

It's a bot mitigation tool called Anubis. Something like cloudflare without the MITM part. If I don't have this I end up serving 50 queries/sec to scrapers.