integrate anubis

[austin-weeks]

Take a look at anubis. There may be an opportunity to incorporate this into miasma such that any non-ai scrapers are ignored. The goal being that we don't waste resources responding to scrapers that aren't going to actually do anything with the poisoned training data.

[austin-weeks]

After some research, it seems like the reverse it true - users can use anubis to detect bots and redirect them to a running instance of miasma. I'll include a guide on how to do this in the documentation #72