Fetch llms.txt

[DavideDelbianco]

It would be nice if you attempted to fetch llms.txt the same way you fetch robots.txt
It is still a proposal, but more and more sites are starting to adopt it.

https://llmstxt.org/