Skip to content

Tighten robots.txt to cut duplicate bot crawl - #3654

Draft
ChristopherChudzicki wants to merge 3 commits into
mainfrom
robots-txt-crawl-rules
Draft

Tighten robots.txt to cut duplicate bot crawl#3654
ChristopherChudzicki wants to merge 3 commits into
mainfrom
robots-txt-crawl-rules

fix: address review comments on robots rules and test env handling

c6942e9
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar