Replies: 1 comment
|
Went full circle on this. I think the issue with scrapy is that out of the box with proxies, it will not help you vs AWS WAF. I think Scrapling MIGHT be the better option if you are scraping sites with WAF. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I have a distributed scraper that uses scrapy (i've been using it for 8+ years). It works.
I am also the type to have shiny new object syndrome. Curious if anyone who has used both. What improvements do you see?
All reactions