After #4104, JSDOMCrawler and LinkeDOMCrawler are thin subclasses of DomCrawler that only plug in a parser. Keeping them in the monorepo no longer buys us anything: jsdom and linkedom are heavy dependencies that every workspace install and CI run has to pay for, and the two crawlers are rarely used yet tied to the release cadence of everything else. Anyone can now provide the same thing as an external package through the parser option, without special hooks in @crawlee/http.
Proposal for 4.0:
- move
@crawlee/jsdom and @crawlee/linkedom to their own repositories, https://github.com/apify/crawlee-jsdom and https://github.com/apify/crawlee-linkedom (created, private until they have content), with @crawlee/http as a peer dependency
- stop re-exporting them from the
crawlee meta-package
- keep the npm package names, so existing imports keep working after a version bump
- mention the move in the 4.0 upgrading guide
Blocked by #4104.
After #4104,
JSDOMCrawlerandLinkeDOMCrawlerare thin subclasses ofDomCrawlerthat only plug in aparser. Keeping them in the monorepo no longer buys us anything:jsdomandlinkedomare heavy dependencies that every workspace install and CI run has to pay for, and the two crawlers are rarely used yet tied to the release cadence of everything else. Anyone can now provide the same thing as an external package through theparseroption, without special hooks in@crawlee/http.Proposal for 4.0:
@crawlee/jsdomand@crawlee/linkedomto their own repositories, https://github.com/apify/crawlee-jsdom and https://github.com/apify/crawlee-linkedom (created, private until they have content), with@crawlee/httpas a peer dependencycrawleemeta-packageBlocked by #4104.