Skip to content

[Lustery] fix Cloudflare CDP scraping (#2838) - #2839

Open
HornyQT wants to merge 2 commits into
stashapp:masterfrom
HornyQT:patch-3
Open

[Lustery] fix Cloudflare CDP scraping (#2838)#2839
HornyQT wants to merge 2 commits into
stashapp:masterfrom
HornyQT:patch-3

Conversation

@HornyQT

@HornyQT HornyQT commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Lustery is now behind Cloudflare's bot-challenge and as such useCDP is required to reach the site.
That in turn brakes the old scrapeJson, since Chrome wraps a direct navigation to a JSON endpoint in a bare <pre> tag rather than leaving it as a raw document, and scrapeJson needs the fetched body itself to be valid JSON.

Converted to scrapeXPath: selectors grab that <pre> element and JSON.parse() it in a javascript postProcess step, which also handles unicode/escape unescaping correctly for free.

Also:

  • sceneByURL/URL now match lustery.com/video/ instead of the old /video-preview/ path, since you get redirected to that when opening link.
  • Performers now resolve real individual first names (via the primary couple's /info endpoint) instead of the combined couple display name.
  • Image URLs use width=8000,quality=100 instead of a fixed width/quality, so they always return the source's native resolution (confirmed the resize proxy accepts up to ~8000-10000 wide and simply clamps rather than upscales beyond native).
  • Fixed the performer Image host: static.lustery.com no longer resolves in DNS at all; img.lustery.com is the current CDN host serving couple images.
  • Code field dropped: its source, video.thumbnailsBasePath, is no longer present in the API response.

Note: When a scene has multiple couples the scraper can only add the first couple, second couple gets added as an URL so that it doesn't get overlooked.

@Maista6969

Copy link
Copy Markdown
Collaborator

Was this issue not resolved by 9ad6f17 ? I'd really rather avoid making more scrapers depend on CDP since it's a common stumbling block for users

@HornyQT

HornyQT commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

I tried this and it did not work for me. Even when i use a "webbrowser" user agent i get the same 403 error.
Just tested again on my other machine with curl -s -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0 Safari/537.36" "https://lustery.com/api/video/a-perfect-match" and i get a cloudflare challange as response and no json.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants