General refactor, GUI works, site testing and updates - #182
General refactor, GUI works, site testing and updates#182hollisticated-horse wants to merge 26 commits into
Conversation
- Add error check when finding chapter list - Normalize URLs with proper trimming and validation - Add fallback for missing select box - Add empty chapters validation - Fix variable naming (issueUrl -> issueURL)
- Add detailed debug logging with base64-encoded snippets - Log fetch URL and request failures - Show obfuscated entry count and decoded link preview - Limit snippet size to prevent log overflow
- Import golang.org/x/image/webp for WebP decoding - Handle WebP format in SaveImage function
- Set DisableLevelTruncation to true for better readability
- Distinguish between invalid semver tags and actual errors - Log debug message for invalid tags, error for real failures
…ror on broken encoding
|
The only comment I have is that I wanted to try and switch off of soup. I found their api to be more limiting then helpful in complex scrapping jobs. I hadn't gotten around to it in my branch, but since your introducing new sources in yours, I figured it was worth mentioning. |
Let's make a new PR on top of our changes then. I'd prefer making targeted PRs (unlike my current one...). But sounds good. Do you have a replacement for |
|
Yea a targeted pr is fine with me. I had https://github.com/PuerkitoBio/goquery in mind because of its support for selectors |
|
Since this is such a big pr, do you want to wait until this is merged before replacing soup, or merge that into this branch? |
|
I'm thinking of splitting this PR up actually, then re-evaluate the smaller bags of changes. Maybe after that change to soup ? What do you think ? Sorry about the merge delays, I'll start doing some cleanup and then get those PRs merged. |
f7d5987 to
bb2d3e1
Compare
|
I'd prefer to just land the bigger architectural changes first as they are more significant blockers. Whatever helps work through it is good to me. As for the delays, its not a massive issue for me, so I'm more then willing to accommodate whatever timeline. |
2 participants
No description provided.