Add Social_Bookmark_extraction.R for parsing social bookmark HTML exports - #3
Add Social_Bookmark_extraction.R for parsing social bookmark HTML exports#3charles-hebert wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b1f6954ccb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Convert relative URLs to absolute | ||
| url = url_absolute(url_raw, base = base_url), | ||
| # Remove tracking/query params (everything after '?') | ||
| url = str_remove(url, "\\?.*$") |
There was a problem hiding this comment.
Keep post-identifying query strings when normalizing URLs
Stripping everything after ? removes required identifiers for query-based post URLs (for example Facebook story.php?story_fbid=...&id=...), so different posts collapse to the same normalized URL and are then dropped by distinct(url, .keep_all = TRUE). This silently loses data whenever exports contain such links; the normalization should remove only known tracking parameters rather than the entire query string.
Useful? React with 👍 / 👎.
Motivation
Description
Social_Bookmark_extraction.Rwhich usesrvest,dplyr, andstringrand defines an editablehtml_pathinput variable.<a>tags and filters likely social post links using a configurablepost_patternwithfilter(str_detect(...))(examples includestatus,feed/update,posts,story.php,permalink).url_absolute()using afile://base, and removes URL query/tracking parameters withstr_remove("\\?.*$").div/span(or nearest ancestor with visible text) as the title/snippet via XPath; assigns"No Title Found"for empty or missing titles; returns a clean tibble and writesbookmarks_cleaned.csv.Testing
Rscript -e "parse(file='Social_Bookmark_extraction.R')", but this failed becauseRscriptis not available in the current environment.Rscriptexecutable.Codex Task