Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions Social_Bookmark_extraction.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Social Bookmark Extraction Script
# Extracts links and nearby title/snippet text from a local HTML export.

library(rvest)
library(dplyr)
library(stringr)

# ---- Input ----
# Update this path to point to your exported HTML file.
html_path <- "path/to/your/bookmarks_export.html"

# ---- Optional platform-specific pattern ----
# Examples:
# twitter_pattern <- "status"
# linkedin_pattern <- "feed/update"
# facebook_pattern <- "posts|story.php|permalink"
post_pattern <- "status|feed/update|posts|story.php|permalink"

# ---- Read HTML ----
page <- read_html(html_path)

# Use file location as base URL for resolving any relative links.
base_url <- paste0("file://", normalizePath(html_path, winslash = "/", mustWork = FALSE))

# ---- Extract all <a> tags, not class-dependent ----
all_links <- page %>% html_elements("a")

extract_nearest_text <- function(node) {
# Try parent span/div first
parent_text <- node %>%
html_element(xpath = "./ancestor::*[self::span or self::div][1]") %>%
html_text2(trim = TRUE)

# Fallback to nearest meaningful ancestor containing visible text
if (is.na(parent_text) || parent_text == "") {
parent_text <- node %>%
html_element(xpath = "./ancestor::*[normalize-space(text()) != ''][1]") %>%
html_text2(trim = TRUE)
}

if (is.na(parent_text) || str_squish(parent_text) == "") {
return("No Title Found")
}

str_squish(parent_text)
}

bookmarks_cleaned <- tibble(
url_raw = html_attr(all_links, "href"),
title = vapply(all_links, extract_nearest_text, character(1))
) %>%
filter(!is.na(url_raw), url_raw != "") %>%
mutate(
# Convert relative URLs to absolute
url = url_absolute(url_raw, base = base_url),
# Remove tracking/query params (everything after '?')
url = str_remove(url, "\\?.*$")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep post-identifying query strings when normalizing URLs

Stripping everything after ? removes required identifiers for query-based post URLs (for example Facebook story.php?story_fbid=...&id=...), so different posts collapse to the same normalized URL and are then dropped by distinct(url, .keep_all = TRUE). This silently loses data whenever exports contain such links; the normalization should remove only known tracking parameters rather than the entire query string.

Useful? React with 👍 / 👎.

) %>%
# Keep likely social post links only
filter(str_detect(url, regex(post_pattern, ignore_case = TRUE))) %>%
mutate(
title = if_else(is.na(title) | str_squish(title) == "", "No Title Found", title)
) %>%
distinct(url, .keep_all = TRUE) %>%
select(title, url)

print(bookmarks_cleaned)

write.csv(bookmarks_cleaned, "bookmarks_cleaned.csv", row.names = FALSE)