Add R project for sitemap-driven recipe scraping with y/n selection - #11
Add R project for sitemap-driven recipe scraping with y/n selection#11charles-hebert wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fdb122bca6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| if (length(selected_urls) == 0) { | ||
| warning("No rows selected (selected='y'). Writing an empty output file.") | ||
| write_csv(tibble(recipe_url = character(), title = character(), ingredients = character(), instructions = character()), output_csv) |
There was a problem hiding this comment.
Ensure empty-output branch creates parent directory
When no URLs are selected, this branch writes output_csv immediately, but the parent directory is only created later in the non-empty branch. If a caller provides an output path in a directory that does not already exist, write_csv fails and the script exits without producing the expected empty CSV. Create the parent directory before this write (or before the branch) so both execution paths behave consistently.
Useful? React with 👍 / 👎.
Motivation
selectedy/n CSV column.Description
recipes_scraper/includingrecipes_scraper.Rprojand.gitignoreto house the tooling and ignore R artifacts.R/sitemap_utils.Rwithsafe_read_xml,extract_locs_from_sitemap,filter_recipe_urls,initial_recipe_table, andscrape_recipe_pageto parse sitemaps and extract recipe fields.scripts/01_extract_recipes_from_sitemap.Rto produce a CSV ofrecipe_url+selected(defaults ton) from a sitemap andscripts/02_scrape_selected_recipes.Rto read that CSV and scrape only rows withselected == 'y'.README.mdand make the CLI scripts executable (chmod +x), with example commands usingRscript.Testing
Rscript -e "parse(file='recipes_scraper/R/sitemap_utils.R'); parse(file='recipes_scraper/scripts/01_extract_recipes_from_sitemap.R'); parse(file='recipes_scraper/scripts/02_scrape_selected_recipes.R')", but the check could not run becauseRscriptis not installed in the environment (test failed due to missing runtime).Codex Task