Skip to content

feat: support source.filter for URL and archive path sources - #2608

Merged
wolfv merged 2 commits into
mainfrom
claude/github-issue-2581-m0i9s9
Jun 30, 2026
Merged

feat: support source.filter for URL and archive path sources#2608
wolfv merged 2 commits into
mainfrom
claude/github-issue-2581-m0i9s9

Conversation

@wolfv

@wolfv wolfv commented Jun 30, 2026

Copy link
Copy Markdown
Member

Summary

This PR extends the source.filter field to support url sources and path sources that point to archives. Previously, filtering was only available for git sources. The filter is now applied to the contents of extracted archives, allowing large source tarballs to be trimmed down to only the parts needed for the build.

Key Changes

  • Recipe Schema Updates: Added filter field to UrlSource in both stage0 and stage1 recipe structures with proper serialization handling
  • Parser Enhancement: Extended the URL source parser to recognize and parse the filter field alongside existing fields like url, sha256, and patches
  • Source Extraction Logic: Modified copy_from_cache() to accept and apply glob filters when copying extracted archive contents to the destination directory
  • Path Source Archives: When a path source points to an archive file, it's now converted to a temporary URL source with the filter applied during extraction
  • Variable Evaluation: Added filter evaluation in the stage0 evaluation logic to support variable interpolation within glob patterns
  • Documentation: Added comprehensive examples in recipe_file.md showing both simple list-based and include/exclude mapping-based filter syntax for URL sources
  • Tests: Added unit tests for parsing URL sources with filters and an end-to-end test with a real archive demonstrating filter functionality

Implementation Details

  • The filter is applied at the extraction stage using the existing copy_dir::CopyDir builder with the new with_globvec() method
  • Filters support both simple list syntax (["cmake/*", "clang/*"]) and structured include/exclude mappings
  • The implementation reuses the existing IncludeExclude and GlobVec types already used for git source filtering
  • Snapshot tests were updated to reflect the new filter: [] field in serialized output

https://claude.ai/code/session_01RxSDwdYTY8h7T8VRHR5wmD

Extend the `filter` field (already available on `path` sources) to `url`
sources. The filter is applied to the contents of the extracted archive,
which lets large source tarballs be trimmed to just the files needed for
the build (e.g. selecting only certain subprojects from a monorepo or
excluding bundled `third_party` directories).

The filter is wired through stage0 parsing/evaluation into the stage1
`UrlSource`, and applied via `CopyDir` when copying the extracted source
out of the cache. This also makes the filter effective for `path` sources
that point to an archive, since those reuse the same extraction code path.

Closes #2581

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxSDwdYTY8h7T8VRHR5wmD
@wolfv wolfv changed the title Add filter support for URL and archive path sources feat: support source.filter for URL and archive path sources Jun 30, 2026
@wolfv

wolfv commented Jun 30, 2026

Copy link
Copy Markdown
Member Author

I also submitted a CEP update: conda/ceps#176

Following CEP-0014 (conda/ceps#176), which specifies `filter` on `url`,
`path`, and `git` sources, extend filter support to git sources as well.
The filter is applied to the checked-out tree when copying it into the
work directory, mirroring the url/path behavior.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxSDwdYTY8h7T8VRHR5wmD

wolfv commented Jun 30, 2026

Copy link
Copy Markdown
Member Author

Thanks for the pointer to conda/ceps#176. Since CEP-0014 specifies filter on url, path, and git sources, I've pushed a follow-up commit (41f28d5) adding git source support too — the filter is applied to the checked-out tree, mirroring the url/path behavior. Parser tests and an end-to-end check against a local git repo confirm it works. The PR now covers all three source types from the CEP.


Generated by Claude Code

Hofer-Julian pushed a commit to prefix-dev/recipe-format that referenced this pull request Jun 30, 2026
Move the `filter` glob field from LocalSource up to BaseSource so that
url, git, and path sources all support include/exclude glob filtering of
the extracted source, matching prefix-dev/rattler-build#2608.
@wolfv
wolfv merged commit 29438d4 into main Jun 30, 2026
17 of 19 checks passed
@wolfv
wolfv deleted the claude/github-issue-2581-m0i9s9 branch June 30, 2026 14:29
@h-vetinari

Copy link
Copy Markdown

Super nice! 🤩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants