Maintenance/code cleanup 2026-05 Phase IV (pluggable resolver abstraction)

[matteius]

Summary

Phase IV of the 2026-05 modernization track. Four workstreams converge here:

• Initiative D — Project sub-system extraction (T_D.5, T_D.6): split Project into focused sub-systems (Lockfile, Pipfile). Initiative D is now structurally complete.
• Initiative E — Drop the legacy requirementslib shim (T_E.4): move the last two functions (unpack_url, get_http_url) to pipenv/utils/unpack.py and delete pipenv/utils/requirementslib.py entirely.
• Initiative F — Pluggable resolver-backend scaffolding (T_F.5, T_F.5a doc sign-off): introduce a Backend protocol + registry + a refactored pip backend. Re-scoped per the T_F.5a design-doc sign-off to ship groundwork only; the second backend (uv) and any user-facing backend selection are deferred to a follow-up.
• Phase-IV regression fixes caught during integration: T_F.7 log-capture scope, Windows path-separator portability in test_lockfile, and a stale project.build_script call site missed during the T_D.6 cut.

No user-facing API change. Lockfile format unchanged. CLI surface unchanged. The Initiative F backend selection mechanism is in-tree but not yet exposed via Pipfile or CLI — that gets its own PR after a backend is ready to consume the abstraction.

What's in this PR

T_E.4 — legacy requirementslib.py retired

• New: pipenv/utils/unpack.py (the relocated unpack_url + get_http_url + their VCS-link divergence from upstream pip, preserved with explicit tests).
• Deleted: pipenv/utils/requirementslib.py.
• Sole caller (pipenv/utils/dependencies.py:determine_package_name) updated to import from the new module.
• Tests: tests/unit/test_unpack.py pins the move and the load-bearing divergences from upstream pip (VCS-link File(...) return, globally_managed=False temp-dir lifetime).

T_D.5 — Lockfile sub-system extracted

• New: pipenv/utils/lockfile.py (Lockfile class) + tests/unit/test_lockfile.py.
• 12 Project methods moved (lockfile_location, lockfile_exists, load_lockfile, as_dict, meta, hash, write, …) — all access via project.lockfile.*.
• project.py gains a deprecation-style comment block enumerating the moved names with their new homes; no __getattr__ shim — callers are migrated cleanly.

T_D.6 — Pipfile sub-system extracted

• New: pipenv/utils/pipfile.py (Pipfile class) + tests/unit/test_pipfile_subsystem.py.
• ~30 Project methods/properties moved (parsed_pipfile, pipfile_location, pipfile_exists, read_pipfile, get_pipfile_section, write_toml, build_script, proper_names, calculate_pipfile_hash, add_package_to_pipfile, recase_pipfile, …) — all access via project.pipfile.*.
• Cross-subsystem reads documented in the module header (e.g., Lockfile.meta() reads project.pipfile.calculate_hash()).
• Migration map for every renamed attribute included as a module docstring for searchability.
• Initiative D is now structurally complete — Project is a coordinator for Pipfile, Lockfile, VenvLocator, Sources, Settings.

T_F.5 — Pluggable resolver-backend scaffolding

• New: pipenv/resolver/backends/__init__.py (registry), base.py (Backend protocol), pip.py (refactored existing behaviour).
• pipenv/resolver/core.py:resolve_for_pipenv dispatches via get_backend(name) — pip is currently the only registered backend and the only default.
• Re-scoped per the T_F.5a design-doc sign-off in docs/dev/initiative-f-backends-design.md:
  • Ships now: the protocol + registry + pip backend.
  • Deferred: the uv backend, the [pipenv] resolver_backend Pipfile field, the --backend CLI flag, lockfile compatibility shims.
• News fragment: news/T_F.5.feature.rst.
• Tests: tests/unit/test_resolver_backends.py.

Phase-IV fixes during integration

• T_F.7 log-capture scope (886eea99): the phase-3 resolver-log capture was leaking ERROR records through pytest's default root-logger, causing T_F.7 + caplog tests to false-fire. Scoped to INFO + propagate=False so caplog ignores it.
• Windows test_lockfile_location_is_pipfile_plus_lock (9d9808b8): the test hard-coded f"{tmp_path}/Pipfile.lock". Lockfile.location is plain string concatenation of pipfile.location + ".lock", so on Windows the final separator is \ while the test expected /. Fixed with str(tmp_path / "Pipfile.lock") so the comparison stays platform-native.
• Stale project.build_script call site (22708197): T_D.6 moved build_script onto project.pipfile, but the pipenv run path in pipenv/routines/shell.py:126 and the corresponding integration test were missed. Fixed by migrating to project.pipfile.build_script(...). Surfaced by the CI deploy step; do_run already used project.pipfile.project_directory three lines earlier, so the migration target was provably reachable.

Test plan

• Full unit suite (860 tests) green locally.
• Resolver protocol golden tests pass.
• Targeted parity check: pipenv lock / pipenv install / pipenv run against a 30-package Pipfile produces the same lockfile hash before and after, with empty-dev-packages, populated-dev-packages, and Pipfile-edit relock cases all verified.
• Smoke / Ubuntu / 3.12 — green after the Windows-test fix in 9d9808b8.
• Smoke / Windows / 3.12 — should clear with the path-separator fix.
• Smoke / MacOS / 3.12.

Companion docs

• docs/dev/initiative-f-backends-design.md — T_F.5a design doc, sign-off recorded.
• docs/dev/modernization-plan.md — T_E.4, T_F.5, T_D.5, T_D.6 all marked complete in the plan.
• The post-merge perf work building on this PR lives at maintenance/code-cleanup-phase5-perf-2026-06; ~7-11 % real-world CI gains on the bench suite are already pushed there.

Migration notes

None. All renames are internal:

• project.parsed_pipfile / pipfile_location / pipfile_exists / read_pipfile() / get_pipfile_section(...) / write_toml(...) / has_script(...) / build_script(...) / proper_names / register_proper_name(...) / pipfile_build_requires / calculate_pipfile_hash() / all_packages / packages / dev_packages / get_editable_packages(...) / get_package_name_in_pipfile(...) / get_pipfile_entry(...) / remove_package_from_pipfile(...) / add_package_to_pipfile(...) / recase_pipfile() / ensure_proper_casing() / proper_case_section(...) and ~10 others now live on project.pipfile.*.
• project.lockfile_location / lockfile_exists / load_lockfile() / as_dict() / meta() / hash() / write(...) etc. now live on project.lockfile.*.

No __getattr__ shim — third-party tools that reach into Project internals would need the same migration. The boundary inventory is documented in pipenv/utils/pipfile.py and pipenv/utils/lockfile.py module docstrings.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Phase IV of the 2026-05 modernization track: it completes the Project subsystem extractions (Pipfile + Lockfile), removes the final legacy requirementslib shim by relocating pip-internal unpack helpers, and introduces initial “pluggable resolver backend” scaffolding (registry/protocol + pip backend) while updating call sites and tests across the codebase.

Changes:

• Extracted Pipfile and Lockfile functionality out of pipenv.project.Project into pipenv/utils/pipfile.py:Pipfile and pipenv/utils/lockfile.py:Lockfile, migrating internal call sites and updating tests accordingly.
• Retired pipenv/utils/requirementslib.py by moving unpack_url / get_http_url into pipenv/utils/unpack.py with new unit tests that pin Pipenv-specific divergences.
• Added resolver-backend scaffolding (Backend protocol + registry + pip backend adapter) and updated schema/options to carry a backend selection field.

Reviewed changes

Copilot reviewed 64 out of 64 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/unit/test_utils.py	Updates mocks to use project.pipfile.* accessors.
tests/unit/test_update.py	Updates update routine tests for project.pipfile.* API.
tests/unit/test_unpack.py	New tests pin pipenv.utils.unpack behavior and relocation.
tests/unit/test_settings.py	Adjusts Pipfile-cache invalidation tests for project.pipfile internals.
tests/unit/test_resolver_parent_dispatch.py	Updates resolver-parent dispatch tests for pipfile subsystem access.
tests/unit/test_resolver_backends.py	New tests for backend registry/dispatch scaffolding and selection precedence.
tests/unit/test_pylock.py	Updates tests to use project.lockfile.* subsystem API.
tests/unit/test_project_caching.py	Migrates cache behavior tests to project.pipfile.parsed.
tests/unit/test_pipfile_subsystem.py	New comprehensive tests for pipenv.utils.pipfile.Pipfile subsystem.
tests/unit/test_locking_no_mutation.py	Updates “no mutation” locking test to new Pipfile API.
tests/unit/test_lockfile.py	New unit tests for pipenv.utils.lockfile.Lockfile subsystem.
tests/unit/test_lock_sync_uninstall_context_routing.py	Updates routine routing tests for pipfile/lockfile subsystem APIs.
tests/unit/test_do_update_context_routing.py	Updates do_update routing tests for lockfile subsystem.
tests/unit/test_do_install_context_routing.py	Updates do_install routing tests for pipfile subsystem.
tests/unit/test_dependencies.py	Updates dependency-cleaning tests to use project.pipfile.project_directory.
tests/unit/test_dependencies_bridges.py	Updates legacy requirementslib expectations to module non-existence.
tests/unit/test_core.py	Updates tests to import NON_CATEGORY_SECTIONS from pipenv.utils.pipfile and use pipfile subsystem.
tests/integration/test_run.py	Migrates script resolution calls to project.pipfile.build_script.
tests/integration/test_pylock.py	Migrates integration tests to project.lockfile.*.
tests/integration/test_pipenv.py	Migrates proper-names assertion to project.pipfile.proper_names.
tests/integration/test_lockfile.py	Migrates lockfile load calls to project.lockfile.load.
tests/integration/test_install_twists.py	Updates commented references to the new pipfile writer location.
tests/integration/test_install_markers.py	Migrates hash comparisons to project.lockfile.hash() / project.pipfile.calculate_hash().
pipenv/utils/virtualenv.py	Migrates project path/name/pipfile references to pipfile subsystem.
pipenv/utils/venv_locator.py	Migrates pipfile-derived reads to project.pipfile.*.
pipenv/utils/unpack.py	New module hosting pip-internal fork helpers (relocated from legacy shim).
pipenv/utils/toml.py	Migrates category iteration to project.pipfile.get_package_categories().
pipenv/utils/sources.py	Migrates Pipfile/Lockfile reads & writes to subsystem APIs.
pipenv/utils/settings.py	Reads [pipenv] settings via project.pipfile.parsed; adds Settings.resolver accessor.
pipenv/utils/resolver.py	Adjusts Pipfile/Lockfile access; adds request option field for backend selection.
pipenv/utils/pylock.py	Adds deferred TODO marker for resolver metadata in pylock output.
pipenv/utils/project.py	Migrates ensure_project checks to project.pipfile.*.
pipenv/utils/pipfile.py	Major: introduces Pipfile subsystem; renames legacy plette-wrapper to PlettePipfile.
pipenv/utils/locking.py	Updates to use PlettePipfile for plette lockfile construction.
pipenv/utils/lockfile.py	New Lockfile subsystem handling lockfile IO/meta/hash and pylock seams.
pipenv/utils/environment.py	Uses project.pipfile.project_directory to locate .env.
pipenv/utils/dependencies.py	Imports unpack_url from new module; uses pipfile subsystem for project directory and batch-add.
pipenv/routines/update.py	Migrates lockfile/pipfile interactions to subsystems.
pipenv/routines/uninstall.py	Migrates lockfile and pipfile mutators to subsystems.
pipenv/routines/sync.py	Switches lockfile presence check to project.lockfile.any_exists.
pipenv/routines/shell.py	Migrates project-directory and script resolution to pipfile subsystem.
pipenv/routines/scan.py	Migrates pipfile and lockfile path/exists checks to subsystems.
pipenv/routines/requirements.py	Migrates lockfile/pipfile reads to subsystems.
pipenv/routines/outdated.py	Migrates category and Pipfile lookups to pipfile subsystem.
pipenv/routines/lock.py	Migrates locking flow to use new subsystems (pipfile, lockfile.as_dict/meta/hash).
pipenv/routines/install.py	Migrates package add/remove + hash checks + lockfile checks to subsystems.
pipenv/routines/context.py	Adds ExecutionOptions.resolver field to carry backend selection intent.
pipenv/routines/clean.py	Migrates lockfile and pipfile hash/name reads to subsystems.
pipenv/routines/check.py	Migrates pipfile/lockfile reads to subsystems.
pipenv/routines/audit.py	Migrates lockfile reads/exists checks to lockfile subsystem.
pipenv/resolver/schema.py	Adds additive ResolverOptions.backend and suppresses it from wire JSON when unset.
pipenv/resolver/core.py	Refactors canonical pip resolve flow into _pip_resolve and adds backend dispatch.
pipenv/resolver/backends/pip.py	New pip backend adapter calling _pip_resolve.
pipenv/resolver/backends/base.py	New Backend protocol + shared registry object.
pipenv/resolver/backends/init.py	New backend registry and get_backend() helper.
pipenv/project.py	Removes Pipfile/Lockfile-related methods in favor of project.pipfile / project.lockfile subsystems.
pipenv/help.py	Migrates diagnostics printing to subsystem paths/exists checks.
pipenv/environments.py	Adds PIPENV_RESOLVER env var read into settings.
pipenv/environment.py	Migrates default pipfile injection to project.pipfile.parsed.
pipenv/cli/options.py	Adds --resolver CLI option and plumbs it into State.
pipenv/cli/command.py	Passes resolver selection into context for install/lock; migrates various project accessors to subsystems.
news/T_F.5.feature.rst	News fragment describing resolver backend scaffolding and new knobs.
docs/dev/modernization-plan.md	Marks Initiative D and related tasks complete; documents scope changes.
docs/dev/initiative-f-backends-design.md	Adds maintainer sign-off notes and rescoping to scaffolding-only delivery.

Comments suppressed due to low confidence (1)

pipenv/resolver/backends/base.py:76

• REGISTRY is annotated as dict[str, Backend], but it is populated with backend classes (e.g. PipBackend) and tests also patch in backend instances. This is inconsistent with the type annotation and the module docstring, and will confuse type-checking and future maintainers. Consider annotating it as something like dict[str, type[Backend] | Backend] (and update get_backend()’s return type/docs accordingly).

# Single shared registry.  The ``backends/__init__.py`` populates this
# on import with the in-tree backends.  Keeping it here (rather than on
# ``__init__``) lets test code patch via ``mock.patch.dict`` against a
# single canonical reference no matter which module the patch targets.
REGISTRY: dict[str, Backend] = {}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[matteius]

Restacked onto current main (579de8fc) with the already-upstream Phase 3 logging fix dropped during rebase. Conflict resolution preserved the merged no-argument pipenv update relock behavior while migrating it to the extracted lockfile API and retaining resolver-backend propagation.

Local verification: 877 unit tests passed, 9 platform skips; full pre-commit suite passed.

[Copilot]

Pull request overview

Copilot reviewed 64 out of 64 changed files in this pull request and generated no new comments.

Suppressed comments (3)

pipenv/resolver/backends/base.py:75

• REGISTRY is documented/used as a name -> backend class registry (e.g. REGISTRY["pip"] = PipBackend in pipenv/resolver/backends/__init__.py), but it’s annotated as dict[str, Backend]. With mypy enabled in this repo, that annotation is inconsistent with the actual stored values (classes and sometimes instances in tests) and will likely trigger type-check failures. Widen the annotation to accept both backend classes and pre-instantiated backend singletons.

# Single shared registry.  The ``backends/__init__.py`` populates this
# on import with the in-tree backends.  Keeping it here (rather than on
# ``__init__``) lets test code patch via ``mock.patch.dict`` against a
# single canonical reference no matter which module the patch targets.
REGISTRY: dict[str, Backend] = {}

pipenv/routines/uninstall.py:72

• resolver = exec_opts.resolver is later passed into venv_resolve_deps(..., resolver_backend=resolver), but the CLI entrypoint currently builds the uninstall RoutineContext without setting execution_options.resolver (see pipenv/cli/command.py cmd_uninstall: RoutineContext.from_cli(...) lacks resolver=state.resolver). As a result, pipenv uninstall --resolver NAME will be silently ignored and always fall back to env/Pipfile/default selection. The uninstall context should plumb state.resolver into RoutineContext.from_cli(resolver=...) so CLI selection works.

    system = target.system
    pypi_mirror = target.pypi_mirror
    pre = policy.pre
    lock = policy.lock
    resolver = exec_opts.resolver

pipenv/routines/update.py:124

• do_update() forwards exec_opts.resolver into upgrade(..., resolver=...), but the CLI entrypoint currently creates the update RoutineContext without setting execution_options.resolver (see pipenv/cli/command.py cmd_update: RoutineContext.from_cli(...) lacks resolver=state.resolver). This means pipenv update --resolver NAME will be ignored even though the plumbing exists below this layer. Pass resolver=state.resolver into the update context so the CLI flag actually selects the backend.

                lock_only=policy.lock_only,
                extra_pip_args=extra_pip_args,
                resolver=exec_opts.resolver,
            )

[matteius]

Merged current main (including the v2026.7.0 version bump) into Phase IV and fixed the subprocess protocol test to patch the resolver core dispatch hook actually shared by the python -m child. Full unit suite: 878 passed, 10 skipped.

[Copilot]

Pull request overview

Copilot reviewed 65 out of 65 changed files in this pull request and generated no new comments.

Suppressed comments (3)

pipenv/utils/pipfile.py:719

• get_package_categories() builds package_categories from a set, so the order of custom categories is nondeterministic (depends on hash iteration order / PYTHONHASHSEED). This can cause unstable behavior/output whenever callers iterate categories. Preserve file order (or sort) to keep behavior deterministic.
  pipenv/utils/lockfile.py:311
• Lockfile.package_names indexes self.content[category] for every category present in the Pipfile. If a project defines a custom Pipfile category (e.g. [docs]) but the lockfile doesn't contain that section (common until first lock / after partial locks), this will raise KeyError. Use a defensive .get() fallback (or ensure categories exist via as_dict) so package_names is stable and callers like pipenv clean don't crash.

        for category in project.pipfile.get_package_categories(for_lockfile=True):
            category_packages = get_canonical_names(
                self.content[category].keys()
            )
            results[category] = set(category_packages)

pipenv/resolver/backends/base.py:75

• REGISTRY is typed as dict[str, Backend], but the registry intentionally stores backend classes (e.g. REGISTRY["pip"] = PipBackend) as well as instances (for tests/singletons). The current annotation is inconsistent with actual usage and will confuse type checkers/readers. Widen the type to allow both classes and instances.

# Single shared registry.  The ``backends/__init__.py`` populates this
# on import with the in-tree backends.  Keeping it here (rather than on
# ``__init__``) lets test code patch via ``mock.patch.dict`` against a
# single canonical reference no matter which module the patch targets.
REGISTRY: dict[str, Backend] = {}