Skip to content

Migrate constraint generation to constraints output type in DFG #311

Description

@KyleFromNVIDIA

Background

Until this point, we've been using the requirements output type to generate constraints files. The biggest problem with this approach is that the requirements output is also used to install dependencies for devcontainers. Because constraints don't support extras, and the requirements output type currently has to support constraints as well, this precludes us from installing extras in devcontainers dependencies.

One possible way to work around this is to add a constraints or similar matrix selector in order to differentiate requirements and constraints. However, requirements and constraints are fundamentally different, even though they have the same file format. Therefore, I think it makes sense to add a separate constraints output type as a first-class citizen to DFG.

Design Considerations

There are two ways to handle the migration from requirements to constraints:

  1. Keep them entirely separate and require projects to add constraints where needed. This is cleaner, but requires more work downstream.
  2. Allow DFG to automatically use requirements if constraints is specified but no constraints are listed in the dependency set. In this case, projects that don't need to differentiate requirements and constraints don't need any changes at all. This means less work downstream, but is more complicated for developers to understand.

I propose we go with the first option, which is currently the status quo for rapidsai/dependency-file-generator#197.

Proposed Migration Procedure

  1. Merge feat: Add constraints output type dependency-file-generator#197.
  2. Add a flag to rapids-generate-pip-constraints called --output constraints or similar, which causes it to use the constraints output type rather than requirements. It will continue to use requirements by default, thus preserving the status quo while allowing downstream projects to opt-in to the new system.
  3. In each downstream project:
    1. Add the constraints output type to any dependency set that's used to generate a constraints file.
    2. Add the --output constraints flag to any instance of rapids-generate-pip-constraints to indicate that dependencies.yaml is ready for the new system.
    3. Add --output constraints to any other direct or indirect usage of DFG that generates a constraints file.
  4. Once all projects have been migrated to use --output constraints, remove the flag from rapids-generate-pip-constraints, make it the default, and remove all usages of it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions