Improve project dataset association and visibility options for large data collections

[charlesbrandt]

A user from BC2 raised a use case where a project operator wants to give a platform user access to nearly all available raw_data. Currently, this would require manually adding almost all datasets to a project, which is impractical.

In CfN-Dap, there is support for associations, where operators can automatically associate datasets with a project based on shared properties (e.g., protocol). This allows automatic inclusion of datasets matching certain criteria, rather than adding them individually.

This issue can explore extending similar functionality or adding other ways to handle broad dataset access and visibility in Bioloop.

---

Current behavior

• In the core Bioloop implementation, datasets must be added to a project manually, via the “+ Add” button (limited to selecting up to 10 at a time).
• Associations exist only in CfN-Dap and have not been merged into the Bioloop core.

---

Proposed directions / discussion points

1. Automatic associations

   • Allow operators to define association rules (e.g., by protocol, assay type, or metadata filters) so that datasets meeting criteria are automatically linked to a project.

2. Bulk dataset addition improvements

   • Enable selection or filtering to add more than 10 datasets at a time.
   • Possibly use search/filter + “Add all results” for bulk actions.

3. Dataset visibility rules

   • Consider whether Bioloop instances need broader dataset visibility rules not tied strictly to project membership.
   • This might align with Andrea’s comment about non-operator users needing different access patterns.

FYI @aiavenak

[charlesbrandt]

Another important aspect of this issue is the ability to "bulk stage" multiple datasets. This could be managed through a "select all" option with individual check boxes for each dataset to manage deselection.

We're also hearing use cases where the staged collection of datasets ultimately need to be delivered elsewhere (e.g. different slate project directories, etc).