[SOP Request] Dataset Withdrawal from Node FAIR Data Point

[M-casado]

SOP topics

Technical infrastructure & software development

SOP type

Node-specific

SOP Title

Dataset Withdrawal from Node FAIR Data Point

Detailed Description

Derived from #55. This SOP would be triggered at Step 7 (see PR #62)

The SOP will define how a node’s FAIR Data Point (FDP) checks for the presence of a dataset requested for withdrawal and removes it if found.

Foreseeable steps (not limited to!) based on my limited knowledge about the FDP:

1. Identify dataset records

   • Query FDP metadata catalogue using available information (e.g., dataset PID, accession, or title)
   • Locate all distribution entries (API endpoints, data download URLs) that expose the dataset.

2. Assess withdrawal scope

   • Soft or hard deletion:
     • If soft-deletion, take into account the accessioning of new dataset IDs (versioned).
   • Full or partial withdrawal:
     • Full withdrawal: all catalog records and distributions.
     • Partial withdrawal: only the metadata elements or distributions corresponding to the specified subset (e.g. a single participant).

3. Execute removal

   • Unpublish or delete the catalog record(s), based on scope.
   • Remove or disable linked distribution endpoints.
   • If soft deletion requested, mark record status = “withdrawn” and hide from public search while retaining metadata internally for audit.

4. Update dataset index

   • Trigger re-indexing of FDP catalogue so the withdrawn dataset no longer appears in search or APIs.
   • Log the version change if partial withdrawal produces a new dataset version.

5. Audit and logging

   • Record the action (who, when, what removed, scope, reason) in FDP audit log.
   • Provide log excerpt to the GDI Virtual Helpdesk (VHD).

6. Report completion

   • Post confirmation in the GDI VHD ticket, including any new version PID if applicable.

Motivation

A node’s FAIR Data Point (FDP) is often the first public-facing catalogue where a dataset becomes discoverable. Without a standard withdrawal procedure, withdrawn datasets could remain findable or downloadable, undermining GDPR compliance and user trust. This SOP ensures each node follows a uniform, auditable process that aligns with the overarching dataset-withdrawal SOP (SOP0009).

Existing Procedures or References

• GDI SOP0009 “Withdraw a dataset” (European-level). It has the overarching workflow. See issue [SOP Request] Withdraw a dataset #55 and PR SOP0009: Withdraw a dataset #62.
• Technical FDP docs: https://www.fairdatapoint.org/

Impact

Implementing this SOP will guarantee that any dataset withdrawn centrally is also removed from every node FDP catalogue, preventing stale or orphaned metadata. It benefits data controllers (compliance), end-users (accurate catalogue), and node operators (clear checklist and audit trail).

Stakeholders

• Node data-management team
• Node FDP administrators
• WP4 Technical Infrastructure leads
• GDI Virtual Helpdesk (VHD)
• OC / SDPC representatives
• Dataset controllers and submitters
• Pillar I for legal compliance (?)

Additional Information

No response

Requester GDI role

Yes

Requester GDI Node

EMBL-EBI

Confirmation

• I have searched the existing SOPs and this request does not duplicate an existing SOP.
• I understand that submitting this request does not guarantee the creation of the SOP.

[M-casado]

Additionally, it is imperative to include a retained metadata stub for findability (where appropriate), and require documenting relations between old and new dataset IDs. Essentially letting data consumers know that there was something (i.e., the record) there, but it's been withdrawn, and if there's a new version of the dataset, a link to it.

This helps to trace different versions of the datasets after withdrawal and maintain PIDs.

[M-casado]

Document to work on the SOP created.

[hcvdwerf]

I filled in the template document. Please review!

[hcvdwerf]

This PR now intentionally covers only the FAIR Data Point soft-withdrawal subset of the original request.

Covered in PR #71:

• locate the dataset record in the node FDP
• identify linked distributions
• apply soft withdrawal through adms:status
• support full dataset withdrawal and distribution-specific withdrawal
• verify FDP visibility and tombstone behaviour
• report completion back through the VHD workflow

Not covered in PR #71:

• hard deletion
• participant-level or arbitrary metadata-element withdrawal
• dataset versioning / replacement identifier handling
• downstream withdrawal actions in other systems beyond the FDP

The PR title/body and SOP text have been updated to reflect this narrower scope. If needed, the remaining scope can be covered in a future follow-up update or separate SOP work.