Embargo for metadata

[6tova]

@MichalMed
I would like to address the call for embargo on (specific) metadata. While this may not apply to CCMM metadata itself, it might be essential for potentially sensitive metadata within extensions—for example, metadata describing methodology. Is there a way to implement this functionality within the metadata model? (Thank you)

[petulica]

To better understand your use case, I need more details about the need. Do you need to set an embargo date, or would you prefer to permanently restrict the metadata? While there is an embargo/restricted access, is there anyone (specific group or individual) who should be able to read such metadata? Should all the embargoed metadata attributes use the same date, or can they vary per attribute?
One more question that might seem stupid, but why is methodology sensitive in your case?

[david-antos]

You made me a bit suspicious mentioning "specific" metadata. Metadata of a record can be handled as a unit regarding access permissions, it can be either public, or have controlled access, or accessible only by the record owner.
On the other hand, it is not possible for individual metadata fields to differ.

In addition to that, kindly note that metadata in the NRP should never contain sensitive data (such as, e.g. parts of medical records). Metadata is kept in a database and indexed by external systems. Truly sensitive data should be kept encrypted at rest, this is not feasible to achieve in a metadata database.

[MichalMed]

anyway, we need some specific usecases, that are relevant, until we get them, there is no way in which we can implement anything

[6tova]

Hi, after a consultation with Marek Cebecauer, we prepared an exhausting answer for @petulica questions. For @david-antos - it should hopefully also clarify what we mean be "sensitive" in our case:

a) Do you need to set an embargo date, or would you prefer to permanently restrict the metadata?
Both options, "permanent restriction" and "embargo date" for metadata would be desirable for different use cases. The embargo option is our main focus at the moment and the way we think about it is with limited time period (maximum 3-5 years) with the possibility of extension. Permanent restriction is then regulated by attached licenses and is also at the edge of Sensitive (Confidential) data topic.
Reasoning: The embargo period in data repository is need to support early deposition of the data. Ideally directly from scientific instruments (after enrichment of metadata of course). In most of the cases, metadata can be visible but there are use cases when even the title may say too much to the competitors. Combination of a material and a method could be sensitive.
Permanent restriction then applies to data which were “published”, which means that these data were used in a publication or for a patent. Again, there could be a combination of metadata which is confidential. This will be probably less urgent to implement.

b) While there is an embargo/restricted access, is there anyone (specific group or individual) who should be able to read such metadata?
Definitely, first of all, the data curator and repository administrator should have permission. Ideally, the depositor should have an option to grant access to other authorized members of their team to edit the record during the submission process and view it after publishing in a repository.
Reasoning: Data curator and repository administrator are responsible for data quality and FAIR maturity, as well as preservation. To enable access for collaborators would be a very useful option and a way how to market the best data management practice in our communities.

c) Should all the embargoed metadata attributes use the same date, or can they vary per attribute?
There is no need for different embargo dates per attribute. However, it would be beneficial to have an option to distinguish between the metadata that should stay open and those that should be embargoed.
Reasoning: Only single “public opening” date would apply to embargoed metadata in general. However, not all the metadata need to be embargoed. After the release from embargo period, licenses shell become the primary regulation for accessibility of (meta)data (in case the license can be applied to the metadata). In some case the embargo must be extended due to objective reasons. This is ideally done by data curator or repository administrator.

d) One more question that might seem stupid, but why is methodology sensitive in your case?
It is definitely not a stupid question. Some projects may take couple of years to finish with proper publication or patent. With the right data management culture, you deposit your results to the repository as they are created (ideally some logical units) to avoid loss of information about the data (rich metadata). However, it is not desirable to share information about their existence yet as they could be used by competing groups before the project is officially finished and without proper reference.
Reasoning: Some projects work with non-confidential samples; however, the fact that a specific analytical method is applied to a given material can itself be sensitive from an IPR perspective. The combination of material and method may reveal the material properties under investigation, which should remain confidential until a scientific article is published or a patent is filed. In patenting cases, any public disclosure prior to filing can jeopardize patent approval.
Similarly, information about the use of a specific instrument may provide clues about research direction—especially when combined with advanced AI/ML tools—potentially exposing research interests before proper publication and attribution.
In some regions (mainly China), individuals actively monitor publicly available information to identify research directions that can be replicated or accelerated using extensive local resources (especially in China). Such information may then be shared with research groups capable of advancing the work more quickly, potentially undermining the original researchers’

[MichalMed]

OK, I get it. Is there any major limitation why this shall be part of metadata and not part of data/metadata management on the repository/catalogue level? As I see it, there might be a need of adding new type of date into the vocabulary (embargoed until), but it basically means that all the metadata is there, just the metadata record or some of its parts is not published yet. Moreover, if this attribute is part of non-published metadata record, then it is not of course visible to public, therefore no one can see that this metadata record even exists. So basically I do not really think that this is an issue of metadata profile/schema. But feel free to convince me.

[motyc]

I can just add my two cents on the topic - we work with such metadata as well (either embargoed or with limited accessibility). What we plan to do is that we just won't ever disclose them to CCMM OAI-PMH, until free publishing requirements are met. Until then, metadata will be available in the native API behind AAI. If the state of things changes, the records gets updated and should be reharvested approprietly, propagating the newly available parts. This just supports what Michal told, ie. that this could be possibly handled on the repository level. I am not sure if transfering such complex logic into CCMM would be beneficial.

[michal-klodner]

If I may add something too, I would distinguish two parts - one is implementation of these mechanisms in the source repository and accessing the dataset metadata in the source repository, the second is interoperability of such record, it means transferring the record to some other system and providing the same protection elsewhere. If we talk in the context of CCMM, assumed can be we talk about interoperability.

To understand further - it seems the issue is part of the metadata being sensitive. If there is full record sensitive, there is no publishing involved. In case of publishing and reporting, there is usually PID needed and PID request requires the PID metadata to be public, so the part of the metadata for assigning PID - generally for principal identifiability and distinguishability from other datasets, has to be in the open part. The sensitive part can contain additional metadata and possibly there can be CCMM application profile created for that, where the added attributes would be masked when transferring to some other system or encrypted. The @motyc description of OAI metadata exposure is also highly relevant for that, i.e. the source repository decides when the public CCMM metadata are enriched with the previously sensitive parts.

[MichalMed]

we still do not have any use cases which shall be solved on the metadata level. I'm putting this on long term list.