Complete dcat:Dataset headers in all NetworkCode cimxml files

[benceszirbik]

Complete dcat:Dataset headers in all NetworkCode cimxml files
All 36 NetworkCode cimxml files across Belgovia, Espheim, Galia, Svedala,
and DC-Espheim-Svedala now conform to the header structure in
docs/HeaderExample.xml. Addresses issue #157 (langString usage).

Changes applied to all files:

• Removed unused xmlns:eumd namespace declaration
• Added xmlns:dcatcim and xmlns:adms where missing
• Added: dcatcim:alternativeVersionOf, dcterms:accrualPeriodicity,
  dcterms:hasPart, dcat:temporalResolution, dcatcim:preferredVersionOf,
  dcterms:references, dcterms:type, adms:versionNotes xml:lang="en"

File-specific fixes:

• Belgovia_CO: added dcat:endDate; fixed dcat:replaces -> dcterms:replaces
• Belgovia_SIS: added dcterms:publisher and prov:wasGeneratedBy
• Svedala_AS: fixed xmlns:dcat-cim -> xmlns:dcatcim; added xml:lang to
  dcterms:description and adms:versionNotes; added dcat:endDate
• Svedala_FAP_PV: added dcterms:license and dcterms:requires; removed
  unused xmlns:eu; dcterms:type Activity preserved as-is

[Sveino]

Data Instance Issues

There is currently a mix of metadata issues and instance data issues. This list focuses only on the instance data issues.

1. Prefixes

• All instance files shall use the same namespace prefixes.
• Only the prefixes included in docs/HeaderExample.xml shall be used.

2. Remove the xmlns:eumd Namespace

• Remove all xmlns:eumd namespace declarations.
• Remove all eumd instances from the header.

3. Properties That Shall Not Be Included

The following properties shall not be included in any instance file header:

• dcatcim:alternativeVersionOf
• dcterms:accrualPeriodicity
• dcterms:hasPart
• dcat:temporalResolution
• dcatcim:preferredVersionOf
• dcat:inSeries

4. adms:versionNotes

All files shall include:

<adms:versionNotes xml:lang="en">...</adms:versionNotes>

5. dcterms:accessRights

All files shall include:

<dcterms:accessRights rdf:resource="https://energy.referencedata.eu/Confidentiality/Public"/>

6. dcterms:conformsTo

All files shall include dcterms:conformsTo.

The referenced value shall be validated against:

Instance/referenceData/NetworkCode/cimxml/ConformanceReleaseScheme-NCP-2-4-1_RD.xml

7. dcterms:description

All files shall include:

<dcterms:description xml:lang="en">...</dcterms:description>

8. dcterms:identifier

All files shall include dcterms:identifier containing a unique UUID.

For example:

<dcterms:identifier>6b7387f2-c7f9-11ed-afa1-0242ac120002</dcterms:identifier>

9. dcterms:issued

All files shall include dcterms:issued.

The value shall use UTC and include the Z suffix.

For example:

<dcterms:issued>2024-04-10T10:00:00Z</dcterms:issued>

10. dcat:isVersionOf

All files shall include dcat:isVersionOf.

The referenced value shall be validated against:

Instance/referenceData/NetworkCode/cimxml/Test-ModelScheme-NCP_RD.xml

11. dcat:keyword

All files shall include at least one dcat:keyword.

12. dcterms:license

All files shall currently include:

<dcterms:license rdf:resource="https://creativecommons.org/licenses/by/4.0/"/>

This is planned to be changed to use a value defined in:

Instance/referenceData/Grid/cimxml/LicenseScheme_RD.xml

13. dcterms:publisher

All files shall include dcterms:publisher.

The publisher URI shall follow this pattern:

https://energy.referencedata.eu/test/party/TSO-Svedala

The following rules apply:

• TSO publishers shall start with TSO-.
• RCC publishers shall start with RCC-.
• Jotunheim shall use an RCC publisher identifier.

14. dcterms:requires

We currently use dcterms:requires.

The references shall be reviewed and changed to dcterms:references where appropriate.

15. Rights and Rights Holder

All files shall include:

<dcterms:rights>Copyright</dcterms:rights>
<dcterms:rightsHolder>ENTSO-E</dcterms:rightsHolder>

16. dcterms:spatial

All files shall include the relevant spatial reference.

For example:

https://energy.referencedata.eu/test/frame/Galia-Transmission

The correct spatial reference shall be selected based on the model name and the values defined in:

Instance/referenceData/NetworkCode/cimxml/Test-FrameScheme-NCP_RD.xml

17. dcat:startDate

All files shall include dcat:startDate.

18. dcterms:type

All files shall include:

<dcterms:type rdf:resource="https://energy.referencedata.eu/type/CIM-PowerSystemModel"/>

19. dcat:version

All files shall include dcat:version.

The default value is:

<dcat:version>1.0.0</dcat:version>

20. prov:generatedAtTime

All files shall include prov:generatedAtTime.

21. prov:wasGeneratedBy

All files shall include prov:wasGeneratedBy.

The activity identifier shall follow the rules defined in:

docs/ActivityNamingRecommendation.adoc

It shall also follow the accompanying Excel spreadsheet used to generate:

Instance/referenceData/NetworkCode/cimxml/ActivityScheme-NCP_RD.xml

The current activity naming recommendation is also available at:

https://github.com/entsoe/relicapgrid/blob/281-add-activity-naming-recommendation/docs/ActivityNamingRecommendation.adoc

22. dcterms:title

All files shall include dcterms:title.

The title shall follow the rules defined in:

docs/DatasetNamingSpesification.adoc

23. File Name

The file name shall be derived from dcterms:title.

The following exception applies:

• Template and default files shall not include the date component from dcterms:title.
• For all other files, the complete applicable title shall be used as the file name.

[benceszirbik]

Thanks for the detailed list. I've pushed fixes for everything that was mechanical or had an explicit target value — eight commits on fix-header-issues, one per review point (referring to your numbered points as "point N" below, to avoid auto-linking repo issues):

• Point 2 — removed xmlns:eumd + the eumd:applicationSoftware element
• Point 3 — removed the six disallowed properties (dcatcim:alternativeVersionOf, dcterms:accrualPeriodicity, dcterms:hasPart, dcat:temporalResolution, dcatcim:preferredVersionOf; dcat:inSeries wasn't present)
• Point 1 — dropped the now-dead xmlns:dcatcim everywhere, plus euvoc/md in the DC file
• Point 13 — normalised publishers to TSO-/RCC- (Jotunheim→RCC-Jotunheim, etc.)
• Point 18 — set dcterms:type = CIM-PowerSystemModel in all files (including Svedala_FAP_PV, which was type/Activity)
• Points 4/5/7/12/15/19 — completed the missing fixed-value fields in Svedala-Espheim_RA, and language-tagged the Svedala-Belgovia_CO versionNotes
• Point 22 — fixed the Belgovia_PS title copy-paste (_AE_→_PS_)

Before I touch points 6, 10, 16, 21, I need to know your plans for the reference data, because the current schemes don't yet contain entries for most of our parties/frames/models/activities, so I can't "validate against" them without inventing values. Specifically:

1. Point 16 — dcterms:spatial → Test-FrameScheme-NCP_RD.xml — the scheme only defines Svedala-Transmission, Galia-Transmission, Espheim-Portheim-Transmission. Instances point at Test/Frame/<Area>-Power-Transmission-System, and there is no Belgovia or Jotunheim frame at all. Are you adding the missing frames, and should the instances adopt the scheme's .../test/frame/<Name>-Transmission form?
2. Point 10 — dcat:isVersionOf → Test-ModelScheme-NCP_RD.xml — scheme has only TSO-Galia_IGM-ER and TSO-Svedala_IGM-EQ; instances use Test/Model/<Area>-<Profile> (e.g. Belgovia-CO), none of which exist. Will the scheme be extended per profile, and what's the canonical model-naming pattern?
3. Point 6 — dcterms:conformsTo → ConformanceReleaseScheme-NCP-2-4-1_RD.xml — the release collection lists nc/2.4 + EquipmentReliability/2.3 + AssessedElement/2.4, but instances reference many other profiles/versions (Contingency/2.3, RemedialActionSchedule/2.4, SteadyStateHypothesisSchedule/1.1, ObjectRegistry/2.2, Provenance/1.0, …) and use cim4.eu/ns/nc/2.4# with a trailing # vs the scheme's no-#. Which profile+version set is authoritative for NCP-2.4.1, and is the # intended?
4. Point 21 — prov:wasGeneratedBy → ActivityScheme-NCP_RD.xml + ActivityNamingRecommendation.adoc — the naming doc lives in branch 281, not here, and the scheme only has IGM-EQ/ER/EQ-VF/RASR; instances use Activity/CGM-*, OPC-AS, PV, IGMx-CO. Can we merge the 281 doc first? (Note Svedala-Espheim_RA also has a malformed prov:wasGeneratedBy — a text value with a stray quote instead of rdf:resource — I left it for this pass.)

Related, also blocked on the above:

• Point 13 party entries — Test-PartyScheme only has TSO-Galia/TSO-Svedala; it needs TSO-Belgovia, TSO-Espheim, RCC-Jotunheim. And who publishes DC-Espheim-Svedala_CO? It currently uses test/party/DC-Espheim-Svedala, which is neither TSO nor RCC — I left it untouched.
• Point 14 requires vs references — many headers now carry both dcterms:requires and dcterms:references pointing at the same UUID (from the earlier pass). Which should stay? (Svedala-Espheim_RA uses prov:references.)
• Points 22/23 title & filename — proper titles/filenames need the publisher-party, spatial and activity tokens above, so I'd rather redo them once the schemes are settled rather than churn twice.

Once you confirm the scheme extensions (or point me at where they'll live), I'll do points 6/10/16/21, the remaining point 13 party entries, point 14, and the points 22/23 rename pass.

[benceszirbik]

Follow-up — widened scope (eumd + Jotunheim/Belgovia-version files)

Pushed six more commits:

• Point 2, repo-wide — the eumd prefix was declared but unused across the whole repo, so I removed the dangling xmlns:eumd / PREFIX eumd from all 149 Instance XML + TriG files (Grid, NetworkCode, GridSituation, boundaryData, commonData). The only two files that actually used it — Svedala_RAS_proposal.trig and Svedala_RAS_acceptance_proposal.trig — had their eumd:applicationSoftware statement removed and the block reterminated.
• Extended the header review to the Jotunheim NetworkCode IAM/SAR files and the Belgovia CO version files (Dataset_version_dependency/):
  • Point 13 — Jotunheim→RCC-Jotunheim, Belgovia→TSO-Belgovia
  • Point 18 — added dcterms:type = CIM-PowerSystemModel (all 29 were missing it)
  • Point 4 — added xmlns:adms + adms:versionNotes xml:lang="en" (all 29 were missing it)

Two correctness bugs fixed along the way:

• 2D_IAM_1430Z_IAM_.xml had a stray double quote in the ImpactAssessmentMatrix conformsTo (.../2.4""/>), which made it invalid XML. Note: that broken value has propagated into the generated Jotunheim/NetworkCode/manifest.ttl and Jotunheim/GridSituation/manifest.ttl (as ...2.4\", plus some empty dcterms:description ""@en), so those manifests should be regenerated from the corrected instance.
• The Jotunheim SAR headers declared xmlns:dcterms="http://purl.org/dc/terms/#" (trailing #), which resolves every dcterms:* term to the wrong IRI. Fixed in both the NetworkCode and GridSituation copies.

Newly-surfaced items for these files, still blocked on the reference-data decisions above:

• The Jotunheim IAM/SAR datasets have no dcterms:spatial at all (point 16).
• dcat:isVersionOf targets Jotunheim-IAM, Jotunheim-SAR, Belgovia-CO — none in Test-ModelScheme (point 10).
• dcterms:conformsTo uses ImpactAssessmentMatrix/2.4 and SecurityAnalysisResult/2.5 — not in ConformanceReleaseScheme (point 6).
• prov:wasGeneratedBy uses Activity/CGM-IAM, CGM-SAR, CGM-CO — not in ActivityScheme (point 21).
• RCC-Jotunheim / TSO-Belgovia still need Test-PartyScheme entries (point 13).

One question: Instance/Jotunheim/GridSituation/cimxml/ holds duplicate copies of the IAM/SAR datasets with the same header issues. So far I've only applied the eumd + SAR-namespace fixes there. Are those generated from the NetworkCode copies (so I should leave them and let regeneration handle it), or should I mirror the header fixes into them too?

[Sveino]

I have made some comments - but not run through all the files. After we have made all the update to the dcterms:title we need to update the file name (and remove the dateTime).

[Sveino]

Suggested change

	<dcterms:spatial rdf:resource="https://energy.referencedata.eu/Test/Frame/Belgovia-Power-Transmission-System"/>
	<dcterms:spatial rdf:resource="https://energy.referencedata.eu/Test/Frame/Belgovia-Transmission"/>

In the Frame https://github.com/entsoe/relicapgrid/blob/cgmes-3.0_ncp-2.4_tc-1.1/Instance/referenceData/NetworkCode/cimxml/Test-FrameScheme-NCP_RD.xml
we shorten it to "-Transmission" - this apply to all.

[Sveino]

Suggested change

	<dcterms:title>20250520_Belgovia_CO_1.0.0</dcterms:title>
	<dcterms:title>202505181800_TSO-Belgovia_SR-CO</dcterms:title>

The first date should match the dcat:startDate (but not sure why it start at 18:00, Structural data should only change on a daily resolution. SR-CO is following the activity naming standard. When it is version 1.0.0 we do not need to include this - as this is the default.

[Sveino]

Suggested change

	<dcterms:references rdf:resource="urn:uuid:886d71fa-6ca7-4be9-a55d-04c3c49c987c"/>
	<dcterms:requires rdf:resource="urn:uuid:886d71fa-6ca7-4be9-a55d-04c3c49c987c"/>

[Sveino]

we should use dcterms:requires rather than dcterms:references - we might need to update some to dcterms:references later.
I do see that some is change to requires - so it could be that this is correct.

[Sveino]

Suggested change

	<dcat:isVersionOf rdf:resource="https://energy.referencedata.eu/Test/Model/Belgovia-ER"/>
	<dcat:isVersionOf rdf:resource="https://energy.referencedata.eu/Test/Model/TSO-Belgovia_SR-ER"/>

See https://github.com/entsoe/relicapgrid/blob/cgmes-3.0_ncp-2.4_tc-1.1/Instance/referenceData/NetworkCode/cimxml/Test-ModelScheme-NCP_RD.xml

[MateZsebehazi]

Applied the changes:

1. dcterms:spatial — dropped Power- and -System from frame URIs

2. dcterms:title — rebuilt as {startDate as YYYYMMDDHHmm}_TSO-{Entity}_SR-{ActivityCode}, matching dcat:startDate instead of an unrelated date, and dropping the version suffix when it's the default 1.0.0

3. dcterms:references — removed wherever it duplicated an already-present dcterms:requires pointing to the same UUID (true in all 34 occurrences in scope). dcterms:requires was left as the single source of truth.

4. dcat:isVersionOf — renamed the trailing {Entity}-{Code} segment to match the new title convention, per Instance/referenceData/NetworkCode/cimxml/Test-ModelScheme-NCP_RD.xml