Units for experimental Young's modulus data

[eriktamsen]

While working on the calibration PR, I had a look at the extracted experimental data for the Young's modulus test.
The units are part of the name of the columns in the CSV file. This is not ideal, or usable.
As I have lost the overview of what is happening in which branch and issue, I am not sure if this has already been adressed.
As far as I can remember, all relevant fields in the ontology have units linked.
I assume therefore the data extraction script needs to be improved, to solve this, so that the units are availiable in a machine readble format. I would suggest to add them to the metadata yaml (or json if that has been changed).
It is possible, that the units dont change as our data extraction is only for one specific machine. If this is the case, this would still require the information to be availiable. Meaning, e.g. just hard coding this data in the meta data file.

@AidaZt, @ThiloMuth, @StephanPirskawetz: Please check if any change is required and if so, make a plan of how to best implement this.

[AidaZt]

There is no placeholder currently in knowledge graph templates to store the information of the units. This ones mentioned in #118 , I think Alina is looking into it. @alFrie right?

[eriktamsen]

Then there are two independent steps to be done

• include the units in the ontology / the placeholder in the KG
• improve the extraction script to save the values in a file

[alFrie]

There is no placeholder currently in knowledge graph templates to store the information of the units. This ones mentioned in #118 , I think Alina is looking into it. @alFrie right?

Correct, there are no placeholders for units.

[alFrie]

As far as I see it we have four options:

1. Extract units via metadata extraction, save them raw in json, map them raw to the KG
2. Extract units via metadata extraction, save them raw in json, convert & map them via mapping script
3. Extract and adjust units via metadata extraction, save then in json, map them to the KG
4. Extract and adjust units via metadata extraction, save then in json, not map them to the KG but give maybe a single placeholder saying we use IS base units or whatever

Could you just chose one of them or mention another if I forgot an option? @eriktamsen @joergfunger

Edit: And another question, the validation of the units, if they make sense and so on, is then part of the Shacl validation and not the extraction/mapping, right?

[joergfunger]

not sure I understand 1/2/3, 4 is not optimal. From my point of view, the unit should always be in the metadata.json (in case this is existing in the raw data, then use that value, if it is not there, we will fix this within the metadata extraction script which is specific to a defined raw file with an implicit assumptions on what units have been used). Then we will map the units similar to all other metadata files in the KG templates.

[alFrie]

What I ment with raw units/ converted units is that the original script by Ilias contained a function to check f.e. lengths above 100 and assuming then that the unit is cm and converting it to m and so on.

[joergfunger]

I think this automatic conversion creates more problems than it solves. I would rather have an error message (e.g. in the SHACL validation when certain assumptions (see above) are not fulfilled.

[alFrie]

That means you would prefer option 1: extract the units like they are given (raw) and map them raw to the KG, later check in the SHACL validation if they make sense, right?

Edit: While units are given for the mix metadata in the xlsx files (meaning we could just hardcode the units in the KG for this case), there are none given for the specimen.dat ... what to assume in this case? And also hardocde it?

[alFrie]

So Aida and I just had a meeting with Mattheo and talked about the implementation of units in the template. 
Two things that we talked about: 

1. Doesn't adding not-unified units affect the possibility to query? If I query for a 1kg heavy specimen, I might not find 1000g heavy specimen. This means that the units should be definitely unified.
2. It is not as easy to map units as it was with previous values, where we could just replace a placeholder with a value-string. Units in ontologies have their own category. It would be easier to pre-define in the template a unit for each instance. This would mean we could just keep mapping the non-converted data as it is, not change anything about the mapping script and later validate with shacl and if neccessary correct the value to fit the unit. Either way, it's your call @joergfunger and @eriktamsen . Please decide on a way of handling the units.