Integration

[meehkal]

This subcorpus should not grow separated from our "data backbone". Ideally, we maintain and develop a complete set of data, metadata and annotations for in a private folder and derive the gold subcorpus from there automatically. However, until the full implementation of this interaction, we build this gold subcorpus separated from the other data repositories.

Ideas:

• For gold corpus annotations, we use separate ELAN tiers because they need to be added or corrected manually; the annotations of these tiers may undergo permanent changes (additions, corrections, etc.), but these changes are only done in the "data backbone working repository"; in the future, the present repository get its data automatically from the working repository
• tiers: ref@ orth@ ft-eng@ word-gold@ lemma-gold@ pos-gold@ morph-gold@
• For gold corpus metadata, we define a minimal subset of metadata categories; they are also derived automatically (see above) and included completely into the EAF file (either as attributes or in tier names)
• Since our session names are unique we can include a link to the corresponding sessions in the CMDI catalogue in order to provide more complete metadata as well as multimedia or additional corpus annotations (if available)

[nikopartanen]

I think the cleanest solution to this is to have a separate public branch with only specific files being automatically added and pushed there when they change in the private working branch. It is somewhat similar to what is described here with small adaptations. I'm not sure yet what is the best way to specify which files should be in the public branch though, and how to do that transformation Micha described most conveniently.

Micha, would these gold-tiers be somehow manually corrected? It may be difficult to keep them synchronised with the private branch if there are any changes on orth-level. The way I see it the whole file would just be created again every time it changes in the working branch, although it could also be done so that there is a list of differences somewhere and that is manually maintained, which is in many ways more complex path to take, but I guess only viable one with manually corrected tiers.