Criteria for including issue + single component file categorization

[matyaskopp]

@TomazErjavec
affected partners: @maria-gav @E2GG @Dandelliony @dareqx

After today's discussion, two related issues have arisen, so I am merging it in one:

1. What single component file contains?

• single newspaper issue (CZ)
• one article (UA)
• small random fragment (PL)

Here, we want to somehow encode what is present in the file. Shouldn't we introduce some taxonomy that says simply what is inside a file (/TEI/@ana) or part of a newspaper (//div/@ana)?

2. What type of "text input" should be included?

• newspaper
• newspaper supplement like attachments
• magazines

We need to set stricter boundaries on what should be included in corpora, and if we want to also include the edge types like newspaper supplements, then we need to categorize them, to be filtered out if users do not want them.

[TomazErjavec]

1. What single component file contains?

This is actually specified in the documentation.

* single newspaper issue (CZ)
* one article (UA)
* small random fragment (PL)

i.e. any of the above, as long as a date can be tied to the component. I don't think it makes sense to be more specific than that, as this will differ too much between the corpora (as the 3 cases above demonstrate).

Here, we want to somehow encode what is present in the file.

Currently, the documentation specifies that div/@type should be used to classify the divisions, and this could easily be used to classify a whole component, by having one divisions inside body with the appropriate @type. The documentation also gives a list of possible types, although this, of course, could be extended or changed.

Shouldn't we introduce some taxonomy that says simply what is inside a file (/TEI/@ana) or part of a newspaper (//div/@ana)?

This would certaily be another way of doing it. The downside is that it is more complicated, the upside that we can then have localised names of the various component / division types.

I would ask the involved parties to give a preference, and, if you want a taxonomy, I can implement it.

2. What type of "text input" should be included?
* newspaper
* newspaper supplement like attachments
* magazines

I would not include magazines, as we are, after all, making newspaper corpora. But supplements, sure.

We need to set stricter boundaries on what should be included in corpora, and if we want to also include the edge types like newspaper supplements, then we need to categorize them, to be filtered out if users do not want them.

OK, so these would be either further div/@type values or further categories in the taxonomy. So let's first decide which approach to use, and then discuss the values.