Tuning / tweaking Finto AI to give more creative or less-apparent subject headings suggestions

[tpalonen]

1. Currently, many of the subject headings suggestions given by Finto AI are terms that occur in the actual text. As an alternative, it would be interesting to have Finto AI give suggestions a bit above/below the text surface. Suggestions that might appear more creative, derivative, interpretative. It would be nice to have a kind of a knob in the user interface to allow the user to choose between very literal or more creative results or something in between. Technically, the more creative results could perhaps be produced by stressing MLLM and/or Bonsai more in the process.

===========================

2. As an addition or as a separate suggestion to the above: it would be interesting to have an option to filter out the most commonly used subject headings from the suggestions. For example, "history" might be one the most commonly used subjects headings in the context of humanities. And if the text in question is a doctoral thesis in history, the subject heading "history" would be a very unsurprising and therefore not a very helpful result particularly if the user is looking for more creative suggestions. The challenge here is the context, as the most commonly used subject headings are domain-specific. Nevertheless, it might be interesting to see what the results would look like if a multidomain dataset, such as Melinda, could be used to identify and filter out the most common subject headings. These could be semi-automated, ie. first identified statistically from a dataset and secondly reviewed by an expert to see which subject headings are truly & globally generic and which ones just happen to stand out in the given dataset. Alternatively, Finto AI could perhaps have several domain-specific lists of unwanted subject headings. Finto AI could then first identify the text's domain and then apply the unwanted subject headings list of that domain to exclude the respective subject headings from the suggestions.

[juhoinkinen]

1. A simple way to implement tuning the relative probabilities of literal suggestions (those appearing in the input text) and "more associative suggestions" (coming from associative backends) would be to allow tuning the weights of lexical (MLLM) and associative (Omikuji and FastText) backends in an ensemble project. It is already possible on Annif side when running suggest function from CLI with the --backend-param/-b option, the similar functionality should be added to REST method.
2. The ability to tune from general to specific suggestions sounds really nice. There are a few related issues of Annif: Use subsets of vocabularies in ensembles Annif#596, Dealing with overrepresented concepts / blacklisting Annif#735. The feature could also work by utilizing the ontology hierarchy: a specificity index of a concept could be calculated from its position in the tree (the number of concepts above vs below), and the indices could then be taken into account in the suggestions.

[tpalonen]

True, ontology hierarchy might be of help here. But the position in the hieararchy is not always a good indicator of how specific a concept is. Consider the subhierarchy of the YSO concept properties. Most of the subclasses are pretty specific (inexperience, heat resistance, predictability etc) even though they appear very high in the hierarchy (ie. there's only one level up in the hierarchy above them). https://finto.fi/yso/fi/page/p8691?clang=en

In many contexts, the YSO hierarchy is pretty flat, actually.

Me and the rest of the YSO maintenance team would be happy to help here.