At the moment, TextRank on n-grams (adjectives+nouns) is used for keywords extraction.
collect_np function in util.data_preprocessing.py for n-grams collection and util.textrank for TextRank scores computation
This approach leads to very long unconnected n-grams as output keywords. For example, 'вопрос доставка мигрант белоруссия представитель пресс-служба еврокомиссия стефан' becomes one keyword.
The question is how to split these long keywords. Should we collect noun phrases another way (collect_np function in util.data_preprocessing)? Or should we process output of the current TextRank algo to split these long keywords after we already got them?
At the moment, TextRank on n-grams (adjectives+nouns) is used for keywords extraction.
collect_npfunction inutil.data_preprocessing.pyfor n-grams collection andutil.textrankfor TextRank scores computationThis approach leads to very long unconnected n-grams as output keywords. For example, 'вопрос доставка мигрант белоруссия представитель пресс-служба еврокомиссия стефан' becomes one keyword.
The question is how to split these long keywords. Should we collect noun phrases another way (
collect_npfunction in util.data_preprocessing)? Or should we process output of the current TextRank algo to split these long keywords after we already got them?