Skip to content

Recommendations ignore descriptions in Chinese, Japanese and other scripts without letter case #1395

Closed

Description

@A-EDev

NeuroTokenizer.extractDescriptionKeywords drops every description line that equals its own uppercase (line.trim() == line.trim().uppercase(), meant to skip shouted lines like "SUBSCRIBE NOW"). Chinese, Japanese, Thai and other scripts have no letter case, so every line passes that check and is dropped. Descriptions in these languages never give keywords.

Found by @Chiehx0220 while working on #1366.

Where: app/src/main/java/io/github/aedev/flow/data/recommendation/NeuroTokenizer.kt, the content-line filter in extractDescriptionKeywords.

Expected: only skip a line as all-caps when it has cased letters, e.g. when line.any { it.isLetter() && it.isUpperCase() } and no lowercase letters.

Related: #1341.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions