Skip to content

feat: implement tokenized serach for news filtering by text #5

Description

@zefr0x

Current State

We have a simple substring matching with SQLite's LIKE keyword.

Possible Solution

The SQLite FTS5 Extension with its trigram tokenizer looks like a suitable option.

It should be able to tokenize text of all UTF-8 characters, not just ASCII.

Issues to Solve

Currently in the get_news_with_filter method we are using FirstFetchedAt as a cursor, however, with FTS5 we will get a ranked list of best matches first, so we need to use a different type of cursor and a special pagination for news searching.

Open Problems

Highlighting search matches in the displayed results is great for UX, but I'm not sure if this is Aleym Core's issue or the presenter's. If it was the presenter's then it will need an identical algorithm to the one used by FTS5 and it will do some redundant calculations that should be cheap.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions