[Looking for beta testers] Sort tags by frequent use

[DominikDoom]

TL;DR: I am looking for feedback on a big new feature. Try out the https://github.com/DominikDoom/a1111-sd-webui-tagcomplete/tree/feature-sort-by-frequent-use branch if you want to help and tell me if you face issues with it.

ℹ️ About the branch

For the last few months, I have been sporadically working on a big new feature:
Sorting tags, extra network models, and most other stuff tag autocomplete supports by usage.

It drastically changes the way you can interact with the extension, as tags you use frequently will be sorted higher than other matches for the same search phrase. This makes the search feel much smarter and reduces the amount of typing, as you don't have to scroll down to get to a tag you like or get more specific just because an overall more popular tag has a higher count by default. Negative prompts are kept separate, so you can have those "bad" tags sorted without contaminating your normal prompt completion.
This all gets recorded and stored locally in an SQLite database file in the extension's "tags" folder.

Ex.: You want to type "solo", but "sole_female", an alias for "1girl", has a higher count and always comes first, requiring navigating a few entries down or typing out the full word. With frequency sort, "solo" will rank higher after you used it enough times, enabling direct completion after typing just "s".

Some example images of the feature in use

A tag sorted higher due to frequency will be marked with the ✨ symbol. It can also be reset at any time:

"solo" ranks higher than "sole_female"/"1girl":

"standing" ranks first despite a much lower post count:

Also works for embeddings:

And for lora:

New settings:

There are various settings to configure it further, as usual (also shown in the example images):

• "Locally record tag usage and sort frequent tags higher"
  • Turns the feature on/off
• "Function to use for frequency sorting"
  • This determines how strong the influence of usage count on the sort order should be.
• "Minimum number of uses for a tag to be considered frequent"
  • As the name suggests, usages lower than this will just be ignored.
• "Maximum days since last use for a tag to be considered frequent"
  • This will not consider tags you haven't used for a long time to prevent having to manually reset them. It will continue as normal from the old count after you first use it again.
• "Frequency sorting matches aliases for frequent tags"
  • Normally, aliases are ignored during frequency sorting since they would often disrupt what you intend to type, due to the main tag the alias links to ranking higher again (e.g. with the "solo" example). This option can be used to turn it on in case you want this.

⚠️ Feedback wanted!

The feature is already very usable and now nearing completion for the first release, but I still have some concerns I want to collect feedback for.
If you want to help, checkout or clone the https://github.com/DominikDoom/a1111-sd-webui-tagcomplete/tree/feature-sort-by-frequent-use branch. It is on par with main, so you can switch without losing bug fixes or recent new features.

Since this feature uses a local database, I am especially interested in the following points:

• Fresh install or first start after updating, do any path or SQL errors occur?
• Different path setups, e.g.
  • Linux vs. Windows
  • Symlinks in use
  • User data directory (extensions, models, etc.) in a separate folder or another drive
  • Installed inside Docker container
• Performance
  • As this adds additional overhead to the sorting, it might introduce some small lag while typing.
  • Is this lag noticeably worse than with the feature disabled on your system?
  • Does it hinder you while typing, and is it "worth it" if you can reach your wanted tag faster?

As a whole, I am just looking for some more feedback from different setups, especially on weaker systems.

Of course, I am also open to suggestions if you have other ideas to improve the feature.
For example, one thing I'm considering for a future release is a script that can pre-populate the usage database from your already generated images. Instead of slowly adapting during use, it could use the PNG info from a folder you provide to count the tags you used in these images.

[Symbiomatrix]

Hey dom. I've setup a small custom sort function for myself (of course humbled by this project), from which I've learned this one major usage concern: it's inevitably going to bury any new models one obtains beneath a (huge) pile of most frequently used items, even if limited to just one-two days. Perhaps a user set hard cap on the number of frequency favoured tags / models at the top of the list, followed by usual order? Sure, one could switch frequency <-> date modified back & forth, but a full refresh takes a while and is a distraction. I've only skimmed the code, sorry if I missed something that could obviate the problem.
Since reaction speed is somewhat essential given the live update, you could probably update the database asynchronously (frequency counting may not be that important in the short term but for future reference), and reorder per score counts client side periodically during inactivity or on manual refresh. Or cache the updates.

[DominikDoom]

Perhaps a user set hard cap on the number of frequency favoured tags / models at the top of the list, followed by usual order?

Yes, that would be a possibility. I'm still not 100% happy with the behavior myself which is why it isn't merged yet. Burying new or infrequently used tags is something I noticed as well. It would definitely have to be an optional thing though, as for infrequent use the date method works well enough on its own and a hard cap or more aggressive usage filtering might remove more than the user wants.

Since reaction speed is somewhat essential given the live update, you could probably update the database asynchronously (frequency counting may not be that important in the short term but for future reference), and reorder per score counts client side periodically during inactivity or on manual refresh. Or cache the updates.

Caching is something I'm already working on and want to implement before merging, I've just gotten a bit busy with work and can only continue sporadically here. The increment itself is already fire-and-forget and not awaited, so the database update is not a factor in speed. But querying is always fetching live data right now and pretty inefficient. My current idea is to load the frequencies once at startup and then increment the loaded cache alongside the real DB. That way you'd get instant feedback without having to wait for the network requests and the DB would be more of a permanent offline storage. Since querying the usage count doesn't need much SQL, it would probably still be pretty efficient and only need a few KB more memory to keep the client-side copy.

[Symbiomatrix]

The only other method I could think of would be to give ranking to new unused items, but that's both more of a hassle to implement and may likely still result in tweaking annoyances (Missed the time window by a day, or don't want all that random stuff cluttering the top slots? Tough!).
Well, and in the case of my custom sort, various files are nested in folders (aa*, bb*, zz*) so that new can be accessed via a simple "zz*", and I've also removed the selectedTag nullifications so that I can always continue browsing from where I was before easily even after the menu is hidden, but those modifications are hardly everyone's cup of tea and may take some mental adjustment. Now, if you could save a dictionary of positions for each query, that'd be really something, but I see nothing less of an overhaul in that and a myriad of problems. Or maybe preview going by cursor hover. But I digress.

Oh, indeed? Hadn't seen any threading py side, so I guess the async call handles that. Any locking to prevent collisions, or does it just display / suppress some sorta html error? What if the db is kept locked by some naughty user who forgot to commit?
Aye, updating cache side by side is exactly what I had in mind.

[DominikDoom]

The only other method I could think of would be to give ranking to new unused items, but that's both more of a hassle to implement and may likely still result in tweaking annoyances (Missed the time window by a day, or don't want all that random stuff cluttering the top slots? Tough!).

Yeah I don't think that would work well, most of the time the new stuff I drop in is just random neat stuff picked up by chance on Civitai, not necessarily something I want to use instantly. A simple max amount of recommended tags from the first suggestion would still work best for all cases IMO: If you frequently use new stuff you can set it to a low cap, and if you infrequently use tags or models other than your existing favorites, setting it to a high number replicates the current behavior.

Oh, indeed? Hadn't seen any threading py side, so I guess the async call handles that. Any locking to prevent collisions, or does it just display / suppress some sorta html error? What if the db is kept locked by some naughty user who forgot to commit?
Aye, updating cache side by side is exactly what I had in mind.

All API calls on the python side are async by design, so as long as these don't await, they shouldn't block anything else. In python return implicitly waits for the result (although return await ... would work the same and would be necessary if the secondary function is itself async). So API calls that return something are awaited and those that don't are fire-and-forget. Currently I just handle all writes to the DB as fire-and-forget since I don't need confirmation if it succeeded. It's not a critical function, so if it fails, that's sorta ok (assuming the rest of the extension still works in that case). Those that do return something and fail would fail the same as anything else. The JS fetch would recieve null either instantly or after a timeout, and most likely throw an error trying to access nonexistent JSON properties. Admittedly I haven't tested much yet if the rest of the extension would continue working in that case.

As for the DB lock, it's all transaction based. Each call is atomic and closes the connection afterward. So long as the whole process doesn't crash (which would require a restart anyway) the lock will clear automatically. Of course that doesn't take into account if the file is write-protected by the system or something like that, but again, the app wouldn't work at all in that case.