super high memory usage doing rescan

[unoukujou]

I notice from doing the initial rescan just 2 folders of about 50 total pictures, photofield in a matter of seconds went up to about 1.2GB ram usage.

Is that normal or maybe some memory leak?

Because that's only from 50 pictures just for testing. So I wonder what would happen if I point it to my real picture collection with thousands of pictures.

Maybe there is some setting to tone it down a little?

I tried cache max_size 256Mi but did not do anything, still goes over 1GB in just a few seconds after hitting rescan on 50 pics

[SmilyOrg]

Hi! Thanks for being so active in the issues first of all, I appreciate it!

1.2GB is not too unexpected, it could be the caches, but also the temporary garbage created while parsing photos. Go tends to keep the memory already allocated for some time to prevent it fluctuating too much.

I should take a look to minimize this, but I wouldn't worry too much about running it on more than 50 files. Memory shouldn't grow too much more than that. I have about 700k photos in my collection for reference :)

[unoukujou]

Alright I'll try with 100 photos next time to see what happens.

I also thought it can be the Go garbage collection and already tried with GOMEMLIMIT=768MiB and GOGC=20 but it did nothing noticeable.

I don't use docker, just running it as systemd service so I tried to set memory limits through systemd but it cause photofield to freeze and I had to force stop it.

For now I'll just make sure that I always have a lot of free ram available before I ever do rescans. Because when I was trying it for the first time, I was already at around 80-85% used. And then running photofield rescan got me up to 99%