Consider optional zstd compression instead of gzip

[ChloeSartorelli]

Zstandard (aka "zstd") is a newish (2016) compression algorithm from Facebook that boasts significantly faster compression and decompression times than gzip, often with smaller resulting files. It also has adjustable compression levels that allow you to choose between and tune for maximum speed or minimal file size.

It might be a good option to add to this utility, as it would make creating and restoring snapshots faster and usually result in smaller files.

[axllent]

Absolutely (I love zstd) - however that would break format compatibility with the official sspak format/tool would it not?

[ChloeSartorelli]

That's one of the main reasons I recommend it as an option rather than the-way it's done.

But also, sspak is effectively abandoned right now so I don't think compatibility with that tool should remain a long-term goal of this tool at the expense of missing out on opportunities to be improved.

[axllent]

That's one of the main reasons I recommend it as an option rather than the-way it's done.

That's fair - an optional flag (with note that it would break compatibility).

But also, sspak is effectively abandoned right now so I don't think compatibility with that tool should remain a long-term goal of this tool at the expense of missing out on opportunities to be improved.

Oh, I was not even aware of the "abandonment" (lack of love), but looking at the commit log that appears to be the case. Assuming you have more insight that I do, is this because SS has a better way to manage snapshots, or just because the current (official) tool still works?

I will look into adding zstd support in the relatively-near future as an optional flag, thanks for the suggestion!

[ChloeSartorelli]

Assuming you have more insight that I do, is this because SS has a better way to manage snapshots, or just because the current (official) tool still works?

It's a combination of the team that defacto owns it being too small to really take care of it (we have a whole framework and CMS and supported modules to look after, and two people to do it with), and the tool working kinda-almost well enough (i.e. there are no active fires to put out).

I suspect that if it outright stops working (or if people get fed up with it already not working on large data) then the solution would be to look for alternatives rather than doing dev work on sspak, though I can't say for sure.

[axllent]

I've been doing some experimenting with my local development version to get an idea of what the result would be if I introduced a flag to use zstd compression - focussing (for this test) on just the assets.

One point to note is that the Go implementation of zstd encoder I'm using only has two compress settings; fastest and default - I'm just using default here (you''ll see why below)... The following test compares the output size and time taken to create an sspak from a large (8.9GB) assets folder, first with the default gzip compression, and then with zstd:

gzip:

$ time go run . save --assets /tmp/example.com/ /tmp/gzip.sspak -v
2026/02/06 10:39:52 Parsing /tmp/example.com/.env
2026/02/06 10:39:52 Creating temporary directory '/tmp/ssbak-00b80c37bb79'
2026/02/06 10:39:52 Calculating size of '/tmp/example.com/public/assets'
2026/02/06 10:39:52 Compressing '/tmp/example.com/public/assets' (8.9GiB) to '/tmp/ssbak-00b80c37bb79/assets.tar.gz'
2026/02/06 10:42:27 Wrote '/tmp/ssbak-00b80c37bb79/assets.tar.gz' (8.7GiB)
2026/02/06 10:42:27 Creating SSPak archive `/tmp/gzip.sspak`
2026/02/06 10:42:36 Wrote '/tmp/gzip.sspak' (8.7GiB)

real    2m45.494s
user    1m46.640s
sys     0m56.958s

zstd:

$ time go run . save --assets /tmp/example.com/ /tmp/zstd.sspak -v -z
2026/02/06 10:42:53 Parsing /tmp/example.com/.env
2026/02/06 10:42:53 Creating temporary directory '/tmp/ssbak-f53001c16bc8'
2026/02/06 10:42:53 Calculating size of '/tmp/example.com/public/assets'
2026/02/06 10:42:53 Compressing '/tmp/example.com/public/assets' (8.9GiB) to '/tmp/ssbak-f53001c16bc8/assets.tar.zst'
2026/02/06 10:43:05 Wrote '/tmp/ssbak-f53001c16bc8/assets.tar.zst' (8.3GiB)
2026/02/06 10:43:05 Creating SSPak archive `/tmp/zstd.sspak`
2026/02/06 10:43:13 Wrote '/tmp/zstd.sspak' (8.3GiB)

real    0m20.948s
user    0m8.053s
sys     0m16.469s

So from 2m45s down to just 21s, including an additional ~400MB savings. That is a pretty phenomenal improvement! Anyway, I'll keep working on this feature.

[axllent]

@GuySartorelli I'm sorry it took so long - I ended up in hospital with a septic shoulder shortly after our previous conversation, and I'm just returning to normal again now.

I have just released a new beta version of ssbak which includes support for zstd (enabled with -z/--zstd while saving a backup [add -v flag to see more info]). I decided not to implement a zstd compression setting, as the default is a good mix of compression and speed. This also included a significant rewrite of large portions of the code, so extensive testing is definitely needed. I have done my own testing, but I would really appreciate it if you could on your end too and provide any feedback?

[axllent]

@GuySartorelli ?

[ChloeSartorelli]

Hi, sorry, I had a whole bunch of personal things all at once in April and am only just coming back to work now part time. I don't think I'll have time to test this sorry.

[axllent]

@ChloeSartorelli - sorry to hear that, and thanks for getting back to me. Not a problem, I will proceed with the optional -z/--zstd flag as I know that approach has merit, I just need to invest extra time to do more testing.