Zstandard v1.5.4 Unveiled: Faster Compression with New Improvements

Zstandard v1.5.4 was just released, and it promises to decompress and compress files even faster than previous versions.

More than a year in the making, Zstandard version 1.5.4 has finally been released, bringing with it faster compression speeds and a number of improvements to the compression algorithm with new features and a little bit of work on bug fixes.

In terms of performance, Zstandard version 1.5.4 has seen a 10–20% speed improvement over the previous version, with an increase in performance optimization across various platforms and architectures.

which you can clearly see in one of the test results that I pulled from the changelog for you, where they report an 8% performance increase in  compression speed when compared to v1.5.3

M1 Prodecompresssilesia.tar1370 MB/s1480 MB/s+ 8%

Aside from that, using the middle compression levels (5–12) will result in a 1–10% speed improvement as well as significant changes in dictionary compression.

Thanks to the new asynchronous I/O capability, you will find improvements in the zstd cli tool, which is active by default, and if you don’t want to use it, you can use the –[no-]asyncio flag to disable it.

The performance of versions 1.5.4 and 1.5.2 on different platforms is compared in the following table, along with the asyncio default state of enabled and disabled for enwik8 and silesia corpus.

platformcorpuszstd-1.5.2zstd-1.5.4-no-asynciozstd-1.5.4-asyncio1.5.4-asyncio / 1.5.2
Xeon D-2191A CentOS8enwik80.3410.3410.294x1.16
Xeon D-2191A CentOS8silesia0.6660.6690.523x1.27
i7-9700K Ubuntu20enwik80.1590.1580.115x1.38
i7-9700K Ubuntu20silesia0.2960.2980.204x1.45
MBP M1enwik80.2230.1300.117x1.91
MBP M1silesia0.4350.2310.202x2.15

With the help of a new advanced registration function, libzstd now supports external sequence producers and enables you to register unique sequence producers so that libzstd can make use of Intel® QAT hardware acceleration for matchfinding.

This is a significant advancement as it enables the libzstd library to use hardware acceleration to improve compression ratios and speeds.

Aside from this, there are also a number of improvements and bug fixes, which you can read about in the latest version’s release notes.

Download Latest Zstandard or zst

You can use the download link down below to get the most recent version of Zstandard or the zst CLI program, which will take you to the release section where you can choose the right file for your operating system.

Source: Official Changelog

Leave a Reply