LZ4 v1.10 Introduces Multi-Threading Support For Major Compression Speedups
([Free Software] 5 Hours Ago
LZ4 Multi-Threading)
- Reference: 0001480321
- News link: https://www.phoronix.com/news/LZ4-1.10-Multi-Threading
- Source link:
Yann Collet released LZ4 v1.10 today as a major update to this extremely fast compression algorithm. Most significant with LZ4 1.10 is adding multi-threaded compression support for much faster performance with today's modern multi-core processors.
LZ4 1.10 has been dubbed the "multi-cores edition" with this version adding multi-threading support to help speed-up compression now that modern I/O storage with NVMe is so much faster there's a real need to make compressing data even faster.
Benchmarks by Yann Collet show LZ4 1.10 with multi-threading being 5.4~8.0x faster with various data and across Intel / AMD / Apple processors on Windows / macOS / Linux. LZ4 compression with multi-threading on any modern CPUs should now be much, much faster than prior versions. LZ4 decompression can also be ~60%+ faster with overlapping decompression but not as important as the compression speed-ups.
LZ4 1.10 also promotes its dictionary compression from being "experimental" to a fully supported feature. There's also a new "Level 2" compression mode for LZ4, build system improvements, various CLI fixes, and a variety of other optimizations and code improvements.
Downloads and more details on this big LZ4 v1.10 update via [1]GitHub . I'll have out some new [2]LZ4 benchmarks soon across different processors in looking at the multi-threaded compression performance gains.
[1] https://github.com/lz4/lz4/releases/tag/v1.10.0
[2] https://openbenchmarking.org/test/pts/compress-lz4#results
LZ4 1.10 has been dubbed the "multi-cores edition" with this version adding multi-threading support to help speed-up compression now that modern I/O storage with NVMe is so much faster there's a real need to make compressing data even faster.
Benchmarks by Yann Collet show LZ4 1.10 with multi-threading being 5.4~8.0x faster with various data and across Intel / AMD / Apple processors on Windows / macOS / Linux. LZ4 compression with multi-threading on any modern CPUs should now be much, much faster than prior versions. LZ4 decompression can also be ~60%+ faster with overlapping decompression but not as important as the compression speed-ups.
LZ4 1.10 also promotes its dictionary compression from being "experimental" to a fully supported feature. There's also a new "Level 2" compression mode for LZ4, build system improvements, various CLI fixes, and a variety of other optimizations and code improvements.
Downloads and more details on this big LZ4 v1.10 update via [1]GitHub . I'll have out some new [2]LZ4 benchmarks soon across different processors in looking at the multi-threaded compression performance gains.
[1] https://github.com/lz4/lz4/releases/tag/v1.10.0
[2] https://openbenchmarking.org/test/pts/compress-lz4#results
scottishduck