News: 0001583163

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Meta Opens Up OpenZL As Format-Aware Compression Framework

([Free Software] 4 Hours Ago OpenZL)


Meta already has [1]Zstd (Zstandard) compression while this week they announced the release of OpenZL as a new open-source, format-aware compression framework.

OpenZL aims to provide high compression ratios while maintaining high speed. High compression and high performance typically don't go hand-in-hand for generic compressors while OpenZL is format-aware in trying to leverage the best of both worlds using a specialized compressor depending upon the format.

OpenZL focuses on lossless compression for structured data and the performance of format-specific compressors but with just a single executable binary.

They explained in their announcement that OpenZL comes after hitting rather a wall in Zstandard development:

"However, while it was improved over time, remaining within the Zstandard framework offers diminishing returns. So we started looking for the next great leap in data compression.

In this quest, one pattern kept repeating: Using generic methods on structured data leaves compression gains on the table. Data isn’t just byte soup. It can be columnar, encode enums, be restricted to specific ranges, or carry highly repetitive fields. More importantly, it has predictable shapes. A bespoke compressor that leans into that structure can beat general-purpose tools on both ratio and speed. But there’s a catch — every bespoke scheme means another compressor and decompressor to create, ship, audit, patch, and trust.

OpenZL is our answer to the tension between the performance of format-specific compressors and the maintenance simplicity of a single executable binary."

Those wishing to learn more about OpenZL can do so via [2]this Facebook Engineering blog post . The OpenZL code is BSD licensed and is available from [3]GitHub .



[1] https://www.phoronix.com/search/Zstd

[2] https://engineering.fb.com/2025/10/06/developer-tools/openzl-open-source-format-aware-compression-framework/

[3] https://github.com/facebook/openzl



The salesman and the system analyst took off to spend a weekend in the
forest, hunting bear. They'd rented a cabin, and, when they got there, took
their backpacks off and put them inside. At which point the salesman turned
to his friend, and said, "You unpack while I go and find us a bear."
Puzzled, the analyst finished unpacking and then went and sat down
on the porch. Soon he could hear rustling noises in the forest. The noises
got nearer -- and louder -- and suddenly there was the salesman, running like
hell across the clearing toward the cabin, pursued by one of the largest and
most ferocious grizzly bears the analyst had ever seen.
"Open the door!", screamed the salesman.
The analyst whipped open the door, and the salesman ran to the door,
suddenly stopped, and stepped aside. The bear, unable to stop, continued
through the door and into the cabin. The salesman slammed the door closed
and grinned at his friend. "Got him!", he exclaimed, "now, you skin this
one and I'll go rustle us up another!"