Updated XZ Code For The Kernel Looks Like It's Ready For Linux 6.12
([Linux Kernel] 5 Hours Ago
Linux Kernel XZ Update)
- Reference: 0001485157
- News link: https://www.phoronix.com/news/XZ-Update-Linux-6.12-Queue
- Source link:
The past few months have seen [1]patches for updating the XZ (de)compression code within the Linux kernel that's been a lengthy process in part due to [2]the XZ backdoor situation earlier this year and the reputable XZ developers in turn being busy cleaning up that mess. In any event it appears that the updated XZ code for the Linux kernel is now on track for mainlining come Linux 6.12.
This code adapts to the upstream XZ Embedded shifting from public domain to the BSD Zero Clause License, drops the notorious "Jia Tan" malicious contributor from being a maintainer, and Image.xz support is added in the process for ARM64 and RISC-V kernel builds.
This updated XZ code for the Linux kernel also now is set to use a 128MB dictionary and force XZ's single-threaded mode. Developer Lasse Collin explained of that alteration:
"Desktop kernels on x86-64 are already around 60 MiB. Using a dictionary larger than 32 MiB should have no downsides nowadays as anyone building the kernel should have plenty of RAM. 128 MiB dictionary needs 1346 MiB of RAM with xz versions 5.0.x - 5.6.x in single-threaded mode. On archs that use xz_wrap.sh, kernel decompression is done in single-call mode so a larger dictionary doesn't affect boot-time memory requirements.
xz >= 5.6.0 uses multithreaded mode by default which compresses slightly worse than single-threaded mode. Kernel compression rarely used more than one thread anyway because with 32 MiB dictionary size the default block size was 96 MiB in multithreaded mode. So only a single thread was used anyway unless the kernel was over 96 MiB.
Comparison to CONFIG_KERNEL_LZMA: It uses "lzma -9" which mapped to 32 MiB dictionary in LZMA Utils 4.32.7 (the final release in 2008). Nowadays the lzma tool on most systems is from XZ Utils where -9 maps to 64 MiB dictionary. So using a 32 MiB dictionary with CONFIG_KERNEL_XZ may have compressed big kernels slightly worse than the old LZMA option.
Comparison to CONFIG_KERNEL_ZSTD: zstd uses 128 MiB dictionary."
These XZ updates for the Linux kernel have landed in Andrew Morton's [3]mm-everything branch . With working their way now into an MM Git branch, they are likely -- barring any last minute issues -- to be submitted by Morton for the Linux 6.12 merge window opening in mid-September.
[1] https://www.phoronix.com/news/XZ-Linux-Kernel-Patches-v2
[2] https://www.phoronix.com/news/XZ-CVE-2024-3094
[3] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/log/?h=mm-everything
This code adapts to the upstream XZ Embedded shifting from public domain to the BSD Zero Clause License, drops the notorious "Jia Tan" malicious contributor from being a maintainer, and Image.xz support is added in the process for ARM64 and RISC-V kernel builds.
This updated XZ code for the Linux kernel also now is set to use a 128MB dictionary and force XZ's single-threaded mode. Developer Lasse Collin explained of that alteration:
"Desktop kernels on x86-64 are already around 60 MiB. Using a dictionary larger than 32 MiB should have no downsides nowadays as anyone building the kernel should have plenty of RAM. 128 MiB dictionary needs 1346 MiB of RAM with xz versions 5.0.x - 5.6.x in single-threaded mode. On archs that use xz_wrap.sh, kernel decompression is done in single-call mode so a larger dictionary doesn't affect boot-time memory requirements.
xz >= 5.6.0 uses multithreaded mode by default which compresses slightly worse than single-threaded mode. Kernel compression rarely used more than one thread anyway because with 32 MiB dictionary size the default block size was 96 MiB in multithreaded mode. So only a single thread was used anyway unless the kernel was over 96 MiB.
Comparison to CONFIG_KERNEL_LZMA: It uses "lzma -9" which mapped to 32 MiB dictionary in LZMA Utils 4.32.7 (the final release in 2008). Nowadays the lzma tool on most systems is from XZ Utils where -9 maps to 64 MiB dictionary. So using a 32 MiB dictionary with CONFIG_KERNEL_XZ may have compressed big kernels slightly worse than the old LZMA option.
Comparison to CONFIG_KERNEL_ZSTD: zstd uses 128 MiB dictionary."
These XZ updates for the Linux kernel have landed in Andrew Morton's [3]mm-everything branch . With working their way now into an MM Git branch, they are likely -- barring any last minute issues -- to be submitted by Morton for the Linux 6.12 merge window opening in mid-September.
[1] https://www.phoronix.com/news/XZ-Linux-Kernel-Patches-v2
[2] https://www.phoronix.com/news/XZ-CVE-2024-3094
[3] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/log/?h=mm-everything
caligula