On-disk format change beckons for brave early adopters of Bcachefs
(2024/02/28)
- Reference: 1709121444
- News link: https://www.theregister.co.uk/2024/02/28/new_vers_bcachefs_zfs/
- Source link:
New versions of the two leading next-generations filesystems are coming: both OpenZFS 2.2.3, and some time afterwards, an improved bcachefs.
Kernel 6.7 [1]only appeared last month and as we wrote at the time, it finally included the next-gen bcachefs filesystem. We say finally because it [2]didn't make it into 2023's kernel 6.5 , just as it [3]didn't make it in 2022 when we looked at its features and what it promised to deliver.
Well, now, Torvalds and his army of techie termites are labouring away on kernel 6.8, and lead developer Kent Overstreet just submitted a [4]revised bcachefs which makes some improvements to the version in 6.7. The patch notes in the link describe what this version changes, focussing on how it handles disk space accounting. It notes that the old way was simple and fast, but "limited," noting "lack of scalability", and that it was "a pain to extend." There's some more information about this in the [5]official project roadmap .
[6]
In the new version of bcachefs, free space counts are moved into BTREE_ID_accounting keys in the [7]B-tree data structure that gave the "B" in the project's name. (If you don't follow the Wikipedia explanation, we don't blame you at all: it's a complex subject, as this [8]request for a simpler explanation demonstrates.)
[9]
[10]
This improvement does come with a significant cost, though:
this is in no way compatible with the old disk accounting on-disk format, and it's not feasible to write out accounting in the old format.
[11]One of the changes modifies bcachefs's [12]own fsck tool , so that it will convert the old disk-space accounting structures to the new ones when the volume is mounted. This should be a one-time operation, and happen the first time that a volume with the old format is mounted by a newer version of the filesystem. Such are the costs of using a very new disk storage subsystem. We suspect that it will be worth it in the long run.
Due to the extent of the changes, it's possible that they won't make it into the forthcoming 6.8 release, and might have to wait for kernel 6.9.
OpenZFS gets updated too
There is also a new release of OpenZFS, the Solaris-derived next-generation filesystem used in FreeBSD and some Linux distros. Since we [13]covered OpenZFS 2.2 back in August , a serious – if very unlikely – [14]data-destroying bug was uncovered in late November, and fixed in [15]two point-releases the following month.
[16]Could immutability be a Leap too far for openSUSE users?
[17]Biggest Linux kernel release ever welcomes bcachefs file system, jettisons Itanium
[18]Linux will soon offer switchable x86-32 binary support
[19]FreeBSD 14's RC2 dances to the tune of OpenZFS 2.2
The OpenZFS project now arguably needs to win some confidence back, and the newly-released [20]version 2.2.3 doesn't have any large, scary changes, just lots of bug fixes, which look nicely innocuous. About a dozen of these refer to the [21]ZFS Test Suite , known as ZTS, which presumably reflects the amount of testing that's been done since the problems at the end of 2023.
This release is compatible with Linux kernels from 3.10 (released in mid-2013) all the way up to the current 6.7, and also has some preliminary compatibility with the still-work-in-progress kernel 6.8. Given the timing, we suspect this version will be included in Ubuntu 24.04. OpenZFS 2.2.3 also works with FreeBSD back to 12.2, released in October 2020, up to the current FreeBSD 14.
Bootnote: Is counting free space so hard?
It may not sound difficult, but keeping track of free space is not a trivial task on a modern storage subsystem, which juggles multiple drives of different sizes and speeds, takes and manages snapshots of the contents, and selectively compresses some of that storage.
This is a [22]key weakness of principal rival Btrfs. That's especially important because if a Btrfs volume fills up with snapshots, which is easily done, writing to it is almost certain to corrupt it, and the repair tools are risky, as the warnings in the [23]official docs and [24]Btrfs advocate SUSE's docs show. The Reg FOSS desk has lost his entire OS to corrupted Btrfs partitions several times, and remains extremely wary of it as a result. As such, we feel confident that the [25]bcachefs slogan "The COW filesystem for Linux that won't eat your data" is a poke at Btrfs. ®
Get our [26]Tech Resources
[1] https://www.theregister.com/2024/01/10/linux_kernel_67/
[2] https://www.theregister.com/2023/07/10/linux_6_5_rc1_bcachefs/
[3] https://www.theregister.com/2022/03/18/bcachefs/
[4] https://lore.kernel.org/lkml/20240225023826.2413565-1-kent.overstreet@linux.dev/
[5] https://bcachefs.org/Roadmap/#Disk_space_accounting.3A
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Zd9muy7vvyePyvsHIMWiiwAAAdA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[7] https://en.wikipedia.org/wiki/B-tree
[8] https://www.reddit.com/r/explainlikeimfive/comments/wfz52/eli5_btree/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zd9muy7vvyePyvsHIMWiiwAAAdA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Zd9muy7vvyePyvsHIMWiiwAAAdA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://lore.kernel.org/linux-bcachefs/20240225023826.2413565-15-kent.overstreet@linux.dev/
[12] https://bcachefs.org/Fsck/
[13] https://www.theregister.com/2023/08/16/openzfs_zfsbootmenu_2_2/
[14] https://www.theregister.com/2023/11/27/openzfs_2_2_0_data_corruption/
[15] https://www.theregister.com/2023/12/04/two_new_versions_of_openzfs/
[16] https://www.theregister.com/2024/01/17/opensuse_confirms_leap_16/
[17] https://www.theregister.com/2024/01/10/linux_kernel_67/
[18] https://www.theregister.com/2023/10/30/linux_kernel_6_7_rundown/
[19] https://www.theregister.com/2023/10/24/freebsd_14_rc2/
[20] https://github.com/openzfs/zfs/releases/tag/zfs-2.2.3
[21] https://github.com/openzfs/zfs/releases/tag/zfs-2.2.3
[22] https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/FAQ.html#Help.21_Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21
[23] https://btrfs.readthedocs.io/en/latest/btrfs-check.html
[24] https://www.suse.com/support/kb/doc/?id=000018769
[25] https://bcachefs.org/
[26] https://whitepapers.theregister.com/
Kernel 6.7 [1]only appeared last month and as we wrote at the time, it finally included the next-gen bcachefs filesystem. We say finally because it [2]didn't make it into 2023's kernel 6.5 , just as it [3]didn't make it in 2022 when we looked at its features and what it promised to deliver.
Well, now, Torvalds and his army of techie termites are labouring away on kernel 6.8, and lead developer Kent Overstreet just submitted a [4]revised bcachefs which makes some improvements to the version in 6.7. The patch notes in the link describe what this version changes, focussing on how it handles disk space accounting. It notes that the old way was simple and fast, but "limited," noting "lack of scalability", and that it was "a pain to extend." There's some more information about this in the [5]official project roadmap .
[6]
In the new version of bcachefs, free space counts are moved into BTREE_ID_accounting keys in the [7]B-tree data structure that gave the "B" in the project's name. (If you don't follow the Wikipedia explanation, we don't blame you at all: it's a complex subject, as this [8]request for a simpler explanation demonstrates.)
[9]
[10]
This improvement does come with a significant cost, though:
this is in no way compatible with the old disk accounting on-disk format, and it's not feasible to write out accounting in the old format.
[11]One of the changes modifies bcachefs's [12]own fsck tool , so that it will convert the old disk-space accounting structures to the new ones when the volume is mounted. This should be a one-time operation, and happen the first time that a volume with the old format is mounted by a newer version of the filesystem. Such are the costs of using a very new disk storage subsystem. We suspect that it will be worth it in the long run.
Due to the extent of the changes, it's possible that they won't make it into the forthcoming 6.8 release, and might have to wait for kernel 6.9.
OpenZFS gets updated too
There is also a new release of OpenZFS, the Solaris-derived next-generation filesystem used in FreeBSD and some Linux distros. Since we [13]covered OpenZFS 2.2 back in August , a serious – if very unlikely – [14]data-destroying bug was uncovered in late November, and fixed in [15]two point-releases the following month.
[16]Could immutability be a Leap too far for openSUSE users?
[17]Biggest Linux kernel release ever welcomes bcachefs file system, jettisons Itanium
[18]Linux will soon offer switchable x86-32 binary support
[19]FreeBSD 14's RC2 dances to the tune of OpenZFS 2.2
The OpenZFS project now arguably needs to win some confidence back, and the newly-released [20]version 2.2.3 doesn't have any large, scary changes, just lots of bug fixes, which look nicely innocuous. About a dozen of these refer to the [21]ZFS Test Suite , known as ZTS, which presumably reflects the amount of testing that's been done since the problems at the end of 2023.
This release is compatible with Linux kernels from 3.10 (released in mid-2013) all the way up to the current 6.7, and also has some preliminary compatibility with the still-work-in-progress kernel 6.8. Given the timing, we suspect this version will be included in Ubuntu 24.04. OpenZFS 2.2.3 also works with FreeBSD back to 12.2, released in October 2020, up to the current FreeBSD 14.
Bootnote: Is counting free space so hard?
It may not sound difficult, but keeping track of free space is not a trivial task on a modern storage subsystem, which juggles multiple drives of different sizes and speeds, takes and manages snapshots of the contents, and selectively compresses some of that storage.
This is a [22]key weakness of principal rival Btrfs. That's especially important because if a Btrfs volume fills up with snapshots, which is easily done, writing to it is almost certain to corrupt it, and the repair tools are risky, as the warnings in the [23]official docs and [24]Btrfs advocate SUSE's docs show. The Reg FOSS desk has lost his entire OS to corrupted Btrfs partitions several times, and remains extremely wary of it as a result. As such, we feel confident that the [25]bcachefs slogan "The COW filesystem for Linux that won't eat your data" is a poke at Btrfs. ®
Get our [26]Tech Resources
[1] https://www.theregister.com/2024/01/10/linux_kernel_67/
[2] https://www.theregister.com/2023/07/10/linux_6_5_rc1_bcachefs/
[3] https://www.theregister.com/2022/03/18/bcachefs/
[4] https://lore.kernel.org/lkml/20240225023826.2413565-1-kent.overstreet@linux.dev/
[5] https://bcachefs.org/Roadmap/#Disk_space_accounting.3A
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Zd9muy7vvyePyvsHIMWiiwAAAdA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[7] https://en.wikipedia.org/wiki/B-tree
[8] https://www.reddit.com/r/explainlikeimfive/comments/wfz52/eli5_btree/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zd9muy7vvyePyvsHIMWiiwAAAdA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Zd9muy7vvyePyvsHIMWiiwAAAdA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://lore.kernel.org/linux-bcachefs/20240225023826.2413565-15-kent.overstreet@linux.dev/
[12] https://bcachefs.org/Fsck/
[13] https://www.theregister.com/2023/08/16/openzfs_zfsbootmenu_2_2/
[14] https://www.theregister.com/2023/11/27/openzfs_2_2_0_data_corruption/
[15] https://www.theregister.com/2023/12/04/two_new_versions_of_openzfs/
[16] https://www.theregister.com/2024/01/17/opensuse_confirms_leap_16/
[17] https://www.theregister.com/2024/01/10/linux_kernel_67/
[18] https://www.theregister.com/2023/10/30/linux_kernel_6_7_rundown/
[19] https://www.theregister.com/2023/10/24/freebsd_14_rc2/
[20] https://github.com/openzfs/zfs/releases/tag/zfs-2.2.3
[21] https://github.com/openzfs/zfs/releases/tag/zfs-2.2.3
[22] https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/FAQ.html#Help.21_Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21
[23] https://btrfs.readthedocs.io/en/latest/btrfs-check.html
[24] https://www.suse.com/support/kb/doc/?id=000018769
[25] https://bcachefs.org/
[26] https://whitepapers.theregister.com/
Re: I don't trust btrfs either...
Spazturtle
Yeah I have lost systems to btrfs before as well, every time they promise it is now stable some new data eating bug pops up.
I don't trust btrfs either...
I've had multiple VMs get totally corrupted when trying to take a snapshot before the upgrade from Suse SLES 12.5 to 15.x, which they say you can roll back to if there's a problem. Not too likely. I had several systems work... but several also went completely sideways and couldn't be recovered. So I'm *really* leary of btrfs with my data. Right now I stick with ext4 and xfs, but hoping bcachefs becomes stable and reliable and performant all at once. snapshots and sub-volumes are going to be really useful down the line.