NixOS and the changing face of Linux operating systems
- Reference: 1638530710
- News link: https://www.theregister.co.uk/2021/12/03/nixos_linux_os_design/
- Source link:
For DevOps folk, this was tantamount to clickbait: Nix and Docker are different tools for different jobs, and anyway, it's possible to use Nix to build Docker images.
The distro, which hit [2]version 21.11 on the last day of November, was built around the purely functional Nix package manager.
[3]
Nix is one of several responses to the [4]many complexities of Linux software packaging . Given the issues around dependencies, updates, and the many problems they can cause – which in some contexts is becoming a [5]joke – some of the more alternative-minded Linux developers thought that the best way to tackle this would be to re-engineer how OSes are built and installed.
[6]
[7]
There are many such efforts, and functional package managers lie at one extreme. Nix is the most mature such tool, dating back to 2003; the conceptually similar [8]GNU Guix is a decade younger. Nix uses its own definition language, while the GNU Project uses the [9]Guile dialect of Scheme.
Both profoundly modify the traditional Unix directory hierarchy. Instead, they take a formal specification of a desired system state, written in a declarative, functional language, and then solve this automatically. The approach has a number of benefits: reliable, reproducible system builds; guaranteed consistency; atomic upgrades with rollback; the ability to have multiple versions of the same programs installed concurrently without clashing; and more.
[10]
But the side effect of this is too much for some techies to swallow: the resulting filesystem layout is no longer easily human-readable. Programs live in folders with names based on cryptographic hashes. There are other costs, too, such as slow build times, although there are binary-caching tools to help.
It's not necessary to use either distro to get the benefits of the packaging system – both can run on other distros and indeed operating systems, including macOS.
The other extreme of this scale is [11]GoboLinux . The same age as Nix, this experimental distro also completely discards the traditional Unix filesystem layout, but this time it's replaced with a more human-readable directory tree with plain-English names.
[12]The rocky road to better Linux software installation: Containers, containers, containers
[13]Utopia? Echoes of Delphi and Dreamweaver in new visual editor for React
[14]Crims target telcos' Linux and Solaris boxes, which don't get enough infosec love
[15]Undebug my heart: Using Cisco's IOS to take down capitalism – accidentally
[16]Wine pops cork on version 6.0 of the Windows compatibility layer for *nix systems
The traditional Unix directory layout combines unrelated files according to their type, or the system roles for which they're required, into a semi- [17]standardised [PDF] tree. The result is large, complex, and somewhat cryptic – although [18]there are efforts to simplify it – but millions of Unix users are familiar with it.
Gobo turns this layout on its head. Each app, along with all of its components and dependencies, is kept in its own separate directory tree. Because directory names are versioned, different versions of the same app can co-exist. Installation is as simple as copying a directory, and de-installation just means deleting one. It resembles the macOS way of doing things, but it's more radical, restructuring the entire OS far below the GUI level.
[19]
Sadly, GoboLinux has never really caught on, and functional package managers seem to be just a bit too radical for most distro maintainers – just as functional programming languages are for many programmers. Although Nix or Guix definitely offer advantages, to use them, overworked sysadmins have to learn a lot of complex new tools and methods. In contrast, tools like Docker are relatively simple to understand and to use.
So, instead of re-architecting the way distros are built, vendors are reimplementing similar functionality using simpler tools inherited from the server world: containers, squashfs filesystems inside single files, and, for distros that have them, copy-on-write filesystems to provide rollback functionality.
All the enterprise Linux vendors are working away on this. The goal is to build operating systems as robust as mobile OSes: periodically, the vendor ships a thoroughly tested and integrated image which end users can't change and don't need to. In normal use, the root filesystem is mounted read-only, and there's no package manager.
Red Hat was working on an immutable OS with transactional updates under the banner of "Project Atomic", but then [20]it acquired CoreOS . This resulted in considerable overlap, and the Atomic Host edition of RHEL is now "sunset" without a clear replacement. In the interim, the Fedora project offers [21]CoreOS for servers hosting containers, an [22]Internet of Things variant , and desktop editions with either [23]GNOME or [24]KDE .
Because Red Hat [25]doesn't currently use a filesystem with snapshots , it had to invent a transactional-install system (think "Git for binaries") called [26]OStree and layer it on top of a conventional filesystem.
The openSUSE Project has [27]MicroOS , and SUSE recently launched its corporate sibling [28]SLE Micro . Both focus on servers running container workloads, but you can [29]install a desktop on MicroOS if you want.
SUSE uses Btrfs and leans heavily on its snapshot and copy-on-write functionality, meaning that it can offer transactional updates and rollback, a root filesystem that in normal operation is mounted read-only, but also end user package installation and updates, without needing OStree or anything like it.
Since [30]Canonical cancelled its touch UI and phone/tablet OS , the company's only immutable OS is [31]Ubuntu Core , its IoT distro.
It doesn't get much airplay, but there is already a mature immutable-image-based desktop Linux, based around OStree and Flatpaks: [32]EndlessOS . Since there's no package manager, it's somewhat moot, but Endless is in fact based around a Debian core. It has a read-only root filesystem, a version of GNOME Shell slightly customised to be a little more Windows-like, and all applications are managed as Flatpaks.
The design of Linux distros is changing, and as with packaging systems, the lightweight efficient approaches look set to lose out against simpler but more pragmatic designs… but the evolutionary pressure from tiny smart devices with very limited storage and bandwidth might yet change that. ®
Get our [33]Tech Resources
[1] https://blog.replit.com/nix-vs-docker
[2] https://nixos.org/blog/announcements.html#21.11
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YaqhpevOVYhw3OlrfsTDeQAAAUk&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2021/11/26/linux_software_installation/
[5] https://xkcd.com/1987/
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YaqhpevOVYhw3OlrfsTDeQAAAUk&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YaqhpevOVYhw3OlrfsTDeQAAAUk&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://guix.gnu.org/
[9] https://www.gnu.org/software/guile/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YaqhpevOVYhw3OlrfsTDeQAAAUk&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://www.gobolinux.org/
[12] https://www.theregister.com/2021/11/26/linux_software_installation/
[13] https://www.theregister.com/2021/06/17/utopia_react_tool_alpha_release/
[14] https://www.theregister.com/2021/10/20/linux_solaris_under_attack_at_telcos/
[15] https://www.theregister.com/2021/08/02/who_me/
[16] https://www.theregister.com/2021/01/15/wine_6/
[17] https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.pdf
[18] https://www.theregister.com/2016/11/24/debian_testing_merged_codeusrcode
[19] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YaqhpevOVYhw3OlrfsTDeQAAAUk&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[20] https://www.theregister.com/2018/01/31/red_hat_coreos_acquisition/
[21] https://getfedora.org/coreos?stream=stable
[22] https://getfedora.org/iot/
[23] https://silverblue.fedoraproject.org/
[24] https://kinoite.fedoraproject.org/
[25] https://www.theregister.com/2017/08/16/red_hat_banishes_btrfs_from_rhel/
[26] https://ostreedev.github.io/ostree/
[27] https://microos.opensuse.org/
[28] https://www.suse.com/products/micro/
[29] https://opensuse.github.io/openSUSE-docs-revamped-temp/microos_getting_started/
[30] https://www.theregister.com/2017/04/05/ubuntu_euthanizes_unity/
[31] https://www.theregister.com/2019/01/22/ubuntu_core_18/Snap
[32] https://endlessos.com/
[33] https://whitepapers.theregister.com/
Re: All to be superseded
. . . and Kernel
. . . and text editor!
Re: All to be superseded
An in kernel text editer genius idea i'll get right on it, right after i've released emacs as an OS
Re: All to be superseded
Little known fact, systemd and emacs are actually the same piece of software just started from a different slice of functionality.
Re: All to be superseded
Nah. EMACS is both functional and a learning tool. The systemd-cancer, not so much.
Re: All to be superseded
And floor-wax! And dessert topping!
It all started going downhill when people started putting stuff that wasn't users' home directories into /usr
ITYM ...
It all started going downhill when users were allowed to access systems.
Re: ITYM ...
It all started going downhill when the idea of a consumer Os being read only died with the Commodore 64.
It all started going downhill when people changed /u back to /usr ... Us lazy bastards at Berkeley had changed AT&T's /usr to /u in the name of brevity, but apparently it made things hard to understand for newbies.
/u (or /usr) always had non-home directories & other stuff in it ... Source code, documentation, the man pages, user installed binaries available to everybody, and other useful tat like vi, EMACS, UUCP and the BSD games pack.
It was both useful and logical to split it into /usr and /user when the system grew large enough, and had many users. Then some bright spark decided that /usr and /user was too complicated due to their similarity, and thus /user became /home ... except in the appleverse, where they "simplified" it to /Users (Caps in a system directory name? WTF‽‽‽).
There is a special place in hell for people who put capitals and whitespace in directory names.
Interesting fact many filesystems support / in a file name although very little software does.
Define rules and Bork early is my approach.
> All the enterprise Linux vendors are working away on this... a thoroughly tested and integrated image which end users can't change and don't need to.
That the end user CAN'T change. How do they square that with the principles of free software?
At the very least they are removing users' freedom to change and improve the software and to distribute copies of modified versions to others. Half the four essential freedoms of free software!
How do they license this resulting code?
The catch is that Linux lives in different worlds, each with its own (often legal) requirements.
Changing the software and stuff like is part of the hobbyist and testing world. These often live in small, mostly-controlled environments where the consequences of Things Breaking are minimized if not controlled.
But in an enterprise setting, stability is paramount. Downtime costs money, and "Things Breaking" can result in bills if not lawsuits, so some of those freedoms have to go in the name of legal compliance (which trumps the Four Freedoms).
As a programmer of 40 years, commercially for 25, you don't need to explain the corporate IT world to me. My current employer has contracts with everyone from Santander to NASA.
Having a stable platform does not require removing user freedoms. In this case the user being the enterprise still deserves the same freedoms as a lone user at home.
one freedom that open source software brings to the enterprise world includes the ability to FIX THINGS YOURSELF.
So any package wrapper that includes all versions of all libraries that are natively compiled and guaranteed to work together needs a SOURCE PACKAGE to build them all together, as well as run them all together. I am not familiar enough with these systems to know whether or not they already do this, but it would be akin to using the 'ports' system on FreeBSD to do 'build from source' on EVERYTHING.
Now if an executable ships as binary with a container, the GPL'd lib sources and binaries would need to be there too, so you can fix things. Making a "source package" for the entire container should actually make GPL compliance easier (and more portable).
The right to fix things oneself doesn't necessarily accord the ability along with it. If you believe that, consider the adage of "If you want something done right..." against cryptography. Simply put, it's not for everyone, especially if you have to deal with Stupid every day (the kind of Stupid that makes the "Who, Me?" section here at El Reg).
The point is ...
... it is up to me to decide if I have the ability or not, not the likes of Apple or Canonical. Indeed, some would say that that is the entire point of FOSS.
Re: The right to fix things oneself doesn't necessarily accord the ability
Wrong diddly wrong wrong.
The right to fix things, made fixers out of all of us.
@AC - Well, there's always Windows for that
or AIX, or Solaris, or HP-Unix. You have stability, compliance and whatever you might need for an enterprise.
From what I can see they are wanting to achieve is an ISO image which doesn't allow the end user to make changes. Much like how Android phones don't by default give the end user root access to the filesystem. But yet the AOSP source code is still available under an Apache and GPLv2 license so you can modify the code and compile your own if you wanted.
Er, I think you might be mixing apples and oranges here.
The four freedoms apply to source code, to the ability to develop and distribute your own software based on others work.
What they're trying to prevent is people like me having a decaffeinated moment and typing rm -rf * at the root directory and then screaming in panic when they realise what they've just done.
So I can easily imagine a "DontDeleteStuffYouShouldntByAccident" package being GPL... at least until Skynet becomes self-aware, starts listening to The Cure and then decides to delete itself, that is.
rm -rf /*
> [ ... ] typing rm -rf * at the root directory and then screaming in panic [ ... ]
There's a perfectly valid prevention for that particular accident: rm -rf or rm -f does nothing when getcwd(3C) is "/" or when the removal target is immediately below "/".
The command that can delete in "/" is called rootremove or slashremove or some other such unintuitive thing that can't be typed accidentally.
Solaris has been doing this for many years. I don't quite get why Linux didn't adopt the idea. Most Linux distros rely on aliasing rm , and that is very brittle.
Re: rm -rf /*
No matter how unintuitive you try to make a History Eraser button, someone will still find a way to do it, due to either being duped into doing it by a jerk, or by a Captain Peachfuzz who can stumble onto it in spite of hell or high water.
@Charles 9 - Re: rm -rf /*
Shaw's Principle: Build a system that even a fool can use, and only a fool will want to use it.
Re: @Charles 9 - rm -rf /*
But everybody plays the fool sometimes.
Re: @Charles 9 - rm -rf /*
"But everybody plays the fool sometimes."
Yes. It's part of the learning curve. Learning can be painful sometimes, it's part of what makes us human.
Re: @Charles 9 - rm -rf /*
It's the people who aren't just playing you need to worry about.
Re: @Charles 9 - rm -rf /*
Everybody's playing, at some level, for various values of playing.
@Charles 9 - Re: rm -rf /*
Yeah but preventing this would go against that natural selection for sysadmins.
Re: @Charles 9 - rm -rf /*
Look, if you don't find a way to fix Stupid, you're gonna end up with Stupid taking the rest of us down with them.
Re: @Charles 9 - rm -rf /*
Nah. I admin my own computers. The only person who can take me down is me[0]. And if I do, so be it ... at least I'll have learned something. Having proper backups and automatic fallover hardware redundancy in key systems helps with piece of mind.
[0] To be perfectly honest, the Wife has access to the safety deposit box with the BigBookO'Passwords[tm], just in case I step in front of a bus. She's *nix literate (finally!) and knows how to access everything, but she's never seen reason to use root, even on her own computers (Slackware and BSD), so she doesn't.
Re: @Charles 9 - rm -rf /*
But that's you, and you're in the minority. I routinely deal with people who when told to look left, look right, and if told to turn around, spin 360 degrees. That's why I'm constantly reminded of Stimpy and the History Eraser button.
Re: @Charles 9 - rm -rf /*
So you're an advocate of stupiding the world down to the lowest common denominator?
When I was younger I thought I could change the world. Now that I've been teaching on and off for about 45 years, I've come to the realization that probably nine out of ten humans are ineducable beyond "eat here, sleep there, bathe occasionally & don't poop in the living room".
I can live with that. Just don't ask me to join them in their mire in the name of you separating them from their money. That's between you and them, kindly keep me out of it.
... and it irritates the pig.
1. http://www.islinuxaboutchoice.com/
2. It's all 100% open source software. You can do whatever you want with it. This is about solving problems and improving operating system design, not restricting freedom.
3. As a user you can still change it. Commands and documentation are provided. The point is you don't want to.
Creating a strong separation between "OS" and "Applications" is the end goal here.
With traditional Linux distribution tools and approaches if you want to downgrade your version of Libreoffice or get compatibility with a different version of Blender3d the only way to end up with a supported configuration is to re-install your operating system.
That is a complete shit design. Nix helps solve this. OSTree and Flatpak helps solve this, so does OCI containers. There are a variety of different approaches with their different merits, but apt-get and rpm and dnf definitely do not solve this.
"With traditional Linux distribution tools and approaches if you want to downgrade your version of Libreoffice or get compatibility with a different version of Blender3d the only way to end up with a supported configuration is to re-install your operating system."
Horseshit. Total, utter and complete horseshit.
After much wracking of brain cells
Reminds me of SmartOS where the OS is in memory and sort of immutable
Back to the past
GoboLinux looks like the setup we had at work in the early 1990s.
Each workstation had mounted a NFS share containing all the company and public domain (now open source) executables and libs. All the shares were kept in sync. So releases were built and copied to one place by a central source code team and processes rdisted them around.
So easy release process and all users had the same versions or the end apps but each app could use different versions of libraries - so a new library could be released without forcing everything to upgrade.
The build logs would also show the exact version of the libraries used so a repeatable build if needed.
Re: Back to the past
Common to a number of London offices of investment banks and, as I understand it, all the fine work of the same person.
In the future when everything is machines talking to machines...
An OS made by machines, for machines. Sounds like a great thing ... for machines.
I, personally, lost interest at "no longer easily human-readable."
Re: In the future when everything is machines talking to machines...
Yes, "... in folders with names based on cryptographic hashes" means they are reproducing Windows registry type problems and all the other problems created when people say "let's use binary!" instead of text.
In case you haven't heard 'accessibility' is high priority. Making the sighted feel blind is not helping that.
All to be superseded
I’m just waiting for the Systemd package manager and file-system re-organizer. Only a matter of time.