Make Linux safer… or die trying
- Reference: 1676378532
- News link: https://www.theregister.co.uk/2023/02/14/make_linux_safer_p1/
- Source link:
Why is Canonical pushing Snap so hard? Does Red Hat really need all these different versions of Fedora? Why are some distros experimenting with ZFS if its licence is incompatible with the GPL? Is the already bewildering array of packaging tools and file systems not enough?
No, they aren't. There are good justifications for all these efforts, and the reasons are simple and fairly clear. The snag is that the motivations behind some of them are connected with certain companies' histories, attitudes, and ways of doing business. If you don't know their histories, the reasoning that led to major technological decisions is often obscure or even invisible.
[1]
The economics of the computer software industry has changed massively since some now-widespread tools were originally invented. Techniques and methods that made good commercial sense decades ago don't any more, and some of this applies to Linux more than it does to Windows. Modern Windows is based on Windows NT, the first version of which was released in July 1993 and was a modern, hi-tech OS from the start. Its developers had already learned lessons from its forerunners: less DOS and 16-bit Windows, more as OS/2 1.x and Digital Equipment Corporation's VAX/VMS.
[2]
[3]
Linux is quite a different beast. Although many Unix fans haven't really registered this yet, it's a fact: [4]Linux is a Unix now . In fact, arguably, [5]today Linux is Unix .
As a project in its own right, Linux is roughly the same age as Windows NT. Linux 0.01, the first public version, appeared in late 1991, it went GPL with version 0.99 in late 1992, and version 1.0 was released in March 1994. [6]FreeBSD is about the same age , and so is NetBSD. All of them are fairly traditional, monolithic, Unix-like OSes in design. This means that it inherits many of its design choices from earlier, mostly proprietary Unix OSes.
[7]
The thing is, solid, carefully made decisions that worked for commercial Unix in its heyday may not be such a good fit any more. In the 1970s and 1980s, proprietary Unix boxes cost lots of money. The companies that bought them – and it was a big-business level of expenditure – could afford to pay for highly trained specialist staff to tend and nurture those machines.
Windows NT came out 30 years ago and created a lively commercial market of relatively inexpensive 32-bit PCs, powered by x86 processors, open-standard fast expansion buses and low-priced mass storage. Cheap mass-produced PCs were just about good enough, and so were cheap mass-developed OSes for them.
Since then, Windows has been good enough, and it runs on commodity kit. So the commercial mainstream, always looking for savings, moved to Windows. The result is that Windows tech staff became cheap and plentiful – which implies fungible – while Unix techies remained more expensive.
[8]
This cheap, mass-market hardware in turn has aided the evolution of open source Unixes. Linux has done well partly because its native platform is the same cheap kit that was built to run Windows. This is a huge and vastly diverse market and, as we recently described, [9]software is a gas : it expands to fill the hardware. The result is that, to support the most diverse computer platform ever, Linux is big and complicated.
Yes, it's a Unix-like OS, and Unix has been around for over 50 years. But Linux isn't just another Unix. It's free for everyone, and the same kernel runs on everything from $5 SBCs to $50 million supercomputers. Proprietary Unix was expensive, exclusive, and mostly ran on expensive, high-quality hardware that was designed for it, while Linux mostly runs on relatively cheap devices that were designed to run Windows.
[10]Spotted in the wild: Chimera – a Linux that isn't GNU/Linux
[11]Don't bore us, get to the Horus: Elementary OS 7 is here and looking good
[12]The Balthazar laptop: An all-European RISC-V Free Hardware computer
[13]Open Source Policy Summit: Where FOSS and government meet
When Unix ruled the datacenter, computer resources were limited, and proprietary platforms strictly controlled what was on offer. Now that disk and memory are cheap, the PC hardware is uncontrolled and proliferates [14]as wildly as kudzu . Linux supports most of it, meaning that it's much bigger and more complex than any proprietary Unix ever was… and to a good approximation, nobody fully understands the entire Linux stack: it's just too big. Real experts are scarce, and that means that they command top dollar.
But the mass adoption of Linux has changed the economics somewhat. While the top-tier gurus remain pricey, ordinary mortal techies aren't. Smart curious folks who can work out how to stack some components together like construction toys, and get it more or less working. Then you push it out into someone else's datacenter, add some tools that will arrange for it to scale out – if you're lucky enough to need it, and for it to work… those folks aren't so pricey. Which implies that the building blocks of that stack need to be tough, to match the levels expected over in Windows land, and they need to just plug together.
The flipside of this coin is the famed DevOps model: [15]treat servers as cattle, not as pets . It's not all about servers – but it's server distros that pay. So desktop distros use lots of tools designed for servers, and phone distros are being built from the same components.
When the software and the hardware are cheap, but the skills are expensive, the cost centers become support and maintenance – which is a large part of why the big enterprise Linux vendors sell support, not software. The software is free, and if you don't mind compiling it yourself, you can have the source code for nothing. To get the ready-to-use version, though, you have to buy a support contract.
What that means is that the evolutionary selective pressure is to reduce the cost of providing that support in order to maximize the profitability of the support contracts. That requires making the OSes as robust as possible: to prevent faults from occurring, so you don't have to pay someone to fix them. If possible, to prevent whole categories of system failures. Better still, to make the OS able to recover from certain types of fault automatically, without human intervention.
If you want to deploy a lot of a cheap or free OS, without hiring a lot of expensive gray-bearded gurus, a core part of the economic proposition is to build Linux distros that can cope, even thrive, without constant nurture. For example, making them able to fetch and install their own updates. The goal is to make them able to cope with their own problems, and heal their own injuries, just as farm animals must in their short, miserable lives.
One aspect of this is visible as multiple parallel efforts to contain and manage the vast and ever-growing complexity of modern Linux: to encapsulate it, and if possible, even eliminate parts of it. This shows up in several places. The first was in file system design, but the first set of such changes was relatively minor and caused little disruption. Now another round of modernization is being worked on. There are also major changes in how software is packaged: how packages are built, how they're distributed, and how they're stored, installed, and upgraded. A further aspect is how they are uninstalled again or upgrades reverted.
This is a complex, interlocking set of problems, and not only is there not one single best way to tackle it, but the approach each company takes is guided by the tools which it has or favors. For various reasons, not all vendors are spending their R&D money in the same directions. Some are working on file systems, some on packaging, some on distribution, some on more than one of these at once.
In the second half of this feature, we'll offer an executive briefing on the different efforts, and why different distro vendors are addressing the problems in different ways. ®
Get our [16]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y@u@M-glO2SXLhuhI@eRsAAAAE8&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y@u@M-glO2SXLhuhI@eRsAAAAE8&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y@u@M-glO2SXLhuhI@eRsAAAAE8&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://www.theregister.com/2022/07/13/xorg_servers_updated/
[5] https://www.theregister.com/2023/01/17/unix_is_dead/
[6] https://www.theregister.com/2022/05/20/freebsd_131/
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y@u@M-glO2SXLhuhI@eRsAAAAE8&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y@u@M-glO2SXLhuhI@eRsAAAAE8&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2023/01/11/software_versus_hardware/
[10] https://www.theregister.com/2023/02/13/chimera_non_gnu_linux/
[11] https://www.theregister.com/2023/02/10/elementary_os_7_horus/
[12] https://www.theregister.com/2023/02/09/balthazar_free_hardware_laptop/
[13] https://www.theregister.com/2023/02/09/open_source_policy_summit/
[14] https://www.theregister.com/2008/06/26/magic_biofuel_boom/
[15] https://www.theregister.com/2013/03/18/servers_pets_or_cattle_cern/
[16] https://whitepapers.theregister.com/
Re: Rhythm is a Dancer
I'd settle for a fingersnap!
Re: Rhythm is a Dancer
How about two finger snaps, on Wednesday (Icon).
Re: Rhythm is a Dancer
No can do - it ain't got the Power.
Re: Rhythm is a Dancer
Presumably it would also play a few bars of "I've Got The Power" every time you did a 'sudo' or 'su root'
Re: Rhythm is a Dancer
That article was worrying, NT is years old, which makes me feel ancient.
Technology & Economics
1.) The Linux kernel can be stripped much smaller than the WNT kernel, as the latter has graphics, font rendering and several networking stacks baked-in. Mind you: a single kernel exploit is Game Over.
2.) Windows has automated a limited number of tasks with user-friendly GUIs. As soon as you need advanced things or if you want to automate/mass operations: same effort as Linux command line. Capable Windows Admins are not cheaper than Linux Admins, as they all must be semi-programmers using bash, perl, python or PowerShell.
3.) The men running AWS or Google Cloud must be true experts, their economy comes from the scale of their operations.
4.) If you want to see the future of OSs, look at minimalist microkernels:
https://sel4.systems/
http://sappeur.ddnss.de/L4gegenueberLinux.html
(Maybe it is not fair to compare L4 to Linux at the moment, but in the future it could be like the picture)
https://github.com/AmbiML/sparrow-manifest
Like a warship, seL4 can take hits in base modules, and still be overall secure.
Re: Technology & Economics
4.) If you want to see the future of OSs, look at minimalist microkernels:
I wouldn't bet on it. That's the exact reasoning people gave back in the 90s when saying "Linux will never catch on" and yet here we are. We all should have learned by now that the future belongs to the most successful path not the technically most efficient / reliable / advanced one, and success is just as often determined by what is easy as what is good.
Well
It is good to know how to systemically fix the challenges of Big Kernels.
Re: Technology & Economics
3.) The men running AWS or Google Cloud must be true experts, their economy comes from the scale of their operations.
That's rather misogynist!
4.) If you want to see the future of OSs, look at minimalist microkernels:
HURD has entered the chat,,,
Men
I was told that "man" and "men" has always been used as "Mensch" in German. If that offends COMINTERN tools, even better.
Re: Men
Wouldn't that translate to "the humans... "? Which in the current context may be debatable - or perhaps the Yiddish "mensch" arguably even more so.
@AC - Re: Technology & Economics
Pardon us. From now on, we'll use comrades instead. Like in "the comrades running AWS or Google".
Satisfied ?
The companies that bought them – and it was a big-business level of expenditure – could afford to pay for highly trained specialist staff to tend and nurture those machines.
Or entities, shall we say, could use their existing staff with an aptitude, hence I got to feed a Siemens Nixdorf upright freezer-sized box with Sony tapes, reset luser's passwords and stuff in a Framed-Access Command Environment front end, and gradually learn some more stuff like vi and cpio for restoring luser's oops-I-deleted-this-file-can-you-restore-it-from-backup cockups. NT servers were prettier (but carrying too many DLTs up and down stairs can really knacker your thumbs, kids), but any form of Windows never really floated my boat. And then I saw Knoppix, and saw what a POS XP was, and that was that.
Pedantic note
ZFS isn't really "new tech", it's over 15 years old. Btrfs is only slightly younger, but vastly more exciting as you never know whether your data is really safe or not. /ZFS fanboy
Re: Pedantic note
No data is "safe" unless you have backups.
Untested backups are not backups.
Re: Pedantic note
Well, ZFS fails miserably, just like btrfs, when flames destroy the drives. This will always end in tears.
Most other filesystems are less robust and, as a consequence, the sysadmin(s) will take regular backups and simply restore after the burn.
/me, the sysadmin with some experience restoring all those carbonized drives
Snap is terrible.
Slow to install, slow to start, proprietary backend, and italso break installs due to a fundamental limitation on how it handles home folders.
It is a problem looking for another problem to make even worse.
"making them able to fetch and install their own updates"
I don't quite understand this. You wouldn't let Windows do this so why would you want Linux to? On my home machine I run enterprise for this exact reason. Server I run Debian and chose when and how I update. In both environments I think it's always best to check said updates beforehand. Like windows there can be many moving parts to an OS and while an update may resolve one problem there is always the worry it's going to create one.
I agree with you... on my own boxes. On my technophobic wife's box, the alternatives are:
1. Try to teach her to do as I do. Ha ha, that's not going to work.
2. Tell her to always blindly accept the "updates are available" prompt. Which she rarely notices, so security patches go unapplied.
3. Set up autoupdates for her.
What you describe is the gold standard, but not all computers are administered by folks who agree. Let's autoupdate those boxes, without taking away the ability for you and I to choose.
The problem is desktop components on servers
The recent trend on Linux in my experience is that an OS in theory aimed at a server comes with so many mobile end user system components, some of which are even harder to strip out than ever before that I feel like I'm running a laptop, not a server. I've done the exercise many times of sit down and justify every package installed or remove it on a few Linux distributions, and often end up stripping at least fifteen daemons, some of which are network related, that I couldn't justify ever existing on a server. (Yes, said systems ran in production for years in various functions without needing said packages re-installed.)
More recent trends in Linux only accelerate this tendency to treat the entire OS as a laptop, to the point that I've argued the people making the decisions for some Linux distributions are only using it on their personal laptop and think no one ever uses the OS on a server.
Re: The problem is desktop components on servers
"...I've argued the people making the decisions for some Linux distributions are only using it on their personal laptop"
You don't have to skirt around it, you can just say "Poettering", I don't think it summons him...
Re: The problem is desktop components on servers
That depends on the distro. If you preferred distro isn't giving you an installation option that doesn't leave you with all that laptop nonsense, then it's time to try another distro.
Unix was always diverse
Because it was open (specifications more important than code) it has always been possible to replace components. So people did. Sometimes the replacements improved things, sometimes they did not.
So there was diversity and experimentation. In a Darwinian way the better alternatives usually** won out after several years, so Unix systems gradually evolved to use better components. The same is happening today but without the benefit of hindsight today's diversity just looks like a mess. In a few years time what is considered a mess will be something different.
** "Usually" - large company marketing and techie conservatism sometimes meant staying-with/adopting non best solutions.
Still Is
MacOS, iOS, Android, FreeBSD, OpenBSD.
If Linus turns nuts tomorrow, we will simply switch to them.
Re: Unix was always diverse
"Unix was always diverse"
I fully agree there and what really did it for the commercial Unices was the huge and extortionate licence and royalty fees that came with them. As soon as the free and open source upstart Linux cousin came along, that marked the end of Unix domination and if they were creatures, they'd have been put on the risk of extinction list now.
Chrome OS?
Maybe I have misunderstood something, but Chrome OS (a flavour of Linux) seems to tick the boxes:
* regular unobtrusive updates from Google that just work
* install apps from Google Play, and uninstall them when you don't want them any longer
I understand that if you've got a job that requires power (e.g. full time video editing (occasional video editing is absolutely fine on a Chromebook)) or something else that's special, you'll choose a different device - but in terms of what this article is about, it seems perfect!
Re: Chrome OS?
An excelllent option with just two minor drawbacks for some of the commentards on here (me included):
1) All the unremovable snooping baked into it, and
2) All the unremovable snooping baked into it.
I realise that technically speaking that's just one drawback, but I thought that it was such a big one that it was worth mentioning twice.
Micro-kernel
Linux will either have to be re-written as a microkernel OS or it will die. I'm pretty sure the U.S. Government will mandate microkernels for most of its branches, since their improved security is proven better than anything Linux can offer.
Also, I predict the U.S. DoD and NASA will mandate the use of the Rust programming language for their systems and embedded software in a couple of years.
Re: Micro-kernel
They won't, unless you have an OS and matching applications for it in wide use.
Microkernels might catch on for IoT and similar but the effort of rewriting an OS and porting applications, or even just trying to make the API completely compatible is huge . It is why Windows is still in common use, because XYZ business demands ABC package and that is all that matters. Linux has taken a lot of areas, most cloud and web servers for example, and it is what I use myself for almost everything, but it has not replaced it for many and never will completely while something, somewhere, needs win32 compatibility to some odd or undocumented aspect.
Add a new OS, rinse and repeat after 15 years.
ZFS solves a problem that needs solving - raid 5 write holes. Copy on write, checksumming, very large file sets, all useful good stuff. Btrfs does the same thing and despite being complex as hell. I'll even allow that systemd is a good thing overall, despite it's tendancy to reach out tendrils everywhere, it does solve problems.
What precisely does the abominal problem children called snap and flatpack solve? They bloat stuff up, and take control away from me. I want updates WHEN I choose - not when some faceless gnome decides to push shit out to MY servers and desktops. They offer nothing beyond apt or yum for my convenience.
Rhythm is a Dancer
I'd accept snap if it played a few bars from Rhythm is a Dancer each time a package gets updated or installed.