The rocky road to better Linux software installation: Containers, containers, containers
- Reference: 1637947506
- News link: https://www.theregister.co.uk/2021/11/26/linux_software_installation/
- Source link:
[1]Fraser wrote in a blog published on 23 November that "these are not the future of desktop Linux apps," citing a litany of technical, security and usability problems. His assertions about disk usage and sharing of runtimes between apps were hotly disputed by Will Thompson, director of OS at Endless OS Foundation [2]a day later in a post titled: "On Flatpak disk usage and deduplication," but there is no denying it is horribly inefficient.
Most people don't care about that any more, one could argue. But they should.
[3]
The Linux world has been trying to invent a cross-platform packaging format for years, but leading contenders – the [4]older, vendor-neutral older, AppImage format as well as [5]Ubuntu's Snap and Fedora's Flatpak – all have serious issues.
[6]
[7]
They [8]may be revolutionary and mean Linux becomes easier to develop for, but it's enough of a mess that some mainstream distros [9]avoid the whole thing .
In comparison, installing software on Windows is easy. Download an installer, run it, and you have a new app. The snags are that it means trusting unknown binaries from the internet – and that it teaches Windows users that this is fine and a perfectly normal thing to do.
[10]
Plain Win32 programs have unfettered access to your computer. This is why Microsoft invented the Windows Store: it would contain only safe, vetted, approved, [11]"modern" apps written in " [12]managed code ". (And of course Microsoft [13]got to keep a cut of the revenues .)
The plan hit [14]a few problems , and in the end Win32 apps [15]were allowed in too .
How about them Apples
It's not unfair to say that everyone is trying to catch up with Apple. Not the App Store – that's indispensable [16](and extremely lucrative) on iOS, but you can pretty much ignore it on a Mac, if you wish. No, the target is macOS's .app application bundles, which macOS inherited from its 1989 ancestor NeXTstep – although they're usually delivered in a classic MacOS-style .dmg disk image file.
Applications on macOS are a specially structured folder, containing all of the program's supporting "resources" and compiled binaries for as many CPU architectures as the app creator supports. It works pretty well, but not without snags. For instance, there's no global way to update all your apps (unless you got them from the App Store). Apps tend to be big – but that's fine, because if you can afford Macs, you can afford a big disk and fast broadband, right?
Ironically, Linux could easily have had much the same because all the functionality already exists in [17]GNUstep , the venerable FOSS rewrite of NeXTstep's core libraries. Unfortunately, no mainstream Linux uses the GNUstep desktop, and the [18]Étoilé project to modernise it and make it a bit more Mac-like is moribund. The [19]superficially Mac-like Elementary OS would have been richer and more capable if its developers had started from Étoilé or GNUstep, but they didn't – it's just a facelift for GNOME 3, like [20]Mint's Cinnamon and [21]Zorin OS .
[22]
A lot of the developers behind [23]Flatpak are from the GNOME and Fedora communities, or their corporate backer Red Hat, but it's a desktop-independent effort and comes installed by default on some Debian and Ubuntu derivatives. It uses Red Hat technologies such as OStree to manage binaries in a similar way to Git.
Somewhere deep within your OS is a complex directory tree full of your Flatpak applications and all their dependencies, as opposed to somewhere in full view where you can interact with it, as on macOS or GNUstep.
Rather than a macOS-style directory full of files, Ubuntu's [24]Snap format compresses an application and all its dependencies into a single, compressed [25]SquashFS file, which is loop-mounted as the system boots.
Flatpak and Snap have a fair bit in common. Both keep your apps inside /var/lib (although Snap makes them visible at /snaps , and Flatpak will let you install into your home directory if you prefer). Both require you to install a supporting framework. Both do some degree of sandboxing of apps, but aren't as secure as their publicity might lead you to believe. Both do things like run silent automatic scheduled updates in the background, in a very Windowsy fashion – bad news if you're on a metered connection. It also means that if you update your OS with a shell command, then these apps won't be included.
The [26]AppImage format has similar pros and cons to macOS' app bundles – such as lacking a global update mechanism – because it grew out of tech of a similar age. AppImage's developer took the [27]ROX Desktop 's [28]AppDirs and put each one inside a SquashFS (which probably inspired the Snap developers). ROX and AppDir are FOSS recreations of [29]Acorn's RISC OS , which appeared slightly before NeXTstep in 1987. It needs no supporting frameworks and you can keep your AppImages anywhere you like.
All three share a weakness in that they include almost all of an application's supporting libraries and other dependencies in its package, so packages tend to be very large – in the order of hundreds of megabytes – and so do updates. Installing large Linux apps from the command line generally takes in the region of seconds to tens of seconds, but installing a Snap or Flatpak, even on a fast connection, can take many minutes – and of course ignores any local mirrors you may have configured for your distro's built-in package manager.
[30]You GNOME it: Windows and Apple devs get a compelling reason to turn to Linux
[31]helloSystem: Pre-alpha FreeBSD project chases simplicity and elegance by taking cues from macOS
[32]Snapping at Canonical's Snap: Linux Mint team says no to Ubuntu store 'backdoor'
[33]Reviving a classic: ThinkPad modder rattles tin to fund new motherboard for 2008's T60 and T61 series of laptops
Endless's Thompson makes a good point, though. Flatpak's format allows something single-file formats can't do: if files in different Flatpaks are identical, OStree can reduce duplication by hard-linking them together… although in principle, a smart enough filesystem could do the same at the block level.
What's next?
There are alternative cross-distro systems. Several avoid installing apps into the OS at all, and just fetch them off the internet into your home directory when needed. Examples include [34]0install , from the developer of the ROX Desktop. That inspired [35]AppFS – which does something similar to CERN's unrelated [36]CernVM-FS . And then there are functional package managers, which are a whole other type of software-management tool which we'll come back to in another article. Suffice it to say there are multiple vendor-neutral ways of distributing Linux software that predate the big three. All of them are lighter-weight, more efficient, the packages are generally much smaller, and all are seriously obscure and you'll never encounter them in a mainstream distro.
Naturally, because of the rampant Not Invented Here Syndrome of the Linux industry, all of these systems totally ignore one another. Which does at least have the benefit that you can install most of them side-by-side on the same OS and try them out, with no real penalties except using a lot of disk space. But at least that's cheap these days.
There are so many alternatives vying for space that it's hard to pick winners. This is partly due to rivals building their own tools rather than cooperating, and partly because there's almost no money to be made from desktop Linux, only servers – so there's little investment, and engineers' occasionally sketchy prototypes end up getting shipped.
What does seem likely is that the lean, efficient but daringly unconventional tech won't go anywhere, while the inefficient and space-hungry variants will be pushed heavily and widely used.
This being so, AppImage probably won't get much bigger, because it doesn't have a big company behind it. Ubuntu's Snap system has some advantages over GNOME's Flatpak, such as being useful on servers and so on… but Ubuntu runs the only Snap Store, and the only open-source back-end is obsolete and has been [37]deleted .
By contrast, Flatpak, by its own [38]admission , is a desktop tool – but anyone can host their own [39]repos , and several already [40]exist .
With Red Hat's considerable clout behind it, Flatpak's chances look good. But whoever wins, in time we'll probably have to get used to distros occupying terabytes of disk and need hundreds of gigs of regular updates… and the memories of the small, efficient systems that went before will be lost to history. Isn't progress great? ®
Get our [41]Tech Resources
[1] https://ludocode.com/blog/flatpak-is-not-the-future
[2] https://blogs.gnome.org/wjjt/2021/11/24/on-flatpak-disk-usage-and-deduplication/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YaFnHalFr-2jLW3EFp83KgAAAAc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2021/02/12/freebsd_that_looks_like_macos/
[5] https://www.theregister.com/2017/04/28/snap_flatpacks_the_future_of_desktop_linux/
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YaFnHalFr-2jLW3EFp83KgAAAAc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YaFnHalFr-2jLW3EFp83KgAAAAc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2018/01/08/desktop_linux_open_source_standards_accessible/
[9] https://www.theregister.com/2020/06/02/linux_mint_team_snap/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YaFnHalFr-2jLW3EFp83KgAAAAc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://www.theregister.com/2012/09/13/metro_apps_are_windows_store_apps/
[12] https://www.theregister.com/2004/05/06/microsoft_managed_code_rethink/
[13] https://www.theregister.com/2018/05/07/microsoft_wants_serious_nongaming_devs_to_make_more_money/
[14] https://www.theregister.com/2015/05/28/windows_store_spring_cleaning/
[15] https://www.theregister.com/2020/05/19/project_reunion_microsoft/
[16] https://www.theregister.com/2021/06/04/apple_vs_epic/
[17] http://www.gnustep.org/
[18] http://etoileos.com/
[19] https://www.theregister.com/2021/08/12/elementary_os_6_odin/
[20] https://www.theregister.com/2021/03/02/linux_mint_fix_for_memorygobbling/
[21] https://www.theregister.com/2021/08/10/zorin_os_16_pro/
[22] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YaFnHalFr-2jLW3EFp83KgAAAAc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[23] https://flatpak.org/faq/
[24] https://snapcraft.io/about
[25] https://tldp.org/HOWTO/SquashFS-HOWTO/whatis.html
[26] https://appimage.org/
[27] http://rox.sourceforge.net/desktop/
[28] http://rox.sourceforge.net/desktop/AppDirs.html
[29] https://www.theregister.com/2018/10/23/risc_os_open_source/
[30] https://www.theregister.com/2018/01/08/desktop_linux_open_source_standards_accessible/
[31] https://www.theregister.com/2021/02/12/freebsd_that_looks_like_macos/
[32] https://www.theregister.com/2020/06/02/linux_mint_team_snap/
[33] https://www.theregister.com/2021/11/26/thinkpad_motherboard_mod/
[34] https://0install.net/
[35] https://appfs.rkeene.org/web/home
[36] https://cernvm.cern.ch/fs/
[37] https://github.com/noise/snapstore/
[38] https://flatpak.org/faq/#Can_Flatpak_be_used_on_servers_too_
[39] https://docs.flatpak.org/en/latest/hosting-a-repository.html
[40] https://gist.github.com/intika/8e4a7faeb72c3a393e42ac9af85b62b7
[41] https://whitepapers.theregister.com/
At least there are lots of words in this article.
From the article,
In comparison, installing software on Windows is easy. Download an installer, run it, and you have a new app. The snags are that it means trusting unknown binaries from the internet – and that it teaches Windows users that this is fine and a perfectly normal thing to do.
Er, MS did introduce signed applications, where Windows will show you confirmation of the signature before the installer runs. It'll also make it clear if you're installing something that isn't signed. So there's no excuse for unknowingly installing software from an unknown or spoofed location. Furthermore with Windows networks you can enforce practising of safe hex at the domain level.
All bets are off if you download, compile and run unknown source code, but that's true of anything including Linux.
Impossible Goal with RedHat in Control Of So Much
From other El Reg articles that cast a poor light on the state of Gnome and the ever-changing state of the GTK libraries, it seems to me impossible to have a sensible, sane installation mechanism on Linux distros that use it. If you can't count on the host OS and desktop having stable APIs, there's no choice but for an app to provide its own versions of the API libraries and rely on the lowest common denominator - the Linux syscall API (which Linus is famously fastidious about not breaking) and glibc.
Sure, other distros don't use Gnome / GTK, but then if they do achieve a sane installation mechanism it's only a partial win, because other distros do use Gnome.
My biggest issue with snaps and flatpaks - both of which I keep off my computers - is that they don't update dependencies unless the maintainer tells them to, Which the maintainer rarely does, and who can blame them? Instead of updating just their bit in the knowledge that all the libraries used will be updated as and when bugs show up or improvements are possible, the poor sod(s) have to keep an eye on everything.
Here, for example, is the list of dependencies for Chromium, snappage of which is why from Ubuntu 20.4 is why I am sitting resolutely at 18.04:
Depends: libasound2 (>= 1.0.16), libatk-bridge2.0-0 (>= 2.5.3), libatk1.0-0 (>= 2.2.0), libatspi2.0-0 (>= 2.9.90), libc6 (>= 2.27), libcairo2 (>= 1.6.0), libcups2 (>= 1.7.0), libdbus-1-3 (>= 1.9.14), libdrm2 (>= 2.4.38), libexpat1 (>= 2.0.1), libgbm1 (>= 17.1.0~rc2), libgcc1 (>= 1:3.0), libglib2.0-0 (>= 2.39.4), libnspr4 (>= 2:4.9-2~), libnss3 (>= 2:3.22), libpango-1.0-0 (>= 1.14.0), libwayland-client0 (>= 1.0.2), libx11-6 (>= 2:1.4.99.1), libxcb1 (>= 1.9.2), libxcomposite1 (>= 1:0.3-1), libxdamage1 (>= 1:1.1), libxext6, libxfixes3, libxkbcommon0 (>= 0.5.0), libxrandr2, libxshmfence1, libxtst6, bash (>= 4), libx11-xcb1, xdg-utils, chromium-codecs-ffmpeg-extra (= 95.0.4638.69-0ubuntu0.18.04.1) | chromium-codecs-ffmpeg (= 95.0.4638.69-0ubuntu0.18.04.1)
Maybe a large project like Chromium can keep an eye on all of these and update as required, but smaller ones can't and won't, so my computer would fill up with increasingly out-of-date and vulnerable libraries.
I had to install ntfs3g (FUSE) on a 4 year old Mac OS X install the other day. Nothing worked, the whole process was terrible.
Did it on Linux in a couple of seconds.
The ad-hoc installer approach on macOS is absolutely terrible. In Windows it oddly works quite well (though cleanly uninstalling is impossible). Linux is infinitely ahead of the game here. Actually the game is moving in the incorrect direction entirely.
The only thing that beats traditional Linux package management (in terms of maintenance) is possibly ports collections (like Portage, FreeBSD ports, AUR). This gets the benefits of distributed (from the vendors website) *and* the benefits of platform specific patches.
We have been faffing with computers for over half a century now and we have solved very little. Granted scum like Microsoft and Apple have been holding back the industry for decades.
Personally I disagree with the whole concept of "vendor neutral" packaging for most of what I use on a system. I need the database as close to the metal as possible for performance. I need everything as close to the metal as possible for that reason. The ONLY thing I use snaps for are tools like Netbeans, where performance isn't an issue.
Call me old school (and I am), but I cut my teeth doing things like getting the perversion of Oracle 6 running on a Netware server... I do NOT like having things "just happening" without me knowing about it and having control over whether that default is applied or not. :)