They worked fine, and I don't feel like having multiple types of containers and update methods and mounted image file systems really made anyone's life better.
What you are really saying is you want all software developers to publish Debian packages (no such thing as "apt" packages) compatible with your particular Debian distribution, even though they might be using a different Debian based distribution, a completely different altogether, maybe even a different architecture, even though that compatibility isn't their primary concern.
The original model is that software developers don't package their software for distributions at all, at best they provide helper scripts that the distribution maintainers can use to do that. That's why distributions are typically either giant volunteer-run organizations like Debian or companies like RedHat or Canonical.
The problem with that model is that it puts the burden on the distro maintainers to package every possible application for their distro. And application developers have to essentially wait for each distro to repackage their app before it becomes available on that distro. Or start messing with alternate repositories for each distro they want to support.
The old model works for established software, but breaks down a little now that it becomes easier to write software applications.
Personally I use my distro packages for foundational stuff (DE, systemd, tools and utilities) but I use an alternative package manager (flatpak) for most other applications.
What Flatpak, Snap and AppImage try to achieve is to limit the packaging burden for both distro maintainers and application developers, so that end-user applications can become immediately available on a wide variety of distros.
>> The problem with that model is that it puts the burden on the distro maintainers to package every possible application for their distro.
That's kinda what the distros ARE. Also, if you're debian based and debian packages are not compatible with your distribution you're actively fucking something up for "reasons" - stop doing that.
If an app can't use a standard .deb or .rpm then the distro is doing something wrong. If dependency version management is too hard, someone is doing something wrong - not sure who, could be a library maintainer or an app maintainter. Let's not ship half an OS with every app to avoid this stuff.
Have you ever actually worked on distribution packaging? There is no such thing as a "standard" .deb or .rpm. Unless you're statically linking against distro policy they have dependencies on the particular version of the distribution they are built for.
You can't take a "standard" .rpm from the Fedora repositories and install it on CentOS. You can't take a .deb from Debian 11 and install it on Debian 10.
>> There is no such thing as a "standard" .deb or .rpm. Unless you're statically linking against distro policy they have dependencies on the particular version of the distribution they are built for.
If you're using a "Debian based" distribution, the "standard" .deb is the one shipping with Debian. If it doesn't work on the derivative distro, they are doing something wrong. Or like I said, maybe the dependencies are doing something wrong.
Find a deb packaged for an older version of Debian and see if it runs, there's no guarantee, if the software was closed-source you're basically out of luck, unless people statically link all of glibc, ssl etc into their application which is a big no no.
This is the problem that Flapak, Snap etc try and solve. I won't put AppImage onto that list because it actually doesn't solve the problem it just makes it worse.
So, Flatpak and Snap solve the problem of wanting to run stale software? That ought to be a very niche requirement, I would think.
I was under the impression that Debian already solved that problem by allowing you to deploy an older Debian version in a chroot with debootstrap? As long as the Linux kernel is binary compatible, that should work fine. Although I have to admit I don't use stale software that often, so I have little experience in that area.
They solve the problem of being able to run software against a known set of dependencies instead of depending on the versions that come with the distro. Older packages can still run even when the dependency version the distro provides has changed in a breaking way (as parent suggests), but also the other way, new packages can ship features using newer dependency versions that the distro might not have yet.
> You can't take a .deb from Debian 11 and install it on Debian 10.
I mean, you can, if you also install its dependencies. And you may end up with a weird franken-system, but you can. You can even automate it and set preferred distros with pinning, it's how people run things like hybrid testing-unstable distros.
That's the point though: this sort of thing is not ridiculous on most OSs. I should be able to use old versions of software (or new ones for that matter) without having to worry about causing my system to catch fire and explode.
That just requires the ability to have multiple versions of a dll, or to install specific versions with an app, or to statically link. Lots of ways to deal with that, don't distributions support any of those options?
Oh, not all libraries support that. They need to...
You're shifting more work onto distros and users that shouldn't be work in the first place, and basically preventing non-linux literate people from using their OS.
If I install software on MacOS or Windows, I don't have to care if it was packaged for an older version etc, or that my distro may not package a dependency.
It seems very much intentional. You could just keep multiple different, vulnerable versions around and keep everything working. Instead distros say "Nope. We support exactly one version. Update or die."
That is also why you have runtimes, grafting, support sunset,... . I agree that a different trade off makes much more sense for desktops. For servers though...
Most updates aren't security updates. Not all vulnerabilities in a library affect all consumers of that library. Distros don't have every library packaged. Distros often are not often willing shipping patched versions of dependencies. Distros often offer out of date versions of libraries.
No. The libraries are not "out of date" but intentionally static. These static foundations are what companies pay lots of extra money for with windows ltsc, red hat, oracle, SuSE etc.
> distros don't have every library packaged.
Exactly. And for those that are packaged they say "these are the versions we support. If you want to us to do the support work, use these". Again for stuff like windows ltsc that means I install version X now and want this to be supported for the next 5 years. If I instead install a consumer version of windows it means X will be out if support by then and I am expected to have upgraded to X+1, X+2, X+3 during these 5 years.
Case in point, Firefox has multiple current versions: 102 ESR and 111. Both get regular updates and neither is "out of date".
> Unless you're statically linking against distro policy they have dependencies on the particular version of the distribution they are built for.
The irony here is that we’re discussing flatpak/snap, which take the idea of static linking to the absolute extreme by doing something closer to a container where every dependency is part of the package. Maybe static linking being “against distro policy” is tossing the baby with the bath water by causing maintainers to reach to a much worse packaging method (snap) because the distro policy is just too obnoxious.
There’s no good reason you couldn’t just statically link (or relocatably copy) your dependencies into your .deb except the distro maintainers being purists. It would make the process of building a deb (or RPM or whatever) trivial because you’re using it as a dumb archive format for your build artifacts, similar to how a container works.
Static vs dynamic linking is the core of the packaging debate isn't it? Like, distro maintainers say that dynamic linking is better because it lets them swap out libraries underneath the app in case of like security vulnerabilities and stuff. Devs don't like that because inevitably minor version changes break stuff unexpectedly, plus devs prefer to use shiny and new libraries that distros often don't have. Containers were IMO primarily a packaging and deployment solution, and container-style package formats like flatpak and snap are efforts to force distros to use static linking and stop breaking the app's libraries. IMO distro maintainers should realize that their advantages in security aren't as useful as claimed, and their claimed advantages in distro coherence are only relevant to maintainers, not devs and rarely users.
For the base system and libraries, yes. But why should the distro maintainers be burdened with additional work for every possible enduser application out there? If I write a GTK app and want to make it available for Ubuntu/Debian users through official repositories, I need to make sure it gets added to the official package list, and every time I make a new release, someone else somewhere else has to do additional work just to repackage the application so it is available in the repository.
Maybe a far-fetched analogy, but imagine if browser maintainers have to perform additional work every time a new website launches before it is available to its users.
Also, in this system, the application developer has a lot of extra work for making the application run and build against older versions of its dependencies. If I want to make my app available for Ubuntu 22.04 LTS which has libadwaita 1.1, I cannot use any of the new widgets in libadwaita 1.2 (released 6 months ago) or 1.3 (released earlier this month). I can use those widgets but I'll have to write ifdefs and provide fallback behavior/widgets when building/running against these older versions. I will also have to manually track the various distro versions to detect when I can remove the fallbacks from my codebase.
This is what Flatpak is for. Using Flatpak I can target a specific GNOME SDK version and make use of all the new code immediately, without having to write fallbacks. The downside is that when a user downloads my application through Flathub or another Flatpak repository, it might have to download the additional libraries as well, but they will be the correct versions and they won't be downloaded if that SDK is already available due to other Flatpak applications already installed.
Essentially, something like Flatpak is a middle-ground solution that trades of some disk space for the benefit of less work for distro maintainers (so they can focus on core packages) and less work for application developers (that can use more recent dependency versions and don't have to worry about the versions in other distro's)
> If I write a GTK app and want to make it available for Ubuntu/Debian users through official repositories, I need to make sure it gets added to the official package list, and every time I make a new release, someone else somewhere else has to do additional work just to repackage the application so it is available in the repository.
Yes, and that's a good thing for a whole bunch of reasons.
If you don't want to leave it up to the distro maintainers, nothing's stopping you from standing up your own repo to distribute your software to a particular distro. It's a one-liner for users to add your repo to their list so they can use their package manager to install and update your software as usual.
> Essentially, something like Flatpak is a middle-ground solution
Yes, I get that it's convenient for maintainers. But it kinda sucks for users (at least for me), which is why I avoid using software packaged that way.
I'll give flatpack this much credit, though -- it's not a nightmare like snaps are.
> Yes, I get that it's convenient for maintainers. But it kinda sucks for users (at least for me), which is why I avoid using software packaged that way.
My browser, mail client, rss reader, music player, video player, image viewer, steam, ... all have been working incredibly well as Flatpaks. I also get free sandboxing for all of them, so for example Steam and its games, don't have access to my ssh and gpg keys or documents.
The only applications which don't work that well with Flatpak are things like my IDE or file manager.
What things are can change, sometimes for the better. Imagine if distros maintainers could spend their time doing something more productive than doing the same work as hundreds of others are doing.
No, users don't do it. The application developers do it in their CI pipelines. Application developers should be the ones building and testing the app, not distro maintainers responsible for a dozen other applications.
Why should a developer of a free and open-source application, provided free of charge and without any guarantees, have any obligations to package and even test their software on random, thirdparty distributions?
If a distro wants to include their application they have every right in the world to do so. So its up to them to do what ever is necessary to enhance their product with the freely available product of the unpaid developer who created it.
History has shown that application developers are very bad at releasing good deliverable without too much security holes in the packaged libraries or bad practices. And the sandboxing in flatpak is actually meant to protect users from harm done by clueless devs but it fails because devs can actually build non sandboxed flatpaks and they will do it because they don't care
History has shown that distro maintainers aren't perfect at patching security vulnerabilities either and that sandboxing is useful regardless. It also shows that user want working software and will go through the effort of inventing new package formats like flatpak to work around distro maintainers. Maintainers now have a choice between complaining that everyone else is doing it wrong and eventually becoming irrelevant, or getting with the program and maybe even offering their expertise to accomplish what people want to do
Package it for what? There are a lot of distros. Should the dev be packaging it for every one of them? Debian, red hat, suse, arch, other more esoteric ones? Which distro versions? How many years back should they be maintaining the packages?
It’s called a distribution. Literally distributing the software. The distro deals with integrating all the packages into a single compatible system. This includes setting options to maintain system compatibility.
Packaging is not required for testing individual applications. That happens at build time and the developer writes the tests. These are not distribution specific.
The separation of concerns is very clear. If a distribution doesn’t package the code then the user is left to build the application themselves. It’s impractical that a developer would build and maintain their own packages for every flavor of every distribution.
The discussion is mostly about what a distribution should contain. I don't think a distribution has to contain all the possible software applications in existence.
Instead, I think distros have to provide the base packages like desktop environments and related software. All configured for compatibility and complying with the distro philosophy.
But third party desktop applications that are not directly related to the desktop environment are a different category. There is an endless amount of them with varying quality and resources. You cannot expect distro maintainers to spend time on all these random applications.
However, if a third party app is not included in a distro, it does not mean users have to build the software by themselves. That is the problem that Flatpak and Snap and others are trying to solve. They provide sets of distro-agnostic libraries that developers can target instead of having to target each distro separately.
This way a developer can only package the app once, distro maintainers don't have to do extra work, and users can install applications without having to manually configure and build them. Everyone is happy.
That’s a reasonable position but it puts developer and maintainer experience ahead of user experience.
Flatpak and friends are a pain in the ass to use and offer a shitty UX. Having a single point of contact and well understood mechanism for software management is a feature for users.
I don’t expect my distribution to have every software package ever. I do expect it to fulfill my needs. As long as there are applications in the repository that do what I need I am happy.
This. Also void and alpine. Which have simple, no-nonsense package formats. We get it that deb and rpm are a hassle to learn and write, but it's completely a false dichotomy to say snap or flatpak are the only alternative. In fact they push a widely different model of software distribution, one that completely destroys "user unions" which is a crucial aspect of what distributions are.
> We get it that deb and rpm are a hassle to learn and write,
They solve problems that arch/pacman didn't start even thinking about. Like reliably updating an installation, that wasn't kept in a tight loop with the upstream repo.
> false dichotomy to say snap or flatpak are the only alternative
we are slowly moving into the world of immutable base systems, like fedora silverblue for example. The last thing you want is for a random app package to modify your base system. Separating system and apps is a good thing.
> They solve problems that arch/pacman didn't start even thinking about. Like reliably updating an installation, that wasn't kept in a tight loop with the upstream repo.
So they've decided to degrade the baseline UX because they want to optimize for people who don't keep their system up to date? As someone who has no problem keeping my system fresh, this isn't a use-case I want prioritized in my package manager.
> we are slowly moving into the world of immutable base systems, like fedora silverblue for example. The last thing you want is for a random app package to modify your base system. Separating system and apps is a good thing.
The "last thing I want" is a package manager that's invasive to use, doesn't have the latest software and is slow. Immutable systems can be a nightmare to actually use. Wrote your own software? Copying to /usr/local/bin is no longer an option, hope you like packaging up your one-off tool!
> So they've decided to degrade the baseline UX because they want to optimize for people who don't keep their system up to date? As someone who has no problem keeping my system fresh, this isn't a use-case I want prioritized in my package manager.
So, they decided that the update path is always defined, from any state to the latest, without having to update the packages in specific order, where some steps needed may disappear. You know, being robust.
If the year of linux desktop has to happen, not borking the system during updates is a requirement. You don't have a problem with daily updates? Congratulation, but your grandma probably has.
> mmutable systems can be a nightmare to actually use. Wrote your own software? Copying to /usr/local/bin is no longer an option, hope you like packaging up your one-off tool!
Immutable system does not prevent writable /usr/local/bin. Your one-off tool has no business messing with /usr/bin or /usr/lib.
Immutable systems are also minimal; they don't care about your additional software, as it is separated from the base system. You can update your software at any pace you want; nightlies if you want. It just cannot touch anything in /usr (with /usr/local being exception).
I actually don't want a package manager tuned for "my grandma" (the white whale of the linux community). I'm a professional dev, Arch perfectly suits my needs where as Flatpak/Snap... cause excruciating pain every time I'm forced to interact with them (something I go out of my way to avoid at this point).
> Immutable system does not prevent writable /usr/local/bin.
I can't speak to the others, but Nix most definitely does not make /usr/local/bin writable. You have to package up your tools to use them.
So you have particularly skewed perspective; you should try a bit of operations, or at least devops, to normalize it.
> Arch perfectly suits my needs
Good for you. You just need to realize that you are minority. Do you notice that Arch is not the dominant distro? There's a reason for that.
> where as Flatpak/Snap... cause excruciating pain every time I'm forced to interact with them
You are holding it wrong ;)
> but Nix
Nix is not exactly a typical immutable system; it has strong opinions about many things, that other systems don't. You can't evaluate other systems through assuming they are like nix.
Nowadays, even MacOS is immutable and people live with it just fine.
> So you have particularly skewed perspective; you should try a bit of operations, or at least devops, to normalize it.
I didn't say that you should use Arch on the server, but it's great for the desktop.
You can refer to this entire thread to see how well Flatpak/Snap... are working out. Pretty much universally reviled by developers (people who actually use Linux on the desktop).
> I didn't say that you should use Arch on the server, but it's great for the desktop.
And I said that apt/dnf solve problems that pacman didn't start thinking about yet. So we are in agreement here.
Flatpak seems to be working out pretty fine.
> Pretty much universally reviled by developers (people who actually use Linux on the desktop).
Two things:
1) Reviled by some very conservative types, often not willing to consider things from different perspective than what they are used to. If they would build the desktop, it would end up like Homer's dream car.
2) Don't you think that it is a problem when only developers use the desktop as it is? (Not really true, but let's consider that for the sake of argument).
> 1) Reviled by some very conservative types, often not willing to consider things from different perspective than what they are used to. If they would build the desktop, it would end up like Homer's dream car.
I would consider myself non-conservative to the extreme, which is exactly why I don't like Snap since it's built out of fear of problems that I simply don't have. I want the latest versions of any given piece of software and I want my package manager to be performant. I don't think those are conservative values.
> 2) Don't you think that it is a problem when only developers use the desktop as it is? (Not really true, but let's consider that for the sake of argument).
I don't think developers having a great computing environment is a problem at all, quite the opposite. I think it's very desirable and I think efforts that hamper that for the mythical "grandma" are indicative of the core UX problems that plague the "desktop linux" community.
AppImage is not like the other two, it is fundamentally just a way to make a self-contained binary directory into a single runnable file - a way to avoid having to tell the user to extract a tarball and run a particular file inside. Convenience, nothing more. The other two define entire alternate runtimes.
No, an Appimage still ships a dynamically linked executable + lots of dynamically linked libraries, that only link against system libraries when strictly needed, e.g glibc and libGL. Those files are bundled in a squashfs and have a script setting a custom LD_LIBRARY_PATH to the squashfs content for the binary. Snap does use this technique, too, but does more, for worse.
AppImages are not comparable with statically linked binaries.
I can't make out what you're disagreeing with - you seem to be arguing against something I didn't say? The point is that the contents of an AppImage - "a dynamically linked executable + lots of dynamically linked libraries" - works just as well without all the squashfs backflips. If you can ship an AppImage, you can ship a regular ol' tarball with a binary named "RunMe" inside. The purpose of an AppImage is simply to condense such a tarball to a single runnable file, for convenience - nothing more.
Meanwhile, Snap and Flatpak are package managers - what's more, they're invasive heavyweights with permanent costs that are even worse than distro package managers. Snap permanently runs a daemon, and Flatpak requires you to run programs with "flatpak run" - yuck! They are both trying to impose some dubious vision of sandboxed apps that require bureaucracy to communicate with each other, instead of just tackling the core issue of software distribution. Maybe you even like sandboxing! But I've seen no justification why that should be co-mingled with package management.
> Flatpak requires you to run programs with "flatpak run" - yuck
To begin with Flatpak makes .desktop files so no one should be needing to use that command manually.
Secondly, Flatpak has an option folder you can add to your path that lets you run applications by running their FQDN. e.g. org.gnome.Gimp myFile.png rather than gimp myFile.png
> But I've seen no justification why that should be co-mingled with package management.
Building sandboxing on top of package management makes a lot of sense because you want sandboxing to work by default, and for that you need to identify the sandboxable things without making the user point to each one individually.
First of of all, I don't like Snap or Flatpak. I'm only really interested rpm-ostree as used in Fedora CoreOS and Silverblue. I don't hate Appimages, you're claim that they're "fundamentally different" is just hypocritical.
> The purpose of an AppImage is simply to condense such a tarball to a single runnable file, for convenience - nothing more.
This still forces the user to learn the internals if the AppImage doesn't work. E.g. if MATLAB would use Appimage, I'd have to extract squashfs contents, fix the broken libraries inside and the repackage it. Or I would have to write a script to start the executable outside. It's a simpler from a pure technical standpoint when it's a tarball + wrapper script.
> Snap permanently runs a daemon, and Flatpak requires you to run programs with "flatpak run" - yuck!
Snap has many issues Canonical just refuses to solve (e.g. users without home under /home), so I just ignore that. What flatpak does is arguably exactly what Appimage does, a wrapper script. Maybe more complex, but with additional features and the it's the same script for all packages. If you have 100 AppImages installed, you have the same thing as "flatpak run" in up to 100 slightly different versions. I can't see how that reduces complexity.
The problem with AppImage is that it doesn't tackle the core issue of software distribution: how to build the software so that it runs on all the systems you want it to run on.
Flatpak and Snap have SDKs for this purpose, but with AppImage an ISV is forced to guess which libraries need to be bundled and which may be dynamic link dependencies from the OS.
Not to mention the requirement for fuse2, which is being replaced with fuse3.
> And application developers have to essentially wait for each distro to repackage their app before it becomes available on that distro.
Nonsense. Developers can do their thing, and distro maintainers can do theirs. The two have very different priorities and goals.
I'm perfectly happy downloading the latest version of hip package X directly from the developer's website while relying on distro packages for all the boring, stable stuff.
The whole "it's either all hip or all boring" is a false dichotomy.
> The problem with that model is that it puts the burden on the distro maintainers to package every possible application for their distro.
You know, that was never a problem when we had to package the other 500000 packages available in debian, just when users want seemingly popular packages updated more than once per decade. Firefox for starters.
For base packages like desktop environments, tools, libraries, popular applications, etc, yes. But I don't think they should be responsible for every end-user third party application. It's a massive waste of time for everyone involved.
For me, the "stop the world" distribution model is pretty broken, compared to FreeBSD Ports or similar. I don't ever want a maybe-years-old version of software, even with random security patches applied to it.
Setting aside that those developers are not good team players / open source citizens, they always have the option to ship a regular tarball with everything inside, a la Blender / Firefox / VSCode / Arduino etc...
I resent having to install and maintain another package manager, and another set of base runtime libraries. I won't do it. Give me standalone binaries, give me source that compiles, but don't give me a link to some third party app store thingy.
Indeed. This seems like a tooling issue, and one that strikes me as more worthy of community attention than reinventing the package manager for the nth time.
Perhaps. Extrapolating the idea that .tar.gz application packages should be standalone, a better system than an external updater might be that the aforementioned build tooling includes an "update" script inside the .tar.gz. Then the user can run them at their own convenience. This is similar to how Firefox's updater works, except Firefox itself does double duty as the update script.
>Setting aside that those developers are not good team players / open source citizens
...yeah, no.
Open sourcing your code doesn't mean you necessarily want people packaging it up for distribution elsewhere, because when they inevitably fall behind the release schedule and people get on old versions, you are often the one who gets the support triaging work. It is entirely reasonable to not want other people doing this.
Nodejs v4.x was "new" when Ubuntu 16.04 LTS came out. It was added to its apt repos, LTS releases are supported for 5+ years, and LTS policy is not to update major versions of software within a release.
So while nodejs was pumping out new major versions every 6-months, people running Ubuntu 16.04 and installing "apt-get install nodejs" were stuck on the same ancient version.
This is what is expected on LTS releases and is what is expected by people that highly value long term support releases. That said, I think modern security and modern software development practices have obsoleted a lot of the thinking behind LTS releases.
> but the need for real LTS releases is stronger than ever
Actually, I think the LTS mentality is one of the bigger problems in security right now. The hardest problems I've had to deal with in tech all stem for LTS:
* Getting an not-substantial budget to update an essential but forgotten server with custom software and an unpatched heartbleed problem.
* Convincing developers to even look at old web services that have massive SQL injection and were built with libraries with known (six years ago) exploits, all running on some 13 year old version of RedHat.
* Inevitable meetings where you try your best to avoid saying "I told you so" when a disclosure, cryptolocker or malware infestation happens because of the above. These are no fun because they quickly devolve into career-end bingo.
(This entire comment is about my use of my own machines, not about the use of machines in an enterprise setting. In the enterprise, much of this is very, very different)
From a security point of view, yes, you have a point.
But I blame the problem on the industry shift to lumping security and feature updates together. I hate, and prevent, automatic software updates because I don't want feature changes to happen except if/when I'm ready for them. Feature updates are very disruptive, and sometimes break things horribly.
If I could just get security updates, I'd allow those to automatically happen without thinking twice. LTS releases were a (poor) compromise to accomodate those of us who can't, or won't, take on random feature updating.
Sadly, the LTS time periods are getting so short that they're often not effective for this purpose anymore -- so in those cases, I resort to blocking updates entirely until I'm ready for them.
That's also a bad security place to be. I just don't see any other way to handle it aside from separating security and feature updating, like we used to do. But that's not going happen. So all I'm left with is LTS releases and blocking updates.
That is very much missing the point. They were not "stuck", they explicitly asked to keep using that version and get support for it. Same reason Windows ltsc exists and is popular with such customers.
Such developers can stand up their own repository and distribute their software there, where they have complete control. And users can still install and update that software as normal.
The last app image I've installed took 800 MB. It was approximately 720 MB of libraries, and just 80 MB of core program. Using self-contained images is simply not scalable.
I have some experience with packaging, and if we talk about standalone applications, dependencies aren't a big deal; producing deb packages for different targets is not difficult. The last (only) breaking change across different targets I can remember was old graphic libraries (I think it was gtk2-based WxWidgets, removed in newer Ubuntu versions).
> The last app image I've installed took 800 MB. It was approximately 720 MB of libraries, and just 80 MB of core program. Using self-contained images is simply not scalable.
This is one of the points they are missing with snaps and flatpaks.
These formats might be OK for desktops, but are absolutely awful for servers, containers, etc., which, incidentally, are where Linux dominates.
Do I want to spend my time worrying about setting up a package cache, because a handful of updates are costing me a small fortune in transfers? Why would I need to think about what snap is doing in my small instance with a 16GB volume, when I use it to run a certainly small application?
Snaps are an absolute nightmare in these scenarios.
Not sure about snap, but at least for Flatpak, it has runtimes and runtime extensions that Flatpaks can share, which already reduces the individual Flatpak size. But further more, all runtime, extensions and Flatpaks live in a single OSTree repository on the system, which does automatic de-duplication on the file level.
While probably still less efficient than a properly maintained and integrated distro, where all software uses a single set of shared core libraries, its actually (in the Flatpak case) quite optimized & has additional benefits, such as installing a set of apps that don't share a common set of required library versions, which would normally be impossible.
> These formats might be OK for desktops, but are absolutely awful for servers, containers, etc., which, incidentally, are where Linux dominates.
Containers have already won the war on the Linux server, there's no point Flatpak etc trying to compete here. Additionally, desktop software is a complete design paradigm to server software. I wouldn't even say they make sense to be in the same package manager.
Although they do share tech, ostree, namespacing etc.
> Containers have already won the war on the Linux server, there's no point Flatpak etc trying to compete here.
There are things that you just cannot run inside a container.
Also, in cloud environments, default provisioning is not done with containers, which makes sense, but directly causes the issues I mentioned, e.g. Amazon provisioning SSM on Ubuntu Server instances.
Is there a distribution (or whatever one would call it - a Linux) where the aim is to not be managing the whole end-user system of OS+Apps, but just the OS. E.g. similar to Windows, or Android?
No distribution forces you to use package managers for everything. Even before flatpak/snap, distributing (especially commercial) software as just archives or executable installers was/is perfectly possible.
The caveat is, there's no common baseline for libraries. Graphics libraries, the libc itself, GUI widget libraries etc. will appear in different version combinations in different distributions, since they have different release schedules.
So if you want to go the Windows/MacOS route and ship a readily compiled program for Linux, you have to vendor in a loooot more libraries, and they have a higher risk of breaking due to incompatibilities (kernel interface for your graphics driver changed, configuration file syntax for your font renderer changed, etc.)
Flatpaks/snaps try to solve this (poorly) by vendoring the entire kitchen sink and then some, which just creates more bloat and a worse DLL hell than Windows could ever have dreamed of. So using it for everything isn't really feasible still.
> they have a higher risk of breaking due to incompatibilities (kernel interface for your graphics driver changed, configuration file syntax for your font renderer changed, etc.)
Simply don't vendor glibc and graphics drivers and you will be fine. Vendoring drivers doesn't make sense anyway as your application will be obsolete by the next HW cycle.
glibc breaks regularly, as does mesa occasionally, so it would realistically be required if you wanted win32-style 20+ years of backwards compatibility.
And there is a lot of more obscure examples: fontconfig e.g. at some point changed their config file format in a not backwards compatible way, and now some Steam games crash on startup because the developers earlier vendored it to get around its ABI breaking repeatedly.
And that sort of software doesn't really obsolete. Steam and GoG allow games to have a 10+ years long tail of slow but steady sales that don't really justify constant maintenance releases but still both let people enjoy good games (those don't really obsolete), and serve as advertisement for the developers' newer games.
> glibc breaks regularly, as does mesa occasionally
Neither of this is true.
> And there is a lot of more obscure examples: fontconfig e.g. at some point changed their config file format in a not backwards compatible way, and now some Steam games crash on startup because the developers earlier vendored it to get around its ABI breaking repeatedly.
Bundling fontconfig but not it's internal config is retarded, yes. Really, games have no business looking at system fonts at all.
Calling everyone except yourself too retarded to figure out compiling Linux software is not a productive discussion. And glibc/mesa related breakages are well documented, if you care.
Arch Linux is pretty faithful to shipping software exactly or as close as possible to as the upstream authors intended. It's on a rolling release which means you kind of get a firehose of nonstop major version updates though, whether you want them or not.
upstream didn't build with all the dependencies at the same version as the latest in the arch repo. Nor are most upstream developers bumping all their dependency versions constantly
if you're a big player like ubuntu/debian you can go to the authors/upstream and ask them to fix the program to work with their dependency versions (in say.. an LTS release). most would be glad to bc it's just something upstream needs to do once every couple years and doesn't need maintenance. If even thats too much and yyou want esomething even more handsoff/timeless then you make an Appimage. It'll work for ever
With Arch youd need to constantly monitor the thing doesn't break？Are they just #yolo'ing dependency versions most of the time ？ (itd prolly work 99% of the time..)
In practice it works because GP is exaggerating. Arch does hold back library and/or program updates until they work together, or simply ships multiple major releases of libraries, and have a separate testing branch.
Right now e.g. there's openssl (still at version 3.0.x, because nobody supports 3.1.x yet) and openssl-1.1 (1.1.x-y) to handle the rather slow migration of upstreams to the new major release.
I stopped distro-hopping a few years ago after landing on Silverblue a few years ago (back when it was still called Fedora Atomic Workstation). While it doesn't satisfy all use cases, I view it as a boring (in a good way) 80-percent solution that normally just works.
Fedora has always been quite bad at branding and making their distro appealing.
It's probably one of the most solid Linux distros at the moment and their website makes it seem.. boring.. No screenshots, features, mentions of whats available (like apps), that Fedora uses newer graphics/drivers kernels, that you can run Steam, that apps are sandboxed etc.
There are operating systems - like FreeBSD - which have a very strong and clear distinction between the Operating System base and third-party software.
I'm curious what a Linux would look like that behaved more like windows or android in the sense that there isn't such a blurry line between the OS and its applications. For example, installing a C# or Go compiler on Linux is great with a package manager. But doing the same with a C or Python is different? This is basically the Unix "I'm not just an OS, I'm a platform for building and running C programs" heritage that isn't properly hidden I think.
We've reached the point where the C ecosystem has so given up on having a package manager that isn't Linux distributions that Microsoft added WSL and Docker(-compatible) Linux images are the most widespread distribution format. I don't think we can still figure out a way to decouple the two again, and without that, we can't really create a Linux environment that isn't a package manager with an email client.
And to a degree, the line is blurry with Windows as well, what with how deep MSVC/win32 is integrated into the system. .NET took decades to dis-entangle itself from Windows and become a proper standalone platform, and Microsoft's attempts at getting people to use UWP instead of win32 is ongoing and not very successful so far.
> in the sense that there isn't such a blurry line between the OS and its applications.
Interesting. It never occurred to me that there's a blurry line between the two. It seems to me that there's a very hard line between the two. I'm not sure I understand what you're saying here.
> For example, installing a C# or Go compiler on Linux is great with a package manager. But doing the same with a C or Python is different?
I'm confused by this question, so forgive me if this answer isn't really addressing it. A package manager is a convenience, nothing more. Using one means makes your life easier in a number of ways. But not using one isn't really that much more difficult. Package managers are 100% optional.
Hot take: package distributors are 'fake jobs' that don't need to exist, or exist only for the make-work of packaging. Tools like flatpak, snap, appimage, containerization, etc. have made the need for adapting software to different distributions unnecessary.
Yeah, if you don't mind hugely increased start-up times, RAM usage and less curation and more crap, sure no need for package distribution.
I personally do mind very much. Just the differences in startup times between apt and snap applications are huge and I would absolutely despise working with such a sluggish system. I would rather build everything myself from source if forced to.
Not to mention it introduces a single point of failure, that once compromised can start pushing malware directly to user.
With maintainers in the loop, there is at least one more person that can notice something is fishy. Not to mention there is usually so time before packages are updated, so there is more time to notice an attack.
And yet the upstream developers themselves never incorporated the change, it was entirely because an unnecessary third party middle man made unnecessary changes.
The mass proliferation of electron and other such webpages-as-desktop-applications would seem to indicate that the average user doesn't care for any of those things.
First of all, there is no such thing as an average user. Also, it is not relevant to the discussion. I care about what I want in a system, not what some imaginary users want.
Now, we could stop there but I will play: Electron solves a real problem for developers. Writing cross-platform GUI apps is a real pain. Yes, there are native solution but ensuring the user has the exact same experience on every platform tends to be orders of magnitude more costly compare to web based solutions. (That or they are exotic options like Lazarus with Free Pascal that have a much lower Developer pool.) Many apps wouldn't even have Linux ports if they were not written on Electron.
Now, why do users accept them? Why are they not out-competed by native solution? Oh, honey. Why do I have Microsoft Teams installed? Because I like it? Hell, no! Because I need it for work. Why do I have the Discord Client? Because it is great? Nah, I long for good old IRC but Discord it there the people currently are. Did I think the Epic Games launcher is such an great app? Nah, they bought me with offering free games.
Users tolerate shitty software for many reasons, mostly because they have to. It does not follow from that, that they don't mind software being shitty.
>First of all, there is no such thing as an average user. Also, it is not relevant to the discussion. I care about what I want in a system, not what some imaginary users want.
In which case I hope you are prepared for software to get many times worse, because the software industry doesn't give a tuppeny fuck what you want in a system, they care what sells to the lowest common denominator. And that means slow, bloated electron web sites shoehorned into the desktop because the pool of mediocre JavaScript developers that can extrude a minimum viable product is huge compared to the pool of native developers of any language.
And it will continue this way for as long as it's accepted. So, forever basically, because the average user you claim doesn't exist will put up with anything placed in front of them without significant enough complaint to impact profits.
You keep trying to hand wave this relatively straightforward point: users are already used to, and accepting of, slow and bloated applications. Therefore, the downsides that app containerization introduces are irrelevant to most people that will use them.
I agree but I don't think the software packages themselves should be held back by the available time and attention of packagers. In the new world a packager or distro creator can just pick and choose what flatpaks, snaps, appimage, etc. sources they deem good enough to push to users. They shouldn't be a roadblock in the way of users getting the latest version of software.
Then you install the new version, find out it requires a new version of python and all its libraries, but you can't install it because pytorch only works on old versions of python.
That doesn't happen when apps are packaged in their own containers with all dependencies. The new version you installed has its own python dependencies.
I agree. Devs would like to statically link and have their app work. Maintainers don't want that to happen for IMO reasons that have far less benefit than claimed. flatpak and snap are ultimately efforts to get around the vortex of maintainers, maintainer efforts to get things not to be statically linked, and all the complexity that entails. I think there's still currently value in distro maintainers that do things like getting KDE running on Wayland using pipewire, but IMO that role should be limited to configuration and compilation only. There should not be any distro-specific patches in existence
I don't want debian to be like the android app store, where there are thousands apps that work badly and overflow me with ads.
I much prefer the f-droid model, of having curated repositories to keep crap outside.
Also, I can't understand why people on the internet think that upstream developers are omniscient. They make lots of mistakes and errors. Distribute maintainers fix a lot of things, and send the fixes to the authors.
There's nothing stopping someone from making a flatpak or other similar tech feed or 'store' that's only the curated apps they deem appropriate for users.
Bugs should be fixed upstream, not kept in distro specific silos. There's no reason why only a packager can fix some upstream issues or become a contributor. On the contrary shipping your app as a universal tech like flatpak means Redhat, Debian, Arch or any other user can use it, develop for it, and send fixes upstream.
The strawman sounds compelling on the surface, but that's not what happens in practice: Bug fixes are sent upstream by all relevant distributions, and they regularly cooperate with each other directly, as well as upstream. The "new world" of users of all distributions working together has been reality for the past 20+ years.
> Bugs should be fixed upstream, not kept in distro specific silos.
This is mostly a Debian issue fwiw. Debian is literally notable because they're obsessed with making sure that any package in their repositories is kept with the same "API"[0], no matter how old the software is. The result is that Debian packages can end up hugely derivative compared to the equivalent of upstream and other distros, but it's usually also because the software in question is half a decade old.
With other distros, packaging changes to upstream usually just reflect the preference to match a certain style of configuration (to pull another example from Debian: nginx ships with sites-{enabled,available} folders and is configured to load from sites-enabled by default. This is to match the same configuration style that's used for apache2 and that it's associated tools assume you configure apache2 with, even though nginx just uses a conf.d folder and has no extra tools to facilitate anything fancy with sites-{enabled,available}).
The extreme end is nix, which actively requires you to have the upstream written with nix in mind because nix will basically demand you configure the source code in ways to accommodate for it.
[0]: This includes actual software-intended interfaces and the ones hacked together by users by ie. reading out logfiles.
And even Debian tries to upstream fixes where feasible, which isn't too uncommon for security vulnerabilities, they tend to lurk in old parts of codebases that haven't been refactored for a while.
> The extreme end is nix, which actively requires you to have the upstream written with nix in mind because nix will basically demand you configure the source code in ways to accommodate for it.
I will agree though that applications which have their own update mechanism or do other things that make reproducibility harder are much more difficult to create a Nix expression for.
It's best practice not to have any shell tools in your app container, including package managers. It bloats the image and can be a security vulnerability if a zero day exploit hits the app. Ideally a container is something like distroless which just has the libc and dependencies you care about and nothing else, not even bash.
If you're shipping modern code like go or rust you have a static build with no real dependencies. If you're shipping a scripting language like python you're probably going to use their base images, and if you're shipping native C/C++ you have to figure out your risk tolerance for trusting a distro to ship good dependencies vs. just building them yourself. It's not hard to build all your deps in a container, and arguably is the best security practice so you have total control and knowledge of their versions.
As long as there is unavoidable software made by organizations like Mozilla that don't care for their users's choices there is an important role for distributions: To make sure what you get is what you want.
I agree. Maintainers are useless middlemen (at best) who only exist because of Linux userland's particular diseases and a desire for distros to rule over their own little repo fiefdoms.
No other desktop OS has done it like Linux and for good reason. People have been citing this as a reason they don't want to use Linux as a desktop for decades to mostly deaf ears, who then turn around and wonder loudly why no one wants to use their OS. Hell, even Linus Torvalds himself complained about it.
I have no earthly idea what you are talking about.
For decades, Linux package managers have been the killer app for Linux. They made installing and updating every single one of your applications trivial. You didn't google for sketchy download sites and unsigned exe's. You didn't have to fight the system to cleanly uninstall things. Even release upgrades were the smoothest thing ever. In 25 years, I've never had a Debian release upgrade go wrong.
Anyone bitching about package managers as user hostile is a flat out idiot.
What's wrong isn't so much the package managers as the necessity for them and for an army of third party volunteers to maintain packages and all the problems that predictably arise from that. Linus famously complained about how things work at DebConf 14[0], but I guess he's an idiot for doing so? That's a pretty hot take but whatever.
If a package is not in the repo? Sorry, you have to compile from source. Want a newer version? Compile from source and hope that the build environment dependencies are in the repo. Want an older version for some reason? Break out docker or KVM so you don't break your system.
None of this is fundamental to the model, that much is true, but in practice it is how all Linux distributions using a package manager/repo model without things like Snap, AppImage, and Flatpak work.
Here's the best part though: Even with Flatpaks and AppImage you can still use a repo! In fact Fedora Silverblue, which uses an immutable base system and installs everything through Flatpak and Toolbox, uses a Fedora controlled Flatpak repo by default.
If you want the dystopian hellhole you seemingly long for, just use Android and enjoy the ad-infested crapware? No reason to moan about things you seemingly don't understand.
There are plenty of projects to semi-automatically produce packages for a large number of distributions starting from just a git repo of the software to package.
Sure, packaging software is a bit of a thankless task, but with enough automation, packaging thousands of bits of software on every git commit should be doable by just a few volunteers.
1) a unified database for package names (it's all unnecessarily ad-hoc right now, with different distros having different policies for capital letters, whether libraries are prefixed with "lib", whether python packages have "py" or "python" or "py3" etc...
2) a standard format for declaratively describing how to build a package.
Basically, we have FreeBSD ports, Arch PKGBUILDs, Void templates, Gentoo ebuilds - they all do more or less the same thing in the same way, they all work really well, and they're all incompatible for sheerly incidental reasons. I'd like to see these incompatibilities papered over in such a way that I can write one template and generate all the others.
There are already many standard build specification formats: CMake, autotools, Meson. For each, most distro's package managers can build the software with very little configuration, e.g. Debian's debhelper will cover most standard build systems and Gentoo has eclasses that you just need to inherit in a package. I'm not sure that adding another makes things easier.
Configuration options will need to be specified manually if you want any but I don't see a way around that as they are both package-specific and also something where distros want to differ.
For dependencies there are usually language-specific registries and pkg-config for C/C++-land. Some distros (e.g. RPM-based-ones) already support specifying dependencies by the pkg-config names or library SONAMEs. There is no distro-agnostic way to specfy your dependencies except for a README as far as I know - this would be something worth specifying.
You'll invent new standard and have N+1 standards.
It's not as if package formats can't be declarative - just that some issue always turns up in package X or Y that the declarative format didn't anticipate.
I think there may indeed be tools to convert these formats but in the end the devils are always in some distro-specific set of details that make life miserable. e.g. your library has some option enabled or doesn't and some other package in that distro needs it and so on and on.
Not sure how you figure that. Containers are completely orthogonal to both of my requirements, which again are 1) a globally canonical package namespace, and 2) tooling to convert some standard recipe file format into distro-specific recipe files (which would require point 1 to work).
I take your point (it's the general point many use to argue for Snap/Flatpak/Nix/AppImage/0install/Homebrew/&c.) but if you want to ensure you're not replacing per-distro-package-manager-fragmentation with completely-arbitrary-chaotic-package-manager-fragmentation there needs to be some unity & consideration for what users want in order to ensure they'll willingly subscribe to your system of choice.
While I wouldn't call Flatpak "popular" with users per se, it's probably one of the least-worst alternatives that have come out. The horse Ubuntu has backed (Snap) may be the most used by virtue of being rammed down user's throats by projects with existing user capture (e.g. LetsEncrypt), but that's not going to make the debate go away: it just strengthen's the argument to return to distro packaging.
I just mean distributing software exclusively through 1 channel (or at least only officially supporting 1).
This wouldn't be the worst situation (devs needing to more time preparing myriad distribution formats is a valid problem) if the specific 1 channel being used wasn't a broadly resented channel.
> What you are really saying is you want all software developers to publish Debian packages
That's exactly what I want. Developers and Linux distribution maintainers should be working more closely with one another instead of reinventing static linking with "snaps" or whatever just to avoid working with the community.
It's not actually that hard for cross platform apps, at least. We do this in Conveyor and can build debs and apt repos, with inferred dependencies, even from Windows and macOS:
It works by downloading the Ubuntu package index and using it to look up ELF dependencies, devs can also extend the inferred list with other package names if they like. The deb installs an apt sources file and public key so users can stay entirely in the GUI. The control scripts are auto-generated but can have extra code appended to them via the config file if the developer so wishes.
It works pretty well. Conveyor is used primarily for cross-platform apps that use {Electron, JVM, Flutter, C++/Rust} and those don't normally have very complex Linux specific dependencies. Also, in practice Ubuntu and Debian and its various forks are close enough that package metadata is compatible in basically all cases.
People do ask us for Flatpak sometimes. Probably RPM is a higher priority though. Those two get most Linux users, they're stable, there are portable libraries that can make them and they can package anything without code changes. Flatpak requires mastering a new file format and the mandatory sandboxing is guaranteed to cause problems that will end up in the support queue of the deployment tool.
As someone who does this for a lot of distros (ZeroTier), I can say that it is hell and I understand why devs don't want to do it.
Two specific examples:
(1) For some reason a lot of Debian distros decided to rename and/or re-version-number OpenSSL for no good reason (pedantry is not a good reason), meaning if you depend on OpenSSL/libcrypto your packages will mysteriously break all over the place because you're not using the versioning scheme or naming convention. We're not doing it now but we may switch to statically linking this.
(2) We use a UPnP library called miniupnp in the current version. We have to statically link it because of issues like (1) and also because some distros were for a time stuck on a version that had security bugs that they would not upgrade because it would break other packages.
So imagine (1) and (2) ad infinitum times a hundred little distributions and Debian forks and... it's completely untenable. Static linking solves many of the problems but that defeats some of the purpose of package management.
We do it for all the major Debian and Ubuntu distributions by using a qemu-chroot based build farm that builds on each distribution+architecture combo with actual qemu binary emulation. It's over 200 gigabytes of chroots and takes two hours on a 64-core Threadripper to build all the artifacts. We tried cross compilation and had too many problems: it builds fine, seems to run fine, then users complain that some mysterious glibc symbol was missing on their system. We switched to building on actual chroots and most of those problems went away. Most of them.
Snap, FlatPak, and Docker are all variations on the same basic conclusion that many developers reached long ago: "fuck it, just distribute software in the form of tarballs of entire Linux installs."
The only thing worse is Windows MSI installers and Windows drivers, though with those at least there's less platform variation so once you battle those horrors the result mostly works on most systems. Mostly. But whoo boy is the Windows installer system horrific. I shall quote Pinhead: "I have such sights to show you..."
Apple is the easiest because they have less legacy baggage than Windows and only one "distribution." They do have some horrors around signing and notarization and we don't distribute in the Mac App Store (yet?).
A major reason SaaS delivered through browsers is eating the world is the immense and totally unnecessary pain of distributing apps for major OSes. Even Apple is quite a bit harder than making a web site that you visit in a browser. All this pain is 100% the fault of OSes not caring about developer experience and in the case of Linux the multiplication of endless "vanity distributions" to no benefit to the community.
If you want packages instead of tarballs of OSes, Linux distributions could (1) consolidate and (2) invest some time in the very boring totally un-sexy work of massive amounts of clean-up and standardizing things as much as possible. But it's more fun to make another vanity distribution I guess. You'll get it right this time and everyone will switch to your distribution.
> Snap, FlatPak, and Docker are all variations on the same basic conclusion that many developers reached long ago: "fuck it, just distribute software in the form of tarballs of entire Linux installs."
..wait a minute
> (2) We use a UPnP library called miniupnp in the current version. We have to statically link it because of issues like (1) and also because some distros were for a time stuck on a version that had security bugs that they would not upgrade because it would break other packages.
Wouldn't this solve the same problem without adding snap/etc?
It solves some of the problems, but not all, and it requires a lot of testing and trial and error and setup. Sometimes libraries are hard to statically link. Sometimes you have problems with different glibc versions.
I forgot to say: some distros that people still use are too old for us to build for, so we build fully static binaries for those. But you can't really statically link glibc, so we build them on Alpine statically linked with musl. That usually works okay. It's the best we can do for the old stuff.
We've invested a lot of time in this. For most developers I totally understand "fuck it."
I have had a lot of success with building against old glibc version and then bundling all required libraries except system ones (glibc, libGL, libX11 etc). Anything optional needs to be loaded with dlopen/dlsym but for many things that detail is already handled by libraries like SDL. That's generally what seems to be the common solution for games (well used to be at least, today most just build against the Steam runtime and call it a day). It's really not that different from Windows IMO except that the set of base system libs that you can rely on to be backwards compatible and don't need to bundle is smaller.
You can still pacakge that distro-agnostic build up into distro-specific package formats to make updates easier for users without needing zillion of different build roots just for Linux.
Building apt packages isn't the issue, libraries is. Debian ships very old libraries, forcing applications to run in docker. Or bring their own copies of system libraries.
Look at Arch Linux, we just write a short AUR script and the package is integrated. Once the script is written, everyone can use it. This is possible because Arch always ships recent libraries.
Conversely, `pacman -Syyuu` doesn't complete reliably without user intervention, which is far from ideal. I say this as an Arch user - the expectation that I'm going to manually attend to package upgrades in practice leads to me using outdated packages
I haven't had it often but I've had stuff break sufficiently badly that I couldn't boot to a graphical environment before.
I agree with the other posters that a lot of the stuff that breaks is in the AUR, but practically most users rely on the AUR so if the AUR is prone to breakage so is the typical user's arch Linux environment.
I'll also freely concede that part of this stems from the common practice of treating the AUR and pacman packages interchangeably through helpers, but that's a practice that's almost necessary for reasonable use and is present on the official wiki.
I think ultimately Arch is just a UX which was designed in a simpler time when users could reasonably expect to account for all the packages they installed
It's probably because you run updates every couple of days that you don't have the issue. For me, I have a laptop that can go untouched for up to a month, and it's not fun catching up on a month's worth of updates. Lots of packages have been replaced, I usually have to futz with the keyring, etc.
Also worth pointing out - the fact that you can so easily perform a partial update with pacman and totally break your system is infuriating to me. If an upgrade fails, it should revert to a cached package database. Otherwise an update will fail, the user will go to install something else, and all of a sudden nothing can link to libcrypt because you installed ntpd, which upgraded all of your auth stuff after an openSSL vuln was discovered, and everything is hosed.
> If an upgrade fails, it should revert to a cached package database.
I'm prettttty sure it does now? When did you run into this problem? It definitely used to be a problem, a long time ago (I ran into it once when I ran out of disk space mid-upgrade), but I think system updates are atomic now.
Happened to me a few months ago (whenever that big SSL 0-day was announced).
I use a 3rd party repo for ZFS drivers, which gets checked for compatibility with each new kernel release by the maintainers, so ZFS frequently stops me from upgrading, and crucially it stops me after I've already fetched the new package databases.
Running pacman -Sy and then installing an individual package isn't supported and it's understandable why.
Running pacman -Syu and having it break sticks you into this limbo where if you install anything before finishing the upgrade, you risk shooting yourself in the foot.
Interesting! How does the ZFS upgrade "stop" the package upgrade process, I wonder? Might be worth reporting a bug, maybe pacman can handle that kind of failure better, or maybe the ZFS package could be changed to fail more gracefully. I think the pacman devs would agree leaving the system in a partial-upgrade state is a bad thing to do.
The official release of ZFS on Linux I believe only supports kernel 5.x. Since arch is always on the cutting edge, the repo maintainers need to manually test the driver with each new kernel release before pushing it out to the world. They stop you from borking your system in the interim by having a strict version requirement for Linux on the zfs package. Pacman only does that check, though, once the -Sy operation has finished
I do have a shell VM (accessed via SSH & xpra) that I update much less frequently and still don't have issues.
I think the main reason I have fewer issues compared to other comments here is I try to minimise my use of the AUR, and don't install third-party repos.
I also suffered from the grub issue mentioned here elsewhere, had to rescue my laptop after it became unbootable (though it was very simple to Google the problem and to fix it). I don't think using grub is particularly unusual choice in packages.
I also had to downgrade Samba and wait a couple of weeks for a fix to a bug it had mounting password protected shares. Again, Samba isn't exactly an exotic package.
Also had to downgrade something in my audio stack (I forget if it was pipewire, pulseaudio or a kernel issue that was the root cause) and wait a couple of weeks for a fix due to bluetooth issues.
There's also the occasional update that requires manual intervention.
All of that said, I don't think I cumulatively spent all that much time on maintaining my Arch systems as all these errors are spread across a long period of time and didn't take that long to handle. I probably spent a lot more time and nerves on Ubuntu as I don't think I've actually ever had a dist-upgrade work flawlessly and each time takes a lot of effort and mental energy.
Those are all upstream bugs, not Arch's. That's the trade off when you're running bleeding edge, but you can't blame Archlinux for packaging a buggy application.
I'm no really blaming anyone, I'm thankful for Arch, I love it, and I recommend it to other people who ask (well, at least those that can handle it :)).
> Those are all upstream bugs, not Arch's.
IIRC the grub issue was in the update script, so it may have been an Arch bug rather than an upstream bug?
Still, it would also be nice if a package is known to be broken due to an upstream bug it would get rolled back, so once the breakage is known, no one else will update into a broken state. That would save some time over each person individually updating to a broken state, debugging for a bit and then downgrading the broken package and then also paying attention not to reupdate it each time they update the system until the problem is resolved.
But again, not trying to complain or assign blame, I was just responding to a question in a parent comment.
EndeavorOS is basically vanilla arch with an installer, I just posted their thread because they were open and transparent about the issue while the arch sub did very little.
When I raised this with an arch dev, I was told they purposely shipped grub's master branch (not release) because they didn't feel like backporting a security fix, and if I couldn't fix a grub issue with no working computer I probably didn't have any business using arch.
Ok then. Went and downloaded popOS from a friends house and haven't touched arch since.
I may be misreading this comment, but it feels a little stuck up. Arch is created for Arch developers. This is well-known and they have never hidden that. The wiki states as much somewhere in the installation guide. That the distribution turned out to be useful to many others is purely coincidental. If you tried it and decided you're not one of them, no need to pour shit on devs as if they owed you something and didn't fill their part of the bargain.
I agree, it is a little stuck up, on the part of the elitist devs.
If this was a little hobbiest distro, I'd agree with you, but this is one of the most popular distros out there. There is a certain expectation of competency and quality.
I would argue that shipping the master branch of a bootloader isn't competent when there are stable releases available, and not addressing the issue when literally thousands of users are having the problem isn't very respectful of your userbase.
FWIW, I did eventually fix it, but the endeavour os folks were much more understanding and helpful with the issue. The arrogance I saw on r/arch was insane and offputting.
I think it depends on when folks were using Arch. I've been using it since 2007, and back then before the "conf.d" configuration file convention became common, it was very frequent that you would need to manually merge your local config files with the upstream one. If you screwed up or forgot, your system would be some variety of hosed. It's become much, much less common in the past ~10 years. I'd hazard a guess I only have to manually attend to an update once or twice a year now, where it used to be almost daily.
Yeah, `pacman -Syyu` works fine most of the time, except when you use a lot of AUR `devel` packages, which can break because they often do not have strict dependencies definition.
AUR packages are not part of the distribution though, so that's understandable - they're explicitly user contributed packages and are very much "here there be dragons" and "if it breaks, you get to keep both pieces".
Without having looked closer at how to build either of snap and flatpack; if you can static link or bundle libraries with your snaps/flatpack packages, what stops you from doing the same in a .deb?
It's just a convention not to, no? IIRC the Steam .deb comes with pretty much everything statically linked. Works fine.
But what's the point? You install debian stable, where every library is at around 2 years old. Now you want run SomeApplication (Blender, Gimp, video games) as that's what you actually use your computer for.
If you wanted to run a 2 year old version of SomeApplication, that would work just fine. Is that what you really want? Is that what most users want?
If you instead install a snap/flatpack of the latest version of SomeApplication, you are installing also new libraries instead of the good old stable libraries your distro provides. So what's the point then?
But that means I have to update my OS and my application when I want a new version instead of just my application. Say what you want about Windows, but I can download and run the latest version of all my applications on Windows 7 without it missing a beat.
Yes, because either you, the user, must track down .NET Framework X.Y (less of a problem now that they're stabilized) or because the application packs all of it dependencies and shoves them into \Program Files or \Windows\System32 until it's eventually 30gb with 1000 copies of msvcpXXX.dll
That's not that different from the distribution model of Flatpaks/AppImage (or APK, or .app on MacOS/iOS). It's more that, traditionally, Linux packaging solutions try to only have ONE copy of glibc or any other library, and packages are recompiled from source so the symbols resolve. Something which isn't an option on Windows, as a generality.
I don't think Windows is a good comparison, since major upgrades for it tend to be vastly more invasive than major upgrading the average Linux distribution. Debian 6→7, ten years ago, was a really massive infrastructure upgrade, but ever since it's been pretty smooth sailing. Ubuntu is a bit bumpier, but it's still only on the level of Windows' "major feature upgrades (that totally aren't service packs because we don't want to extend your warranty)" updates that Windows 10 and newer get every 6 months.
And Windows 7 is extremely old and only works with new software because, and as long as, developers go the extra mile to make their software work with its old APIs. Valve recently announced that they'll drop support for it next year, and other companies will follow soon. It's not too much different from the situation of, say, Debian Oldstable or Ubuntu LTS: Outdated, but popular enough that people tend to put in the effort anyway.
This puts too much unnecessary burden on developers and maintainers. It doesn't make sense to adapt software to tens of distributions (and then to different OS like FreeBSD).
Apt should be used only for system software (like init system, system libraries) but not for applications.
Also apt allows to run scripts as root during installation and this is not secure.
There's a Pareto effort, of course. If you release Debian and Redhat packages, you cover 80% of the user base. It's not hard to maintain, if you compare to building msi installers. Further, the process of generating packages for derivatives of debian or redhat can be mostly automated, so user coverage is bound to grow if your package is popular.
As for root requirements, what you are asking for are (non privileged) user installable packages. Packages that install only to the user account. This feature doesn't exist, but it'd be a much saner approach than snap/flatpak.
> If you release Debian and Redhat packages, you cover 80% of the user base.
That means you need three builds to cover Ubuntu and its derivatives (20.04, 22.04, 22.10). Two to cover Debian (10 and 11). Three to cover RHEL (7,8,9) and also a few more if you want to support Fedora. If you want to support multiple architectures you get at least twice as much builds.
No matter how you put it, that's a lot more work than maintaining a simple flatpak manifest, which allows you to target all those platforms with one automated build for each architecture. And you also get the benefit of being placed in the app store and not being hold back by whatever API is the lowest-common-denominator between all those platforms or having to clutter your code base with `ifdef`s; if you want your app to use an API which was only added in GLib 2.74, then you can just do that and it'll work.
I go to cool app website, I click download, I get a '.deb' file, I click .deb file, it opens some installer GUI (instead of Ark because I'm lucky today), I click install, I get error, I google error, I find cryptic commands that modify my system in weird ways just to get the app to install, I install app, I open app, it crashes, I open firefox, I get error "error: libX.so.1.2 not found", I can no longer google the error.
The system package manager is convenient when it works, because it's already there. But that's about it. Using it to install any random apps is a recipe for disaster, it leads to fragmentation since everyone on a different distro uses different commands/workarounds to fuck up their systems in different ways when trying to install poorly packaged software.
If everyone just used Debian, we wouldn't need Flatpak, but obviously that's not the case. Whenever you find a "linux app" that's packaged for distroA, but you're on distroB, there's a chance that it will work. That is 100% luck and coincidence, because most linux distros just so happen to ship the same family of system software/libs, sometimes even with the same versions.
Rather than leave it up to the undefined behavior lottery, a standardized non-system packaging format can guarantee that things will work across any distro. That's better for everyone involved: users, sysadmins, distro maintainers, and developers. Whether Flatpak is that format idk, but IMO it's the best overall out of the three main contenders (Flatpak, Snap, AppImage)
> It's not hard to maintain, if you compare to building msi installers.
I don’t think building MSI installers is especially difficult; it’s kind of a set-it-and-forget-it thing. Configure Wix or Inno Setup once and then you’re good to go (on every Windows machine, no need to worry about multiple distros).
Snaps/Flatpacks/other binary-containerised stuff is one of the major reasons why I abandoned Linux distributions after 22-odd years (1999-2022), and am now a happy Apple user (the other one being abandoning X11 in favour of Wayland, and a switch to laptop-first usage, on which Linux still has horrible power-consumption).
Linux distros have a history of abandoning useful, well-understood technology for fads that then cause power users all kinds of headaches later-on (e.g. by cluttering output of useful tools like mount). Eventually, power users just lose the will to adapt, and move on.
Could I have just switched distros, or compiled my own stuff? Sure. But I am no longer a student. I am become a mid-40s guy who is becoming aware his lifetime is not unlimited and can be spent better than learning the ropes of a new system.
In the time of Jenkins et al the burden of creating bistro-specific packages is negligible.
> Snaps/Flatpacks/other binary-containerised stuff is one of the major reasons why I abandoned Linux distributions after 22-odd years (1999-2022), and am now a happy Apple user (the other one being abandoning X11 in favour of Wayland, and a switch to laptop-first usage, on which Linux still has horrible power-consumption).
Flatpaks/Snaps are almost exactly the same, conceptually, as apps on MacOS. Go look inside something in /Applications/... The .app is a folder, with all of its deps.
> Linux distros have a history of abandoning useful, well-understood technology for fads that then cause power users all kinds of headaches later-on (e.g. by cluttering output of useful tools like mount). Eventually, power users just lose the will to adapt, and move on.
"Power users" have a history of complaining about distro maintainers abandoning useful, well-understood techology because the "power users" don't actually understand the technology, nor do the understand the enormous headaches it causes distro maintainers to put a sane face on 30 year old design philosophies and continue moving forward.
The goal of distro maintainers is to STOP investing endless man hours in trying to make X work with high DPI displays, scaling, high refresh rates, a fundamentally insecure design model, and so on. The goal of systemd is/was for distro maintainers to STOP wasting endless man hours on edge cases because sysvinit didn't have reasonable mechanisms for "if anything in this dependency chain is restarted or changed, re-evaluate it" or "don't try to start this service if the mount isn't available so systems stop hanging on startup if some NFS server isn't available" or whatever.
> In the time of Jenkins et al the burden of creating bistro-specific packages is negligible.
Under the assumption that they are ABI stable, which is a bad one. Otherwise, it's the same old kaleidoscope of mapping out whether it's `libfoo-dev`, `libfoo-devel` for build requirements, whether it's a distro with a completely different naming system, dissecting the ELF header or reversing depchains in Python/Node/whatever to try to find deps and what THOSE are named in some distro, or whether they're even packaged at all, then building for every possible supported version of RHEL, Debian, Ubuntu, Fedora, SuSE, and down the line.
This is why "power users" aren't an important user model. Arch, Gentoo, Nix, and others exist if you want to be a "power user". Otherwise, extremely naive assumptions about the amount of effort expended by distro/package maintainers hand-wave the complexity away with "duh, just put it in Jenkins."
Flatpak/Snap are essentially a re-invention of putting things in /opt and using LD_LIBRARY_PATH or chrooting. Disk space is much less of a concern than it was before, and sacrificing a little space to AVOID the mess of shit above is a good investment.
Mac and iPhone apps don’t automatically update unless you tell them to.
The goal of systemd is to make everyone do everything the way Lennart Poettering likes it on his personal laptop. Perhaps some of it is nice but also some of it is not nice. And his holier and smarter than thou attitude is off putting and rightly so.
And don’t handwave away wasting my resources just so you can avoid work. That’s how we end up with Microsoft Teams.
The goal of systemd as an *init system* is not the same thing as the goal of some of the systemd umbrella projects, and they shouldn't be conflated. systemd as an init system is leaps and bounds ahead of sysvinit, openrc, and upstart for distro maintainers, large scale sysadmins, etc. No more need for supervisord, random scripts to flag off and on was part of VPN connection, convoluted "meta" scripts which carefully restart 5 different services in the same order and a huge mess of shell.
That said, no, Lennart did not/does not do things that way on his personal laptop. His position is that users shouldn't need to know how to configure dnsmasq to have a local caching DNS server, that 99% of the options for dhcpcd aren't used by 99% of users (who are perfectly happy to simply get an address in a fraction of the time), that most users don't need to know how to configure /etc/sysconfig/network-scripts/ifcfg-* or /etc/network/interfaces/* for their use case.
If you do, you can disable those things. You can think this is a good opinion or a bad opinion, but at least he's pushing towards some kind of solution which isn't "RTFM". If you think his ideas are bad, propose new ones. Start a project which does it better. "Just don't change anything" is not a meaningful or productive way to design software or operating systems.
Some interesting gaslighting here, complaints about the forced systemd ecosystem can be deflected by pointing at the init system. Which, just like the rest of the ecosystem, improves some parts and deteriorates others like the absolutely worthless journaling log replacement.
Anyway I’m not about to engage the systemd evangelization task force, thanks. Good luck elsewhere.
> This is why "power users" aren't an important user model. Arch, Gentoo, Nix, and others exist if you want to be a "power user". Otherwise, extremely naive assumptions about the amount of effort expended by distro/package maintainers hand-wave the complexity away with "duh, just put it in Jenkins."
Surely distros could be based on top of Nix with carefully curated sets of packages with less effort than it takes to package everything for Debian.
And the advantage of doing this over letting application authors maintain their own packages and deptrees with flatpaks/appimage/whatever is...?
They could be based on top of Nix. In practice, it's more likely they'd be based on top of rpm-ostree. But that doesn't do anything to close the gap between the wild west of copr/ppa/aur and "get your application accepted into mainline repos" for someone who wants to distribute their app.
Translating from Nix to another packaging ecosystem usually entails removing specificity. This is why going in the opposite direction is harder. So I suspect we can provide escape hatches from Nix into Debian/RPM/Arch/Gentoo so that one would only have to maintain one fully specific package, but get easy translators for the other ecosystems.
It worked fine? Did you ever apt install nextcloud-client? Then find out (after pulling your hairs out) that it doesn't work because it is some ancient (and I mean years old) version?
And this is just one example. Many devs warn you out right about package manager's versions on their websites.
But exactly this example got fixed! I maintained "inofficial" debian packages for quite some time, but since Debian bullseye came out `sudo apt install nextcloud-desktop` just works :-)
When Let's Encrypt changed the installation instructions to heavily recommend Snaps, I was quite disappointed. I'd been using their apt package for years already, across my fleet, and to have them suddenly change tack and all but disavow their apt repo made me question my choice in them for the first time.
Would their apt repo suddenly disappear? Probably not, but who knows.
They don't work fine when they're months and sometimes years behind the current versions. It's a huge maintenance burden for everyone--debian package maintainers, upstream maintainers fielding tons of confused user questions, etc.
This is what I like, as a user. When it works, and it usually does, it's pretty slick. I hear people saying it only works when things are new and fresh but when you try to install old packages or new packages on older systems, things break. I guess that's true, and that's unfortunate. But things break for me using Snap/Flatpak anyway, new or old. And I find it's easier to debug distro packages breaking than application images.
The developer experience on Linux I find is much better than Windows and macOS. But putting my 'user' hat on, the experience on Linux in general is still pretty poor, despite all the progress that's been made over the decades. And package management is one example of this. I don't think it's fair to ask users to work with a new paradigm of maintaining software, even for a "free" OS, and despite how hard it is on developers to maintain distro packages.
Moving away from Debian packages would probably be better. Packaging a .deb is an absolute chore, with a bunch of tools and configuration files, while packaging for Arch or NixOS is almost trivial by comparison.
When your break your fancy apt-architecture beyond repair aka You have held broken packages (whatever it means), you will be very thankful for snaps and flatpaks.
Thats a bug of apt... There should be an apt fix-my-system command that just looks at the state of everything, and figures out how to get everything back towards a working system.
And make sure it keeps a log of everything it did so it can be rolled back if it doesn't work as promised.
Aptitude does total fix-my-system. I have tried it several times. The suggested fix is always: remove everything and then reinstall the broken package. It does not even try to reinstall those packages it has just destroyed.
I never broke a Debian install beyond repair, and I've used it for a couple of decades. Even when doing some really off the charts stuff, including powering off a system mid dist-upgrade. Apt is really really solid and well documented/community supported.
sergio@sergio-laptop:~ > sudo apt-get install wine
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
fonts-liberation fonts-wine libcapi20-3 libodbc2 libosmesa6 libwine libz-mingw-w64 wine64
Suggested packages:
odbc-postgresql tdsodbc ttf-mscorefonts-installer q4wine winbind winetricks playonlinux wine-binfmt dosbox exe-thumbnailer | kio-extras wine64-preloader
Recommended packages:
wine32
The following NEW packages will be installed:
fonts-liberation fonts-wine libcapi20-3 libodbc2 libosmesa6 libwine libz-mingw-w64 wine wine64
0 upgraded, 9 newly installed, 0 to remove and 15 not upgraded.
Need to get 105 MB of archives.
After this operation, 700 MB of additional disk space will be used.
Do you want to continue? [Y/n]
No need to overcomplicate things by going multi-arch.
I found a better example. Only solution aptitude ever gives is not to install paprefs.
Please solve that.
$ sudo aptitude install paprefs
Keep the following packages at their current version:
...
11) paprefs [Not Installed]
...
Accept this solution? [Y/n/q/?]
Don't know your install, but Debian by default uses Pipewire, not Pulseaudio. If you are using Pipewire, indeed you can't install a Pulseaudio application. That's expected.
You can replace pipewire with pulseaudio. It's doable, just not trivial enough for an HN comment. In general, you'd have to force removal of pipewire without removing all dependent packages, install pulseaudio and then have apt check that all dependencies are ok. Then, having Pulseaudio in the system, you can install paprefs, a Pulseaudio application.
As I said, it's a solid system, rather impossible to break into an unrecoverable state.
There's work to do that since 8.0. Current version is 8.4 but I haven't been following, don't know the status. There's a HN thread about the 8.0 release with 201 comments.
They worked fine, and I don't feel like having multiple types of containers and update methods and mounted image file systems really made anyone's life better.