I’m honestly surprised Immutable distros is so controversial. I get why people choose not to do it, but I don’t know why I see so much hate towards them in a lot of Linux communities.
SteamOS is immutable and incredibly successful. macOS (not Linux of course) is also immutable and very successful.
As long as the OS’s have a concept of overlays, an immutable system rarely gives up much in the way of flexibility either.
macOS hides its immutability pretty well, with fine grained image immutability and keeping the behavior mostly unchanged.
Immutable Linux distros, esp. NixOS pull all kinds of shenanigans (ELF patching to begin with) to achieve what they want to achieve with a complete disregard how they change the behavior and structure of the system. What you get is something which resembles a Linux distro, but with monkey patches everywhere and a thousand paper cuts.
When a Linux distro can become transparently immutable, then we can talk about end user adoption en masse. Other than that, immutable distros are just glorified containers for the cloud and enthusiast applications, from my perspective.
> but I think that’s true of any aspect of a distro.
That's true, but these problems are worked on for a quite long time, and the core ethos of a Linux distribution is being able to be on both sides of the fence (i.e. as a user and as an administrator which can do anything).
For example, in macOS, you can't customize the core of the operating system since the eternity, and it's now sent you as an binary delta image for the OS, and you have no chance to built these layers or tweak them. This increases the ergonomics a ton, because you're not allowed to touch that part to begin with.
However, with Linux, you need to be able to change anything and everything on the immutable part, and this requires a new philosophy and set of tools. Adding esoteric targets like "I really need to be able to install two slightly different compilations of the same library at the exact same version" creates hard problems.
When these needs and the mentality of "This is old, let's bring it down with sledgehammers. They didn't know anything, they're old and wrinkly people" meets, we have reinventions of wheels and returning to the tried and working mechanisms (e.g.: Oh, dynamic linking is a neat idea. Maybe we should try that!)
Immutable systems are valuable, but we need less hype and more sane and down-to-earth development. I believe if someone can sit down and design something immutable with consideration to how a POSIX system works and what is reasonable and what's not, a good immutable system can be built. Yes it won't be able to do that one weird trick, but it'd work for 99% of the scenarios where an immutable system would make sense.
> However, with Linux, you need to be able to change anything and everything on the immutable part, and this requires a new philosophy and set of tools.
Taking macOS as a North Star as a successful mutable Os,
most people don’t need to, and shouldn’t be, touching the immutable parts. I know the assumption is that you need to do that on Linux, but I don’t think a successful distro should require most casual users to go anywhere near that level of access.
If they do for some reason on Linux specifically, why would overlays not be sufficient? They achieve the same results, with minor overhead and significant reliability gains.
> I believe if someone can sit down and design something immutable with consideration to how a POSIX system works and what is reasonable and what's not, a good immutable system can be built.
But someone has done that. macOS is POSIX. SteamOS just works for most people.
I think there are a couple of problems with taking macOS as the so-called North Star of the immutable OSes.
First of all, macOS doesn't have package management at the core OS. You can't update anything brought in as the part of the OS from libraries to utilities like zsh, perl, even cp. .pkg files bring applications in to mutable parts of the operating system, and can be removed. However, OS updates are always brought in as images and applied as deltas.
In Linux, you need a way to modify this immutable state in an atomic way, package by package, file by file way. That'd work if you "unseal, update, seal" while using EXT4 FS. If you want revisions, you need the heavier BTRFS, instead, which is not actually designed for single disk systems to begin with.
You can also use overlays, but considering the lifetime of a normal Linux installation is closer to a decade (for Debian, it can be even eternal), overlays will consume tons of space in the long run. So you need to be able to flatten the disk at some point.
On the other hand, NixOS and Guix are obsessed with reproducibility (which is not wrong, but they're not the only and true ways to achieve that), and make things much more complicated. I have no experience with RPM-OSTree approach.
So, if you ask me we need another, simpler approach to immutable Linux systems, which doesn't monkey patch tons of things to make things work.
> But someone has done that. macOS is POSIX.
Yes, but it's shipped as a set in stone monolithic item, with binary delta updates which transform it to another set in stone item. You can't customize the core OS. Can you? Even SIP sits below you, and has the capability to stop you with an "Access denied." error even if you're root on that system. It's a vertically integrated silicon to OS level monolithic structure.
> SteamOS just works for most people.
SteamOS is as general purpose as a PlayStation OS or car entertainment system. It's just an "installable embedded OS". Their requirements are different. Building and maintaining a mutable distro is a nightmare fuel already, making it immutable yet user customizable is a challenge on another level.
It's not impossible, it's valuable, but we're not there yet. We didn't find the solutions, heck even can't agree on requirements and opinions yet.
Even though macOS won’t let you replace the actual binaries that ship with the OS, you can still replace the binaries that get resolved when they’re called via brew/nix/macports etc.
I again disagree with your assertions that you need to replace the OS contents. You just need to be able to reflow them to other ones. That’s the macOS way, the flatpak way etc..
I think the issue is that you are claiming that you MUST be able to do these things on Linux, and I’d push back and say no you do not.
And your comparison of SteamOS to a console or in car entertainment is flat out incorrect. Have you never booted it into desktop mode? For 90% of users, what are you expecting they need to do in that mode that it’s not “general purpose” enough to do?
Yes, it’s not impossible. It’s been done multiple times , successfully as well.
> you can still replace the binaries that get resolved when they’re called via brew/nix/macports etc.
Yes, but this doesn't make my point invalid and irrelevant. $PATH overriding is as old as this planet itself, and its primary purpose is not to make immutable systems more user-friendly. It's just a way to use your own binaries if you want to override itself.
Being able to use flatpak, etc. on a system doesn't and won't allow you to change the core OS. Being able to reflow things doesn't solve anything about system management, esp. on the long run.
> I think the issue is that you are claiming that you MUST be able to do these things on Linux, and I’d push back and say no you do not.
I'll politely won't move and say that it depends. If you're building appliances or embedded systems, that might be discussed if there's an absolute requirement to remove the update mechanism, but for a server and end-user system which can live more than a decade, this is a core requirement which can't even be discussed, full stop.
> And your comparison of SteamOS to a console or in car entertainment is flat out incorrect.
I don't think so.
> Have you never booted it into desktop mode? For 90% of users, what are you expecting they need to do in that mode that it’s not “general purpose” enough to do?
No, but even Valve doesn't say what you say. What Valve says is "Users should not consider SteamOS as a replacement for their desktop operating system. SteamOS is being designed and optimized for the living room experience." (emphasis mine)
[0]
Its desktop mode is akin to Browser sessions present in consoles, or "Other OS" which used to be in Playstation. IOW, a nicety, not a feature.
> Yes, it’s not impossible. It’s been done multiple times , successfully as well.
Yes, to answer different set of requirements with set of different features. We still have a long way to go with "An immutable, daily driveable, desktop and server distribution which can be maintained, used and updated for a decade" (a-la Debian).
For a couple of years now, macOS has an RO System volume image, and then mounts an RW data volume on top of that, similar to how overlayfs works on Linux. That system volume isn't modifiable. If it is, then the system won't boot.
So I'd say it's a little bit immutable.
Uh... yeah it is. Have you ever switched it into desktop mode? I haven't pushed my Steam Deck as hard as my daily driver Linux system, but I've done all sorts of fun things on it like run a web server and write new Python scripts directly on the device. You can hook up a keyboard and mouse and monitor and use it like any other desktop Linux environment. It's basically just an Arch distro with KDE and some extra stuff on top to make it easy for people to run games.
For a long time Windows has dominated PCs, and all the software that run on it come packaged in their own little installers and with their own little updaters to manage versions, leaving users free to not care and focus on just using them. For us old desktop users, Linux's "enforced" centralization of software packaging and distribution is just too divergent of a concept to get immediately used to. Immutable distros take the restrictions even further, and they make one think you might as well just have an Android/Chromebook at that point.
I switched to Ubuntu after witnessing Windows 11 and am seeing there's now yet another confusing delivery channel (snap) added on top of what was already an overcomplicated system (apt). At least it still allows single installers (.deb files) so that works for now.
I agree, in a sense. Windows has had mostly .exe (.msi) packaged installers that run their own install scripts to place files and modify the system to run the software. The software vendors all used the same format. Installing software was a practised exercise of double-clicking on a installer.exe file and then hitting next a bunch of times.
OSX (before the App Store model) had .dmg disk images and installer files, where you drop the entire application into a folder and that .app file contained everything the application needed to run. Easy enough to install applications.
But, what happened when you wanted to update a program or application? You went an manually downloaded the application or found the update button in the context menu and updated. This meant that many users would not update very often. Because the process was manual and because you were never sure what would work or when the new version would refuse to work with your hardware or OS version. Even with an app store, you still need to open the store app and click update or look through menus to find which software packages need updating.
For a long time, as both a Windows and OSX user, I saw the benefits of both approaches. Now, as a Linux user, I can update my entire system, with one bash alias I can update my chosen installed applications, but also my operating system files, and my flatpfak apps or containers and even my firmware (fwupd). It has changed the paradigm of computing for me. It makes me feel like a superhero. I don't have to worry about an update breaking my system. Better still, I don't have to manually update my operating system separately from each application I've chosen to install. I can do it all from the terminal and I can see all of the changes my system will make. It has been a great experience. Do things sometimes break? Yep. But, they broke on the other OS's too. I take it as a learning experience.
Maybe I've just talked myself into trying an immutable os.
/ being mounted ro with /etc and /home being mutable... kind of ruins the point? like you can still mutate whatever, you just have to install it as a user, and if you have overlays then what gain is there?
The advantage is you always have a core system that you can revert into by removing any problematic overlays, and you can always quickly verify that a system is in a verified and unmodified state.
This is how macOS works with SIP, and how it handles rapid response updates for example.
It greatly reduces the ability for user space to compromise the system.
Ah, so malware can no longer manipulate boot state, just steal all of your passwords and credit cards and cryptocurrency and make user-level persistent processes.
> Ah, so malware can no longer manipulate boot state,
Which is an immense benefit.
> just steal all of your passwords
2FA, often now thanks to a HSM (Hardware Security Module) shielding your secrets precisely should your account be compromised (Yubikey, passkeys, ...)
> and credit cards
2FA. My credit cards companies (EU) ask me to sign on a physical hardware device the bank gave me any transaction I make with my credit card when it's either above a certain amount of to an unkown vendor (or both).
> and cryptocurrency
2FA. Cryptocurrencies hardware wallet use an HSM which shields the secret from attackers.
> and make user-level persistent processes.
Which you can detect from root, but only as long as root ain't compromised too.
A local exploit which can be detected and patched is bad but it's not anywhere near as bad as a root exploit which could potentially control the entire boot chain (maybe not SecureBoot if it's setup properly) and lie to you about everything.
Put it another way: it's precisely because a local exploit is not a root exploit that a system can be configured in such a way that should a local exploit happen, the system can make sure that that local exploit doesn't get to stay persistent.
A non root exploit cannot lie to root, which is why there's a distinction between a local exploit and a root one.
Now we begin to have the possibility to boot a minimal immutable Linux distro (maybe even from a read-only medium like a DVD [1]) , maybe from a UKI and a signature enforced by SecureBoot, and from that minimal immutable system, maybe launch something like a VM and/or containers (I prefer my containers to run inside VMs but YMMV).
For example we can begin to envision the following:
SecureBoot -> signed UKI -> Proxmox -> VM -> stateless containers
I am very excited that this now begins to be possible.
Don't you see any value in that?
I don't run an immutable distro yet but I already have throwaway user accounts, mounted on temporary and "noexec" mountpoints.
If you tell me: "Here's a system where it's guaranteed a malware can never ever manipulate boot state", I'll manage to find a way to build a system on top of that where local exploit cannot possibly persists.
If that’s your view, then why have any security at all?
Why not just go back to the days when every process could access each others memory?
Let’s just let every process run as root too while we’re at it, right?
Heck, you can still crash a car, so why wear seatbelts or have airbags?
This is the problem with strawman arguments like yours. They’re not rooted in reason and extrapolate infinitely.
Imperfect safety is better than no safety. More safety is better than less safety.
It also completely ignores that you can have different approaches to security for different parts of any system. You don’t just have single silver bullets
yeah, it have lots of advantages, and that's why it was the default decades ago for everything. (windows, bsd, etc)
then people would have lots of trouble installing different software or updating security issues. so we invented package managers and took all the time in the world to make the base as small as possible.
it's advantages still make sense in some places, like modems with old flash memory. openwrt is static base with overlays. but again, it still carries the same downsides but because of the different aspects of the hardware it makes sense.
it would make sense for tech illiterate end users (hence android, ios, chrome os, wii, macos to a degree, etc) and containers (which already have infinite ways to convert from package to static). but anything else will literary harm the distro ability to evolve and adapt to software changes. imagine every change like systemd or a new browser or wm being atomic.
now people forgot decades of history. and it's so tiring.
I don't think I understand any of your objections.
When was Windows ever immutable in the sense of current immutable Linux distros? I wasn't able to find any reference to this ever being the case.
What do package managers and making the base as small as possible have to do with immutable distros? Package managers still exist, and the base is pretty much the same size as the non-immutable version of the same distro.
Why do immutable distros make more sense on modems with old flash memory?
How does being immutable harm the distros ability to evolve?
Either I'm not understanding your position at all, or you have a very different understanding of "immutable" than I do (after using Kinoite as my daily driver for a year).
> but anything else will literary harm the distro ability to evolve and adapt to software changes.
Comparing nixpkgs to the AUR seems to reflect the opposite trend. Arch is hamstrung by a dependence on outdated and poorly maintained repos that cannot coexist with up-to-date packages. Unless you fully embrace static linking or atomic installs, you'll end up with breakage.
MacOS went the static linking route, and Windows wrote back-compat for most old software by hand. The "decades of history" hasn't proven any approach right or wrong. They're all flawed.
macOS does not go the statically linked route. Apple encourage developers towards dynamic libraries (usually in the form of frameworks) in most scenarios.
Can you share an example of when Windows was immutable? I’ve been using windows since 3.1, and I can’t recall a Windows version where I wasn’t able to muck around in the system itself. Closed source != immutable.
On an unrelated note, I despise the constant insinuation that using Linux is an indicator of intelligence whereas users on other systems are tech illiterate.
Actually, I believe your entire argument contradicts itself multiple times because you give examples on both sides, and don’t stop to reconcile your views.
SteamOS is immutable and incredibly successful. macOS (not Linux of course) is also immutable and very successful.
As long as the OS’s have a concept of overlays, an immutable system rarely gives up much in the way of flexibility either.