I'm surprised that microkernel-like IPC still hasn't found its way into any *NIX system. The closest we've gotten is System V IPC, which has never really taken off.
Is this just difficult to design well or are people genuinely okay with socket(AF_UNIX, SOCK_SEQPACKET, 0)?
I don't think it's difficult to design an IPC mechanism. But I DO think it's difficult to design an IPC mechanism that everyone is happy with. Some people want only synchronous IPC, others also asynchronous (out of order responses), some want multicast, some want events without associated request, some want pub/sub, etc.
If at the end people continue building their own IPC mechanisms on top of TCP/IP or unix domain sockets then the in-kernel mechanism will just be another thing to maintain.
There had been some endaveurs to bring new IPC mechanisms into the Linux Kernel (AF_BUS, KDBUS, BUS1). I think those failed for similar reasons (although I'm not sure were BUS1 now is - the others are definitely discontinued).
You can go all the way from the lowest level kernel uses to very high level application constructs with that. Re-inventing existing wheels badly is something the software world excels at.
Not quite. Those functions need established connections, so you need ConnectAttach and ConnectDetach as well. But none of that is useful unless you can identify clients so you also need ConnectClientInfo.
This isn’t a dig, having worked on a custom IPC system I found the QNX approach to be the best of all worlds.
> But I DO think it's difficult to design an IPC mechanism that everyone is happy with. Some people want only synchronous IPC ...
Every form of IPC can be implemented on top of asynchronous message passing. Interface is not the problem. The problem is high performance designs with all the batching, memory mapped buffers, no syscalls, etc.
> Every form of IPC can be implemented on top of asynchronous message passing
Sure, if you want to introduce inherent DoS vulnerabilities into your IPC subsystem, not to mention slow down IPC so much that it's practically unusable. Many early microkernels were asynchronous, and synchronous microkernels like L4 beat them easily every time.
> Sure, if you want to introduce inherent DoS vulnerabilities into your IPC subsystem, not to mention slow down IPC so much that it's practically unusable.
This is nonsense. There could be DoS vulnerabilities in implementations, but they are not inherent to async message passing.
Yes they are. Who owns the buffers needed to store the async messages?
1. If they're booked to the receiver, then clients can easily DoS receivers by flooding them with message.
2. If they're booked to the sender, then receivers can easily DoS senders by blocking indefinitely.
3. If they're booked to the kernel (which is most common for true async message passing, unfortunately), then senders or receivers can DoS the whole system by the above two mechanisms.
And that's only the most basic analysis. I suggest you read the paper I linked and its references if you want a more in-depth analysis of IPC vulnerabilities and performance properties.
You are over simplifying it and dropping too many important details so that it ceases to be a real system and becomes some weird system susceptible to DoS.
In high performance scenario it would be more like this: process shares fixed ring-like buffers with the kernel where it can put messages, messages it can't put there it either accumulates locally until it can or just drops, there would be some kind of polling or event notification mechanism to know when it can put and get more messages into and from shared buffers.
P.S. I can't access the paper, but presumably they are making the same faulty assumptions if they claim the same things you did.
No, you are oversimpifying by assuming mutual trust between processes, which is not a suitable assumption for a system-level message passing system.
> process shares fixed ring-like buffers with the kernel where it can put messages
I am making a claim like "Turing machines can't solve the Halting problem", and you are saying, "If you put a limit the number of computation steps, then the Halting problem is decidable". But such a system is no longer a Turing machine.
What you are describing is not asynchronous IPC. With async IPC, you ought to be able to send a message at any time without blocking. That's what async IPC means.
If you must sometimes block or throttle before you can successfully send a message, even if only in principle, then it's no longer async IPC. It is instead a mixed sync/async system, which invariably becomes necessary in order to address the inherent limitations of async IPC.
> messages it can't put there it either accumulates locally until it can or just drops
So DoS against the sender, like I said. Try assuming a less liberal threat model and see how far async IPC takes you.
There is no assumption of mutual trust between processes, and there is nothing synchronous about it, you still don't know whether any of the messages reached their destinations. This is just backpressure, asynchronously propagated (or synchronously, depending on your interpretation of it). And still no DoS, it's completely up to the application to decide what to do with the messages it generates too fast.
> This is just backpressure, asynchronously propagated
The need to handle back pressure is exactly why it's not pure async IPC.
> And still no DoS, it's completely up to the application to decide what to do with the messages it generates too fast.
And if the program can't discard messages, then it's a DoS. If the program can instead rely on the receiver to keep up so it doesn't need to make this choice, then there's a trust assumption between these processes.
> And if the program can't discard messages, then it's a DoS. If the program can instead rely on the receiver to keep up so it doesn't need to make this choice, then there's a trust assumption between these processes.
It doesn't work like that and is getting into hypothetical non real world systems again. Trust is especially interesting in this context, because if you don't trust other processes, you absolutely have to be able to discard their messages. They can misbehave, crash, stop responding at any time.
But say somehow you can't discard messages and don't trust them. It's still only about backpressure handling. For example, kernel can just refuse to send messages to a particular recipient it knows is not consuming its incoming messages and instead can return messages back to senders into their incoming buffers or rejected buffers or whatever. Senders can decide what to do with that information, wait for a particular recipient to become ready again, waiting is not DoS, stop generating messages for that recipient or accumulate them or just drop them, minimal cooperation is required of course, but not trust, if they don't cooperate they don't hurt anyone but themselves. It's all still pure asynchronous stuff. And in fact, all high performance real world asynchronous communications deal with backpressure all without DoS and trust, they all also can discard messages though.
It's difficult to tack on to an existing kernel syscall interface. You really want a capability based interface to keep most of the permission checks out of the data plane. And even then modern microkernel IPC goes to extreme lengths. For instance the L4s tend to do crazy stuff like not save all of the registers, but make sure to only clobber the registers that can't be arguments for the call itself. You might be able to tack that onto the front of the syscall interface sort of like how objc_msgsend works, but it'd be a huge pain.
Is this just difficult to design well or are people genuinely okay with socket(AF_UNIX, SOCK_SEQPACKET, 0)?