I had been working on Linux professionally for about 6 years at that point, and got burned out on it. I have a lot of thoughts on the success of Linux as a general-use platform but mostly keep them to myself these days.
I still have a deep love for graphics, and have since moved onto working in the games industry, where I am much happier. I also still have a passion for explaining graphics topics.
The basic problem with X11 is it wraps a whole bunch of things into one software package. So in theory you have to know about fonts, OpenGL, graphics drivers & memory management, the window model and networking simultaneously to configure and run the thing.
Wayland is a protocol for a well designed future where all that functionality is broken up into separate pieces of software. Wayland handles the window model and most of the other parts already live in their own software projects. The problem is that Wayland makes some changes to the basic window model on the way through that are a big deal and so far there is no great software written that uses it.
Adoption is very slow because the best case scenario is there a Wayland compositor that has feature parity with an X11 window manage and a bunch of weird problems cropping up because screen recording is a to-be-solved problem.
My position from the armchair is that Wayland really needed a complimentary official protocol for how to request another application's graphical output. Not coordinating that means that anything that supports Wayland is going to be practically inferior to an X11 solution. It'll get fixed sooner or later I expect, it isn't a technically hard problem.
The issue is I need the powerfull features, like I have a script that OCR a region of the screen when I press a button (the coordinates depend of the app that runs) so I do not want a GUI provided by the DE to let me capture the screen.
Anyway the issue with Wayland is that it is not ready, the important screen recording feature is yet not here,I seen also some complaints from gamers and their issues are a won't fix, probably when Wayland will be actually ready we will need something to replace it or a Wayland 2 that learned from first attempt mistakes.
Having recently moved from X to Wayland there are still some things missing sometimes but they're not many. If you want a DE at least GNOME and I think KDE as well are making the switch reasonably seamless. And there are some features that Wayland solves much more thoroughly (e.g., tear free output, fractional scaling). The network transparency part, that is often brought up but rarely used in X is also being delivered:
I suffer of macular degeneration https://en.wikipedia.org/wiki/Macular_degeneration , so I am having issues reading texts , I am using TTS and Zoom features for most applications but the big issue I have is with video games. For open source game engines I can patch them to have the text pushed to my TTS scripts but for closed source game engine OCR or zoom is the only solution but it has a lot of issues.
1 my zoom follows the mouse but this is broken is nome games where the trying to move the zoom area to the edge where the text is would move the camera in game , pausing the game is a solution but some games would put a Pause screen on top of te text I want to read.
The OCR has issues when games use fancy fonts, old games use some pixelated fonts that are impossible to OCR, other games use more artistic fonts that are also hard to OCR, other games use text over semi transp[arent background and it makes hard to separate the text from the background.
For DirectX I am thinking if I could maybe intercept the draw text calls, I am considering making a Stack OVerflow post but I am a bit anxious that I am not sure what the correct title and tags should be and I could get the off topic and the question closed.
Quoth the wiki 'PipeWire is a rather new multimedia framework by GNOME'.
There will be people who don't want to use GNOME frameworks and don't want to use the GNOME window manager.
Wayland is enforcing a policy ('don't access other devices graphics output') where it should be providing capability ('an applications graphics output is private by default but that can be configured'). However it looks like the design of Wayland purposefully omitted that sort of configuration from the protocol. So instead of having a security model that is sufficiently complicated to allow screen recording, the security model is 'No'.
To get a framework independent solution all the compositors would need to implement an identical API for accessing graphics output. That needs to be coordinated in the same fashion as the Wayland spec. As the spec stands, it is just asking for every framework, compositor or application to go implement its own different solution.
The Wayland designers may even have tried to avoid explicitly creating a security model, but in practice it comes with an implicit security model that is inflexible, unconfigurable and difficult to coordinate around.
From what I gather, Pipewire isn't part of Gnome, i.e. it's generic and can be used with any DE.
For example, all compositors are using libinput, which replaces legacy ways of X handling input. It's not mandated by the protocol either. So I don't see why they can't use Pipewire for audio and video needs.
If anyone wants to make a better alternative to standardize on - they are free to propose it.
But libinput is a freedesktop.org project, ie, supported by the same people who are working on Wayland. It was specifically built to support Wayland.
Pipewire looks like some audio-visual framework for GNOME and GTK apps and while it might be fantastic it is both new and not very comprehensibly advertised. And it will probably be very complicated and involve some rather interesting design issues compared to building something in when the Wayland protocol was created.
We're coming out of a world where the standard used to be "run graphics drivers in userspace!" so any weird compromise is workable (including an audio framework deciding on graphics security policy). But ultimately the idea that a GNOME-sponsored effort is going to be what sorts out the security model for desktop apps is weird. I can't find anything to suggest that Pipewire is intended to be the definitive way to do screen recording, it looks to me like that might be a happy side effect. It wouldn't be surprising if it turns into two or three competing ideas.
You're mistaken with pipewire, it's a daemon intending to replace pulseaudio at some point, it's not a gnome component.
Screen recording is also available in wlroot based window managers.
The wayland environment is maturing quite fast now.
So far video applications on Linux have accessed the
hardware directly, meaning that two applications can’t
access e.g. a webcam simultaneously. Solving that
problem, and screen sharing with Wayland, were to my
understanding the main motivations for creating a video
daemon.
So screen sharing on Wayland (that covers capture use case) was one of the main motivations behind it.
If both Gnome and KDE (the most widely used DEs) are behind it, I'd say it's already a major consensus on where things are going.
From my perspective, Wayland "fixes" things that could allow bad things but were also useful. For instance, screenshots/recording took a long time to be implemented, and is still constrained, because of the security risk of arbitrary programs recording the screen. Because that is a risk... but users still want it. As another example, I use xdotool to inject input to programs in X; in Wayland, that would need to be explicitly supported per-compositer.
Well, removing or not implementing such features because of security is always a big fat sign of something sneaky or even worse, outright incompetent going on. As such security risks of features can always be addressed through some sort of authorization mechanism.
I'm running Sway WM right now, a Wayland compositor that implements an i3-compatible window manager, and in fact working on the code (hopefully merging my first contribution soon.)
I'd say the biggest issue is really maturity. I actually find Wayland to be a surprisingly nice piece of software engineering. I don't know if I like the architecture really, but it comes together fairly nicely in the end.
For example, there's unanswered security questions. It would be good if applications couldn't just go and manipulate and spy on other apps like they can with X11. Wayland indeed solves this issue, but it leaves other problems. How do I capture the desktop?
wl_roots offers some extensions that help alleviate the problem, but they don't solve the full problem; for example, you can capture the screen, for which programs and an OBS plugin exists, but there's no upstream Wayland extension that can make WebRTC desktop capture work. I think what we really need is the ability to integrate policy and consent requests, but I'm not sure whether there's progress on that front, or how long it will take. Once the problem is solved, upstreamed to WebRTC, and that upstream version trickles down to Electron apps, Firefox and Chrome stable versions... well, things will be in a better place.
Speaking of web browsers, today Firefox still has some issues to go before it is fully working in Wayland natively. You can do it today, but it is not perfect; there are sometimes places in the code where it assumes it is running on X11, and things simply break. The screenshot tool is just flat out not present when running under Wayland due to the above issue.
Applications are slow on the uptake too. XWayland covers most of them, but it is not perfect. Like anything that captures the screen, of course, is not going to work under XWayland, but also, XWayland apps may behave a little off. In Sway WM, one bug is that when the mouse travels from an X window to a Wayland window, sometimes it behaves as though the mouse is still hovering over the X window at the last position it touched, which leads to tooltips showing up where they shouldn't. There's still a decent amount of GTK2 apps, and even Qt apps aren't all Wayland yet; Krita plans to start working on support I believe next year.
In my case, here's the itch I'm currently scratching: Sway WM also does not have support for drawing tablets. I found this out, by, well, loading up a drawing program and noticing the lack of pressure sensitivity. So, that's what I'm currently working on. Needless to say, this is a non-starter for anyone that relies on a graphics tablet, although if you are just using GNOME 3 it is unlikely to be an issue since I'm sure GNOME 3 supports graphics tablets just fine.
NVIDIA drivers are still a pain in the neck. I gave up and bought an AMD graphics card, and frankly I can't recommend it enough. Basically everything works better on Linux, but certainly Wayland works better. It's worth giving up some raw power. Still, for many people the proprietary NVIDIA drivers are going to need to work, and today support varies from lackluster to poor depending on what compositor you use.
So Firefox supports screenshots when running on the Gnome compositor, but not running on sway? Looks like we're heading into interesting times where applications are getting tied to certain DEs with little to no cross-compatibility.
IDK. Sway is my daily driver. I just checked Firefox, and the screenshot tool works as expected (including clipboard). So unless GP is talking about capturing non-Firefox windows with that tool, I do not have that issue.
This Firefox instance is running on native Wayland. The only issue I have with it is sometimes the coordinates for the context menu are wrong.
But yeah, there might be compat issues, if some protocols are required for stuff. But Wayland itself is the baseline: you'll always be able to draw windows and interact with them.
What's left to extra protocols isn't guaranteed to work: control of the compositor with a client, for instance (color correction, virtual keyboard, screenshots, window placement, screen geometry, window decorations, etc.) is completely up to the compositor (and the protocols it can speak with the client). So is other stuff, like graphics tablet input (with pressure sensitivity, tile, etc), minimizing windows, window miniatures as used for KDE's taskbar as well, I think.
But then, it's not like everything worked flawlessly everywhere on X either :)
And as long as the baseline is solid, we can build on it.
I may have done something wrong. I am not highly knowledgable here. For example, I just learned of Pipewire.
That said, the problem occurred for me when using the NixOS package firefox-wayland. It is possibly better to use upstream firefox with Wayland, but I have not tried; I am just running firefox on Xwayland for now.
I think capturing desktop and screenshots should be addressed by standard Wayland extensions with integration with something like Pipewire as far as I know. I didn't follow the progress of that though.
As for Nvidia - don't waste your time on their proprietary drivers on Linux. They don't to play by the rules (refuse to upstream), so no need to spend time supporting their blobs.
Like I said... I gave up and bought an AMD card. Not everyone is going to do this. It's a step down from my NVIDIA card.
I think there is still hope. Obviously today's reality is still bleak with signed blobs and plenty of question marks, but I think open-gpu-doc is a sign that NVIDIA does actually want to square away this problem, maybe they're just not sure how.
I do believe NVIDIA is mostly responsible for Wayland's poor adoption, and I firmly believe that this also lead to Wayland not getting as much developer attention as it could've gotten. The fact that one company wields this much power is a serious problem. I kind of understand the hostility Linux kernel developers seem to have towards proprietary drivers under Linux now.
Kernel developers are right to push back against blobs. Nvidia is the bad player here.
AMD is doing fine though. I have RX 5700 XT, and besides for some rough edges due to it being very new and drivers still fixing bugs, it's a very good card.
I expect Nvidia usage on Linux to plummet, due to their refusal to provide open drivers. It's already happening, just will take some time for it to drop to the point where they have little influence.
Personally I’m not the biggest fan of wayland (I haven’t found a compositor I like and using OpenGL for everything including VTEs feels like a mistake.) but I will say it’s common knowledge (as far as I’m aware) that nvidia drivers on Linux are pretty bad and you’re not going to have a good experience with them no matter what kind of display manager you use.
In what world are nVidia drivers crap? I'm regularly doing some pretty funky stuff with OpenGL on top of X11 and nVidia and this setup fails me less often and less hard than any equivalent AMD setup has done so far. Unless I'm trying to allocating tons textures or ridiculously sized vertex buffers, I don't see any hickups at all.
In the Linux world. All of their problems (roughly you can call them poor integration with the Linux stack) are consequences of them refusing to upstream their kernel driver.
The fact that Nvidia blob fails you less is not a good indicator for you, unless you test with Mesa as well in each case. Mesa is a lot stricter about OpenGL spec compliance. So if you only test on Nvidia, you likely have bugs that you didn't notice.
Every time I tested with Mesa and had issues, it turned out that Mesa was out of spec and quite specifically lying about that fact. And this was with absolutely essential features like rendering into floating point textures. So no, Mesa is not an indicator for spec compliance, either.
So when was it exactly? I doubt recently. Mesa progressed to the point of full compliance a few years ago. Today, it's trusted a lot more than the blob, due to the later silently swallowing errors frequently and messing around with substituting shaders behind the scenes.
Try testing current Mesa versions. Previous experiences are no longer relevant.
> Sway WM also does not have support for drawing tablets. I found this out, by, well, loading up a drawing program and noticing the lack of pressure sensitivity. So, that's what I'm currently working on.
Hey, thank you for working on this! The other day I plugged a tablet in, and found myself quite dumb with it not working properly. I wanted to have a go at implementing the tablet v2 protocol under wlroots, but it sounds like I'll hack on something else : P
It has been an interesting exercise so far, and frankly a quite pleasant one. Rather than autotools or CMake, many of these projects are using Meson, which is just so much nicer. I also really have learned to appreciate Sway WM's code, which I personally think is quite nice now, after having worked with it (though, admittedly, the ease of segfaulting does make you long for something like Rust.)
wlroots already has first-class drawing tablet support, but the "process input events" parts and the "dispatch input events to clients" parts are separated and it's up to the compositor to connect them in the ways they best see fit.
In the past that was a common reason. Today it's not relevant anymore. And downsides of the blob make buying Nvidia a non starter. You can clearly observe this trend today among Linux users.
For Windows users coming to Linux with Nvidia, this usually simply involves one extra cycle of GPU upgrades. By the next one, it's AMD for them. So in the long run - still not an issue.
I understand X11 has a lot of unfixable problems, but what's wrong with Wayland world?