What’s the future of screen capture on Linux? What’s holding it back? What’s pushing it forward?
A version of this question that might be more accurate/holistic: Do you think there’ll be a standard, high-performance desktop API in our lifetimes?
It feels like there isn’t the same market pressure like there is for Valve and Proton that’ll make a good capture protocol appear. I’d love to eat my words, but I’d also easily bet one trillion dollars[1] that I’ll be dead before anything dethrones GPU vendor extensions.
tangential additions / context for the terminally bored:
For example, Nvfbc+Nvenc is a must-have for game streaming with Moonlight and ALVR (not sure if AMD has anything like that), but it’s my poor understanding that this type of GPU-only framebuffer capture isn’t possible on Wayland for security or other reasons, which is sadly keeping me on X11.
Windows probably has awesome screencapture APIs given that the NVIDIA Capture SDK documentation cites it as why they removed Nvfbc from their Window’s drivers but not their Linux drivers.
The proof that Windows reaped the rewards of unifying all their stuff can be seen in the Lossless Scaling-likes that are on the market (Magpie, Integer Scaling, etc.) while all Linux has is like Gamescope and vkbasalt [2], so apples and oranges.
I dont see anything in your post that isnt already possible on Linux. OBS and GPU-Screen-Recorder both work great, I dont know what features could be needed that they dont offer.
Gamescope is capable of upscaling games with FSR. It doesnt support as many upscaling options as Magpie, but clearly the capability is there and not restricted by anything inherent to Linux.
I’m confused, OBS works well. What is missing you are desiring?
Yeah the protocols are there, portals support it, OBS works. It is great that random applications cannot capture the framebuffer.
The screencast portal has been around for 7 years. How is it not enough? It is very much GPU-only (if the receiving program supports it, which nearly all do), and encoding the image is up to the app and does not depend on the API.
I’d say wl-screenrec (https://github.com/russelltg/wl-screenrec) shows the best screen capture performance on Linux. I guess it could be even better since Wayland has released newer protocols and newer versions of the protocols used in wl-screenrec.
I am using OBS Game Capture plugin and it works perfectly on Wayland. I have yet try it with Wine Wayland, though.
AMD has an nvdec/nvenc equivalent called AMF, on Linux it’s going to be deprecated in months in favour of va-api.
To my knowledge, it does not have an nvfbc equivalent. Which anyway, Nvidia has deprecated on windows in favour of a windows-native screen capture with a name I don’t remember.
For what is worth, va-api encoding + kmsgrab works pretty well for me, it does have some latency, but nothing too unacceptable. Probably less than the one caused by the Bluetooth controller. And none of this is vendor specific, you can get it working on Intel, AMD and Nvidia (Nvidia needs a compatibility layer, but it works). Also, it works on Wayland, but sunshine needs some privileges to work.
Sunshine supposedly supports nvfbc with patched Nvidia drivers, even on Linux, I haven’t tried it, so I don’t know if it works on Wayland. I don’t see why it shouldn’t, as long as you give sunshine privileged permissions (like you need for kmsgrab). Even without nvfbc you can use nvenc, so you don’t need the va-api compatibility layer.
Supposedly, since this Nvidia driver release nvfbc is used as backend for pipewire screen capture, so it should just work for apps like OBS, I don’t know if sunshine has intention to move to it.
In general, screen capture on Linux pretty much works, even on Wayland. The general sentiment that it’s broken is actually old news.
There’s a caveat though. Proprietary apps tend to use outdated stuff (e.g. electron builds from 5 years ago) and thus don’t support screen sharing on Wayland.
