Hello everyone,
As a recent convert to KDE (thanks to the Steam Deck!) and a CachyOS user, I’ve been transitioning my workflow toward a Wayland-native environment. One of the biggest hurdles I encountered was establishing a reliable RDP connection from a Windows 11 client to a CachyOS-based KRDP server.
After significant troubleshooting, I’ve managed to get the native Windows Remote Desktop Connection (mstsc.exe) client working perfectly. I wanted to share these findings to help others bypass the “protocol mismatch” errors and negotiation failures that currently lack deep documentation.
The Setup
-
Server: Beelink Mini PC (Ryzen 5 5800H) running CachyOS (KDE Plasma 6.5.5 / Wayland).
-
Clients: Windows 11 Pro (RX 7800XT), Android, and a CachyOS Laptop.
Summary of Client Compatibility
-
CachyOS (Linux): KRDC and Remmina work out of the box with no issues.
-
Android: The native Microsoft client currently fails due to H.264 decoding limitations. I recommend using aRDP as a stable alternative.
-
Windows (Native): Requires specific configuration changes to handle the H.264 stream correctly.
The Fix: Enabling Windows Native RDP Support
If you are receiving protocol errors or connection refusals from Windows, follow these three steps:
1. Disable Hardware Accelerated Decoding It appears that certain Windows configurations (in my case, using an RX 7800XT) struggle to negotiate the H.264 stream from KRDP via hardware.
-
Open gpedit.msc (Local Group Policy Editor).
-
Navigate to: Computer Configuration > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Connection Client.
-
Enable the policy: “Turn off hardware-accelerated decoding”.
-
Run gpupdate /force in a Command Prompt to apply the change.
2. Adjust Experience Settings In the Windows Remote Desktop Connection client:
3. Credential Handling KRDP can be sensitive to how credentials are passed.
-
Enter your Username in the client, but do not save the password initially.
-
Ensure “Allow me to save credentials” is checked.
-
Manually enter the password only when the Windows security pop-up appears during the connection attempt.
Additional notes
Third-party clients like Thincast worked immediately. Guacamole on docker only works with guacd package built with h.264 support included. I am using mwilson2764/guacd-h264 - Docker Image .
A note on Hardware Encoding: While my Beelink’s Ryzen 5800H supports H.264 hardware encoding (verified via Jellyfin/FFmpeg), KRDP currently seems to default to CPU encoding. If anyone has insights on forcing VA-API or hardware-accelerated encoding within the KRDP server environment, I would love to hear your suggestions.
A thank you to the Devs: The progress made on KRDP is fantastic. We are closer than ever to a seamless Wayland remote desktop experience. My only hope for the future is expanded documentation to help bridge the gap for users transitioning from X11 or Gnome.
I hope this helps anyone else struggling with “Protocol Mismatch” errors!
Disclaimer: Post rephrased using AI. Re-posted on Cachyos forum: Troubleshooting KRDP: A Guide to Connecting from Windows Native RDP Clients - General - CachyOS Forum
5 Likes
@krunchynug8 Thanks for compiling this information, super useful. I was finally able to get connected up to my KRDP server using the Windows remote desktop client thanks to this.
I’ve been looking forward to more KRDP improvements and/or guides like this to bridge the gap. My solution up until this point was installing Windows Subsystems for Linux, installing Fedora, and running KRDC via wsl.exe to be able to connect up to my machine running KRDP. Not very elegant, but at least it worked.
Thanks for this, I was finally able to get connected after finding this post (actually found the one on the CachyOS forums - but this seemed like the better place to reply. I didn’t need 1. or 2., but 3. got it to finally let me connect.
Windows App is replacing Microsoft RDP. RDP is still installed on my Windows 11 Pro. I have KDE installed on Ubuntu Studio. Ubuntu Studio is 26.04. Windows RDP can connect to KRDP using this article. How ever I cannot connect using the new Windows App. This is on my local network.
I looked in the windows registry, not allot there. But I am not sure what to look for. I know the IP address is correct, and everything works with the RDP. Just not windows App.
Does anyone have any ideas?
I was running into this issue, and I finally got it to login with some important qualifiers. This was using the standard Microsoft RDP client.
First of all, like the above says, I specifically needed to check the box to “Allow me to save credentials”. However, I also needed to actually check the “Remember me” option at the login prompt.
In other words, I haven’t found a way to have it prompt me each time for the username/password from a saved RDP file, which would be the ideal.
To break this down, if I don’t check the “Allow me to save credentials” option, it fails to login. But if I do check “Allow me to save credentials” but then don’t check “Remember me” at the username/password prompt, any subsequent RDP sessions will automatically uncheck the “Allow me to save credentials” option again, even if I’ve deliberately saved an RDP file with that box checked.
Basically, it seems that the Microsoft RDP client will silently uncheck that first checkbox on subsequent runs if you deliberately uncheck the “Remember me” option when typing in the username and password. So there is still buggy behavior involved here on one end or the other. There are scenarios where you simply don’t want to have the credentials saved, but still want to be able to double-click on an RDP file each time.
1 Like
Another thanks for this info. I’m new here. I only recently installed KDE Plasma on a Proxmox VM. I wanted to RDP into it. After a few hours playing around, I finally stumbled on the credential issue (point 3); got me going. I thought I’d share this info with the community, but found you have already done an excellent guide here. I’m keen to test out points 1 & 2.
I’m also keen to find a way to do a headless mode, so I don’t have to be logged into the machine. I noticed some other people had info on this.
Thanks again.
I also ran into all of these same bottle necks, and a few others.
With the Windows server, the session could persist, but the local session would lock once the remote session was authorized. This is a critical difference that I hope is being worked on. Straight screen mirroring may be fine for remote support, but it is not good at all for variable remote work.
When I am remote to my workstation , all 3 monitors are trying to be rendered. With the Windows server the desktop is just rendered at whatever resolution the client has available, but currently krdp is trying to do a screen capture of all of the screens. So 1440p main monitor is being pushed to the laptop, or the phone, I stead of just being an actual smaller desktop.
Excellent work, it is something! But hopefully the goal is at least parity with the MS feature set so it can function as a terminal server and not just a remote support app. Rust desk already can fill the remote support role, the Wayland RDP terminal server is the missing piece of the pie!
Great work
Update to original post
I feel like this post needs to be updated, there has been lots of updates pushed out for KRDP and KDE in general since this post. As of today a lot of these tweaks are not required anymore. At least I don’t seem to have those problems anymore.
Firstly the Android Windows App the native android RDP client seems to be able to connect to the KRDP server but the stream hangs a within a few seconds, the screen does not update but I can see that the mouse and touch and interactions are working in the server with my monitor connected to the server. I need to look into this further about what is causing the video to hand in the Android client. I suspect it will be fixed soon or if I get some time to tweak the settings it would work.
Fixes 1 Disable Hardware Accelerated Decoding, 2 Adjust Experience Settings and 3 Credentials Handling are not longer required for my mstc.exe client in the Windows 11 Pro pc. So this post is totally not required anymore at least for me.
Guacamole 1.6.0 can support H264 if correctly configured but since I am not a docker expert I was using a docker image by mwilson2764 that was not the perfect setup I wanted (it required separate containers for the db and does not work with prefixed subdomain) a better docker image that is setup as one container and works with prefixed subdomains and includes a lot extra tooling and codecs included is flcontainers/guacamole on Docker hub or Github.
Finally the hardware encoding problem for my Beelink minipc Ryzen 5800H H264 fixed itself at some point I suspect it was a graphics driver or KRDP update that fixed it.
One thing to note that after one of the updates to KRDP I was unable to authenticate when RDPing in from windows. That was fixed by adding a user and password in the KDE KRDP/Remote Desktop settings window, in the Other Users section this is separate from the already present user in the System Users section.
So anyone reading this I hope this helped you out. All of this is not required anymore. This is mostly due to the awesome KDE devs hard at work, they have solved most of these problems progressively. Although the one thing I’d like to see is better/updated documentation.