Second monitor black after using the KVM briefly

I have a Tesmart KVM Switch and 2 Monitors as well as 3 devices that are connected to that KWM Switch.

When i switch between devices and come back to a running KDE session, i notice that the second monitor is gone and isn’t detected any longer. It stays black and any related application using it before needs to be restarted to appear on the first and active monitor.

Windows 10 handles this more gracefully. The question now is, where in the stack does this problem occur and what can we do about it? The KVM switch model has no edid emulation.