If you run
journalctl --user -b -p warning --no-pager
do you see a bunch of the following hogging up a couple seconds of time?
kwin_wayland[7596]: Failed to open /dev/dri/card1 device
In my case, I left integrated graphics enabled in my bios. After disabling the setting, I went from up to 5 seconds per action (up to 10 seconds or more just switching tty and/or user sessions to almost instant.
And, this is how it should be done.
Does this also affect anyone else?