It seems like ffmpegthumbs plugin for Dolphin does not like .mkv containers with .webp images, trying to display random image from the video stream instead of the embeded cover art.
Unsure how to properly report a bug (provided this is indeed a bug and not me having something misconfigured in some way).
Or if there is something I can try to get rid of this issue other than trying to mass-convert thumbnails to .jpeg/.png and reembed them, as those seem to work just fine with .mkv container.