Multi-Pointer or Multi-Cursor to use with Wayland?

Development
1

I loved and even flexed how I can use multiple mice and keyboards together in Gnome X11 with the xinput create-master [name] and xinput reattach [slave device name or id] [master device name or id] commands.

Making you able to use two mice with each having its own Keyboard focus. This was never or partly possible with a bug in Plasma 5 X11 as the Mouse Cursor did fight with the location.

Its still kind of possible now in Wayland by having a Wacom Tablet and a Mouse, which automatically creates a custom cursor for the Tablet, but it still fights in Plasma 5 Wayland for the position like it does with Plasma 5 X11 custom xinput setup. In Gnome not which shows that the feature should be possible.

It would be great if it would be possible to set any Mouse as a new custom Cursor with each having its own Keyboard focus (to set) like it was possible with Xinput.

Especially for Plasma 6 I can imagine this as a very great feature for being more productive on the same PC or better teaching by using a second input and the application twice next to each other.

2

Additionally, I would even donate 20€ to KDE or smth for this feature, as I see a greater future in Plasma with this feature.

3

Sponsored Work - KDE Discuss :slight_smile:

4

Here is a multi cursor example for wayland, there are few minor bugs but might worth having a look.

githubcom/ozinal/wayland-multi-cursor

5

I mean, while it’s interesting to finally see an alternative to xinput for wayland in Gnome, It’s still hard to imagine that the mouse cursors could render for itself in KDE or even have their own parallel custom focus like in Gnome.

But I created this post to just be able to have two mice like in your github.

I wonder how much work it would be to integrate this feature into kde, as wayland compositors are wildly different afaik.

6

Based on my analysis, integrating multi-cursor into KDE is actually more feasible than you might think - probably easier than your Mutter implementation.

Why KDE might actually be easier:

KWin has better abstractions for this. They already have a CursorItem system that’s designed to handle multiple cursors (they use it for per-screen rendering). You’d basically be repurposing existing infrastructure - creating cursor items per-device instead of per-screen.

The device detection code from your Mutter patch (the libinput/USB bus type stuff) transfers almost 1:1. KWin also already tracks per-device positions for tablets, so you’re not building from scratch.

The realistic path:

  1. Detect USB mice separately
  2. Create multiple CursorItem instances in the scene graph
  3. Wire up position tracking
  4. Fix the bugs (cursor icons syncing, ghost cursors on disconnect)

The KDE devs are pretty receptive to contributions. I’d suggest reaching out on their Matrix/IRC (kwin) with your Mutter work as reference - they’ll probably be excited someone’s tackling this and might even help with the architecture.

7

How does apps handle hover and cursor shape when multiple cursors enter the same window?

8

I’m interested in this too, I was actually working on some enhancement to my mcp desktop controls trying to do just that to invoke a second mouse to see what it’s grabbing, particularly since it has a hard time grabbing the right areas to click about naturally when invoking a remote desktop session for control. It can invoke its own ghost mouse it seems, but not a second visible mouse to control like my own, or so Codex told me… Maybe it was wrong.

9

Does the premise mean that connecting two mice to a single session would merely bind two mice to a single cursor, and that KDE Connect’s remote screen control mode would also do so? I ask because:

  • I believe that TeamViewer (at least, on Windows) can render multiple cursors, and:

  • I’ve some software that changes how it acts, depending upon whether a keyboard or cursor is hovering over its elements. [1]

  1. ↩︎
10

So after my last post, I’ve actually been hacking my own support for this into kwin, and actually have it working reasonably well in a test kubuntu 26.04 with plasma 6.6.4, or at least for my purposes I think before I have to dig into wayland more.

We (Codex and I, mostly Codex) added an api layer for control and mostly tapped into using the existing drawing tablet support to enumerate secondary pointer capabilities, added glue around at-spi2, and ended up “spiking” control that it can click to interact with windows without disturbing my current mouse and control. We’re still working out pulling applications into their own focus, but so far testing is positive this should be doable, and I’m hoping to actually have this done shortly today for some interactive human vs agent control battles and report some success.

Most of this is because I’ve had a good amount of success in getting my mcp project to have full control over my desktop to do stuff, that it’s annoying me when I’m playing a game, and it starts UI testing my desktop menus and moving stuff around testing mouse and dock control.

Bad AI, here’s your own mouse pointer, go play over there while I finish leveling my game on this monitor.

I’ll post some more on this as I test as I’d like to see the work adopted. I think it could be quite useful to folks, and trying to keep things specifically aligned to KDE principals to avoid friction in upstreaming my work.

Just some background what this looks like, I’m up in the 60’s currently, EMCPS is my mcp server:

- [TASK-SAP-000](TASK-SAP-000-project-docs.md): project docs
- [TASK-SAP-001](TASK-SAP-001-ci-task-validation.md): CI and task validation
- [TASK-SAP-002](TASK-SAP-002-kwin-test-harness.md): KWin test harness
- [TASK-SAP-010](TASK-SAP-010-kwin-api-survey.md): KWin API survey
- [TASK-SAP-011](TASK-SAP-011-window-lease-contract.md): lease contract
- [TASK-SAP-012](TASK-SAP-012-window-lease-prototype.md): lease prototype
- [TASK-SAP-013](TASK-SAP-013-live-package-smoke.md): live package smoke test
- [TASK-SAP-020](TASK-SAP-020-emcps-kwin-lease-client.md): EMCPS KWin client
- [TASK-SAP-021](TASK-SAP-021-emcps-session-policy.md): EMCPS session policy
- [TASK-SAP-022](TASK-SAP-022-live-smoke-nested.md): nested smoke tests
- [TASK-SAP-023](TASK-SAP-023-emcps-atspi-target-contract.md): EMCPS AT-SPI target contract
- [TASK-SAP-024](TASK-SAP-024-live-dual-client-smoke.md): live dual-client smoke
- [TASK-SAP-025](TASK-SAP-025-mcp-http-live-regression.md): MCP HTTP live regression
- [TASK-SAP-030](TASK-SAP-030-agent-pointer-model.md): agent pointer model
- [TASK-SAP-031](TASK-SAP-031-agent-cursor-rendering.md): cursor rendering
- [TASK-SAP-032](TASK-SAP-032-multi-agent-limits.md): multiple agent limits
- [TASK-SAP-040](TASK-SAP-040-input-policy-contract.md): input policy
- [TASK-SAP-041](TASK-SAP-041-app-click-fallback.md): app click fallback
- [TASK-SAP-042](TASK-SAP-042-agent-owned-input-spike.md): native input spike
- [TASK-SAP-043](TASK-SAP-043-atspi-app-action-fallback.md): AT-SPI app action fallback
- [TASK-SAP-044](TASK-SAP-044-target-id-normalization.md): KWin target id normalization
- [TASK-SAP-045](TASK-SAP-045-agent-transient-focus-policy.md): agent transient focus policy
- [TASK-SAP-046](TASK-SAP-046-native-click-hardening-matrix.md): native click hardening matrix
- [TASK-SAP-047](TASK-SAP-047-agent-keyboard-text-spike.md): agent keyboard and text spike
- [TASK-SAP-048](TASK-SAP-048-agent-control-session-model.md): agent control session model
- [TASK-SAP-049](TASK-SAP-049-agent-interaction-live-smoke.md): agent interaction live smoke
- [TASK-SAP-050](TASK-SAP-050-kde-rfc.md): KDE RFC
- [TASK-SAP-051](TASK-SAP-051-upstream-test-plan.md): upstream test plan
- [TASK-SAP-052](TASK-SAP-052-upstream-mr-slicing.md): upstream merge request slicing
- [TASK-SAP-060](TASK-SAP-060-lock-screen-secure-surface.md): lock screen and secure surface denial
- [TASK-SAP-061](TASK-SAP-061-xwayland-popup-grab-hardening.md): XWayland, popup, and grab hardening
- [TASK-SAP-062](TASK-SAP-062-multi-output-geometry.md): multi-output geometry and scale
- [TASK-SAP-063](TASK-SAP-063-keyboard-ime-hardening.md): keyboard layout, modifiers, and IME hardening
- [TASK-SAP-064](TASK-SAP-064-package-ci-vm-runner.md): package CI and VM smoke runner
- [TASK-SAP-065](TASK-SAP-065-independent-agent-seat-design.md): independent agent seat design

This mostly began as a project to see if I could, and seems I can to at least some extents.