Is there a better way to obtain the mouse cursor location?

Since Wayland is different from X11, my old mouse clicker (based on pyinput) cannot obtain the correct mouse location (unless I force the program running on XWayland).

After many search, I’ve found that, there’s no uniform way to obtain cursor location with Wayland protocol, but KWin script could obtain the mouse location (e.g., kdotool do such thing with KWin script and DBus commands.)

It is said that, execute a simple KWin script is enough:

# test.js file
console.error("x=" + workspace.cursorPos.x + " y=" + workspace.cursorPos.y)
# bash command
$ export SCRIPT_LOC="/path/to/above.js"
$ export SCRIPT=$(qdbus org.kde.KWin /Scripting org.kde.kwin.Scripting.loadScript $SCRIPT_LOC)
$ qdbus org.kde.KWin /Scripting/Script${SCRIPT} org.kde.kwin.Script.run
$ qdbus org.kde.KWin /Scripting/Script${SCRIPT} org.kde.kwin.Script.stop

But the output directly goes to journalctl, if I need to judge the mouse location rapidly, the journal’s size blows up, and the speed is not very fast.

Is there some better way to obtain mouse location in KWin Wayland? What about running the program as root?

I only know of two ways to deal with mouse input on Wayland:

  • with uinput (that’s how things like ydotool and similar work) + root permissions / user being in input group
  • with the new RemoteDesktop portal that you’d communicate to with D-Bus calls (that’s how libei works for example)

I didn’t know KWin exposed API to do that with kwin scripts.

2 Likes

After some test, I finally realized that:

uinput cannot obtain the mouse location (although under some settings, moving mouse to an absolute position could be done (firstly move mouse to top-left corner, and then move mouse with the relative move event))

Remote Desktop could send command “move mouse to the specific location”, but cannot obtain the mouse location.


In conclusion, the only possible way to obtain mouse location (in KDE Wayland) is KWin script.

I’m trying to read memory and scanning where the pointer’s data is stored in KWin, resulting in something require root permissions:

mod consts {
    pub const POS_OFFSET: usize = 176usize;
    pub const WORKSPACE_OFFSET: usize = 0x0000000000779330usize
}
/// pointer of kwin workspace and its cursor's position
pub mod pointer {
    use crate::consts::*;
    use libc::{iovec, process_vm_readv};
    use std::{ffi::c_void, fs::File, io::Read, ptr};
    pub struct Workspace(i32, *mut c_void);
    impl Workspace {
        /// get workspace from kwin_wayland, the pid should met kwin_wayland's pid, otherwise I cannot tell what happens.
        /// since it relys on reading "/proc/{pid}/maps", root access might be needed.
        pub unsafe fn get(pid: i32) -> Self {
            let mut buffer = String::new();
            File::open(&format!("/proc/{pid}/maps"))
                .unwrap_or_else(|e| panic!("cannot open file (require permissions?)\n{:?}", e))
                .read_to_string(&mut buffer)
                .expect("read maps failed");
            let buffer0 = buffer
                .split_once("libkwin.so")
                .expect("program does not load libkwin.so (is it really kwin_wayland?)")
                .0;
            let buffer1 = buffer0.rsplit_once('\n').unwrap_or(("", buffer0)).1.trim();
            // 70642a400000-70642a54a000 r--p 00000000 103:02 3323906                   /usr/lib/libkwin.so.6.1.4
            let Some((offset, start)) = buffer1.split_once(" r--p ") else {
                panic!("get offset failed, the buffer line is `{buffer1}`")
            };
            assert!(start.trim().starts_with("00000000"));
            let offset1 = offset.split_once('-').expect("maps format error").0;
            let base =
                usize::from_str_radix(offset1, 16).expect("cannot parse to base 16") as *mut c_void;
            let ret = base.byte_add(WORKSPACE_OFFSET);
            println!("base offset: {base:?}, {ret:?}");
            Self(pid, ret)
        }
        /// get mouse_pos offset from pointer of workspace.
        pub unsafe fn get_mouse(&self) -> Cursor {
            let mut addr: *mut c_void = ptr::null_mut();
            let local = iovec {
                iov_base: &mut addr as *mut _ as *mut c_void,
                iov_len: 8,
            };
            let remote = iovec {
                iov_base: self.1,
                iov_len: 8,
            };
            match unsafe { process_vm_readv(self.0, &local, 1, &remote, 1, 0) } {
                8 => assert!(!addr.is_null()),
                -1 => {
                    eprintln!("failed, check errno for more details.")
                }
                x => eprintln!("unknown bytes readed: {x}"),
            }

            Cursor(self.0, addr.byte_add(POS_OFFSET))
        }
    }
    pub struct Cursor(i32, *mut c_void);
    impl Cursor {
        /// read mouse location from kwin workspace (it is read-only object, cannot write back.)
        pub fn loc(&self) -> (f64, f64) {
            let mut xy = [0f64; 2];
            let local = iovec {
                iov_base: xy.as_mut_ptr() as *mut c_void,
                iov_len: 16,
            };
            let remote = iovec {
                iov_base: self.1,
                iov_len: 16,
            };
            match unsafe { process_vm_readv(self.0, &local, 1, &remote, 1, 0) } {
                16 => return (xy[0], xy[1]),
                -1 => {
                    eprintln!("failed, check errno for more details.")
                }
                x => eprintln!("unknown bytes readed: {x}"),
            }
            panic!("reading failed.");
        }
    }
}

fn main() {
    let workspace = unsafe { pointer::Workspace::get(518247) };
    let cursor = unsafe { workspace.get_mouse() };
    println!("{:?}", cursor.loc());
}

The consts module contains data obtained by build.rs:

use std::{env, fs::File, io::Write, process::Command};
fn main() {
    // The bindgen::Builder is the main entry point
    // to bindgen, and lets you build up options for
    // the resulting bindings.
    let bindings = bindgen::Builder::default()
        // The input header we would like to generate
        // bindings for.
        .use_core()
        .header_contents("header.hpp", "#include<workspace.h>")
        .clang_args(
            env::var("KWIN_INCLUDE")
                .as_deref()
                .unwrap_or(
                    r#"
                        /usr/include
                        /usr/include/kwin
                        /usr/include/KF6/KConfig
                        /usr/include/KF6/KConfigCore
                        /usr/include/KF6/KWindowSystem
                        /usr/include/qt6
                        /usr/include/qt6/QtCore
                        /usr/include/qt6/QtDBus
                        /usr/include/qt6/QtGui
                        /usr/include/qt6/QtWidgets
                    "#,
                )
                .split('\n')
                .map(str::trim)
                .filter(|x| x.len() > 0)
                .map(|x| format!("-I{}", x))
                .chain(
                    env::var("CUSTOM_ARGS")
                        .as_deref()
                        .unwrap_or("")
                        .split(' ')
                        .filter(|x| x.len() > 0)
                        .map(|x| x.to_owned()),
                )
                .chain(["-x", "c++", "-std=c++20"].map(ToString::to_string)),
        )
        // Tell cargo to invalidate the built crate whenever any of the
        // included header files changed.
        .parse_callbacks(Box::new(bindgen::CargoCallbacks::new()))
        // Finish the builder and generate the bindings.
        .generate()
        // Unwrap the Result and panic on failure.
        .expect("Unable to generate bindings");

    // Write the bindings to the src/consts.rs file.
    let mut file = File::create("src/consts.rs").expect("cannot save to src/consts.rs");
    write!(&mut file, "pub const POS_OFFSET: usize = {};\npub const WORKSPACE_OFFSET: usize = 0x{}usize;\n",
        bindings
            .to_string()
            .split_once(r#"offset_of!(KWin_Workspace, focusMousePos)"#)
            .expect("cannot calculate offset of focusMousePos.")
            .1
            .split_once("]")
            .expect(r#"do not find line [::core::mem::offset_of!(KWin_Workspace, focusMousePos) - $(SIZE)]."#)
            .0
            .split_once("-")
            .expect("grab offset failed")
            .1
            .trim(),
        String::from_utf8(Command::new("readelf").args(["-WCs", "/usr/lib/libkwin.so"]).output().expect("readelf execute failed").stdout).expect("failed to parse readelf").split_once(r#"KWin::Workspace::_self"#).expect("cannot find KWin::Workspace::_self").0.rsplit_once('\n').expect("cannot read offset of KWin::Workspace::_self").1.split_once(':').expect("parse `:` failed.").1.trim().split_once(' ').expect("cannot parse space").0
    ).expect("write failed");
}

kdotool does the reporting by setting up a DBus interface and call it in the kwin script. I don’t know if that’s faster, but at least it doesn’t blow up your journal.

1 Like

There is another way. You can install a C++ kwin effect, which receives all mouse events. Like:

And since it’s in C++, you can do whatever you like to report the data, not limited to the journal or dbus as in kwin scripts.

You can take this as a boilerplate for 3rd-party C++ kwin effects:

1 Like

My use case is a little bit weird: use kdotool to obtain the mouse location, then move the mouse with my farmilar /dev/uinput event (send relative move with some calculations that ensure the mouse moves to the desired coordinate.)

Although kdotool’s script only uses ~1KB per call, since it might be called rapidly, the journal will finally grow up to a really huge thing.