Definitely! I’ve been lurking on the sidelines and seeing a lot of good work and contributions from talented and dedicated people, it’s really great to see. There’s a small youtuber I watch occasionally who’s girlfriend has cerebral palsy I think?, and it’s motivated them to get involved, and I’ve been thinking in the context of her, voxd, ydotool, and that keyboard, throughout this entire thread. My disability makes me more invested in this than most, but people like her stand to benefit from stuff like this far more than I do, and that’s present in my mind.
It’s not my intention to imply they can’t work stand-alone, I feel like that’d be kinda disrespectful to the work that has and is being done, not to mention just being incorrect
It’s just that they can’t behave nicely together. The input processing software all take exclusive access to the input devices, before the compositor, so that they can block inputs from being seen in apps downstream. Whichever one you start last will ‘grab’ the device and ‘steal’ the input from the others.
The compositor-level inputs float above all that and their inputs don’t reach these kind of applications at all. If you’re using InputActions to fire off keyboard inputs, try running this command:
sudo libinput debug-events
And note that it will catch the inputs you make, but not the inputs from InputActions.
This isn’t something that really matters at all, if you only want to run an OSK or only want to run voice dictation or only want mouse gestures or only want to map a joystick to generate mouse inputs, you choose the one thing you need and it runs and it’s great.
But when we want to mix them up, which is definitely what the people who use these things the most will do, then we hit problems, because the compositor is an arbiter for which app gets all the input, but there’s no real arbiter for which input goes to which app, it’s just a mixer of everything which is left after everything streams into one big pool and some would-be thieves-of-everything battle it out for which one of them actually gets it
Here are a few links for some examples and some background, hopefully they will be interesting.
Here you can see the day I found out about this the hard way:
I originally mistook this for a bug in a specific application before I learned of the over-arching architectural limitation I’ve been describing ITT.
Several developers of similar software assembled to try and tackle this issue (marathon thread):
One is attempting to build a solution which was borne of that discussion:
I’m sure that collaboration from other awesome devs would be very welcome on this front.
It’s good to see this getting attention because I know I’m not alone in being very keen to see this fixed so we can have more cool toys So thanks to OP for a nice use-case and thanks to the devs for popping by to look at it. I’d really like to see voxd or something like it be an integral part of the ecosystem. I wanted to do a potential dev the favour of drawing attention to this potential hidden pitfall of adding new input devices (ydotool) into the mix, but I didn’t really intend for it to detract this much from the original intent of this thread.
Hopefully the links here and the words from the devs inside will explain it well enough and we can leave this diversion to the side while we all think of cool things we could do with voxd. Sorry OP!
