Voxd and Ydotool Incorporation in KDE Linux

The do.

Maybe not perfectly yet but the Plasma keyboard initiative seems to have mad great progress. Many devices with embedded Linux use them as other implementations as their primary means of entering text or numbers.

On Linux (X11 and Wayland) the mechanism is called “Input Methods”.

Programs that can provide text can register as those and get “connected” with text input fields when necessary. At which time they get information about the current content and other values, e.g. whether this is a numerical input field, email, etc.

The most common use cases are on-screen keyboards and tools for entering Asian language characters.

See this topic for a link to a blog and discussions on several other ideas.

A frontend to Voxd (and/or other voice to text systems) could register as such and provide text input capability.

I am pretty sure that all platforms have similar API/systems even if the do such thing.

Key features are provision of the existing text (without the input having to use OCR to extract it from captured pixel data) and context (what the text input is for).

I would be surprised if Android and iOS allow apps unrestricted access to each others’ input and output.

That is a different, more complex, use case than sending keys/text to the currently focused window like keyboard input would be.

Definitely worth exploring but will likely require much more challenging than using existing integration points.

Edit: see also this effort for moving input methods forward

1 Like