New ideas using Wayland Input Methods

Input Methods are a way of allowing third parties programs to manipulate text in text input boxes in applications.


This is a companion discussion topic for the original entry at http://blog.davidedmundson.co.uk/blog/new-ideas-using-wayland-input-methods/
10 Likes

Great work!

These days a lot of clients now have in-built support, including all GTK4 text areas, so it might not be needed. Maybe the ship for this has sailed, or there could be still value in centralising it and having a common pattern.

if we could remove the custom and duplicated code for this feature in NeoChat. Tokodon and KMail, this would make me very happy. One thing that NeoChat and Tokodon have are custom emojis support which might be a bit tricky to implement.

5 Likes

Wow, amazing stuff.

Each of these would be a blockbuster Plasma 6 feature so I sincerely hope at least a few of them manage to make it!

4 Likes

Amazing. I hope we will get a proper emoji picker on Plasma 6 based on this work. This is one of the thing I miss the most in windows. In windows you can just presss “win+.” and type and the selected emoji will automatically paste on the app.
Here is a baby wogue video demonstrating the problem. Ignoring the smugness, the points he pick up on are actually valid.

Direct Diacritic Display is also a fantastic thing as now I can easily and properly name people’s name.

Timely Translation Tasks, Simply Speak will greatly improve accessibility to many users.

All around an amazing job. Will be eagerly waiting for this.

3 Likes

In case you didn’t know, this works in Plasma as well :slight_smile: The UX isn’t the greatest, because it requires you to manually paste it into the application.

3 Likes

Yep. Knew that. But manually copy pasting emoji really sucks.

2 Likes

As another option to enter emoji, the ibus input method supports emoji, with ctrl+shift+e by default IIRC.

1 Like

I wish someone would make Kwin work with xeyes, or a similar alternative.

:eyes:

Handwriting support like in Windows 11 would be a nice - but not very important - additional input method. It probably would mean to include something like tesseract to work.

2 Likes

Do those features conflict with an existing input method? Like if I am already using fcitx5, can I still get those features?

2 Likes

That’s a really good question.
Right now all the examples in the blog post multiplex internally within a single application.

Kwin is set up so you have one input backend. Having two InputContexts alive at once both grabbing and inserting wouldn’t really work.
But being able to switch at runtime seems doable.

Code wise we need 3 steps:

  • seeing if we can port code in my playground to use KDE global shortcuts instead of sniffing keys to activate
  • add some “activate me please” signal to the InputMethod manager class
  • make kwin follow this, replaying as though we had just changed focus out to the old input context, and acting like a focus in to a new context on the new input method
  • then we can test against maliit and my demos
2 Likes

Especially the voice input is one of the features that would increase accessibility by a lot. Desktop Linux is lacking in the accessibility department and that would get us a bit closer. The other features are cool too but I think a system wide stt system would be the most must-have of them all.
In general the demos are sick and I hope at least some of that gets into plasma 6 in a polished state!

1 Like

The first two ideas, “Convenient Clipboard Connections” and “Easy Emoji Entry”, have been provided in Chinese IMEs (e.g. sogoupinyin) for decades. If we are exploring this direction, then commercial CJK IMEs, especially modern smartphone IMEs, would be a good source of inspiration.

1 Like

Another example is ‘fig’, which uses the ibus IME framework to provide autocompletion for Linux shells.

The problem is that to do good autocompletion, you need not only text the user is currently typing, but also surrounding text. fig does it using shell hooks. But what if we do an IME that provides autocompletion for all text boxes? (probably using ChatGPT-like tech, e.g. like what Github Copilot does in vscode)
Then perhaps there should be someway that IME can get “surrounding text” from the app.

1 Like

There is an existing concept of surrounding text in IMEs.
How much surrounding text is provided is rather undefined. Qt gives you the current line.

1 Like

Yes, but wouldn’t it a killer feature to have custom emojis in all Plasma apps, which supports images or (hyperlink-)plugins?