The Most Precious Screen Space: Magic Corners

[ThomasSeeker]

Hey there,

In a certain aspect KDE operates way below it’s potential, in my opinion.

According to “Fitts’s law”, which is (among other things) about Click-Ergonomics, about how easy, fast and convenient a button can be reached:

"Placing layout elements on the four edges of the screen allows for infinitely large targets in one dimension and therefore presents ideal scenarios. Since the pointer will always stop at the edge, the user can move the mouse with the greatest possible speed and still hit the target. The target area is effectively infinitely long along the movement axis. Therefore, this guideline is called “Rule of the infinite edges”. […] This effect can be exaggerated at the four corners of a screen. At these points two edges collide and form a theoretically infinitely big button. […] These four spots are sometimes called “magic corners”. (Fitts's law - Wikipedia) … I’d like to add, that in contrast to the 4 edges (which are infinite too of course), the 4 corners are not just infinite but can also be hit blindly (with turned off screen) with 100% hit accuracy.

And what is KDE doing with these most precious spaces of the whole screen? … All you can do is: Touch them! That’s all! … (via activated “Hot Corners”) … This is such a waste of potential! … We have 4 infinite large click-spaces on our screens, but we can not click them, we can not double-click them, or middle-click them, or long-click them or scroll on them! … And it is not, that we have plenty of other such possibilities! We have only 4 of those magic areas! But we almost don’t use them at all! Why not?

So I tinkered together a working hack for touches, all the different clicks and scrolls. A kwin-script which gets triggered by a hot corner and then calls via dbus an external bash-script. Which then captures/immobilizes the mouse pointer, so that in an area of about 15x15 pixels mouse-movement does not end the listening. And it also hinders other instances of itself from starting. … and then it just listens to the buttons etc. …

And it is just beautiful. I never want to miss it again.

I have these Magic Corners set to be active above full-screen windows too. And a few of my most used actions are: Scrolling on one corner to change the system-volume up/down. Scrolling on another corner to fast-forward/rewind videos (youtube, vlc, …) … and on another to scroll through all tabs (web browser, file browser, editor, …) … and yet on another to zoom in/out (web browser, editor, …). And so on. The possibilities are endless.

Now shouldn’t this be something, everybody should be able to enjoy?
I believe this would give KDE a substantial boost of usability and luxury!

And out of curiosity: Do any other Desktop Environments have this?

And what do you think?

PS: Here is GIF to show 2 examples of scrolling on the corners:

PPS: I want to give credit to “Panel Spacer Extended”, which led me onto this path. It can function as a workaround for the above: Just create new panels with only a “Panel Spacer Extended” widget and a Transparency-Widget on them … and put these tiny panels into the corners of the screen. It also gives you invisible magic corners. … But it has a few downsides and the functionality IMO naturally belongs to KDE’s “Hot Corners”.

[skyfishgoo]

i think you are onto something.

what you suggesting is a mutually exclusive “Custom Kwin Script” function that would be added to this list of options for these corners and edges, then?

This script would then listen for mouse and keyboard events that can be programed to trigger other actions like change volume, FF/REW, change brightness, run a bash script, etc.

in addition to performing any of these other canned options behind a click and/or keyboard modifier

this would expand the utility of these screen edges exponentially…. sort of a meta function.

to cancel or abort, you simply move your mouse away from the edge/corner just like you do for any of the other functions.

[dzon]

This existed before and not just kde. XD-hotcorners is such an example. As a 20 year-ish openbox user you could have hotcorners which executed custom commands. But what is/was not possible is the scroll.

Here’s such a custom corner command for kde ( btw, I think it’s available for plasma 6-ish)

And this is xd-hotcorner which could be used on a variety of desktop environments and non-environments ( floating managers stuff)

I’m actually quite intrested in this scroll ability on a hot corner.

Edit: Yeah, there seems to be a more extensive script available for 6-ish. But no scroll.

https://store.kde.org/s/KDE%20Store/p/2323175

[ratemisia]

I also think ThomasSeeker is onto something, but I think we could go further than exposing a “Custom KWin Script” option. We could do something like we currently do with “Window Behavior”, where different mouse actions are given as options and each can be assigned a function:

This may be excessive for the purpose (it’d require an awful lot of new buttons and dials), but it would make it simple for less technical users to make full use of the functionality. Worth considering?

[skyfishgoo]

settings like these are likely better placed in the settings > window management > kwin scripts > configure dialog, and yes, it could get quite complicated.

the idea behind the simple add to the screen edges page is that you are choosing to execute a script instead of one of the other choices.

configuring what the script does would be on a separate page, but maybe linked on the screen edges page for convenient access when you choose that option.

[ThomasSeeker]

Hey guys,

thank you for your interest in this and for your appreciation!

I find the things you write very very interesting!

I go throgh them one by one:

First: To what extend does this already exist?

@dzon

Edit: Yeah, there seems to be a more extensive script available for 6-ish. But no scroll.

Custom Screen Edges Actions - KDE Store

I installed the KWin Script “Custom Screen Edges Actions”, but it also seems to only trigger the actions through Touching. Just like the original KDE “Screen Edges” but with more available actions.

@dzon

As a 20 year-ish openbox user you could have hotcorners which executed custom commands.

Ah, interesing! From your screenshot it looks like you could assign custom commands to all 3 mouse-buttons. Ok, this goes quite far. But yes, scrolling seems missing and double clicks, long clicks etc. too. … But to use Openbox on KDE would require to replace KWin, right?

@dzon

And this is xd-hotcorner which could be used on a variety of desktop environments and non-environments ( floating managers stuff)

Can you give some more info please, dzon!? There does not seem to be much info about “xd-hotcorner” on the web. Is this connected to Openbox or something separat and usable with KWin too?

And now to a possible implementation:

@skyfishgoo

what you suggesting is a mutually exclusive “Custom Kwin Script” function that would be added to this list of options for these corners and edges, then?

@ratemisia

I think we could go further than exposing a “Custom KWin Script” option. We could do something like we currently do with “Window Behavior”, where different mouse actions are given as options and each can be assigned a function

Now this is really fascinating for me to think about.

If I understand both of you right, there could be this permanent option "Custom Kwin Script” in the “Screen Edges” drop-down-menus for each corner/edge. And from there one would be linked to a settings > window management > kwin scripts > configure dialog which looks somewhat like ratemisias sceenshot of “Window Behavior”.

Yes, that sounds very intuitive to me! … Because users who look for something to do with the hot corners naturally first go to “Screen Edges” and there they would find this interesting new option which then leads them to the full and rich buffet of choices. … and this without overwhelming other users, who are content with just the standard options.

Maybe the name “Custom KWin Script” could give some more hint to the nature of the option. Maybe something like “Custom Click and Scroll Sript” or something like that? … to make it clear, that unlike with the other actions, here it is not just about the triggered action, but also about the trigger-mechanism itself. … What do you think?

[dzon]

@ThomasSeeker

I installed the KWin Script “Custom Screen Edges Actions”, but it also seems to only trigger the actions through Touching. Just like the original KDE “Screen Edges” but with more available actions.

Yeah, I said that.

@ThomasSeeker

Ah, interesing! From your screenshot it looks like you could assign custom commands to all 3 mouse-buttons. Ok, this goes quite far. But yes, scrolling seems missing and double clicks, long clicks etc. too. … But to use Openbox on KDE would require to replace KWin, right?

Openbox is a different beast. For one, a “clean” openbox doesn’t have a desktop, it’s not a DE unlike, say, xfce,kde etc… ( unless you run it in an lx, xfce or rox environment) It’s a floating window manager. But in terms of corner bindings, you could use either scripts and assign them to corners in the autostart OR you could use, for example, hot-corners-xd ( gave the wrong name, my bad) But that one is no longer available. However, I tried it on a KDE 5-ish once and it worked ( I have the deb file somewhere), with more options than default kde. Btw, hotcorners used to come with skippy-xd and I’ve seen ( for the life of me..can’t remeber where) a revival for that. Not sure if it includes hotcorners as well.

There used to be a time you could easily switch a kwin manager with openbox using the openbox-kde-session. Not sure if that’s still the case.

Skippy-xd ( overview-ish thingie) on xfce or mate, not sure:

On openbox:

[Samuele]

Maybe “Custom mouse action” to group any event you could launch with the mouse (click, scroll, double-click…) and to group any output you could pack in this addition: from your screen recording I see actions like “control volume”, “open X application” etc., which the configuration page could provide without the need to pass through a script (also because, how would a Kwin script let you bind mouse scrolling to volume level when in that specific state?); executing a script would just be one of the services offered.

I was skeptical at first because I couldn’t imagine how to use clicks etc. on a corner, but the mouse trap seems a great idea!

[dzon]

@Samuele

but the mouse trap seems a great idea!

Yes, indeed. It could be naais.

[skyfishgoo]

i think “Custom Action” pretty much opens the option up to whatever kind of kwin script the user installs.

the default kwin script provided as part of this new feature would cover the basics of what is possible and if someone wanted expand on it, they could write their own kwin script and use that instead.

better mouse trap, indeed.

[ThomasSeeker]

Thank you, dzon, for the info!
So there seems to be quite a variety of hot corners out there, but so far it seems, that none of them go all the way!

So KDE could be the first one here. And IMO it perfectly aligns with the KDE philosophy of configurability and user friendliness.

Regarding the name, yes, I agree, “Custom Action” seems to includes all of our above ideas.

Regarding the mouse trap:

@Samuele

I was skeptical at first because I couldn’t imagine how to use clicks etc. on a corner, but the mouse trap seems a great idea!

After using it a couple of month now, I think, the different kinds of clicks would actually not even need a mouse trap, because the infinitely large click-area is so ridiculously easy to hit. I don’t even follow the mouse-pointer with my eyes anymore, if I go to the corners. … The only reason, I added a mouse trap to my hack, was the scrolling. Because during longer scrolling up and down (to find the right zoom-level or video-position or …) the mouse-pointer sometimes left the corner a bit and thus stopped the listening. … yes, and this got solved by the mouse-trap very well.

Now guys, I am new to all this! And since we all think, this whole thing would be a nice addition to KDE … what is the next step forward? … Unfortunately I am not a coder and my dirty hack in bash with the help of AI works just good enough for me, but code-wise that’s about all I am capable of doing.

What I could do though, if this would help, I could put the logic into words. But on the other hand it is very very simple, almost a no-brainer: Once a corner is touched → Listen to mouse or keyboard input → When input X do Y → When corner (+15px or so for the trap) is left, stop listening.

So, yes, what would be a possible next step?

[skyfishgoo]

the mouse trap IS the innovation here, you need to keep the trap because clicking or scrolling before you get to the corner can activate any number of things along the path and when you get to the corner on a window frame it can still activate the underlying action rather than your hot corner action.

in this case the hot corner is “within range” as shown by the glow, but i’m still able to activate the shade roll up on the title bar with my mouse scroll wheel

Seems to me the Custom Action would need a “trap” that prevented the underlying action and it would need to be big enough to accommodate accessibility concerns so users can hit it and stay in it before pressing any buttons or keys to modify the Custom Action.

[ThomasSeeker]

@skyfishgoo

Seems to me the Custom Action would need a “trap” that prevented the underlying action

Ok, I agree completely!
It seems I misunderstood, when you guys spoke of the mouse-trap. I thought, it ment only the movement of the mouse. … But yes, regardless of the movement: As long as the corner is listening, every other click-function etc. (for windows below it) must be disabled. (And this is what it does here too.)

But regarding the movement: In my hack the mouse gets only trapped AFTER it has hit the corner. The current hot corners of KDE are just about 2x2 pixels in size, even though the glow starts way earlier. …
On the way to the actual corner I would want the mouse to be able to function normally. For instance in your screenshot, I would still want to be able to hit that close button of that window, even through the glow! … And this feels not as a limitation at all, because since the corners are infinitely large and so easy to hit, they are almost impossible to miss.

[skyfishgoo]

yes, mouse-keyboard events up to activation of the corner should work as normal

then AFTER activation of the corner there needs to be a big enough zone where listening can take place that it can accommodate accessibility concerns, such as moving the mouse when pressing a button, or mouse jitter while reaching for a modifier key with the other hand.

but not so big that becomes a chore to disentangle your mouse from the Custom Corner function, in case you activate it by accident.

perhaps the script should make this “trap” size configurable.

[ThomasSeeker]

Yes, exactly! I completely agree!
And having the trap size configurable would be the best IMO too!

[Samuele]

@ThomasSeeker

The current hot corners of KDE are just about 2x2 pixels in size, even though the glow starts way earlier. …

I think this is very important: corners are useful now, because we can position the cursor on actions that are placed there (for example, Windows’ Start button until 10 was on the corner and was very easy to reach), and Plasma does a great job to redirect any click on corners to the nearest button. The trap is something for after the corner has been activated, but the corner must be somewhat difficult to activate so that you don’t block buttons on those corners. I think it’s okay that KDE hot corners are so small: it would make sense to launch the cursor to one corner to comfortably place the cursor on any button that’s there, and if one needs to activate the hot corner another swift movement would not be too much to handle.

[skyfishgoo]

now that we have some consensus on what the feature request is going to entail, we should discuss what features the kwin script should provide by default and what might be left to 3rd party developers to fill in gaps.

once the corner is active and listening to mouse/keyboard events, what kinds of functionality should it afford?

mentioned so far:

• volume up/dn
• audio track prev/next
• video playback ffw/rew
• video playback frame advance fwd/back
• monitor brightness up/dn

and what sort of mouse button / modifier keys would be needed (if any)?

• Left click
• Middle click
• Right click
• Shift
• Ctrl
• Meta

the default size of the listening zone was mentioned as 15x15, is this a good value?

how configurable does this need to be, could there just be size small, medium, large to accommodate user taste.

[Samuele]

I think it doesn’t really matter what you put at first: the feature is totally new and it would be great to have a proof-of-concept of the engine to gain traction first.

I think it probably depends on screen resolution: escaping from a 15x15 square is easier on higher resolution screens, because the pointer movements usually span more pixel, right?
Using presets would be elegant, but some study on the correct dimensions given mouse/trackpad speed should be done, which brings us to: maybe at first total customizability of this area would avoid the burden choosing good presets.

[ThomasSeeker]

Hey guys, sorry for the delay!

@Samuele

I think it’s okay that KDE hot corners are so small

Yes, I completely agree! It is perfect this way. … This was actually one of the main reasons, why I was not happy with the Extended-Panel-Spacer widget: By its nature, you can not make it smaller than the minimal panel-size, which is about 20x110 pixels! Which is huge compared to 2x2 and almost covers a whole window-close-button. … 2x2 is much better!

@skyfishgoo

now that we have some consensus on what the feature request is going to entail, we should discuss what features the kwin script should provide by default and what might be left to 3rd party developers to fill in gaps.

Ah, thank you, skyfishgoo, for leading the next step! This makes perfect sense!

@skyfishgoo

once the corner is active and listening to mouse/keyboard events, what kinds of functionality should it afford?

Wouldn’t it be best to somehow import all possible D-Bus commands to choose from?
And very important IMO would be an additional option: “Run custom script”.

@skyfishgoo

and what sort of mouse button / modifier keys would be needed (if any)?

I added 4 more:

• Left click
• Double left click
• Long left click
• Middle click
• Double middle click
• Long middle click
• Right click
• Shift
• Ctrl
• Meta

@skyfishgoo

the default size of the listening zone was mentioned as 15x15, is this a good value?

Yes, a played around with this and 15x15 works about best for me here on a 1920x1080 resolution.

@Samuele

maybe at first total customizability of this area would avoid the burden choosing good presets.

Yes, I agree, the most freedom to play around and test … at the beginning.

And one other thing which might need some thought too: In the Screen-Edge-Settings there are 2 settings of potential influence: Translated from German: “Activation Delay” and “Re-Activation Delay”. … I set the first to 0 here … but this can still be up to each user, right? … Well, and the second should not be of any concern, because the mouse-trap hinders fast deactivation anyways. Am I right here?

Thank you guys for being on board with this! …

[ThomasSeeker]

PS:
Another thing, which might be useful for some users:

I play a “click”-sound when the mouse moves into the trap of a corner and a “clack”-sound, when it is released. … Click … Clack. …

For me this gives a nice additional feedback. Also because the optical feedback (although aesthetically very pleasing IMO) is gradual and starts way earlier. So a really sharply defined additional feedback feels somehow satisfying to me. Maybe other users would like this too!?

So maybe we could have a checkbox on our settings-page? :

□ Acoustic Feedback