Please add support for shader effect

Help
1

Basically, allow the user to write a pixel shader in GLSL that would then be applied to all frames of a clip with that specific shader. It would make effect possibilities basically infinite and would work on almost any consumer GPU. The shader would have access to data such as currently rendered frame (both global, of the entire project, and of the clip the shader is attached to) as well as project’s FPS, so that the animated shader effects would be possible. Of course shaders would also be able to set up custom parameters that would show up in the effects panel (and be possible to animate like most effects params can be)

Eventually it would be possible to rewrite most (if not all) MLT and frei0r effects, including generators, to utilize the shader approach (as well as build a shader example library as a nice side effect, to make writing new shaders easier).

2

This is a lovely dream, and I wish it were a reality too, but the actual reality we have is a lot more messy…

Probably the easiest way to do this will be to support movit plugins in MLT, but that has been on the It’s Not That Simple list since at least 2017.

There apparently was some support in MLT for doing this via ‘WebVfx’, and it seems Shotcut exposed that for a brief period - until circa 2020 when Qt dropped support for that and QtWebKit, so it vanished again too …

And movit support has been disabled for quite some time now because it’s Troublesome to keep functional on a few fronts too - so that would be the first thing that needs fixing, and probably is in the scope of anyone who could write an actually usable and More Efficent Than CPU shader to spend some time on improving.

Which in turn brings us crashing into the reality of who would actually do this, and who would actually use it and implement high quality effects with it if someone spent all that time on making it possible, and what effects are currently ‘impossible’ without it, given:

They already are and do work on actually any consumer CPU. You can implement frei0r plugins that will ‘seamlessly’ drop in to any already existing kdenlive build. Which begs the question, why isn’t there already a contributed library of these infinite possibilities using that and what are we desperately missing that is impossible to do that way?

Anything is possible when you qualify it with Eventually, but that doesn’t automatically make it easy or even a Good Idea.

Some of those effects are already troublesome to even multithread on the CPU (hence the warning that enabling that may create rendering artifacts), which pretty severely limits any benefit to pushing them to the GPU (it’s not magically faster, aside from a few specialised operations the gains all come from the ability to run massively parallel implementations of things that can effectively be parallelised).

Writing good, let alone fast, new shaders is never going to be easy. Doing that well probably requires more skill than fixing any of the problems above - and since they aren’t yet fixed, it begs the question of when the people with those skills will Eventually find the time to do some more work on this.

And that’s before we even start on the security considerations involved in that.

Don’t get me wrong, I think this is a fantastic thing to wish for, and improving things in this space is on the roadmap. But there is a lot more work involved in both making it possible and keeping it working in a relatively fragile and fast moving space than just “allowing users to do it”…

Which means that, as always, it’s probably more useful to have and share very concrete wishes like “I would really like an effect that does X” than abstract aspirational ones like “imagine if it was possible to rewrite everything to be blindingly fast and never, ever crash or glitch, and so easy that absolutely anybody could do it!”.

We’re all already wishing for the latter. But we live in a world where the only way to get there is by tackling the former, one clear and present actual need at a time.

3

Olive video editor is working with GPU and shaders. But as I understand with FFMPEG effects only. Maybe it’s worth looking into Olive source code and check if some kind of that GPU acceleration could be applied using the MLT avfilters and maybe MOVIT.