AleChromakey ( a better Chromakey)

Brainstorm
1

Greetings, everyone! Currently, my primary video editor is Lightworks. I appreciate Kdenlive and have even made a donation to the project; however, there is one major reason why I cannot switch the chroma key in Lightworks is astounding. Unfortunately, I do not possess a clean green screen setup, so I require more sophisticated tools. I have explored every option within Kdenlive, but it does not compare to what Lightworks offers. The FX Code being open-source, I copied it here to witness the magic!

// @Maintainer jwrl
// @Released 2023-08-04
// @Author baopao
// @Created 2013-06-07

/**
 This sophisticated chromakey has the same range of adjustments that you would expect to
 find on expensive commercial tools.  It's particularly effective on fine detail.

 NOTE:  This effect is only suitable for use with Lightworks version 2023 and higher.
*/

//-----------------------------------------------------------------------------------------//
// Lightworks user effect AleChromakey.fx
//
// Created by baopao (http://www.alessandrodallafontana.com).
//
// Version history:
//
// Updated 2023-08-04 jwrl.
// User parameters reformatted.
//
// Updated 2023-05-16 jwrl.
// Header reformatted.
//
// Conversion 2023-01-26 for LW 2023 jwrl.
//-----------------------------------------------------------------------------------------//

#include "_utils.fx"

DeclareLightworksEffect ("ALE ChromaKey", "Key", "Key Extras", "A sophisticated chromakey that is particularly effective on fine detail", CanSize);

//-----------------------------------------------------------------------------------------//
// Inputs
//-----------------------------------------------------------------------------------------//

DeclareInputs (fg, bg);

DeclareMask;

//-----------------------------------------------------------------------------------------//
// Parameters
//-----------------------------------------------------------------------------------------//

DeclareIntParam (ChromaKey, "ChromaKey", kNoGroup, 0, "Green|Blue");

DeclareFloatParam (RedAmount, "Red amount", kNoGroup, kNoFlags, 0.5, 0.0, 1.0);
DeclareFloatParam (FgVal, "Fg val", kNoGroup, kNoFlags, 0.45, 0.0, 1.0);
DeclareFloatParam (BgVal, "Bg val", kNoGroup, kNoFlags, 0.25, 0.0, 1.0);
DeclareFloatParam (GammaFG, "Gamma fg", kNoGroup, kNoFlags, 2.0, 0.0, 4.0);
DeclareFloatParam (GammaBG, "Gamma bg", kNoGroup, kNoFlags, 0.4, 0.0, 2.0);
DeclareFloatParam (GammaMix, "Gamma mix", kNoGroup, kNoFlags, 2.0, 0.0, 5.0);
DeclareFloatParam (Despill, "Despill blur", kNoGroup, kNoFlags, 0.5, 0.0, 1.0);

DeclareColourParam (ColorReplace, "Color replace", kNoGroup, kNoFlags, 0.5, 0.5, 0.5, 1.0);

DeclareFloatParam (_OutputWidth);
DeclareFloatParam (_OutputHeight);

//-----------------------------------------------------------------------------------------//
// Code
//-----------------------------------------------------------------------------------------//

DeclarePass (FG)
{ return ReadPixel (fg, uv1); }           // Color FG

DeclarePass (BG)
{ return ReadPixel (bg, uv2); }           // Color BG

DeclarePass (BlurSub)
{
   float4 retval = tex2D (BG, uv3);

   float2 offs = float2 (Despill / _OutputWidth, 0.0);
   float2 xy = uv3 + offs;

   retval += tex2D (BG, xy); xy += offs;
   retval += tex2D (BG, xy); xy += offs;
   retval += tex2D (BG, xy); xy += offs;
   retval += tex2D (BG, xy); xy += offs;
   retval += tex2D (BG, xy); xy += offs;
   retval += tex2D (BG, xy);

   xy = uv3 - offs;

   retval += tex2D (BG, xy); xy -= offs;
   retval += tex2D (BG, xy); xy -= offs;
   retval += tex2D (BG, xy); xy -= offs;
   retval += tex2D (BG, xy); xy -= offs;
   retval += tex2D (BG, xy); xy -= offs;
   retval += tex2D (BG, xy);

   return retval /= 13.0;
}

DeclarePass (Blur)
{
   float4 retval = tex2D (BlurSub, uv3);

   float2 offs = float2 (0.0, Despill / _OutputHeight);
   float2 xy = uv3 + offs;

   retval += tex2D (BlurSub, xy); xy += offs;
   retval += tex2D (BlurSub, xy); xy += offs;
   retval += tex2D (BlurSub, xy); xy += offs;
   retval += tex2D (BlurSub, xy); xy += offs;
   retval += tex2D (BlurSub, xy); xy += offs;
   retval += tex2D (BlurSub, xy);

   xy = uv3 - offs;

   retval += tex2D (BlurSub, xy); xy -= offs;
   retval += tex2D (BlurSub, xy); xy -= offs;
   retval += tex2D (BlurSub, xy); xy -= offs;
   retval += tex2D (BlurSub, xy); xy -= offs;
   retval += tex2D (BlurSub, xy); xy -= offs;
   retval += tex2D (BlurSub, xy);

   return retval /= 13.0;
}

DeclareEntryPoint (AleChromakey)
{
   float4 color = tex2D (FG, uv3);
   float4 colorBG = tex2D (BG, uv3);
   float4 colorBGblur = tex2D (Blur, uv3);

   if (IsOutOfBounds (uv1) || (color.a == 0.0)) { color = colorBG; }
   else {
      float4 colorBGblur = tex2D (Blur, uv3);

      float MixRB, KeyG;

      if (ChromaKey) {                              // Blue key
         MixRB = saturate (color.b - lerp (color.r, color.g, RedAmount));
         KeyG  = color.b - MixRB;
      }
      else {                                        // Green key
         MixRB = saturate (color.g - lerp (color.r, color.b, RedAmount));
         KeyG  = color.g - MixRB;
      }

      float MaskFG = saturate (1.0 - MixRB * FgVal / KeyG);
      float MaskBG = saturate (MixRB / BgVal);

      MaskFG = pow (MaskFG, 1.0 / GammaFG);
      MaskBG = pow (MaskBG, 1.0 / GammaBG);

      float OverMask = 1.0 - MaskFG - MaskBG;

      if (ChromaKey) { color.b = KeyG; }
      else color.g = KeyG;

      float4 retval = lerp (color, ColorReplace + colorBGblur, MixRB) * MaskFG;

      retval += colorBG * MaskBG;

      color = lerp (retval, pow (retval, 1.0 / GammaMix), OverMask);
   }

   return lerp (colorBG, color, tex2D (Mask, uv3).x);
}
2

Where did this code come from? You’ve pasted some source, but I don’t see any licence for it? And without one (that is compatible with the Kdenlive licence), we can’t use it.

It’s not enough that you’ve just found it on the internet somewhere - and if it’s not compatibly licenced, then potentially you tainted our ability to create a clean room implementation of the same algorithm too.

If there’s a known better algorithm we should look at that’s awesome - but if there isn’t a compatible implementation we can use, then someone is going to need to provide a description of the algorithm that someone else can provide a suitable clean room implementation of.

3

i don’t know what kind of license this is. I just know this are community effects and here is the repo → GitHub - fx-planet/lwks-fx-bundle: Synced user effects pack

4

AFAICS nothing in that repository declares any kind of licence at all. Which generally speaking means nobody has any right to do anything at all with what’s in it except the individual authors of the various things in it.

Which I assume is not their intent, so it might be nice to have them clarify that if they actually want them used.

5

If you go to PR page in the repo linked above, you can see that there is an open PR with a link to a forked repo, which contains the license info (new user, unable to add link). I believe the CC BY-NC means that the code isn’t compatible with Kdenlive’s license.

6

Thanks for that! (the forked repo is here: GitHub - LWKS-Software/lwks-fx-bundle: Synced user effects pack)

Though I don’t think that helps us. If we take that page at face value, the choice of CC-BY-NC-SA is not compatible with GPL3 used by Kdenlive and many of its deps. The GPL explicitly permits commercial use and removing that permission is generally accepted to make a licence ‘non-free’.

But it’s not clear to me whether we are actually able to do that either, since it’s not at all clear under whose authority this has been ‘licenced’. To do that legally it would have to be approved by all the authors of the code that effects, and afaics this extra clause in the readme was simply added as an act of fiat by that repositories maintainer. We’d need a compatible licence granted by the original and contributing authors.

7

@Ron I’m still pretty new to video editing, so I don’t have a lot of experience yet. Lightworks’ chroma tools aren’t necessarily superior they’re just more user friendly. I discovered that removing the green border requires a despill step. In my view, Kdenlive feels a lot like Bitwig it offers a collection of straightforward building blocks that you can combine into anything once you know how to use them and what you need.

8

Ok, that’s interesting … so the ALE algorithm you indicated doesn’t actually work better (in terms of the final result you get) - it just does the whole job in one step?

I don’t normally do a lot of chromakey - so I wonder if there is some benefit to keeping those steps separate or if there is some “UI tweak” (which is probably a bit more than just UI if it involves combining two effects, but still …) that we should do to make this more user friendly?

Who here is our chromakey wizard that knows The Answers to that?

9

is fine to have the components separated, because than you are flexible the think is tried the chromakey in Lightworks and Davinci and as noob i was able to get quick results. For my Videos i know what to do now in the Kdenlive. I think this can be a topic again in the far future ^^

1 Like
10

@Ron I’ve put together a new video where I walk through what I figured out on my own and what I’m still exploring. I’m not expect an answer, I just wanted to share my thoughts.

Video => Proton Drive

11

@marix Your audio is crackling because you are recording your mic using Audio Input Capture (PulseAudio). This is a known issue (there are two links). When I tested using PulseAudio, I got the exact same cutouts you did. I’m not sure why you need a separate source to record your mic audio, but if you really need it, there is a very good plugin which I use personally that allows you to record audio sources using PipeWire instead of PulseAudio.

12

thanks i did recognize that i already recording my mic ^^

13

I don’t use OBS to know its quirks, but if you’re on a modern enough distro to have PipeWire, you should be able to pretty much completely remove PulseAudio from your audio chain, even for Apps still using the Pulse API, since it provides a compatibility shim. But you might have to choose your distro base packages carefully to enable that.

As I noted before I’ve not done a lot of chromakey, so there are surely people much better placed than me to offer hints about Best Practice with it. But I do know DSP, so there’s a couple of things I’d note about your observations on this.

I think you’re right about about the ‘blur’ being used to smooth the keyed edge - and if you look very carefully at the lightworks version, you’ll see the ‘shimmering’ of the hard mask edge has actually been replaced with a ‘halo’ where the semi-transparent green forms a highlight around the edge of the pan.

It might be visually less offensive than the shimmering, but that is the tradeoff of the blur, and it still makes it ‘obvious’ that you’re a green screen actor in the scene.

And I wonder if the problem is worse because you’re keying around the polished steel pan, which is reflecting some of the green and being mistaken for it at its very edge.

Do you see the same sort of thing trying to mask a very non-reflective object?

@berndmj is there some easy way we can layer a gentle edge blur on a chroma mask, or is that something we’d really need to build into the effect. Or some better way to handle this?

Thanks for sharing this video though, it is nice to be able to compare things like this.

ed: and just thinking out loud - I wonder if you might get better results of you turn your camera’s “sharpness” control (if it has one) to its minimum or neutral value.

A lot of modern cameras automatically “sharpen” images these days, which could be artificially hardening the edge that the chromakey is trying to detect in ways that doesn’t play nice. So you could also try preprocessing with an ‘unsharp’ filter - but not fighting the camera over applying/unapplying that would still likely be the best approach if that is an option with the camera(s) you use.

14

The only thing I can think of in this context is to use feathering in the Rotoscope effect.

I am really impressed by the way OP managed to get very good chromakeying to work. The small spilling (or triangles as he called them) is not noticable at all. IMO, it is even better than what you can see on German news television around the hair of some of the anchors.