What's the best way to upscale from 720p to 1080p?

Help
1

Okay so for a little context, I have a 1080p monitor, but my computer is not powerful enough to record at 1080p natively, so I have to downscale to 720p while recording.

I’m considering the option of upscaling my videos to 1080p since, after a chat with some friends, I learned that a lot of people see videos not having options above 720p as an immediate click off, so I want to retain those people.

I’m not exactly looking for anything magic, just some advice. I’m currently using the VAAPI AMD H264 rendering profile, with the rescale option set to 1920x1080, and that worked fairly well, the issue is that when I tested it on youtube (720 vs 1080) while the 1080p version did look a bit sharper, when I selected 720p, it looked a little worse than the version that was natively on 720p.

I’m looking for a way to avoid that, have 720p look just as good as before, even if it compromises the quality of 1080p a little. As for rendering times, I don’t care as much, as I render over night, though I do care about file sizes.

This is the render profile itself, fairly fast, surprisingly good quality, and low file size:

ab=192k acodec=aac channels=2 f=mp4 hwaccel=vaapi hwaccel_device=renderD129 hwaccel_output_format=vaapi real_time=-1 scale=1.5 threads=0 vb=30000k vcodec=h264_vaapi
1 Like
2

Honestly … Just Don’t Do That. I’m not sure what people you’re afraid of losing - but if the reason they aren’t watching 720p is because it looks horribly pixellated up close on their 4k or larger monitor, then you’re only going to make it more likely that they’ll switch off immediately and think horrible things about you if you pull this sort of bait and switch on them.

You aren’t going to gain any detail upscaling from 720 to 1080, it’s still going to be the same blurry image at that size, you’re just going to burn more of everyone’s bandwidth sending them pre-interpolated pixels instead of just scaling it up at the viewer’s end. And as you’ve just discovered, it ruins whatever quality you did have for everyone, because you do a lossy transcode and upscale of the original, then utube does a lossy transcode and downscale to provide a 720 version again, losing a lot of the original detail. You’ve basically just reinvented the problems with duplicating analogue recordings.

If you’re starting with a lossy 720p master, then every time you transcode it or re-render it to another lossy format, the quality degrades. If best quality is what you are after, just upload the original. And if that’s not good enough for your target audience, then you aren’t going to be able to fake it - you need to start with a better master. Go buy a cheap 4k camera and downscale to 1080 from that …

3 Likes
3

To be fair, there are some AI technologies around that claim can help upscaling from a lower resolution, but no idea how good they are.

So, yeah, just in case, what @Ron said: starting with the best and highest resolution master possible. Same goes for audio, by the way.

1 Like
4

Ron has a good point here. Although, I can provide some additions/caveats.

Firstly, instead of 4k camera to capture screen, 4k capture card is most likely meant.
Recording screen with camera has many complications and generally not desired.
And unless capture card itself does encoding for you, this may even not help.
But maybe more importantly… are you sure you can’t record in 1080p?..
There are very, very many things to tweak in how video recording/encoding is done.
Are you using encoding for recording in hardware (on GPU) or software (on CPU)?
Switch to other type may bring some benefits (and/or downsides), a consideration.
How about settings of the encoder? Does it use Quality or Speed preset (or similar)?
As example, for software x264 encoder, ultrafast preset, for all its flaws, is ultra fast.
And, my friend records in Xvid (older codec), which is a bit crazy, but also an option.
OBS (you use?) allows (albeit not ideally) for arbitrary ffmpeg video codec to try.
In some configurations, ffmpeg can record video and audio all by itself…
Or for less overhead, GPU Screen Recorder may be a (Linux-specific) option
And… how about frame rate? It’s different thing to sacrifice…
One may prefer (or not) 1080p at 30 FPS instead of 720p at 60 FPS.
What if we overload I/O with chosen bitrate (or adaptive bitrate, quality)?
How about a bit less? Same goes for background processes - less is more.

Following is indeed true:
Every alteration will make video less like original (720p original recording).

However…
Are we sure that bitrate (quality) YouTube provides for 720p is sufficient?
YouTube is fairly stingy with what it provides, I like 1440p videos on 1080p screen.
1440p specifically have another advantage (last I checked) - it forces YouTube to encode VP9 version (more modern video codec) of uploaded video.
And…
Are we sure that watcher’s upscaling would be better than what we would do?
E.g. to watch some 720p YouTube videos, I use ffplay with the following filtering…
-vf scale=1920:1080,cas=strength=0.5 -sws_flags lanczos
Said ffplay is minimal player based on ffmpeg, and you seem to use ffmpeg (in some wrapper, with syntax a bit different) to upscale. You also use VAAPI (hardware accelerated) pipeline, which differs from software (on processor) processing.
Said ffplay output is much sharper than what I get in-browser. Although, sharper is not always better. With great sharpening comes great responsibility, or rather, it can add (or reveal) undesirable artifacts. Lanczos is sharp (and often used) scaling filter. Default is bicubic, which is softer (but with smaller halos). CAS is Contrast Adaptive Sharpening, it’s fancy sharpening filter. Default strength of it is 0 (0.5 is aggressive).

There are a few more moments, which may be not very useful, but interesting…
Due to how important precision is for text, it’s often major pain/distortion to upscale (at least for small text it can be so).
Best upscaling is AI upscaling. Caveat being it’s extremely slow, and may introduce undesired details/style to the image. My go-to AI upscalers are waifu2x-ncnn-vulkan (closer to original) and Real-ESRGAN-ncnn-vulkan (more sharpness/detail). You can try these… on a single image. For fun. I do not joke when speak AI upscaling is slow, it’s slow even on strong PC. Well, small but fast ones also exist (say hi to NNEDI3, for example), but I did not use these, and they are weaker… by design.
I myself record in ffvhuff, which is quick and lossless video codec - generally bad idea: fast disk needed, files are super big.
An interesting option from 720p to 1440p is -sws_flags neighbor, this turns every pixel into 2x2 pixels squares. This may (or may not) be preferable, to keep pixels intact. Albeit, video compression will hurt this 2x2 grid.

So, as a conclusion: It is reasonable to leave things as is, but, there are more options to try out if desired.

1 Like
5

Okay so some added context, no I cannot record on 1080p (granted I didn’t tweak that many settings, but the only settings to tweak were bitrate and the encoder, which are set to 2500 and VAAPI HEVC), otherwise I would, that much should be obvious, the game I’m using as a control is Bug Fables. If I record at a native 1080p, it records at around 45-50fps, if I record at 720p downscaled with bicubic, it records at 60fps stable (the 36 samples one also lags unfortunately). This is on OBS, as I use Linux Mint and I don’t know of other alternatives that are anywhere near as fleshed out.

As for AI upscaling, I’m very anti-AI, so that’s a huge pass, plus it would probably alienate more people than regular upscaling if they notice it, but even if I wasn’t, it is an extra layer of work instead of being integrated in the workflow, and if I had a pc powerful enough to use AI-upscaling, I’d just be recording in 1080p natively or higher, lol.

While upscaling to 1440p sounds interesting, I feel like the filesize might be way too much, I already an 8gb file from a 35 minute episode when upscaling to 1080p, so that’d probably leave me with something like a 15gb file unless I use a better compression algorithm, which is literally the main thing that would hurt the upscaling, lol. For context I only have ~100gb of space which I use to juggle the recordings first and foremost, granted I should be doing a better job of keeping the episodes below 25 minutes but still. Might still mess around with it, though it feels a bit overkill, specially with upload and processing times.

–
I do have a capture card I use on Xbox (can be used on any console) which has “4K” capabilities but it’s probably BS, it can however record 1080p60 with crisp audio, I’m just not sure how I could even use that considering I record on my laptop on a single monitor setup, kinda (I use my TV as a second monitor with the same resolution as my laptop when I’m recording, it has the same performance as just using my laptop). But if there is a way I could record at 1080p without performance issues, I’d really like to know.

When I was discussing about upscaling with friends, a few did say that it was a really bad idea, so, if that is the general concensus, do you guys have any ideas on how to improve the quality 720p as much as possible? Since right now, as shown in the videos I sent, even native 720 is a tiny bit blurry.

Oh yeah, my specs: AMD Ryzen 3 3250U with Radeon Graphics × 2

6

What’s wrong with 45-50fps? This is not a FPS like Call of Duty. When you edit and then export/render your video you will drop to 25-30fps anyway, I reckon.

7

Wh-what do you mean what’s wrong with 45-50fps? Anything below 60fps is lag, I’m not talking about exporting between 30/60fps, I’m talking about frame drops. Not to mention I upload at 720p60 and render at CFR.

8

I think you have to make compromises under the circumstances: either you drop the fps when recording, or drop the quality when upscaling, or get beefier HW, or deal with AI.

Whatever floats your boat …

9

What’s wrong with 45-50fps?

It depends…

The fps metric is an approximation, in reality render takes some milliseconds per individual frames.
The (stable) 60 FPS is 16.6 ms spend on every frame, nice. And (stable) 50 fps is 20 ms per frame.
Sounds good? Now, how about unstable 50 fps with 60 fps target? It can be 180 ms stutter every second.
Or worse, 1 second long stutter every 5 and a half (roughly) seconds. Which observer is bound to notice.
That is why modern (good) reviews often show 1% low (worst) of fps metric or even graph of ms times to render.
There’s also uneven frame pacing due to container, constant (almost always used/preferred) fps of video itself.
It cannot just show (or record) 20 ms frame, it goes only in 16.6 ms chunks, and it’ll add extra delay or skip frame.
Situation is further complicated by individual’s tolerances, Someone will say 20 fps is okay, the other 60 fps is basic.
And, how much of (preferably stable) fps is desired also depends on content, gaps between frames bigger with fast motion.

10

but the only settings to tweak were bitrate and the encoder

That is not true, at least with some of the encoders you may (or may not) choose.
Here, a screenshot of OBS on my system (again, ffvhuff is generally a bad idea).

example 1
example 1747×772 53 KB

…for advanced encoder settings one can consult ffmpeg codecs documentation.
Although, not all ffmpeg encoders work well and OBS syntax differs a bit here.

And here’s setting to change fps of the recording…
example 2
…30 fps (at 1080p) may be considered, since 60 is cleanly divisible by it.

This is on OBS

And OBS is pretty much THE standard for modern era video recording/streaming. It’s nice.

a better compression algorithm
thing that would hurt the upscaling

It depends. Better (be it codec or setting) encoding may give smaller file, better quality, or both.
The classic trade-off is that more modern codecs less mature and higher settings are slower.
And by “less mature” I mean e.g. tendency for AV1 to blur things (bad for high quality video).
Modern codec can be both faster and better, under some settings (say hi to SVT-AV1 vs x265).
There is also a (somewhat) separate thing - how many bits per second (bitrate) codec is targeting.
Or alternatively, which level of quality (e.g. CQ or CRF) to size ratio. These are target, not how it met.

I do have a capture card
I use my TV as a second monitor

And if said card works under Linux and can intercept signal from laptop to a TV, it’s worth a consideration
And, I can’t say more, I never used capture cards and have little to no knowledge about 'em.

about upscaling with friends, a few did say that it was a really bad idea

Again, it depends. What are you trying to achieve with upscaling? Is it worth it?
Upscaling is a damaging/altering process, by design. And should be done only when needed.
My personal opinion is that upscaling is good… if done well and justified. Otherwise, it’s bad.
Also, upscaling and re-encoding are bad for archival, although, this is not what you do here.

improve the quality 720p as much as possible

Well, the aforementioned cas filter of ffmpeg should work on 720p just fine.
Although, of course, another layer of video compression added because of this.
So, it’s more like a trade-off for you, and also, don’t use too high strength of cas.
Other methods include change of color space or downscaling and likely irrelevant.

Finally. Whatever you choose to do (or not to do) with all of this…
1080p vs 720p is not that important in comparison to what videos themselves are.

11

for advanced encoder settings one can consult ffmpeg codecs documentation.

I’m not really tech savvy at all to write my own codecs, I just use what is available through the GUI.

image
image981×982 73 KB

I do have a capture card

I just came back from testing it, aside from being incredibly cumbersome, it was surprisingly to me, exactly the same performance as just recording natively in 1080p. I’m not sure why this was a surprise. The performance drop might have to do with recording the 1080p canvas itself and not with how I render stuff in the canvas.

if done well and justified

I already downscale when recording, so it’s probably not a great idea to add even more artifacts by upscaling back, but still, mainly looking for ideas.

improve the quality 720p as much as possible

I mainly want to return some of the sharpness back from the downscaling, whenever I render the final video I always notice it’s slightly blurry, even when the game looks super sharp.

cas filter of ffmpeg

I’m not really sure what this means, I thought this was a setting from a browser extension. Do I just add cas=strength=0.25 on FFmpeg Options in OBS or on the kdenlive render encoder?

1080p vs 720p is not that important in comparison to what videos themselves are.

I am aware, but it is still a factor, as minor as it is, I will not be able to upgrade my hardware for the foreseeable future, probably stuck with this laptop for the next 5 years or so. Maybe right now being stuck in 720p territory isn’t the end of the world, but YT already doesn’t recognize 720p as HD (hence why a lot of people immediately click off videos if they don’t see 1080p+), who knows what might happen in the next few years.

12

I’m not really tech savvy at all to write my own codecs, I just use what is available through the GUI.

This is not writing your own codec, no-no, it’s just choosing settings. Similar to “render profile itself” of yours.
Albeit, it is indeed convoluted process, 'cause ffmpeg is convoluted (I’m just accustomed to it).

might have to do with recording the 1080p canvas

I’ve had problem with this myself, sorry for thinking that your issue is unlikely to be analogous.
But solution I used is ill-suited for you - switching Linux kernel (to real-time one) is significant risk.
Good news is that Linux 6.6 have lower latency scheduler, which likely to have similar effect…
When Linux Mint will adopt it as a default one (or maybe later version of kernel) - I have no idea.
For now, I suppose, capture card is a solution. And then we’ll hope Linux 6.6 happens soon.
GPU Video Recorder grabs frames less invasively (may help too), but it no OBS, a separate program.

I’m not really sure what this means, I thought this was a setting from a browser extension.

No! This means using https://ffmpeg.org/ffmpeg-filters.html#cas via https://ffmpeg.org/
CAS is a nice sharpening filter developed by AMD, makes image less blurry.
And ffmpeg is an amazing command line tool for media manipulation.
Sorry if ffmpeg is confusing to you, it just what I do know, what I can speak of.

Do I just add cas=strength=0.25 on FFmpeg Options in OBS or on the kdenlive render encoder?

Well, OBS uses ffmpeg and Kdenlive uses ffmpeg… many things media use ffmpeg under the hood.
Anyway, due to this fact, your idea is likely to work. More likely with Kdenlive, since CAS a bit slow.

13

capture card is a solution

It really isn’t, performance is exactly the same, while being more annoying to set up with slightly less quality, since I have to create a feedback loop between my laptop and PC instead of just recording natively (OBS didn’t like my capture card on default settings, may have to do with missing drivers).

More likely with Kdenlive, since CAS a bit slow.

I, have no idea how. I just spent the last hour fiddling with kdenlive and I found absolutely no way. I tried cas=strength=1, cas_strength=1, cas=[strength=1], cas=“strength=1”, “cas=strength=1”, ffmpeg_cas_strength=1, ffmpeg=“cas=strength=1”, amongst many others, and all results were the same. I am at a loss, the documentation didn’t provide any helpful info on how to use the filter on kdenlive either.

Tested with 1 since that would make it pretty obvious if it worked.

14

It really isn’t

Sorry, I understood mention of canvas incorrectly, as if it was the problem and now it is solved via capture card, exactly like for Xbox 1080p capture. I was wrong.

I just spent the last hour fiddling

Sorry, I assumed that it would be trivial change of syntax or that only 1 try (or few) will be attempted. And strength at 1 is beyond obvious (it looks terrible).

Hm… have you tried recording 1080p in OBS via x264 (or libx264, same thing) on ultrafast preset?

example 3
example 3632×607 33.9 KB

And here’s how to use ffmpeg directly (I was writing this, yes)…

ffmpeg -i path_to_your_video -c:v libx264 -crf 18 -c:a copy -vf cas=strength=0.3 path_to_output_video

…assuming that ffmpeg is installed and in PATH (I think will happen automatically, at least upon reboot) of your system.
This is what you need to write in terminal of your choice and then press Enter and wait until it finishes.
The -i flag denotes input, and path is file path, it probably can be done automagically by drag n’ drop of file.
The -c:v denotes codec for video, in our case x264, a software encoder for an old established codec.
Feel free to use VAAPI for better speed here (akin to render profile of yours), this just an example I know.
The -crf 18 chooses Constant Rate Factor of 18, or in other words, high quality. Chose 16 for even higher.
You can add -preset fast after libx264 for better speed, but a bit less quality and a bit bigger file size.
I prefer -preset veryslow, but it is just me thing, it is indeed, very slow (which is often not desirable).
The -c:a copy command to copy the audio stream, this way no additional loses are done.
You can replace it with -c:a aac -b:a 192k to replicate your profile (but why re-encode audio here?).
The -vf starts chain of filters, cas is a chosen filter and weird =strength=0.3 is self-explaining too (chooses cas strength).
And the path to output file does not require special flag, the typical extension for it is .mp4 (this is your choice too, it seems).
Also, paths with spaces in them should be written inside of “double quotes” or ‘quotes’, but drag and drop, if works, likely to do this for you.

This is all I've got. I'm very sorry if my adjacent knowledge was insufficient.
15

*You can get fresh ffmpeg as static build from its website (will need path to executable, not just writing “ffmpeg” in terminal), or just use your distribution package manager or “store”, ffmpeg also may be already installed.

16

But this is for using the ffmpeg library directly through the terminal, I need to use the cas filter through kdenlive, not through the terminal. I’m not gonna render the video and then render it again just to add the filter. I literally don’t have neither the time nor space for that.

Even then, when I try do it without hardware acceleration (libx264), it’s pathetically slow, like 10fps, and when I enable hardware acceleration I get this error. It also happens with h264_vaapi and any other vaapi encoder.

image

Exactly like for Xbox 1080p capture.

I’m, actually not sure if I can record xbox footage on 1080, lol. When I did xbox content, I also downscaled to 720 for consistency’s sake with the rest of my channel.

Have you tried recording 1080p in OBS via x264

I tried just now, and it was unsurprisingly considerably worse than hardware acceleration, however, I completely forgot that due to some issues with CPU management, someone in the Mint forums recommended I upgraded my kernel to the latest version. So now that I did, recording at native 1080p is actually maybe reasonable, I might need to fiddle with some settings still.

It gives me 60fps most of the time, though it still drops to 55 sometimes instead of being constantly on 60 like when I downscale to 720. I also need to speedrun to an area of the game where there are a bunch of particle effects to test if it can handle that on 1080p, so all in all, it’s still a huge maybe.

That being said, what’s the main difference between HEVC and 264? I had the idea that HEVC was better, but 264 seemed slightly more stable during my tests.

17

*Oh, nevermind, I see the difference.

untitled-f001274
untitled-f0012741920×1080 136 KB

18

That being said, what’s the main difference between HEVC and 264?

These are 2 different standards of video compression.
HEVC is a newer one, with more tools in its toolbox.

19

Just saw this post and remembered this topic:

20

The AMD hardware encoders on this “early” generations are - excuse my French - dogshit. It’s hard making a cake squeezing that lemon. CPU recording with x264 is going to look better, but it will tax the system.

Youtube usually reencodes your upload, so if you upload a somewhat “sharper” 1080p the reencoded 720p will look a little bit worse. Nothing you can do about that.

You want to make your source material look as good as possible. And considering the hardware limitations I would go down to 720p 30-50fps fps. Trade resolution and frames for a consistent experience with as less encoding artifacts (blocks/jitter) as possible. The blocks are imho much more jarring than the lower resolution.