[Phredreeke]

Both me and MrFlibble are trying to do the same thing in different ways. His approach is using XBRZ as a pre-processor to smoothen edges, while I use a script that creates an antialiased mask which is then used to soften the image.

[MrFlibble]

 leilei, on 21 April 2019 - 10:58 AM, said:

It's nowhere near as interesting as results with neural scalers (if there is any esrganning in those last 3 pictures, it's eaten off by the brz too much to be visible).

I'll partially agree with that, but for objectivity's sake here's a comparison of the images above done with (a) xBRZ and (b) ESRGAN with multipass preprocessing, both scaled down to 640x480 w/ Sinc interpolation and no further editing or layer blending:
https://imgur.com/a/YjJ0uHl

The difference between the pure ESRGAN result and the blended images I posted above doesn't seem that great to me except the latter feel more blurry; also ESRGAN alone failed to properly process some of the facial features on the Arkania shot, notably the eye of the guy on the far right.

Maybe if the ESRGAN layer is duplicated when blending the result would be sharper but still benefit from the xBRZ layer smoothing some of the rough parts created by ESRGAN?

[Phredreeke]

Can you upload the images without upscaling? I wanna see how my method fares

[MrFlibble]

Good idea, here you go:
https://imgur.com/a/kACYtf9

[Phredreeke]

These are 640x400 because my script does 50% downscale. As expected the texts are mangled.




With less pre-upscale antialiasing



And this is how the last one looks without sharpening prior to downscale (and at the right aspect ratio since I wasn't using my script) For this already fairly contrasty image it ends up looking better



I'm too tired to do my hybrid method right now (requires a bit of pre-setup) but maybe I'll give that a go tomorrow. I do have another image to show off my hybrid method though




(original image)

[MrFlibble]

I gave a quick shot at the Duke3D shareware teaser screen too:

1) Pure ESRGAN upscale of a three-pass preprocessed image, using an ESRGAN/Manga109 0.5 interpolation (scaled down to 640x480 with Sinc)


2) Same as above but scaled to 640x400 for comparison with uncorrected images


3) A result of somewhat improved blending with an xBRZ layer (see below for details)


So I had a couple of crazy ad hoc ideas of how to improve the blending method. First off, I decided in favour of a three-pass 4x image as opposed to a single 4x xBRZ upscale. But then I added a few more steps: first, I resized the blended image from 1280x800 to 1280x960 with nearest neighbour, and then applied the pixelise filter at pixel radius. After that, I scaled the result down to 640x480 with Sinc and then blended it again, now with the pure ESRGAN image that was scaled to the same dimensions using Sinc. After that I sharpened the result a bit by scaling up 4x using nearest neighbour and then back to the original dimensions with Sinc (that produces a sharpening effect of sorts; I could use the sharpen tool but I'm too lazy to try finding a good value ATM).

[Phredreeke]

This is my XBRZ+AA upscaled with ESRGAN using Manga109/PSNR interpolated model without further downscale or processing (linking it because it's huge)

https://i.imgur.com/RErpFb9.png

Here's my tweaked hybrid upscale

[MrFlibble]

 Phredreeke, on 23 April 2019 - 05:34 AM, said:

Manga109/PSNR interpolated model

Yeah, I also decided (before reading your post) to try out PSNR interpolation to get rid of excessive sharpness. It seems give a nice compromise between enhancement and blurring (here I used alpha = 0.8):

[MrFlibble]

Meanwhile someone trained a few more models for ESRGAN specifically for upscaling video game graphics (but not on video game graphics as training data).
https://esrgan.blogs.../blog-post.html

Unlike what we had to deal with previously these can do a pretty good job on 8-bit graphics without any pre-processing at all:

This is the cartoonpainted_80000 model with absolutely no pre-processing whatsoever. Pretty good eh? (yes, I'm aware that it is not perfect in some parts)

NOTE: It seems like the models with the lowest interpolation number (usually 5000) produce the least distorted images.

[MrFlibble]

I interpolated cartoonpainted_80000 with ESRGAN's native PSNR model at alpha 0.8:

net_PSNR_path = './models/RRDB_PSNR_x4.pth'
net_ESRGAN_path = './models/cartoonpainted_80000.pth'

and ran some Daggerfall sprites through it with a single xBRZ pass and no prior sharpening (basically my previous "softening" method):

original image

original image

original image

[Phredreeke]

I decided to try one of the other models (in this case I picked the 5000 iteration Lion King model) and see how it compared with my other methods. In this case there was no preprocessing for the LK model, and the only processing after upscale was a 50% downscale and palettisation. First in these pictures would be my non-hybrid method, followed by hybrid method, and finally LK model.


Non-hybrid method wrecks the access text in the keycard reader, also a lot of small details end up very gritty. Hybrid method preserves the text mostly. The LK model upscale comes out a tad sharper than the hybrid method, and also slightly cleaner!


Here the non-hybrid method comes out the sharpest. The details on the ring of Saturn appears slightly more pronounced. The hybrid method loses a lot of definition, although I would say the alien glyphs come out better in it. The LK model upscale has a lot less noise compared to the other two models. In the case of both the hybrid and the LK model upscale, the outer edges of the ring appear more jagged, something which the non-hybrid upscale smooths over.


Ok. The problems with the non-hybrid upscale is obvious. The text, while readable is fairly irreglar. There's a weird halo effect to the right of the girl with red hair. Most of those issues are gone in the hybrid upscale and the LK upscale. More detail can be made out in the LK model upscale, thanks to it not having to be pre-filtered. I may look into seeing if I can use the hybrid method with the LK upscale instead of the pre-filtered Manga109 one.

[MrFlibble]

Thanks for the test!

By the non-hybrid and hybrid methods, do you mean these are both still created with a Manga109Attempt/ESRGAN interpolation?

BTW, why did you pick the LK model out of the bunch? I didn't have the time to test them all yet, but Donald Duck seemed total garbage to me. From what I tested, I liked the results with the cartoonpainted models as stated above and the custom Manga109 version.

[Phredreeke]

Non-hybrid and hybrid both use Manga109attempt/PSNR interpolation. The difference is that hybrid uses a XBRZ based upscale as a fallback method.

I didn't test every model, LK was the first I tried. Since then I've tried using my hybrid method combining LK model with both Manga109attempt/PSNR interpolation as well as the XBRZ fallback with good results.

[MrFlibble]

I got some very good results with mymanga109_250000 interpolated with ESRGAN at alpha 0.8 (I noticed that PSNR tends to blur out some lines sometimes, ESRGAN gives sharper results most of the time):

More images here:
https://imgur.com/a/YUubwIk

Warcraft

On the other hand, I liked more the cartoonpainted result with Blake Stone:

Same with Realms of Arkania:

[Phredreeke]

This is with my hybrid script using LK and XBRZ upscales and revised post-process sharpening.



Edit: Just processed this using a new script, the best result I got for this tile so far I think

[MrFlibble]

Finally got decent results on all dudes' faces in the Strife test image:


Macil is happy to see you

This is the mymanga109_250000 interpolation.

[MrFlibble]

 Phredreeke, on 30 April 2019 - 01:43 AM, said:

I didn't test every model, LK was the first I tried.

Well, here are test results (each batch from same image pre-processed with a single xBRZ pass):
Strife character sprites
Dagerfall character sprites
Biker Bimbo poster

I have not included results for donaldduck and pinocchio models because the results are garbage.

The results generally seem to suggest that different models may get better results depending on image but of these I liked most cartoonpainted, mymanga and hercules.

[MrFlibble]

Same girls as here but using the Cartoon Painted model (BTW, I just realised why the results are so good with it: the small images were created for the data set using the nearest neighbour method):

Spoiler

Note how the outlines of the sprites are no way better than with Manga109.

And for a test, the same monsters as these:

Spoiler

I think we should no longer be shy of considering the prospects of using 4x upscales instead of 2x.

[Edit] Duh, I intended to post this in this thread but nevermind.

[Phredreeke]

Reminder for anyone making upscales for Build engine games, when you define your upscaled tiles don't forget to put nocompress nodownscale after the file name. I had lots of issues with my upscale packs until I figured that out

[Phredreeke]

Alright, I guess I can scratch this off the list of Build games I never expected to upscale (yes, that is the Doom palette, I was as surprised when I saw it myself)

http://s000.tinyuplo...507524277791292

Attached thumbnail(s)

• 
• 
• 
• 

[NightFright]

Has anyone already posted anything about the ESRGAN Neural Upscale Texture Pack for Jedi Knight?
It only works with JKGFXMod, but as a bonus this also gives you modern effects like 32-bit graphics, bloom, SSAO and parallax mapping, significantly improving the outdated visuals of the game.

Examples (old vs new):






I have tried it with my GOG.com version (latest release from January 2018) and it looks quite fantastic. To make it work, you will need some things, however:
- JKGFXMod
- JKUNOFFICIAL Patch
- Neural Upscale Texture Pack

1) Backup or rename your existing jk.exe (JKGFXMod will only work with the exe from the JKUnofficial patch)
2) Unzip jk.exe and jk-extension.dll from the unofficial patch into your JK dir
3) Unzip JKGFXMod into your Jedi Knight directory (jkgm folder, inject.exe, jkgm.json and render.dll)
4) Extract the "jknup" folder from the Neural Upscale Pack (the complete folder, not just the files inside) into the jkgm\materials subdir
5) Edit jkgm.json and replace default resolution [ 1024, 768 ] with your custom one, e.g. [ 1920, 1080 ] (you can also set hud_scale to something like 3.0 and max_anisotropy to 16.0 to have a bigger ingame HUD and 16x AF)
6) Load the game via inject.exe
7) Make sure you have 3D acceleration activated (it will say 16-bit, but graphics will be 32-bit nevertheless), then go ingame

[MusicallyInspired]

I've seen the screenshots but never tried it. It looks great.

[MrFlibble]

I just ran a quick test with some Doom sprites following the discussion here:

Cartoon Painted 80k results
MyManga109 250k results

I think I like MyManga109 250k (shown above against purple background) more even though it's a bit less sharp. The Doomguy's face looks a lot more like the original low-res image.

UPD: I think I slightly improved the method for getting clean sprite outlines for 2x images:

Spoiler

Basically you take the ESRGAN 4x result, make a copy in mtPaint, apply Kuwahara-Nagao blur, convert to original palette and scale down to 2x (it will be nearest neighbour by default because it's an indexed image).

Then you make another copy, scale it down to 2x using Sinc3 or Bicubic, scale back up in mtPaint using nearest neighbour, and apply all the same steps as above (Kuwahara-Nagao blur etc.).

So now you have two 2x Kuwahara-Nagao'd images, paste them one over another without the background (or with the background colour set to transparency) using any method you choose so that basically you get a blend of the two. Save this image and then open in GIMP, select the background by colour and increase selection by one pixel, then copy onto the image scaled down to 2x using some normal method like Sinc3.

[MrFlibble]

A small test of Hexen sprites. This time I blended the results of MyManga109_250k and CartoonPainted_80k models using the Layer -> Blend [media] function in G'MIC. They look fairly good at 4x and 3x IMO:



comparison 3x nearest neighbour image

And here, 2x versions. This time I like the Cartoon Painted one more:


MyManga109:

I wanted to compare this to the sprites from the Hexen Gigapixel upscale but it uses some counter-intuitive sprite size, something like 3.5x (certainly larger than 3x but smaller than 4x). The sprites obviously have problems with their edges.

[Phredreeke]

I've tackled upscaling an FMV from TekWar


I've made another video describing the process I used here

[MrFlibble]

Just found a wiki listing different models that are available:
https://upscale.wiki.../Model_Database

The CartoonPainted and MyManga models are only listed as an external link though.

[MrFlibble]

So I finally had some time and motivation to test the new models on the Upscale wiki page I linked to above. It's amazing how fast the improvements are made. Most no longer require any image pre-processing to achieve what previously had to be tricked out of the previous models by applying various "smoothing" tricks to source images.

Out of the bunch, I singled the Rebout model, or specifically rebout_interp. Here's what it can do to a sprite sheet from Strife I used for testing some posts above:

Mind you, the source image was not edited in any way.

I was not entirely satisfied with this and interpolated (at alpha = 0.5) rebout_interp with another model, Fatality, which was suggested to me by Phredreeke:

net_PSNR_path = './models/rebout_interp.pth'
net_ESRGAN_path = './models/4x_Fatality_01_265000_G.pth'

Same pic with this model:

More results here:
https://imgur.com/a/A9Jgocl

[MrFlibble]

I further interpolated (again, at alpha = 0.5) the model I used right above with DeToon, which is a model that is supposed to diminish the cartoon feel of the upscaled image, and run a few pic that I had used before (previous results shown earlier in this thread). Please forgive my obsession with Command & Conquer screenshots:

Spoiler

More stuff here:
https://imgur.com/a/t9aMWI8

[Phredreeke]

I'm posting these here as examples of problem frames when upscaling the cheerleader of RRRA.



Any attempts I've made have seriously messed up her face. Here's an example (using a 50/50 interpolation between Fatality and DeToon)

[MrFlibble]

I went back to xBRZ "softening" and the result is slightly better I think:

This is the Fatality/Rebout 0.5 interpolation model I described above, the source image was 4x'd with xBRZ then scaled down to original size with GIMP at Bicubic (no prior blurring this time), then converted to original colours.

Given this is just one frame in an animation I wouldn't worry much about the Nefertiti look.

Interestingly the older Cartoon Painted interpolation seems to do a worse job on the same pre-processed image:

Spoiler