Duke4.net Forums: The ESRGAN AI Upscale non-Duke thread - Duke4.net Forums

Jump to content

Hide message Show message
Welcome to the Duke4.net Forums!

Register an account now to get access to all board features. After you've registered and logged in, you'll be able to create topics, post replies, send and receive private messages, disable the viewing of ads and more!

  • 5 Pages +
  • « First
  • 2
  • 3
  • 4
  • 5
  • You cannot start a new topic
  • You cannot reply to this topic

The ESRGAN AI Upscale non-Duke thread

User is offline   Phredreeke 

#91

Both me and MrFlibble are trying to do the same thing in different ways. His approach is using XBRZ as a pre-processor to smoothen edges, while I use a script that creates an antialiased mask which is then used to soften the image.
0

User is offline   MrFlibble 

#92

View Postleilei, on 21 April 2019 - 10:58 AM, said:

It's nowhere near as interesting as results with neural scalers (if there is any esrganning in those last 3 pictures, it's eaten off by the brz too much to be visible).

I'll partially agree with that, but for objectivity's sake here's a comparison of the images above done with (a) xBRZ and (b) ESRGAN with multipass preprocessing, both scaled down to 640x480 w/ Sinc interpolation and no further editing or layer blending:
https://imgur.com/a/YjJ0uHl

The difference between the pure ESRGAN result and the blended images I posted above doesn't seem that great to me except the latter feel more blurry; also ESRGAN alone failed to properly process some of the facial features on the Arkania shot, notably the eye of the guy on the far right.

Maybe if the ESRGAN layer is duplicated when blending the result would be sharper but still benefit from the xBRZ layer smoothing some of the rough parts created by ESRGAN?

This post has been edited by MrFlibble: 22 April 2019 - 03:19 AM

0

User is offline   Phredreeke 

#93

Can you upload the images without upscaling? I wanna see how my method fares
1

User is offline   MrFlibble 

#94

Good idea, here you go:
https://imgur.com/a/kACYtf9
0

User is offline   Phredreeke 

#95

These are 640x400 because my script does 50% downscale. As expected the texts are mangled.

https://i.imgur.com/KaTY7mb.png


With less pre-upscale antialiasing

https://i.imgur.com/zsyPfGw.png

And this is how the last one looks without sharpening prior to downscale (and at the right aspect ratio since I wasn't using my script) For this already fairly contrasty image it ends up looking better

https://i.imgur.com/fryZChR.png

I'm too tired to do my hybrid method right now (requires a bit of pre-setup) but maybe I'll give that a go tomorrow. I do have another image to show off my hybrid method though

https://i.imgur.com/Hg9kZTO.png


(original image)
0

User is offline   MrFlibble 

#96

I gave a quick shot at the Duke3D shareware teaser screen too:

1) Pure ESRGAN upscale of a three-pass preprocessed image, using an ESRGAN/Manga109 0.5 interpolation (scaled down to 640x480 with Sinc)
https://i.imgur.com/UqEGzbv.png

2) Same as above but scaled to 640x400 for comparison with uncorrected images
https://i.imgur.com/jbi09CO.png

3) A result of somewhat improved blending with an xBRZ layer (see below for details)
https://i.imgur.com/MQnvoU8.png

So I had a couple of crazy ad hoc ideas of how to improve the blending method. First off, I decided in favour of a three-pass 4x image as opposed to a single 4x xBRZ upscale. But then I added a few more steps: first, I resized the blended image from 1280x800 to 1280x960 with nearest neighbour, and then applied the pixelise filter at pixel radius. After that, I scaled the result down to 640x480 with Sinc and then blended it again, now with the pure ESRGAN image that was scaled to the same dimensions using Sinc. After that I sharpened the result a bit by scaling up 4x using nearest neighbour and then back to the original dimensions with Sinc (that produces a sharpening effect of sorts; I could use the sharpen tool but I'm too lazy to try finding a good value ATM).
0

User is offline   Phredreeke 

#97

This is my XBRZ+AA upscaled with ESRGAN using Manga109/PSNR interpolated model without further downscale or processing (linking it because it's huge)

https://i.imgur.com/RErpFb9.png

Here's my tweaked hybrid upscale

https://i.imgur.com/v44YoiC.png
1

User is offline   MrFlibble 

#98

View PostPhredreeke, on 23 April 2019 - 05:34 AM, said:

Manga109/PSNR interpolated model

Yeah, I also decided (before reading your post) to try out PSNR interpolation to get rid of excessive sharpness. It seems give a nice compromise between enhancement and blurring (here I used alpha = 0.8):
https://i.imgur.com/ovwo6IG.png

https://i.imgur.com/s0gbAtu.png

https://i.imgur.com/RWhkVJE.png
0

User is offline   MrFlibble 

#99

Meanwhile someone trained a few more models for ESRGAN specifically for upscaling video game graphics (but not on video game graphics as training data).
https://esrgan.blogs.../blog-post.html

Unlike what we had to deal with previously these can do a pretty good job on 8-bit graphics without any pre-processing at all:
https://i.imgur.com/jQWBnVs.png https://i.imgur.com/tCoRaOm.png
This is the cartoonpainted_80000 model with absolutely no pre-processing whatsoever. Pretty good eh? (yes, I'm aware that it is not perfect in some parts)

NOTE: It seems like the models with the lowest interpolation number (usually 5000) produce the least distorted images.

This post has been edited by MrFlibble: 27 April 2019 - 02:05 PM

0

User is offline   MrFlibble 

#100

I interpolated cartoonpainted_80000 with ESRGAN's native PSNR model at alpha 0.8:
net_PSNR_path = './models/RRDB_PSNR_x4.pth'
net_ESRGAN_path = './models/cartoonpainted_80000.pth'

and ran some Daggerfall sprites through it with a single xBRZ pass and no prior sharpening (basically my previous "softening" method):
https://i.imgur.com/s4qXjRk.png
original image
https://i.imgur.com/vfQhmHZ.png
original image
https://i.imgur.com/09Xv2LV.png
original image

This post has been edited by MrFlibble: 28 April 2019 - 11:26 AM

1

User is offline   Phredreeke 

#101

I decided to try one of the other models (in this case I picked the 5000 iteration Lion King model) and see how it compared with my other methods. In this case there was no preprocessing for the LK model, and the only processing after upscale was a 50% downscale and palettisation. First in these pictures would be my non-hybrid method, followed by hybrid method, and finally LK model.

Attached Image: tile0131-comparison.png
Non-hybrid method wrecks the access text in the keycard reader, also a lot of small details end up very gritty. Hybrid method preserves the text mostly. The LK model upscale comes out a tad sharper than the hybrid method, and also slightly cleaner!

Attached Image: tile0484-comparison.png
Here the non-hybrid method comes out the sharpest. The details on the ring of Saturn appears slightly more pronounced. The hybrid method loses a lot of definition, although I would say the alien glyphs come out better in it. The LK model upscale has a lot less noise compared to the other two models. In the case of both the hybrid and the LK model upscale, the outer edges of the ring appear more jagged, something which the non-hybrid upscale smooths over.

Attached Image: tile0964-comparison.png
Ok. The problems with the non-hybrid upscale is obvious. The text, while readable is fairly irreglar. There's a weird halo effect to the right of the girl with red hair. Most of those issues are gone in the hybrid upscale and the LK upscale. More detail can be made out in the LK model upscale, thanks to it not having to be pre-filtered. I may look into seeing if I can use the hybrid method with the LK upscale instead of the pre-filtered Manga109 one.
1

User is offline   MrFlibble 

#102

Thanks for the test!

By the non-hybrid and hybrid methods, do you mean these are both still created with a Manga109Attempt/ESRGAN interpolation?

BTW, why did you pick the LK model out of the bunch? I didn't have the time to test them all yet, but Donald Duck seemed total garbage to me. From what I tested, I liked the results with the cartoonpainted models as stated above and the custom Manga109 version.
0

User is offline   Phredreeke 

#103

Non-hybrid and hybrid both use Manga109attempt/PSNR interpolation. The difference is that hybrid uses a XBRZ based upscale as a fallback method.

I didn't test every model, LK was the first I tried. Since then I've tried using my hybrid method combining LK model with both Manga109attempt/PSNR interpolation as well as the XBRZ fallback with good results.
0

User is offline   MrFlibble 

#104

I got some very good results with mymanga109_250000 interpolated with ESRGAN at alpha 0.8 (I noticed that PSNR tends to blur out some lines sometimes, ESRGAN gives sharper results most of the time):
https://i.imgur.com/ersVQHr.png
More images here:
https://imgur.com/a/YUubwIk

Warcraft
https://i.imgur.com/MJcQc5M.png
On the other hand, I liked more the cartoonpainted result with Blake Stone:
https://i.imgur.com/ZdHXhJV.png
Same with Realms of Arkania:
https://i.imgur.com/cwfHKES.png

This post has been edited by MrFlibble: 30 April 2019 - 01:58 AM

0

User is offline   Phredreeke 

#105

This is with my hybrid script using LK and XBRZ upscales and revised post-process sharpening.

Attached Image: tile0964_rlt.pngAttached Image: tile0965_rlt.png

Edit: Just processed this using a new script, the best result I got for this tile so far I think

Attached Image: tile0866_rlt.png

This post has been edited by Phredreeke: 30 April 2019 - 03:04 AM

0

User is offline   MrFlibble 

#106

Finally got decent results on all dudes' faces in the Strife test image:
https://i.imgur.com/zgNkTMQ.png

Macil is happy to see you :)

This is the mymanga109_250000 interpolation.
0

User is offline   MrFlibble 

#107

View PostPhredreeke, on 30 April 2019 - 01:43 AM, said:

I didn't test every model, LK was the first I tried.

Well, here are test results (each batch from same image pre-processed with a single xBRZ pass):
Strife character sprites
Dagerfall character sprites
Biker Bimbo poster

I have not included results for donaldduck and pinocchio models because the results are garbage.

The results generally seem to suggest that different models may get better results depending on image but of these I liked most cartoonpainted, mymanga and hercules.

This post has been edited by MrFlibble: 30 April 2019 - 03:42 AM

0

User is offline   MrFlibble 

#108

Same girls as here but using the Cartoon Painted model (BTW, I just realised why the results are so good with it: the small images were created for the data set using the nearest neighbour method):
Spoiler

Note how the outlines of the sprites are no way better than with Manga109.

And for a test, the same monsters as these:
Spoiler

I think we should no longer be shy of considering the prospects of using 4x upscales instead of 2x.

[Edit] Duh, I intended to post this in this thread but nevermind.

This post has been edited by MrFlibble: 30 April 2019 - 07:02 AM

0

User is offline   Phredreeke 

#109

Reminder for anyone making upscales for Build engine games, when you define your upscaled tiles don't forget to put nocompress nodownscale after the file name. I had lots of issues with my upscale packs until I figured that out
1

User is offline   Phredreeke 

#110

Alright, I guess I can scratch this off the list of Build games I never expected to upscale (yes, that is the Doom palette, I was as surprised when I saw it myself)

http://s000.tinyuplo...507524277791292

Attached thumbnail(s)

  • Attached Image: tile0191.png
  • Attached Image: tile0603.png
  • Attached Image: tile0705.png
  • Attached Image: tile2560.png

2

User is offline   NightFright 

  • The Truth is in here

#111

Has anyone already posted anything about the ESRGAN Neural Upscale Texture Pack for Jedi Knight?
It only works with JKGFXMod, but as a bonus this also gives you modern effects like 32-bit graphics, bloom, SSAO and parallax mapping, significantly improving the outdated visuals of the game.

Examples (old vs new):
https://media.moddb.com/cache/images/mods/1/43/42353/thumb_620x2000/fuel_old.1.jpg https://media.moddb.com/cache/images/mods/1/43/42353/thumb_620x2000/fuel_new.1.jpg

https://media.moddb.com/cache/images/mods/1/43/42353/thumb_620x2000/nar2_old.jpg https://media.moddb.com/cache/images/mods/1/43/42353/thumb_620x2000/nar2_new.jpg

https://media.moddb.com/cache/images/mods/1/43/42353/thumb_620x2000/val_old.1.jpg https://media.moddb.com/cache/images/mods/1/43/42353/thumb_620x2000/val_new.jpg

I have tried it with my GOG.com version (latest release from January 2018) and it looks quite fantastic. To make it work, you will need some things, however:
- JKGFXMod
- JKUNOFFICIAL Patch
- Neural Upscale Texture Pack

1) Backup or rename your existing jk.exe (JKGFXMod will only work with the exe from the JKUnofficial patch)
2) Unzip jk.exe and jk-extension.dll from the unofficial patch into your JK dir
3) Unzip JKGFXMod into your Jedi Knight directory (jkgm folder, inject.exe, jkgm.json and render.dll)
4) Extract the "jknup" folder from the Neural Upscale Pack (the complete folder, not just the files inside) into the jkgm\materials subdir
5) Edit jkgm.json and replace default resolution [ 1024, 768 ] with your custom one, e.g. [ 1920, 1080 ] (you can also set hud_scale to something like 3.0 and max_anisotropy to 16.0 to have a bigger ingame HUD and 16x AF)
6) Load the game via inject.exe
7) Make sure you have 3D acceleration activated (it will say 16-bit, but graphics will be 32-bit nevertheless), then go ingame

This post has been edited by NightFright: 20 May 2019 - 07:11 AM

0

User is offline   MusicallyInspired 

  • Buy Mage's Initiation!

#112

I've seen the screenshots but never tried it. It looks great.
0

User is offline   MrFlibble 

#113

I just ran a quick test with some Doom sprites following the discussion here:
https://i.imgur.com/TnHYOJ4.png
Cartoon Painted 80k results
MyManga109 250k results

I think I like MyManga109 250k (shown above against purple background) more even though it's a bit less sharp. The Doomguy's face looks a lot more like the original low-res image.

UPD: I think I slightly improved the method for getting clean sprite outlines for 2x images:
Spoiler

Basically you take the ESRGAN 4x result, make a copy in mtPaint, apply Kuwahara-Nagao blur, convert to original palette and scale down to 2x (it will be nearest neighbour by default because it's an indexed image).

Then you make another copy, scale it down to 2x using Sinc3 or Bicubic, scale back up in mtPaint using nearest neighbour, and apply all the same steps as above (Kuwahara-Nagao blur etc.).

So now you have two 2x Kuwahara-Nagao'd images, paste them one over another without the background (or with the background colour set to transparency) using any method you choose so that basically you get a blend of the two. Save this image and then open in GIMP, select the background by colour and increase selection by one pixel, then copy onto the image scaled down to 2x using some normal method like Sinc3.

This post has been edited by MrFlibble: 25 May 2019 - 12:39 PM

0

User is offline   MrFlibble 

#114

A small test of Hexen sprites. This time I blended the results of MyManga109_250k and CartoonPainted_80k models using the Layer -> Blend [media] function in G'MIC. They look fairly good at 4x and 3x IMO:
https://i.imgur.com/yxU0TVc.png

https://i.imgur.com/jG8T0X5.png
comparison 3x nearest neighbour image

And here, 2x versions. This time I like the Cartoon Painted one more:
https://i.imgur.com/foBHEds.png

MyManga109:
https://i.imgur.com/d5NUaMg.png
I wanted to compare this to the sprites from the Hexen Gigapixel upscale but it uses some counter-intuitive sprite size, something like 3.5x (certainly larger than 3x but smaller than 4x). The sprites obviously have problems with their edges.
2

User is offline   Phredreeke 

#115

I've tackled upscaling an FMV from TekWar


I've made another video describing the process I used here
1

User is offline   MrFlibble 

#116

Just found a wiki listing different models that are available:
https://upscale.wiki.../Model_Database

The CartoonPainted and MyManga models are only listed as an external link though.
0

User is offline   MrFlibble 

#117

So I finally had some time and motivation to test the new models on the Upscale wiki page I linked to above. It's amazing how fast the improvements are made. Most no longer require any image pre-processing to achieve what previously had to be tricked out of the previous models by applying various "smoothing" tricks to source images.

Out of the bunch, I singled the Rebout model, or specifically rebout_interp. Here's what it can do to a sprite sheet from Strife I used for testing some posts above:
https://i.imgur.com/r7WzRpU.png
Mind you, the source image was not edited in any way.

I was not entirely satisfied with this and interpolated (at alpha = 0.5) rebout_interp with another model, Fatality, which was suggested to me by Phredreeke:
net_PSNR_path = './models/rebout_interp.pth'
net_ESRGAN_path = './models/4x_Fatality_01_265000_G.pth'

Same pic with this model:
https://i.imgur.com/dj3zBg1.png
More results here:
https://imgur.com/a/A9Jgocl
0

User is offline   MrFlibble 

#118

I further interpolated (again, at alpha = 0.5) the model I used right above with DeToon, which is a model that is supposed to diminish the cartoon feel of the upscaled image, and run a few pic that I had used before (previous results shown earlier in this thread). Please forgive my obsession with Command & Conquer screenshots:
Spoiler

More stuff here:
https://imgur.com/a/t9aMWI8
0

User is offline   Phredreeke 

#119

I'm posting these here as examples of problem frames when upscaling the cheerleader of RRRA.

Attached Image: tile6678.pngAttached Image: tile6860.png

Any attempts I've made have seriously messed up her face. Here's an example (using a 50/50 interpolation between Fatality and DeToon)

Attached Image: tile6860_rlt.png
0

User is offline   MrFlibble 

#120

I went back to xBRZ "softening" and the result is slightly better I think:
https://i.imgur.com/R7lc9GJ.png
This is the Fatality/Rebout 0.5 interpolation model I described above, the source image was 4x'd with xBRZ then scaled down to original size with GIMP at Bicubic (no prior blurring this time), then converted to original colours.

Given this is just one frame in an animation I wouldn't worry much about the Nefertiti look.

Interestingly the older Cartoon Painted interpolation seems to do a worse job on the same pre-processed image:
Spoiler

0

Share this topic:


  • 5 Pages +
  • « First
  • 2
  • 3
  • 4
  • 5
  • You cannot start a new topic
  • You cannot reply to this topic


All copyrights and trademarks are property of their respective owners. Instead of reading this text, you could be playing Ion Fury! ;) © 2019 Voidpoint, LLC

Enter your sign in name and password


Sign in options