Duke4.net Forums: The ESRGAN AI Upscale non-Duke thread - Duke4.net Forums

Jump to content

  • 11 Pages +
  • « First
  • 3
  • 4
  • 5
  • 6
  • 7
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

The ESRGAN AI Upscale non-Duke thread

User is offline   MrFlibble 

#121

I just tried some Daggerfall sprites with softening, the results are mostly pretty nice:
https://i.imgur.com/fPt8O93.png
https://i.imgur.com/8Yb7Qj3.png
https://imgur.com/a/Ostvgmr
0

User is offline   Phredreeke 

#122

Motherdemon from Doom 64

https://i.imgur.com/ZYtN9wr.png

(used a 50/50 interpolation of Fatality and Detoon, then downscaled to half size and applied the sprite's palette)
0

User is offline   MrFlibble 

#123

Playing around more with Daggerfall stuff I realised that my previous method of extracting sprites from background are not as effective as I thought, which prompted me to come up with a new, albeit a tad more convoluted method (involving a separate upscale for the mask).

The idea is to make one upscale against black background to get the sprites, and another upscale of the same image in grayscale against a coloured background to create the mask. Here's the procedure explained in some detail:
Spoiler

Some results:
https://i.imgur.com/GDOVgHR.png
https://i.imgur.com/ZrTd0lr.png
The model I was using here is an interpolation of (a) Fatality/Rebout interpolation at alpha = 0.5 (described above and (b) 4x_Lady0101_v3_blender.pth at alpha = 0.7.
net_PSNR_path = './models/4x_Lady0101_v3_blender.pth'
net_ESRGAN_path = './models/F_RB_05.pth'

0

User is offline   MrFlibble 

#124

Meanwhile I spent some time to do a more systematic research of what the currently available models can do.

The models that produce good results without pre-processing of the source images are:
  • 4x_Fatality_01_265000_G
  • 4x_rebout_325k
  • rebout_interp
  • 4x_Lady0101_v3_340000
  • 4x_Lady0101_v3_blender
  • 4x_SpongeBob_235000_G

None of these produce acceptable sprite results if used on their own, but can get quite good if interpolated with one another.

Previously I relied mostly on guesswork when interpolating models (using alpha = 0.5 most of the time) but more extensive testing showed that at certain levels (in particular close to the said middle value of 0.5) the models may actually disrupt one another, negating the efficiency with which images are processed. For a quick example, here's the same sprite upscaled with Fatality (left), Rebout Interp (right) and an alpha 0.5 interpolation of the two (centre):
https://i.imgur.com/sDX1cgs.png https://i.imgur.com/ZUy7726.png https://i.imgur.com/MKSktAJ.png
You can clearly see that the priest's staff is a clean diagonal line in all but the centre image where it has become jittery.

With this in mind I conducted further tests using alpha values in the range of 0.1 to 0.3, or, conversely, from 0.7 to 0.9. Note that the alpha values are completely reversible, meaning that an X/Y interpolation at 0.3 is the exact same thing as Y/X interpolation at 0.7. (For clarity in notation, when I say X/Y I mean that in net_interp.py, X is the model in the net_ESRGAN_path and Y in net_PSNR_path).

Here's the same priest sprite with Fatality/Rebout interpolated at 0.3:
https://i.imgur.com/bA1ocGO.png
Another catch is that each model has its own peculiarities:
  • Fatality results are overly sharp and with noticeable ringing
  • rebout_interp fuzzies edges and produces colour banding
  • Lady is rather blurry

For NPC flats, I mentioned above that the Lady models seem to work good. I picked two sprites that have issues with different models to illustrate the results using alpha = 0.7.
Spoiler

I'd probably say that the 4x_Lady0101_v3_blender interpolations with rebout_interp and 4x_SpongeBob_235000_G are the closest to the original image.
0

User is offline   MrFlibble 

#125

I think I may have figured out how to kinda-sorta counter the blurriness of Lady interpolated models' output. There's a filter in Image Analyzer (kudos to Jimmy for suggesting this programme!) called Warp sharpen which produces some rather neat results. Here are some side-to-side comparisons:
https://imgsli.com/NzQ2Ng
0

User is offline   Phredreeke 

#126

https://imgsli.com/NzU4Mw
https://imgsli.com/NzU4NQ
https://imgsli.com/NzU4Ng

This post has been edited by Phredreeke: 23 October 2019 - 04:46 PM

0

User is offline   Phredreeke 

#127

The biggest problem with upscaling sprites have been generating masks for them.

https://i.imgur.com/BMlk2K1.gif

Basically, instead of using non-neural upscaling as before, I've generated an antialiased mask and upscaled with a specific alpha upscaling model. The result was fairly noisy and jagged and needed lots of postprocessing to be useful with the Blood upscale. This ended up deleting lots of fine detail.

Then I got an idea, I had previously been shared a WIP model, which I found good results upscaling text with. So what I did was upscale the antialiased mask 2x size, run it through said model, then downscale to 25% size and posterise it. while the edges aren't as nice as the other model, it preserves more or less all of the smaller details!

Tomorrow I will try to figure out a way to combine the two masks in a script

This post has been edited by Phredreeke: 30 October 2019 - 04:23 PM

2

User is offline   Phredreeke 

#128

Attached Image: spritesheet.png

Upscale using an interpolation between Manga and DeviantArt models, and using the new mask process.
2

User is offline   Phredreeke 

#129

been given access to another new WIP model...

https://i.imgur.com/3nUI2vS.png
0

User is offline   NightFright 

  • The Truth is in here

#130

Looks like after Duke drinking a beer in DNF.
0

User is offline   Phredreeke 

#131

Just upscaled some SW weapons.

https://i.imgur.com/ABCegSP.png
https://i.imgur.com/t0nDNkS.png
https://i.imgur.com/uoVG8WC.png
https://i.imgur.com/GhhvZJ9.png

Used three different upscales utilising five models to make these.
3

User is online   MusicallyInspired 

  • The Sarien Encounter

#132

View PostPhredreeke, on 16 December 2019 - 09:58 AM, said:

been given access to another new WIP model...

https://i.imgur.com/3nUI2vS.png


That forearm tho.

This post has been edited by MusicallyInspired: 16 December 2019 - 05:46 PM

0

User is offline   Phredreeke 

#133

https://i.imgur.com/o565ZZZ.png
1

User is offline   Radar 

#134

Those teeth.

https://i.imgur.com/RrXAy8n.png
0

User is online   MusicallyInspired 

  • The Sarien Encounter

#135

I feel like this model could get really popular for all the wrong reasons lol.
1

User is offline   NightFright 

  • The Truth is in here

#136

Duke Freakem 2D.
1

User is offline   OpenMaw 

  • Judge Mental

#137

The teeth are actually pretty accurate to the original model used for rendering though. If you look at the hologram head which was rendered off the same model, Duke's got his teeth bearing pretty much like that.
0

User is offline   Phredreeke 

#138

Funny that you'd mention that...

https://i.imgur.com/TRBL7mk.png
2

User is offline   Phredreeke 

#139

Here's another one - this time I downscaled back to original size, then ran it through the Fatality-Comix model. Gets a cool Borderlands look.

https://i.imgur.com/Dcs9tDA.png
1

User is offline   Phredreeke 

#140

https://i.imgur.com/8y5lH2Z.png

Used a similar method to the Duke sprite above.

This post has been edited by Phredreeke: 20 December 2019 - 11:13 AM

4

User is online   MusicallyInspired 

  • The Sarien Encounter

#141

Those results are very good!
0

User is offline   MrFlibble 

#142

Lo Wang looks very good, except the shapes of the gun and the sword would probably need some manual touch-up to make them more geometrically consistent.

But the Duke sprite is an ugly mess, sorry, both the "teeth" and even more so the comix version. I'd rather sebabdukeboss20 made a new high-res sprite from a model.

BTW, if it ain't no secret, what was the WIP model trained on?
2

User is offline   Phredreeke 

#143

I posted the teeth version because it was so hideous lol

The comix version isn't something I would use, just thought it looked neat.

The WIP model is trained on faces that been downscaled and had their color depth reduced if I'm not mistaken. It's by twittman who did the fatality models.

BTW here is an example of blending in 25% of the WIP upscale to enhance a sprite. I think that's the key to this (I also used the WIP model to upscale the intro FMV from SW which I grabbed Lo Wang's face from for the sprites above)
Attached Image: tweakedduke.png

This post has been edited by Phredreeke: 21 December 2019 - 06:01 AM

2

User is offline   MrFlibble 

#144

View PostPhredreeke, on 21 December 2019 - 04:39 AM, said:

BTW here is an example of blending in 25% of the WIP upscale to enhance a sprite. I think that's the key to this (I also used the WIP model to upscale the intro FMV from SW which I grabbed Lo Wang's face from for the sprites above)
Attachment tweakedduke.png

That's certainly much better except his chest looks somewhat over-pronounced if you get what I mean. I still believe that some manual touch-ups by an artist would make an improvement, the face and the gun could use some.

Lo Wang evidently has the advantage of having larger sprites in the first place. I'm not sure if Chuck Jones' models for both characters (I don't remember if Chuck did Lo Wang but assume that he did) were originally equally detailed but if yes then it appears that Duke lost a lot of that detail because of the final resolution of the sprites.
0

User is offline   Phredreeke 

#145

I'll update the body for the next release (I had actually made those sprites a while back and had been sitting on it for months)

Right now I'm working on the SW pack though.
0

User is offline   Phredreeke 

#146

Preview of another SW sprite upscale
https://i.imgur.com/q6DMhPD.png
2

User is offline   Phredreeke 

#147

I have no idea why I made these

https://imgsli.com/MTA2NTI
https://imgsli.com/MTA2NTM
https://imgsli.com/MTA2NTQ

They're using a mixture of the follow models: Dedither (as preprocessor), Detoon and Fatality (interpolated at 0.5) and Pixelity-Faces (currently not publicly released) with a set of PSP scripts to composite them together
1

User is offline   Phredreeke 

#148

SW has a LOT of palswaps
Attached Image: spritesheet.png
0

User is offline   MrFlibble 

#149

And there is no way to make the port palswap upscaled sprites even if they're palettised?
0

User is offline   Phredreeke 

#150

I can palswap after upscaling, but I can't palswap in-engine, I have to either provide a separate image for each palswap, or attempt to approximate with tint commands.

BTW. New model I've been testing https://imgsli.com/MTEzNzY
1

Share this topic:


  • 11 Pages +
  • « First
  • 3
  • 4
  • 5
  • 6
  • 7
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic


All copyrights and trademarks not owned by Voidpoint, LLC are the sole property of their respective owners. Play Ion Fury! ;) © Voidpoint, LLC

Enter your sign in name and password


Sign in options