[MrFlibble]

I just tried some Daggerfall sprites with softening, the results are mostly pretty nice:


https://imgur.com/a/Ostvgmr

[Phredreeke]

Motherdemon from Doom 64



(used a 50/50 interpolation of Fatality and Detoon, then downscaled to half size and applied the sprite's palette)

[MrFlibble]

Playing around more with Daggerfall stuff I realised that my previous method of extracting sprites from background are not as effective as I thought, which prompted me to come up with a new, albeit a tad more convoluted method (involving a separate upscale for the mask).

The idea is to make one upscale against black background to get the sprites, and another upscale of the same image in grayscale against a coloured background to create the mask. Here's the procedure explained in some detail:

Spoiler

1. A sprite or sprite sheet is prepared against a black background, and a grayscale version of the same with the black background replaced with blue. A third image contains the original sprites against the same blue background (we'll need it later):

For ease of reference, let's call these source images S1, S2 and S3. Use mtPaint to convert each to indexed palette (exact match).

2. Then apply xBRZ softening (scale up 4x w/xBRZ Scaler Test, then back down to original res w/ Bicubic in GIMP) to S1 and S2 and process them with ESRGAN with Fatality/Rebout/Lady model.

3. With the S2 upscale, apply Kuwahara Nagao blur at 2 pixel radius, convert to the palette of the source image:

4. In GIMP, select the pink background from the result you got in step 2, paste over the S1 upscale, export this image, open in mtPaint and again apply KN blur at 2 pixel radius. Then convert to the indexed palette of S3:

5. Open the result in GIMP, select the pink background and grow selection by 1 pixel. Copy the selected area, then paste over the S1 upscale. You can then re-select the pink background (without growing though so that the copied sprite edges remain) on the S1 upscale and delete it

Some results:


The model I was using here is an interpolation of (a) Fatality/Rebout interpolation at alpha = 0.5 (described above and (b) 4x_Lady0101_v3_blender.pth at alpha = 0.7.

net_PSNR_path = './models/4x_Lady0101_v3_blender.pth'
net_ESRGAN_path = './models/F_RB_05.pth'

[MrFlibble]

Meanwhile I spent some time to do a more systematic research of what the currently available models can do.

The models that produce good results without pre-processing of the source images are:

• 4x_Fatality_01_265000_G
  
• 4x_rebout_325k
  
• rebout_interp
  
• 4x_Lady0101_v3_340000
  
• 4x_Lady0101_v3_blender
  
• 4x_SpongeBob_235000_G

None of these produce acceptable sprite results if used on their own, but can get quite good if interpolated with one another.

Previously I relied mostly on guesswork when interpolating models (using alpha = 0.5 most of the time) but more extensive testing showed that at certain levels (in particular close to the said middle value of 0.5) the models may actually disrupt one another, negating the efficiency with which images are processed. For a quick example, here's the same sprite upscaled with Fatality (left), Rebout Interp (right) and an alpha 0.5 interpolation of the two (centre):

You can clearly see that the priest's staff is a clean diagonal line in all but the centre image where it has become jittery.

With this in mind I conducted further tests using alpha values in the range of 0.1 to 0.3, or, conversely, from 0.7 to 0.9. Note that the alpha values are completely reversible, meaning that an X/Y interpolation at 0.3 is the exact same thing as Y/X interpolation at 0.7. (For clarity in notation, when I say X/Y I mean that in net_interp.py, X is the model in the net_ESRGAN_path and Y in net_PSNR_path).

Here's the same priest sprite with Fatality/Rebout interpolated at 0.3:

Another catch is that each model has its own peculiarities:

• Fatality results are overly sharp and with noticeable ringing
  
• rebout_interp fuzzies edges and produces colour banding
  
• Lady is rather blurry

For NPC flats, I mentioned above that the Lady models seem to work good. I picked two sprites that have issues with different models to illustrate the results using alpha = 0.7.

Spoiler

Series I.
4x_Lady0101_v3_340000 interpolated with: 4x_Fatality_01_265000_G, rebout_interp, 4x_SpongeBob_235000_G:

4x_Lady0101_v3_blender interpolated with: 4x_Fatality_01_265000_G, rebout_interp, 4x_SpongeBob_235000_G:

Series II.
4x_Lady0101_v3_340000 interpolated with: 4x_Fatality_01_265000_G, rebout_interp, 4x_SpongeBob_235000_G:

4x_Lady0101_v3_blender interpolated with: 4x_Fatality_01_265000_G, rebout_interp, 4x_SpongeBob_235000_G:

I'd probably say that the 4x_Lady0101_v3_blender interpolations with rebout_interp and 4x_SpongeBob_235000_G are the closest to the original image.

[MrFlibble]

I think I may have figured out how to kinda-sorta counter the blurriness of Lady interpolated models' output. There's a filter in Image Analyzer (kudos to Jimmy for suggesting this programme!) called Warp sharpen which produces some rather neat results. Here are some side-to-side comparisons:
https://imgsli.com/NzQ2Ng

[Phredreeke]

https://imgsli.com/NzU4Mw
https://imgsli.com/NzU4NQ
https://imgsli.com/NzU4Ng

[Phredreeke]

The biggest problem with upscaling sprites have been generating masks for them.



Basically, instead of using non-neural upscaling as before, I've generated an antialiased mask and upscaled with a specific alpha upscaling model. The result was fairly noisy and jagged and needed lots of postprocessing to be useful with the Blood upscale. This ended up deleting lots of fine detail.

Then I got an idea, I had previously been shared a WIP model, which I found good results upscaling text with. So what I did was upscale the antialiased mask 2x size, run it through said model, then downscale to 25% size and posterise it. while the edges aren't as nice as the other model, it preserves more or less all of the smaller details!

Tomorrow I will try to figure out a way to combine the two masks in a script

[Phredreeke]

Upscale using an interpolation between Manga and DeviantArt models, and using the new mask process.

[Phredreeke]

been given access to another new WIP model...

[NightFright]

Looks like after Duke drinking a beer in DNF.

[Phredreeke]

Just upscaled some SW weapons.






Used three different upscales utilising five models to make these.

[MusicallyInspired]

Phredreeke, on 16 December 2019 - 09:58 AM, said:

been given access to another new WIP model...

That forearm tho.

[Phredreeke]

[Radar]

Those teeth.

[MusicallyInspired]

I feel like this model could get really popular for all the wrong reasons lol.

[NightFright]

Duke Freakem 2D.

[OpenMaw]

The teeth are actually pretty accurate to the original model used for rendering though. If you look at the hologram head which was rendered off the same model, Duke's got his teeth bearing pretty much like that.

[Phredreeke]

Funny that you'd mention that...

[Phredreeke]

Here's another one - this time I downscaled back to original size, then ran it through the Fatality-Comix model. Gets a cool Borderlands look.

[Phredreeke]

Used a similar method to the Duke sprite above.

[MusicallyInspired]

Those results are very good!

[MrFlibble]

Lo Wang looks very good, except the shapes of the gun and the sword would probably need some manual touch-up to make them more geometrically consistent.

But the Duke sprite is an ugly mess, sorry, both the "teeth" and even more so the comix version. I'd rather sebabdukeboss20 made a new high-res sprite from a model.

BTW, if it ain't no secret, what was the WIP model trained on?

[Phredreeke]

I posted the teeth version because it was so hideous lol

The comix version isn't something I would use, just thought it looked neat.

The WIP model is trained on faces that been downscaled and had their color depth reduced if I'm not mistaken. It's by twittman who did the fatality models.

BTW here is an example of blending in 25% of the WIP upscale to enhance a sprite. I think that's the key to this (I also used the WIP model to upscale the intro FMV from SW which I grabbed Lo Wang's face from for the sprites above)

[MrFlibble]

Phredreeke, on 21 December 2019 - 04:39 AM, said:

BTW here is an example of blending in 25% of the WIP upscale to enhance a sprite. I think that's the key to this (I also used the WIP model to upscale the intro FMV from SW which I grabbed Lo Wang's face from for the sprites above)
tweakedduke.png

That's certainly much better except his chest looks somewhat over-pronounced if you get what I mean. I still believe that some manual touch-ups by an artist would make an improvement, the face and the gun could use some.

Lo Wang evidently has the advantage of having larger sprites in the first place. I'm not sure if Chuck Jones' models for both characters (I don't remember if Chuck did Lo Wang but assume that he did) were originally equally detailed but if yes then it appears that Duke lost a lot of that detail because of the final resolution of the sprites.

[Phredreeke]

I'll update the body for the next release (I had actually made those sprites a while back and had been sitting on it for months)

Right now I'm working on the SW pack though.

[Phredreeke]

Preview of another SW sprite upscale

[Phredreeke]

I have no idea why I made these

https://imgsli.com/MTA2NTI
https://imgsli.com/MTA2NTM
https://imgsli.com/MTA2NTQ

They're using a mixture of the follow models: Dedither (as preprocessor), Detoon and Fatality (interpolated at 0.5) and Pixelity-Faces (currently not publicly released) with a set of PSP scripts to composite them together

[Phredreeke]

SW has a LOT of palswaps

[MrFlibble]

And there is no way to make the port palswap upscaled sprites even if they're palettised?

[Phredreeke]

I can palswap after upscaling, but I can't palswap in-engine, I have to either provide a separate image for each palswap, or attempt to approximate with tint commands.

BTW. New model I've been testing https://imgsli.com/MTEzNzY