[Phredreeke]

Paintshop Pro and ESRGAN

[Mav3r1ck]

Is the plan to just do the sprites or is it both sprites and textures?

[Phredreeke]

Both, but I’d like a better way of dealing with palswaps first.

[Mav3r1ck]

I can't download the upscale pack. the download page has connection issues. Is there and alternate download link?

[MusicallyInspired]

Does the link in the OP still link to the most recent update of this pack?

[Phredreeke]

It's the most recent release yes

[Phredreeke]

[Radar]

Great job Phredreeke! The community has really been knocking it out of the park lately. I love how so much effort has been put into mastering the classic visual experience. Polymost is king right now.

[Phredreeke]

As hinted in my last post. I've released a new version of my Total Meltdown upscale pack. https://www.moddb.co...-total-meltdown

New in this release is the battlelord (which uses a recolored version of the upscaled PC sprite), weapons, explosions and fires and a whole lot of decorative graphics and backgrounds, much of it recolored to match the PS1 game. Here's an example of a recolored background



Edit: Here's another example

[MrFlibble]

I've been experimenting with different models to upscale the tally screen with Duke posing, here are two versions that I liked more than others (DUke's face mostly comes out very badly):

DeviantPixelHD/Face 0.7

DigiPaint/DeviantPixelHD 0.6

A while ago Darkus posted a 800x600 wallpaper from the Atomic Edition CD made from the same render which could have been used as a ground truth for comparison but the image hosting site is no longer there and the wallpaper is gone.

A different interpolation, DigiPaint/charsprite 0.2, gives some rather nice results with monster sprites:

And with posters:

[Danukem]

I feel like those are a regression back to the first attempts that looked like someone applying palette knife filter in photoshop.

[NightFright]

Indeed, totally smeared and blurry. Best I have seen so far is nvidia 2x neural upscaling. A full "HRP" with textures upscaled like that would be something I might even consider using, and I have become quite a purist by now when it comes to game enhancement mods.

[Tea Monster]

Personally, I think that someone will probably have to manually tweak some of these - most especially text on posters and signs. Apart from that, and some faces, it looks great and is a significant improvement. I played Heretic recently with an AI Upscale pack and it rocks.

[Phredreeke]

Here are a few comparison shots of different methods I've used for upscaling backgrounds. (I'm fairly happy with my current approach for sprites)




From left to right, top to bottom
1. My first upscale attempt, uses only the manga model (interpolated with PSNR to reduce some of its artifacts)
2. My first hybrid, uses both manga model as well as an XBRZ upscale. This uses an adaptive script that prefers the XBRZ upscale for when there's a large difference between the two (like in the text) and uses the manga model otherwise. This is also what the background upscales in the TM upscale pack are based on.
3. Hybrid between manga model and box model, the latter preprocessed using dedither model. Unlike the above its a 50/50 blend between the two models.
4. Same as above, but the result has been run through a desharpening model before applying the duke palette.
5. Uses an interpolation between fatality and detoon models, preprocessed using dedither model.
6. Same as above except as with 4 it has been run through a desharpening model before applying palette.



Opted to just use 2,4 and 6 for this one.

[MrFlibble]

Trooper Dan, on 21 October 2019 - 12:09 PM, said:

I feel like those are a regression back to the first attempts that looked like someone applying palette knife filter in photoshop.

NightFright, on 21 October 2019 - 12:15 PM, said:

Indeed, totally smeared and blurry.

I assume you'd appreciate these results more if you had seen the really bad ones. These new models have no trouble processing lines and edges at any angle whereas older models required pre-processing for that, otherwise the output would be scarcely better than xBRZ or the like. That alone is a huge step forward compared to what we had a year ago, although that alone does not mean that the upscales become better in all other respects too.

How much the new models "enhance" the image is another matter. charsprite specifically stays very close to the original sprite, and I don't think there's any particular harm in that. Conversely, models that are trained to add extra features sometimes do it in the wrong places and ruin the results.

Tea Monster, on 21 October 2019 - 12:42 PM, said:

I played Heretic recently with an AI Upscale pack and it rocks.

The Heretic and Hexen upscales were made using Topaz Gigapixel, which is a commercial tool for enhancing photographs. It's not exactly suited to upscale video game sprites or textures. It also seems to have a semantic layer in it, meaning that it identifies parts of the photograph (e.g. face, hands etc.) and uses specific modules trained to upscale these particular objects. ESRGAN does not have a semantic layer, which puts it at a certain disadvantage. However the new models can do at least some things better.

Here's an example (I'm sorry for getting a bit off-topic here!). You may notice that the trident logo has some jittery edges on the arrows that are facing down in the Gigapixel version:

Take ESRGAN charsprite/DigiPaint interpolation 0.3 and the logo is just perfect, but the whole texture looks more like it's painted:

Finally the Faces model: it adds some texture to the stones, yet still the lines on the trident logo are cleaner compared to the Gigapixel version.

[Phredreeke]

Upscale I made of the font (actually made the font upscale a long time ago, but the mask I made today and finally got one that turns out ok to me)

[MrFlibble]

Some of the letter shapes seemed jittery to me so I checked if the DigiPaint/charsprite 0.2 interpolation could do better:

Your version:

On another note, here's a sprite upscaling result I got while experimenting:

I first upscaled the raw image with the above DigiPaint/charsprite interpolation, then scaled down back to 1.5 the original size with Bicubic and scaled this image up again with fighteralt2. This was then scaled down to 4x the original size with Bicubic.

To get the mask for the font, I used the same method as described here but Kuwahara-Nagao blur was applied at 1 pixel radius.

[Phredreeke]

Have you got a way to batch process the post-upscale processing? I know GIMP has some sort of scripting but I'm not familiar with it.

Your font upscale does look better than mine.

[MrFlibble]

ATM I can't think of any way to batch process the images other than to make a single sprite sheet, scale it up and then apply all the transformations needed to get the mask. As this involves both mtPaint and GIMP there doesn't seem to be a simple way of streamlining all the steps into a single automated process.

For a further test, I tried to apply the same two-step upscale method as for the monster sprites above to the fonts but there doesn't seem to be any improvement:

DigiPaint/charsprite 0.2:


Two-step with fighteralt2:

[Danukem]

For fonts you are probably better off finding a very similar one, then typing the letters into photoshop to make a sprite sheet. Apply the color, shadows etc. to the layer and then use a sprite sheet extractor.

[MrFlibble]

TBH I believe that fonts are one of the areas where it'd be certainly better to have new high-res ones drawn by an artist altogether rather than upscale the originals. I gather that the HRP already has some pretty nice looking high-res fonts and HUD elements.

It is quite clear now that neural upscales on their own are rarely sufficient, there has to be some manual post-processing one way or another.

From my experience, Manga109Attempt is no longer a good option, not only because of the artifacts but also because it does not handle certain shapes well. With the font examples above, Manga is completely helpless with diagonal lines if the image is not pre-processed:

You can get better results with xBRZ "softening" at the expense of losing some detail to blur:

But you can still see that the diagonal lines of the U and V are not really straight, they're noticeably curvy. DigiPaint/charsprite handles this way better without any preprocessing (same source image as the first Manga result above):

I know you're using different pre-processing methods and interpolate Manga with Box but apparently it does not help much. Model interpolation cannot improve the way a model handles the image, it only makes a blend of the ways two models do stuff. I interpolated DigiPaint with charsprite because they produce fundamentally similar results (charsprite on the left; image picked specifically for diagnostic purposes):

But if you take a model that cannot do diagonal lines similar to that staff above well, like Manga, and interpolate it with a model that does well like DigiPaint, you won't improve the Manga model in this respect; conversely, depending on the alpha, the addition of Manga will progressively distort the diagonal line shape into more of a "pixel staircase". Of course, it will also add other Manga features, like colour alterations that it introduces, but as far as I know you cannot selectively combine features that you want from interpolated models while leaving others out.

[Phredreeke]

Quote

I know you're using different pre-processing methods and interpolate Manga with Box but apparently it does not help much.

Box does not interpolate with anything. Furthermore I've only used Box in a limited number of cases (primarily involving backgrounds). My Box/Manga hybrids do two upscales which are then combined in a postprocessing script.

The version I posted is just manga/PSNR interpolation.

BTW where did you get charsprite and fighteralt models from? I don't find them on the wiki

[Phredreeke]

I've been working on an alternate upscale pack (4x size, different models) here are a few sprites as a preview





For people preferring the look of the 2x pack, don't worry, this won't be replacing that.

[MrFlibble]

Phredreeke, on 30 October 2019 - 05:48 AM, said:

BTW where did you get charsprite and fighteralt models from? I don't find them on the wiki

They are from that folder which you linked to in a PM recently

fighteralt2 makes the sprites look kind of like they were just made from 3D models (I think):

This one was made by the two-step method I described above.

You results kinda look like there's some sort of uneven blur popping out in some of the areas. I haven't had the time to test your preliminary pack and can't tell how this actually appears in the game.

[Phredreeke]

You know I actually had a bit of an epiphany, instead of upscaling from the original size sprite, use my existing 2x upscales as starting point.

https://imgsli.com/Nzg3MQ

[MrFlibble]

Phredreeke, on 04 November 2019 - 08:56 AM, said:

You know I actually had a bit of an epiphany, instead of upscaling from the original size sprite, use my existing 2x upscales as starting point.

Yes, that's exactly the idea behind what I described as the two-step upscale (I considered scaling the first step's output to 2x instead of 1.5x but thought that it'd be more straining for the CPU). When the image is larger the model has more stuff to play around with.

For example, here's what you get if you feed some raw sprites (no pre-processing) to the Faces model:

It's basically garbage no matter how you look at it.

And this is what you get if you scale up the same sprites with DigiPaint/charsprite 0.2 interp first, then scale the result down to 1.5x the original size and upscale this image with the same Faces model (you get output as 6x the original size, I scaled down to 4x with bicubic):

Quite an improvement eh?

[Phredreeke]

I just released the formerly Detoon-Fatality upscale pack, now DTF Edition (because I used more models than those)

https://www.moddb.co...ack-dtf-edition

Some comparisons between the original, my 2x upscale and this





Slide-over comparison between the new and original pigcop

[NightFright]

Especially the Cycloid Emperor looks mind-blowingly good. Never thought one could tickle so much detail out of that low-res sprite!

[Phredreeke]

Thanks, the workflow for this is pretty damn crazy (but fortunately can be be batched)

I will probably keep doing 2x upscale packs going forward though. The 2x upscale sprites don't look out of place in the non-upscaled environments, but once you go 4x there's a rather stark contrast with the non-upscaled environments. That and I don't want to monopolise the space for neural upscales.

[Mark]

The cycloid has so much detail now that you can see he has dandruff.