[MrFlibble]

Phredreeke, on 06 May 2021 - 07:17 AM, said:

The AA sprites are far too different from the originals.

Pixellated faces are also very different from photorealistic images, yet that face model which was shown some pages ago produced rather decent results with pixel art faces from video games.

Or the ESRGAN model you're playing around with here evidently tries to add teeth, eyes and nose in places where they do belong. It makes a lot of errors and produces garbage, but nonetheless it's trying to work in the right direction. It seems not entirely implausible that ESRGAN would learn certain patterns from pairs of vanilla sprites and sebab's higher-res versions and produce something decent or maybe even novel and interesting.

I believe it is worth trying out, if the new sprites are exactly 2x the original size, and someone with a powerful enough GPU for model training is willing to spend time and effort on that. It might fluke just as well, but if the manpower cost of training such a model is not exceedingly high, it seems like worth a try.

Phredreeke, on 06 May 2021 - 07:17 AM, said:

What could be done would be if sebab did two renders at different scales, with the lower resolution renders being used as LR.

It's not a bad idea per se but there are already several ESRGAN models that can make decent upscales of the same kind as you're suggesting here. My entire point is that maybe if the source and target material differ more, the AI will learn to improvise in some consistent manner.

[Phredreeke]

Time to subvert your expectations



Slide-over comparison

[NightFright]

Looks like it's a lot better with more abstract/generic stuff than with faces.

[MrFlibble]

Phredreeke, on 08 May 2021 - 02:02 PM, said:

Time to subvert your expectations

This is not the first time you're posting something like this (an upscale of an image with very low pixel count), but in all honesty I can't say I'm particularly impressed. So the upscale looks like a painting, so what. It's still very low-detailed, but the fact it's supposed to be a painting probably makes you overlook some of the deficiencies.

The tree looks okay-ish I guess, but the house - I believe the light-brown thing behind the tree is supposed to be a house - well, it clearly looks like a mess in the upscale. On the whole, this model appears to like making oblong, slightly curved shapes from pixels of similar colours.

It will probably work as a texture in an upscale pack, as the player is unlikely to closely examine this image, and the original was already low-res and limited in detail. In fact, this kind of decoration is originally more of a symbolic nature, like it's more signifying a painting, rather than being one.

[Phredreeke]

I thought it looked good, as opposed to the recent monstrosities I posted. (Different upscaling model btw)

I guess monstrosities are more well received. Thank you for subverting my expectations

[MrFlibble]

I can count at least three pareidolic creepy faces in that painting upscale.

[Phredreeke]

Spoiler

Are you sure you want to do this?

Spoiler

No, I really mean it...

Spoiler

[NightFright]

Only suitable comment I was able to find:

[MrFlibble]

Now, that was mildly amusing. I don't recognise the source image, it must be something from a CGI intro for a console Duke game I am not familiar with?

Anyway, I feel the need to point out that when there are new posts in this thread, I generally expect some successful results being shown. I don't follow the AI upscale scene closely now but new models keep popping up every now and then, and they produce decent to good results with at least some kinds of images. But as it stands, this thread seems to have become almost exclusively a showcase of ESRGAN failures. Funny? Not sure, but these samples certainly do nothing to reinforce my faith in the capabilities of AI upscales in general.

[Phredreeke]

I wouldn't call the Tekwar painting a failure.

The image is a cropped part of the Land of the Babes intro, upscaled with tinyface.

Right now my focus has been on Duke3D and successful upscales have been posted in the HRP thread.

Anyway, since you want to see something successful, here's something I achieved with a rather tedious setup involving upscaling the tile twice, once normal and once mirrored, then blending the results before downscale and palettising



Models used were Jaywreck2 and Nickelback.
This should be the same tile but only using one of the upscale



Here's another. I think this one used Nickelback as well (doesn't use the mirror process, unsure about what I preprocessed with, probably dedither)




I also posted some new weapon upscales for PowerslaveGDX in the BuildGDX Discord some time ago Here's the link for those I don't think I've posted them anywhere else

(also used Nickelback, with my redither process)

[MrFlibble]

Phredreeke, on 12 May 2021 - 01:53 PM, said:

I wouldn't call the Tekwar painting a failure.

I wasn't referring to it, it was the rest of the bunch really

Phredreeke, on 12 May 2021 - 01:53 PM, said:

The image is a cropped part of the Land of the Babes intro, upscaled with tinyface.

Just asking, what exactly did you expect to get when you ran that image through the tinyface model? A more photorealistic face for Duke?

Phredreeke, on 12 May 2021 - 01:53 PM, said:

Here's another. I think this one used Nickelback as well (doesn't use the mirror process, unsure about what I preprocessed with, probably dedither)

So for now we're still at the stage when "upscale" basically means "depixelising" the image. It's a good result compared to purely algorithmic methods like xBRZ but I feel that AI can do more than that.

I guess my suggestion to try training a model on vanilla sprites and sebab's high-res versions is not going to get any traction, and I have neither a powerful GPU nor the time too do that on my own. But consider this: what ESRGAN and other AI models do is not simply upscaling an image like hq2x or xBRZ do, they actually transform the image, to a greater or lesser extent. It's just that most ESRGAN models are trained so as to be as close to the source material as possible, hence the output is thus similar to algorithmic scalers (but much more advanced). I suppose that if you train a model on more dissimilar source and target images, like the Alien Armageddon high-res enemies, this might produce more drastic, but still meaningful transformations of source images by such a model.

I just checked a sample Octabrain sprite from AA and as I suspected, the dimensions are not exactly 2:1 compared to the original Duke3D sprite. But I think both the source and target images may be put into a uniform correspondence by slightly enlarging the canvas of the high-res counterparts.

[Phredreeke]

I asked about your suggestion in the game upscale Discord and this is the response I got:
Yeah I think that training that would produce nightmare fuel

As for what I expected running that Duke pic through tinyface, I expected an abomination. I wouldn't even say it's that bad given the extremely low quality of the source material.

[Phredreeke]

Using an inpainting model on Ecce Homo (the famous monkey Jesus from a few years back)



I wonder what results I could have gotten from a higher quality source

[NightFright]

Well... way better than what that old lady did to that painting...

[MrFlibble]

I just checked a 8x model called HugePeeps by DinJerr, does some pretty amazing attempts to create realistic faces from pixel art (Daggerfall commoners here).

Spoiler

One more sprite and original low-res images for comparison here.

UPD: Oh, and I couldn't resist trying out this:

Yup, the model saw an extra eye on the bikini top

[NightFright]

What's going on with placing eyes everywhere? It's inspired by Dalí or something?

[Phredreeke]

if a model is trained only on faces then it sees anything vaguely shaped like an eye as one,

[MrFlibble]

NightFright, on 23 May 2021 - 11:22 PM, said:

What's going on with placing eyes everywhere? It's inspired by Dalí or something?

It does not place eyes "everywhere". The model recognises certain patterns of pixels as eyes, this can be clearly seen with male commoner sprites from Daggerfall: these do not have as large and pronounced eyes as the females, but the model still creates eyes for them (e.g. male Redguard). That Shelly sprite just happens to have a similar arrangement of pixels on the bikini top that "looked" like an eye to the model.

[NightFright]

Her cleavage looks like... something else, btw.

[Phredreeke]

brings a new meaning to titty-f*cking...

[jkas789]

Not gonna lie, I once had a nightmare where a eldritch horror catfished me as a blonde babe.

10/10 best upscale ever.

[MrFlibble]

BTW, I was playing with other models as well and realised that DinJerr's 4x_ArtStation1000_Sharp actually produces some fairly decent results (at least, when compared to many other models) with pre-rendered sprites:

Compare to earlier Doom 64 attempts here and here.

Characters from IPOG also came out rather well (never mind the facial expressions though!):


Powerslave:


But sadly Shadow Warrior sprites came out rather poorly:

[Phredreeke]

Pixelperfect -> 25% downscale -> Hugepeeps -> 25% downscale



https://imgsli.com/NTc0NDY

[NightFright]

Lo Wang come for you, little snake coward!

[MrFlibble]

In the meantime someone tested an Nvidia face model that produces photorealistic faces (from Daggerfall character portraits in this case):
http://forums.dfwork...e9ed5e07#p55430

[Phredreeke]

I realised I could use my redithering scripts with other models to add back some texture, in this case a degif upscale which was repalettised afterwards

[MusicallyInspired]

Dang that looks good.

[Phredreeke]

What a wonderful smell I've discovered



This is similar to the last one except instead of DeGif I used Farthestneighbor + Fatal Photo

[Mark]

I was curious what I could do using just a few filters in my ancient Paint Shop Pro 7 program. I wasn't expecting anything as good as the upscale but its not too bad considering the source and only 5 minutes of trial and error with a few simple filters. Mine is on the left. Using a color replacer brush I could have turned those white speck to the colors of the robe but I wanted to see what I could do with using only filters.

Attached thumbnail(s)

• 

[MrFlibble]

Phredreeke, on 08 July 2021 - 03:58 AM, said:

I'm sorry to say this but I don't like this upscale at all. First off, some straight lines are not really straight in the upscale (like the right part of the door handle at the bottom), and secondly, it looks over-sharpened. Maybe it looks okay in-game though, I cannot tell.

Also it looks like your dithering script has a preponderance for creating rectangular patterns or pixels neatly lined up in straight horizontal and vertical lines.

I've run the same image through some 2x models, and while they are not perfect they do not have these two issues that I pointed out above:


The models used here are: 2x_Faithful_v2_130000_G.pth, 2x_FakeFaith-Lite_105000_G.pth, 2x_Faithful-Lite_275000_G.pth

Once again, I'm really sorry to say this, I appreciate your work and the amount of effort you put into this.