[Phredreeke]

Ion Fury HUD image


From this image



From this image


From one of Daggerfall's faces


From one of the SW player sprites


The RRRA cheerleader

[NightFright]

Shelly is actually OK if you cover the left side of her face (or at least the eye). The rest is more or less totally FUBAR.

[Phredreeke]

YARRRR!

[MrFlibble]

Actually for once your Duki Nuki looks like a consistent human face, as long as you look past all the glitches caused by odd shading of the original. I'd rate him right next to Shelly who came out better than most -- but I think that's because her face has the highest resolution among the source images you're shown, perhaps except the meme face one.

Not sure why this GAN thing wants to paint in spectacles everywhere though, but I guess the developers wanted to cover a wide range of faces. Odd that it put only one half of it on the Duki.

[DisgustedGrunt]

Phredreeke, on 18 October 2021 - 07:12 AM, said:

The RRRA cheerleader

It may be accurate, you never know with Redneck Rampage.

[MusicallyInspired]

Is that....Hugo Martin?

[Phredreeke]

[MusicallyInspired]

Oh thank you for doing that. Elaine never looked better!

[Jimmy]

AI "upscales" will only look their best if the original art is edited to account for the quirks of the original art + the quirks of the AI beforehand.

[Phredreeke]

The VGA portraits look good (only preprocessing was resizing them to correct aspect ratio and some blurring)

The EGA portrait was a disaster but it has a certain freakish charm to it.

Edit: more pics






And two more EGA portraits, these came out better, perhaps I used more blurring

[MrFlibble]

Jimmy, on 18 October 2021 - 06:29 PM, said:

AI "upscales" will only look their best if the original art is edited to account for the quirks of the original art + the quirks of the AI beforehand.

Not necessarily, examples like this which I've already posted prove that a good AI can make do with virtually unedited original art pretty well.



One could of course say that GFPGAN produces more "accurate" output on their first try, while pixel2style2pixel/StyleGAN leaps too far ahead and then needs to be interpolated with the original image to get it right.

But on the other hand, it appears that the StyleGAN AI has not made a single error of the kind suitable for the hall of horrors if you look at the many examples. The faces may be off the mark compared to the pixel painted originals but they're unmistakably photorealistic human faces.

[Jimmy]

MrFlibble, on 20 October 2021 - 10:24 AM, said:

Not necessarily, examples like this which I've already posted prove that a good AI can make do with virtually unedited original art pretty well.

In the context of this thread specifically you're missing the point.

These abominations occur because the original art has logic errors that the AI multiplies.

[MrFlibble]

Jimmy, on 20 October 2021 - 11:11 AM, said:

These abominations occur because the original art has logic errors that the AI multiplies.

What do you think is the "logic error" that accounts for duplicate pupils in Shelly's eye?

I think it's a problem with the network/model's architecture that allows for logic errors such unrealistic features of this kind (also just half of spectacles etc.) to end up in the upscaled image. StyleGAN seems to have no errors of this kind.

[Phredreeke]

My understanding is StyleGAN doesn't try to upscale the input image, instead it has its own rules for how to synthesise an image, which parameters are influenced by the input image. And GFPGAN combines that technique with ESRGAN, although I'm not sure how it does that.

[Phredreeke]

This isn't GFPGAN but instead Stylegan3+CLIP

https://cdn.discorda...dladdning_6.mp4

[MrFlibble]

Jimmy, on 18 October 2021 - 06:29 PM, said:

AI "upscales" will only look their best if the original art is edited to account for the quirks of the original art + the quirks of the AI beforehand.

BTW, I'd never argue with the notion itself that editing input images will most likely allow to improve the upscaled results. After all, we've been using all kinds of preprocessing even before ESRGAN for Waifu2x to somewhat improve sprite edges etc. But the problem here is twofold: a) the process of determining which preprocessing steps work best (or have any effect at all) involves a lot of trial and error and b) manually editing input images before upscaling kind of defeats the whole purpose of using AI as it's supposed to be as automated as possible.

It's been a lot of fun messing with ESRGAN for sure, but I have to tell you, if I needed upscales for a project that was on a schedule and I wanted quality, I guess I'd rather commission them from a skilled artist than go through the pain of trying to trick the available models to do the exact thing that I want by editing input and/or output images because none of the models quite do what I need by default.

ESRGAN and other AI methods work very well with upscaling videos though. Pity that the C&C Remastered FMVs came out rather poor with visible artifacts and stuff, because fans have created way better upscales, like these for Tiberian Sun for example:

[Phredreeke]

If you want to recreate assets from scratch then that has been tried already. I think we discovered early on that ESRGAN on its own wasn't going to produce satisfactory results.

To continue the GFPGAN topic, here's the Blake Stone box art



and here's Ecce Homo

[MrFlibble]

Phredreeke, on 21 October 2021 - 08:04 AM, said:

I think we discovered early on that ESRGAN on its own wasn't going to produce satisfactory results.

Wait, what. The early models maybe, but currently quite a few models produce excellent results -- the problem is that apparently, no one bothered to train a model specifically to upscale Build/2.D FPS engine sprites and textures to a level of quality that would be considered "satisfactory". Perhaps this is in part because there is no good data to train such a model on (i.e. high-res pre-rendered sprites), and/or the sample is too small for proper training.

[Phredreeke]

We could attempt to train such a model. Or we get 90% of the way there and finish the last 10% manually. I am doing the latter, but no one is prevented from attempting the former. In fact some of the better results of the upscale packs could arguably be used for training.

[DisgustedGrunt]

Phredreeke, on 21 October 2021 - 08:04 AM, said:

and here's Ecce Homo

Guess not even an AI can improve on perfection.

[MrFlibble]

Phredreeke, on 21 October 2021 - 01:44 PM, said:

Or we get 90% of the way there and finish the last 10% manually.

Admittedly AI upscaling is still in its infancy, but it could achieve so much more than what is available now. Imagine an AI that can essentially do to textures or sprites what SyleGAN now does to photorealistic human face images. I think it's perfectly possible to achieve.

Or, more realistically, it seems reasonable to have a model that produces 2x upscales straight away if that is the target resolution, but since 2x ESRGAN models are scarce we have to make do with 4x and then downsample. But I'd still argue that the 2x Faithful model actually has some advantages over 4x ones, even considering its deficiencies. Like it tries to reproduce certain patterns on a larger scale instead of just sizing those patterns up, e.g. on the teleporter pad:

Phredreeke, on 21 October 2021 - 01:44 PM, said:

In fact some of the better results of the upscale packs could arguably be used for training.

Would be interesting to use for training those flats which have received ample manual edits such as the panels and others that have a lot of minute detail. Sadly I'm of no help here 'cause I don't have a powerful enough CUDA compatible card.

[Jimmy]

MrFlibble, on 21 October 2021 - 03:17 AM, said:

b) manually editing input images before upscaling kind of defeats the whole purpose of using AI as it's supposed to be as automated as possible.

Pipedream. As I've said before, Doom's sprites and textures were a breeze because of how smooth and perfect they are. Many other games from the era are dirty and gritty. Dirty, gritty, and imperfect graphics confuse many of the models.

[MusicallyInspired]

MrFlibble, on 21 October 2021 - 03:17 AM, said:

b) manually editing input images before upscaling kind of defeats the whole purpose of using AI as it's supposed to be as automated as possible.

Yeah I don't agree with that at all. At least not as far as creating a distribution pack goes anyway. If there are legal problems and nobody is ALLOWED to distribute it, then I can see improving a model so much so that anyone can just batch process themselves circumventing that. But otherwise, if the AI needs a bit of help to improve the look I don't see any problem with it. I'm doing the same thing for my Sierra HRPs that I'm making to work with a new fork of ScummVM called ScummVM-ext. I just don't believe there exists a "one-size-fits-all-perfectly" solution when it comes to AI upscaling.

[Phredreeke]

Guys, I made this thread so AI monstrosities wouldn't clog up the other thread


Bubba from a RRRA promo.

[MrFlibble]

MusicallyInspired, on 27 October 2021 - 04:58 AM, said:

I'm doing the same thing for my Sierra HRPs that I'm making to work with a new fork of ScummVM called ScummVM-ext.

I'd love to hear more about what pre-processing steps you take before upscaling the images. Why don't you share that in the ESRGAN thread? Or are you using a different AI framework?

MusicallyInspired, on 27 October 2021 - 04:58 AM, said:

I just don't believe there exists a "one-size-fits-all-perfectly" solution when it comes to AI upscaling.

I did not imply that there is one, although some of the more recent ESRGAN models are a bit more successful on that front compared to previous attempts.

My point though is that the whole ESRGAN thing is a major attempt to use a tool for a purpose completely different from its original intentions. It would be wrong to say that this kind of repurposing cannot or should not be done, but the caveat here is that you end up playing with limitations imposed by the original task, namely solving the SISR problem. That is, restoring a downscaled image back to its original size as faithfully as possible. Whereas many examples of video game pixel art that are being upscaled never had any kind of high-resolution originals at all. And even if there were said originals, they are on a completely different scale of resolutions compared to what the original ESRGAN models were intended to handle.

BTW, it is my understanding that Sierra and Lucas Arts' adventures are a notable exception because precisely there, a lot of art had high-res originals which theoretically could be restored in the SISR solution framework. But here you have another issue, because ESRGAN's authors have not solved the SISR problem, it's just that their model produced results which were closer approximations to ground truth compared to some other works in this field.

The other problem of note is that ESRGAN or Topaz Gigapixel are but two examples of a multitude of AIs. If the photorealistic faces produced by StyleGAN have not convinced you that deep learning technology has a much greater potential than what can be achieved with ESRGAN, well then I guess you'll just have to wait until there are real examples of upscales that go far beyond those available now. I suppose it is inevitable that AI upscaling will be sooner or later employed in commercial remastering of old video games, since there already are examples of that achieved with modern available methods (I think I heard of some console remaster that even used waifu2x). At that point, there will likely be AI networks specifically designed to upscale video game art of various kinds, with possibly much better results compared to anything we have seen so far.

[Ninety-Six]

Phredreeke, on 27 October 2021 - 05:20 AM, said:

Bubba from a RRRA promo.

This looks "real" in the sense that it looks like someone tried to make a clay figure out of him and then kinda botched the painting a little bit.

[Phredreeke]

Hello from Discord

Slap McBald asked me to run this through GFPGAN

I did and this is the result

[MrFlibble]

Captain Bible HD?

[Phredreeke]

Happy Halloween



This is the new beast transform from the upscale pack, upscaled through GFPGAN.

[UltravoxSapphire]

I asked Phredreeke to place some pics of Pete Burns through GFPGAN in PMs.

The results here are... interesting.

Yes, you are looking at a picture of a man. No, he's not a transgender woman. This man 100% saw himself as a man.


Pre-blurring




Pre-blurring (differences between this and the first one are subtle)


Even more pre-blurring...differences aren't so subtle anymore.


Pic of him from the 80s. Taken from the thumbnail of a crusty ass interview posted on Youtube.