[Micky C]

This is fantastic work.

So how automatic is the upscaling process? Would it be feasible to do not only the sprites, but textures as well, ie all the art in the game?

[NightFright]

I think textures are easier than sprites since they would work pretty much like HRP textures, especially since they would maintain their aspect ratios.

[Phredreeke]

It's fairly automated, I put together a folder with images to be upscaled, this is then run through a number of scripts

1. creates a mask from the image in question
2. antialias the edges of the image, this is then fed to ESRGAN
3. the result of ESRGAN is run through a second script that sharpens, downscales and applies respective game's palette
4. finally, the mask from step 1 is applied to the downscaled image

There's actually an additional couple of steps I've used to add the small detail like bloodspatter, but it's rather complicated, but the above gives a fairly understandable picture of the process. For textures step 1 would be omitted, but in step 2 instead the edges would be slightly extended (either by mirroring or wrapping/tiling) to give context to both the antialiasing preprocess but most importantly ESRGAN.

Yes, it can be done to textures, I've got a folder of textures I've done as a proof of concept, HOWEVER, unlike the sprites a lot of them are going to need manual work (see the attached picture for a worst case example)

Attached thumbnail(s)

• 

[Mark]

How does it handle seamless tiling textures? Do texture edges still match up?

[MusicallyInspired]

That would likely require work by hand as well.

[Danukem]

MusicallyInspired, on 28 February 2019 - 05:52 PM, said:

That would likely require work by hand as well.

It had better not, or that would be a deal breaker. Imagine doing hand edits on thousands of images to make them tile properly, and then having to do that all over again every time the upscaling method is improved.

[Mark]

The reason I thought to ask was earlier today I was testing results of converting some seamless high res 32 bit textures to lower res and duke palette. They looked great alone but the tiling was noticably off. I didn't bother to determine if it was the size and/or color downgrade that was the issue.

[Phredreeke]

Basically, if you want the upscaled image to tile seamlessly you include context on the sides of the image when you upscale it then crop it out afterwards

[Marya]

I apologize in advance, but i have a very poor eye for detail. So does this essentially make sprites sharper and cleaner looking?

[Phredreeke]

The goal is to appear as if the game's sprites had originally been at double resolution. I leave it to others to judge how well I've achieved that goal.

[Marya]

Phredreeke, on 01 March 2019 - 07:45 AM, said:

The goal is to appear as if the game's sprites had originally been at double resolution. I leave it to others to judge how well I've achieved that goal.

So does that include the strippers and other girls or are they not updated yet?

[MrFlibble]

Lunick, on 25 February 2019 - 02:05 AM, said:

Made a quick video showing the pack in action

I just realised a thing (Phredreeke had kindly allowed me to check out the pack before the public release), the larger sprites kind of beg to have more animation frames too (like Smooth Doom or whatever that project is called). E.g. take a look at the Enforcer firing at 0:25, it feels sorta jerky. But that would require quite some work yet.

Basically ESRGAN did some very good job on scaling the sprites up, and Phredreeke refined the method to get proper sprite masks almost to perfection. However some manual touch-up could still be used on some frames, there's some colour bleeding out into wrong areas, e.g. the Trooper's eyes have some pixels off in certain frames:

Also some detail on he sprites gets blurry at some angles. For example, the claws/spikes on the Octabrain are sort of not pronounced enough in certain frames.

Pretty sure that an artist of Cage's calibre could fix that in no time (provided there was time and willingness to spend effort on this).

Phredreeke, on 28 February 2019 - 03:11 PM, said:

It's fairly automated, I put together a folder with images to be upscaled, this is then run through a number of scripts

1. creates a mask from the image in question
2. antialias the edges of the image, this is then fed to ESRGAN
3. the result of ESRGAN is run through a second script that sharpens, downscales and applies respective game's palette
4. finally, the mask from step 1 is applied to the downscaled image

If you could share the exact scripts that you use that would be of great help to other gaming communities like Daggerfall Unity who are also doing sprite upscales with ESRGAN.

[Phredreeke]

Upscaled textures, shown next to the original for comparison.






However, I still got issues with tiles with text on it such as this and the order sign above.



If anyone wish to take on the role of "fixing" such problem tiles please either DM me or reply in this thread. (this also applies to Blood as well as potentially other Build games once we got an adequate source port)

[MrFlibble]

Phredreeke, am I right that the anti-aliasing process you do is aimed at properly processing sprite edges and nothing else?

If yes, have you considered cropping the outline of each sprite by one pixel and pasting this over the anti-aliased version to keep the edges anti-aliased while everything else remains sharp? Perhaps that would save you the need to sharpen the image in post-processing, which is a step I'd personally like to avoid as it might result in colour distortion (like what we observed on the border areas between colour-swappable blue and skin colour on the Trooper and dancers).

[Phredreeke]

No, it's not just for the sake of sprite edges. I could have achieved that with a much simpler script.

The neural network upscalers are trained on images downscaled with bicubic filters. This is not representative of the graphics being upscaled. I am not able myself to train a model based on pixel graphics, so instead I alter the source image to look like what the neural network would expect had it been downsampled from a larger resolution original.

IMO the artifacts of the sharpening filter outweighs the artifacts from not preprocessing.

[LeoD]

Phredreeke, on 05 March 2019 - 04:31 AM, said:

The monks are drunk...
This pack fills an ugly little gap in the HRP: decorative wall-aligned textures which show up as models and usually need to be notmd'd via maphacks. Now we have something better than the original ART that used to show up instead. Quite an improvement for just a few extra MB.
Therefore I'd like to ask you to process the Duke ART (1405ff) for this purpose, too.

[Phredreeke]

Right now I'm working on my Blood upscale.

I'm a bit confused over what you're asking for. You want upscales for the sprites that the HRP replaces with models?

[LeoD]

Phredreeke, on 22 March 2019 - 12:11 PM, said:

I'm a bit confused over what you're asking for. You want upscales for the sprites that the HRP replaces with models?

Yes, because of this:

Phredreeke, on 22 March 2019 - 12:11 PM, said:

Right now I'm working on my Blood upscale.

15 years of HRP, 4 years since v5.4, and no new release on schedule - we're not exactly in a hurry here.

[Mark]

never mind, I read the post again and deleted my comments.

[Phredreeke]

Unfortunately Duke is among the sprites that don't upscale well

[Hendricks266]

Upscales have never been allowed in the HRP. You could make your own pack as a complement to it.

At one point I had a setup with Blender and an MD3 importer that actually supported animations and UVs, and I used it to render the HRP's images of all the pickups for use in EDuke32's HUD. You could do that for these cases too.

[Tea Monster]

Someone should start a project where people use their art talents to manually recreate the existing tiles in a higher resolution than what exists in the original game files. You could call it something like "High Resolution Pack" or something.

[LeoD]

Hendricks266, on 22 March 2019 - 02:21 PM, said:

Upscales have never been allowed in the HRP.

Not for reasons that apply here. Not for reasons that apply nowadays. These upscales may only show up when explicitly requested by a notmd maphack command for a specific instance. In my working folder I have indeed put them into the maphacks hierarchy.

[NightFright]

You either make an upscale pack that only includes upscales or you make an HRP with custom made highres artwork. Including upscales would basically be violating the license it's shipped with. We have rejected upscales as HRP entries in the past, and only because their quality got better, it doesn't mean the rules can be changed.

I know how frustrating it is not to get a new HRP for so long, but you shouldn't mix styles for the sake of filling gaps. We ran out of people who want to provide custom made content, but you can't just run stuff through filters now, even if manual adjustments are added. It's different approaches that don't fit together. HRP entries need to be done from scratch.

[MusicallyInspired]

Word.

[Fantinaikos]

Tried to fix the face a little.

[Phredreeke]

I don't know if this was LeoD's intent, but please don't distribute my upscales with HRP assets.

[LeoD]

Phredreeke, on 22 March 2019 - 04:01 PM, said:

I don't know if this was LeoD's intent, but please don't distribute my upscales with HRP assets.

It was, indeed.

NightFright, on 22 March 2019 - 03:10 PM, said:

Including upscales would basically be violating the license it's shipped with. We have rejected upscales as HRP entries in the past, and only because their quality got better, it doesn't mean the rules can be changed.

I'm outnumbered anyway, but please cite me that very passage from hrp_art_license.txt.

[Hendricks266]

The appropriate thing to cite would be 3D Realms Forums discussions on the topic, but I can't look for them right now. Instead, have a look at my NWHRP / VacaHRP submission guidelines (dated 2010) which state the no derivative works rule (#4).

[Jimmy]

There's a lot of discussion in this thread about "errors" that "show up" "because" of the upscaling process and how they need to be fixed afterwards. I've said it a million times, but probably 8 out of 10 of these problems should be fixed before the upscaling process precisely because there are errors in the original sprites/textures that compound into anomalous behaviour during the upscale. The Doom upscale was so successful because Doom's art is much cleaner and clearer.