[MrFlibble]

So I was toying around with various ways to scale up video game sprites and managed to produce these results for Duke3D enemies:

(this is converted to the Duke3D palette, original image scaled twice using nearest neighbour for comparison).

Basically, I noticed that waifu2x does a decent job scaling up pre-rendered images, which would make it good for Duke3D sprites if only sprite edges weren't handled so badly. On the other hand, xBRZ can produce very smooth edges but does nothing to enhance image detail.

The obvious solution was to combine both approaches, keeping xBRZ's smooth edges and the detail that waifu2x can produce.

So the steps I took to produce the above image:

1. Assembled the sprites into a single image (obviously). I also created from it a set of solid colour shapes with the colour replacement tool found in GrafX2.

2. Scaled the sprite sheet to 4x its original size using both waifu2x and the xBRZ Scaler Test. Also scaled up the shape sheet with xBRZ and neutralised all transition colours in the resulting image (again, using GrafX to only keep one colour for shapes and another for the background).

3. Used the scaled up shape sheet to produce shapes that are 3 pixels thinner (note: if you see a blue outline it's an optical illusion). I did this by treating the shape colour as the background colour (using KolourPaint) and placing four copies of the background around the original image -- this thins the shapes by one line of pixels, and repeating this thrice gives a three-pixel thinning result.

4. Pasted the thinned shapes over the xBRZ-scaled sprites, getting a thin outline. I also completely removed the outer one pixel thin outline.

5. Pasted the resulting outline over the waifu2x-scaled sprites, covering the rough edges (resulting image).

6. I did not like how the 4x sprites came out so I loaded them in GIMP and first converted them to the Duke3D palette and then scaled down to 2x the original size using nearest neighbour method. The result is shown above.

I'd say the results are pretty interesting, perhaps such sprites could serve as the base for some true high-resolution sprites after touch-up by an artist?

[Phredreeke]

this is similar to how I did my sprite upscales for Blood, except I just used a nearest neighbour upscale of the original as mask followed by a median filter (which in hindsight wasn't the best idea, as the filter treats red,green,blue channels separately, causing pink fringing around the edges)

for the background upscales, I instead opted to add transparency before upscaling with waifu2x. this causes waifu2x to utterly fail at processing those parts, so I made a new sprite mask by blurring and posterizing the existing opacity.

I followed up both with a noise layer on top of the upscaled tile. for the background I used a slightly more complicated approach...

Edit: Here are your sprites, with a slight noise pass applied. https://i.imgur.com/p7Wnq8E.png

Edit 2: Oh, and PLUs are a pain in the ass. fortunately Duke 3D doesn't use them as extensively as Blood

[MrFlibble]

Phredreeke, on 14 June 2018 - 01:37 AM, said:

Edit: Here are your sprites, with a slight noise pass applied. https://i.imgur.com/p7Wnq8E.png

That looks cool, what exactly did you do to achieve this?

[Phredreeke]

This describes using PSP8 (yeah, I use a 15 year old image editing software) you will have to find equivalent functions in whatever software you use.

1. Create a mask by duplicating the image, filling it with the background color using the difference blend mode, then using a threshold effect set to 1 to make every non-black pixel white.

2. Now, going back to the original picture, use the levels functions and set gamma to 2

3. Now apply a noise filter with a 7% setting gaussian and monochrome.

4. Apply the levels effect again except this time set gamma to 0.5

5. Finally apply the mask you created in the first step.

[MrFlibble]

Thanks! I'll try to figure it out (I'm using GIMP), for now I found an option in GIMP called HSV noise which produces kinda-sorta similar results:


This is at default settings except Holdness set at 1 (default is 2).

Adding noise is a good idea, thanks -- I wouldn't have come up with it

[MrFlibble]

In the meantime I decided to take a jab at some weapon and other item pickups:


This one was a bit more tricky because these have a lot of straight lines at angles other than 45 degrees and xBRZ makes them into squiggly lines that have to be fixed manually. I tried several approaches here but settled upon editing the mask which I used to create the xBRZ outline to paste over the waifu2x'd sprites. Dark purple are lines which I added over the xBRZ squiggly stuff.

I also decided that the mask itself should be edited more carefully. In the previous attempts, I scaled the mask up with xBRZ to 4x the original size and then used GrafX2 to convert any transitional colours at edges into the background colour. This resulted in some detail from the images being eaten up by the background. Now I think it's reasonable to separate transitional colours that are part of the background, and those that are part of the sprite, both by the colour itself (e.g. with a green/purple mask/background, sprite-related stuff will be greenish and background-related purplish), and also by using the scaled sprite themselves as reference.

I also added the HSV noise I mentioned earlier at default settings after scaling the image to 2x the original size. Not sure if it makes a great improvement compared to the no noise stuff.

Also so far I've no idea how to put these scaled sprites to use with EDuke32.

UPD: Take two:


I attempted to improve the pipebomb a little bit, and also fix the edited outlines where some pixels were missing resulting in uneven lines (this should be noticeable with the pistol clip). Probably trying to smoothen the round parts of the pipebomb didn't come out as good as I'd hoped.

[Danukem]

The upscaled enemies are pretty cool. Keep in mind that if you put them in the game, you will have to change some game code and you could get some bad side-effects depending on how you do it. Side-effects could include enemies starting at 2x size and then suddenly becoming normal size when you see them, enemies shrinking to the wrong size (from shrinker ray) and/or growing back to the wrong size, rockets and other projectiles coming out of enemies at the wrong spot, enemies being unable to clear curbs and other short obstacles, and more. You could bypass the need for code changes if you add them in as high-res tiles (HRP style), but then they will not work in classic and they won't change palette properly.

[Hendricks266]

The ideal way to handle these sprites would be to define them as hightile, but with a special keyword for them to use the palette. That way all palswaps, shading, and global palette effects are handled automatically, and no game logic requires any changes.

I will need to revisit our texture manager code in the wake of the recent shader work to see if the task has gotten any easier. I think it has.

[MrFlibble]

Thanks guys, I'll look into that (actually I imagined that handling this kind of stuff would be a bit easier that what it sounds now)

[Jimmy]

I tend to hate scalers, but these are actually pretty damn good. If a talented artist came in on top of them they would be excellent. I would also recommend, if someone was serious about this project, to actually clean up the enemy sprites themselves prior to scaling them. Some of the side frames or animation frames especially have very sloppy things going on, if you cleaned them up before you scaled them you'd end up with better results. For example, the cleanest enemy arts are for the Protector Drone and the Queen. I theorize they would take almost no clean up after scaling.

[Phredreeke]

Thinking about it, wouldn't this cause issues with fullbrights?

[MrFlibble]

I played around with the method a bit more, (also with Daggerfall sprites) and came to the conclusion that it can be improved. Not only waifu2x doesn't process sprite edges properly, but it also creates odd colour artefacts near them, which then have to be masked by importing parts of the same sprite scaled up by xBRZ (which have lower detail anyway, even if it's just an outline that is several pixels thin). Why not eliminate the incorrect processing of edges altogether instead?

Previously I played around with waifu2x and scaled up screenshots from different games, including The Age of Empires. This game me an interesting result, shown below and zoomed 2x (nearest neighbour) for better visibility:


It's plain obvious that the edges of the wall tower roofs are processed smoothly when they are against a noisy background (dirt tiles) and very poorly when against a solid colour (the horse's shadow).

With the sprites easily extractable from the background using the mask, I decided to try scaling them up when not against the empty background but against a texture from Duke3D.

Here's some sprites of Duke himself (cropped using this mask):

no background pattern


wall texture


restroom wall tiles


The difference might not be immediately noticeable but there are artefacts around the edges,including some very odd stuff going on with the boots, when there's no background pattern, and much better results when there's one. Personally I like the restroom wall tiles result a bit more.

[Danukem]

Duke looks like he was run through one of those artistic Photoshop filters (palette knife, maybe?). The earlier bad guys turned out better imo.

[Phredreeke]

I think 1. the face needs touching up, or preferably redrawing. 2. the edges needs to be redrawn. Here's a very rough attempt at the first.

I took Duke's face from tile 3350 and colored it using the player sprite. I then overlaid it on the player sprite and softened the edges a bit, and also added some noise to it.

Attached thumbnail(s)

• 
• 

[MusicallyInspired]

Nice. That looks much better. It's too bad you couldn't fill in some of the detail in the body to equal the face.

[MrFlibble]

Trooper Dan, on 24 June 2018 - 10:48 AM, said:

Duke looks like he was run through one of those artistic Photoshop filters (palette knife, maybe?). The earlier bad guys turned out better imo.

That's because the earlier sprites are 2x and Duke is 4x, I didn't touch up anything to make the difference between the no-background and background pattern versions more visible.

Here's what all the sprites look like when scaled up 4x with the bathroom tiles background (xBRZ only used to scale up the mask, unlike the method described in the first post):


And here's the same image, converted to the Duke3D palette, scaled down to 2x (nearest neighbour method), then adding HSV noise and converting to the Duke palette once more:


Cf. my first post:

MrFlibble, on 13 June 2018 - 05:52 AM, said:

6. I did not like how the 4x sprites came out so I loaded them in GIMP and first converted them to the Duke3D palette and then scaled down to 2x the original size using nearest neighbour method.

UPD: Somehow I uploaded images with a wrong mask applied, now fixed with correct versions of both images.

[Phredreeke]

How is this progressing? I'd love to see more of this.

[MrFlibble]

I haven't had the time to get back at this for a while. Still haven't tried to figure out how to make this actually work with eDuke32 yet.

[Daedolon]

How do you get waifu2x to work with pixel edges? For me it always preserves the original pixels around the edges.

[Phredreeke]

Both me and MrFlibble are processing the masks separately. I use some fairly complicated PSP scripts while MrFlibble uses xbrz for his masks.

Edit: Here are some from a script I'm toying around with

[MusicallyInspired]

Man that's really good. Never seen filters work so transparent.

[MrFlibble]

Daedolon, on 15 August 2018 - 10:02 AM, said:

How do you get waifu2x to work with pixel edges? For me it always preserves the original pixels around the edges.

My method is described in the posts above, I use xBRZ mask plus scaling sprites against a background for better results around the edges.

[MrFlibble]

So I was thinking about scaling up the sprites without resotring to xBRZ at all, here's a preliminary result (4x):

Spoiler

I realised that waifu2x fails to properly process any sharp contrast between colours, so I imposed the sprites on a background and blurred them a bit (Gaussian blur at 0,7 pixel radius in GIMP). I used the same bathroom tile texture from Duke3D for the sprites, but to get a mask I processed another image with the same sprites against grass tiles from Wyrmsun (with the same level of blur):

After scaling up the grassy image by 4x in waifu2x I used the Kuwahara-Nagao blur at 4 pixel radius in MTPaint to get sharper edges, then converted the image to the Duke3D palette with no dithering to get a clean mask to be then imposed on the image processed with the bathroom tile background. Finally I used MTPaint to convert the result to the original palette with the Scattered (effect) dithering method.

I'm still not settled on the optimal method of creating the mask, it might be still needed to clip a one-pixel thick outline to get rid of the halo-like artefacts that arose due to blurring I guess.

[Phredreeke]

Yeah, the mask is the hard part. I look forward to seeing the final result though

[MrFlibble]

Gentlemen, believe it or not but the image below is raw output from waifu2x without any filters or whatever:

Spoiler

I still used xBRZ to produce it, but not in a way that I'd myself imagine.

xBRZ was needed to soften the contrast and blur the sprites into the background. I first tried Gaussian blur as described above, to process both sprites without backgrounds and screenshots from sprite-based games (examples here). However the problem with blur is that it, well, blurs the image, and also doesn't do the job of reducing sharp contrast which results in poor geometry after waifu2x processing quite well. I tried several different approaches and settled with using xBRZ as follows:

• scale up the raw image (sprites on pink background) 4x in the ScalerTest xBRZ tool
  
• load the resulting image in GIMP, apply pixelate blur filter at 4 pixel radius
  
• resize the image back to original dimensions with nearest neighbour interpolation

The image is then fed to waifu2x, the result shown above.

Scaling the sprites against a background might still yield better results, but at least this gives a rather convenient way of creating a mask.

[Phredreeke]

How does it look with mask applied?

[MrFlibble]

Here's what I got from the above image:

I used GIMP to convert it to the Duke3D palette, then used the select by colour tool to select the background, increased the selection size by one pixel to get rid of the blurry edges, then resized to 2x the original size. No noise or anything added this time.

I think the result is pretty decent, especially considering that it does not require messing with parts of xBRZ processed images to get the edges smooth.

[MusicallyInspired]

Very nice.

[Hendricks266]

I presume waifu2x has been trained on anime-like drawings, and I suspect this is why your images look like oil paintings. I would bet the result would be better if you trained the neural network with a custom data set.

[MrFlibble]

Hendricks266, on 03 October 2018 - 04:23 PM, said:

I presume waifu2x has been trained on anime-like drawings, and I suspect this is why your images look like oil paintings.

I'm not sure. First off, waifu2x does a pretty decent job scaling up 3D rendered images and photos without the result looking like oil paintings. It's also worth noting that I tried scaling up Doom sprites, with the results broadly similar to what another user achieved with NVIDIA GameWorks Super Resolution neural network scaling (I doubt that one was trained on anime images).

I suppose this is more to do with the nature of the material which is upscaled, namely 8-bit colour, low resolution images (even if produced from 3D models). Even though smoothing the source images allows to achieve better results, there are still limitations in place, as waifu2x doesn't really add detail — no idea if a neural network trained specifically on sprite sets would.

From my observations, scaling images to twice the original size works pretty well, everything beyond that highly depends on what is being worked with. Existing detail will get more clearly visible, if not too small, but if a certain detail comprises a few pixels that the player basically guesses what it is I'm not sure if there's a way to enhance that in an automated fashion.