[MrFlibble]

I just tried blending the results of ESRGAN and SFTGAN processing for sprites (similar to what I did here), here's a very rough test result (zoomed 2x):

[MusicallyInspired]

That's a lot better!

[Phredreeke]

there's some strong color fringing around the edges though, particularly on top of octobrain's head

[MusicallyInspired]

Nothing you can't pixel-edit. I suspect you're never going to get away with upscaling sprites without doing that.

[MrFlibble]

Phredreeke, on 27 January 2019 - 02:30 PM, said:

there's some strong color fringing around the edges though, particularly on top of octobrain's head

That can be fixed by using different backgrounds, and blending the results as I did with the waifu2x upscale attempt. I just ran a quick test with the green background I used previously.

[MusicallyInspired]

I think he was referring to the harsh red line on the top of the octobrain's head which is not on the background.

[Phredreeke]

Actually I think it may be connected to using a green background. On my older Blood upscales I got greenish fringes on fine detail from the pink background

[MrFlibble]

MusicallyInspired, on 29 January 2019 - 08:19 AM, said:

I think he was referring to the harsh red line on the top of the octobrain's head which is not on the background.

There was no misunderstanding. The wrong colours arise because of the softening procedure I apply to input images using xBRZ. This results in slight blurring, so the colours at sprite edges are affected depending on the background colour.

So far I figured that the least effort-consuming way to counter this is to process separate sprite sheets with different backgrounds (black, white), then blend them as layers. This is even more appropriate here because the method above blends ESRGAN and SFTGAN results anyway.

[MrFlibble]

I did some further tests with ESRGAN and SFTGAN and posted the results in the other thread.

[MrFlibble]

Recently I ran a batch of tests with Daggerfall sprites and concluded that my previous method I called xBRZ softening is rather imperfect (as demonstrated here). Some trial and error later I came up with a better approach which also allows to produce somewhat cleaner sprite edges. As a test, I scaled up some of Duke3D babes:

• 2x the original size
  
  Spoiler

  
  
  
• 4x the original size
  
  Spoiler

  

The images are scaled up with ESRGAN using an interpolated model (ESRGAN4x and Manga109Attempt at alpha = 0.2).

Pre-processing steps:

• use a dark background for the sprites (but not black), preferably if the colour is not used in the images (I used dark green for this batch)
  
• open the image in GIMP, apply Enhance -> Antialiase once, export the result
  
• use the xBRZ Scaler Test tool to scale image up using the xBR scaler (not xBRZ) to 4x the original size
  
• in GIMP, apply Gaussian blur to the xBR-scaled image at 1.5 pixel radius. This is only done to compensate for the sharpening effect produced by Bicubic downsampling (see below)
  
• scale the image down to the original dimensions using Bicubic interpolation

Edge cleanup steps:

• make a copy of the image scaled up with ESRGAN, ope it in mtPaint and apply Kuwahara-Nagao blur at 2 pixel radius with preserve details on (1 pixel radius for the image scaled down to 2x the original size)
  
• load the palette of the original unaltered image and convert to indexed mode with no dithering and colour space set to RGB (in options)
  
• save the image and open both the upscaled original and the version with KN blur in GIMP. Use GIMP's select by colour tool to select the background colour
  
• right-click on the selection and increase selection size by 1 pixel
  
• copy the selected area to the unaltered upscaled image and save it, then open it in mtPaint and convert to the original palette as described above
  
• you can now open the resulting image in GIMP and again select the background colour, then delete it (you'll need to add alpha channel for transparency though first)

This method is not completely perfect but it leaves just a few pixels on the sprite edges that may be cleaned up manually.