Duke4.net Forums: I tried doing neural network upscale of DN3D textures. Who can implement it back into the game? - Duke4.net Forums

Jump to content

Hide message Show message
Welcome to the Duke4.net Forums!

Register an account now to get access to all board features. After you've registered and logged in, you'll be able to create topics, post replies, send and receive private messages, disable the viewing of ads and more!

  • 2 Pages +
  • 1
  • 2
  • You cannot start a new topic
  • You cannot reply to this topic

I tried doing neural network upscale of DN3D textures. Who can implement it back into the game?

User is offline   MF_Kitten 

  • 3

#1

As the title states, I have figured out to export the .BMP files from the .art files, I converted them to .PNG, and then I used Topaz AI Gigapixel to upscale some of it as a test. Results are great, but now the question remains: how does one stuff it back into the game? I know the tool exists to make .art files, but I couldn't get it to work. So basically I can provide the upscaled high res assets, if anyone wants to put it into the game. Any takers?

TL;DR: I can make nice high res versions of the original art assets, if you can put it back into the game!
1

User is offline   Photonic 

  • 1,268

#2

View PostMF_Kitten, on 01 February 2019 - 02:52 PM, said:

As the title states, I have figured out to export the .BMP files from the .art files, I converted them to .PNG, and then I used Topaz AI Gigapixel to upscale some of it as a test. Results are great, but now the question remains: how does one stuff it back into the game? I know the tool exists to make .art files, but I couldn't get it to work. So basically I can provide the upscaled high res assets, if anyone wants to put it into the game. Any takers?

TL;DR: I can make nice high res versions of the original art assets, if you can put it back into the game!

I want this. I'll so gladly do it. It's not hard, so long as you kept the proper tile numbers when exporting, converting and upscaling.

put all pngs in one dir
command prompt. dir>dir.txt
trim garbage keep the file names
notepad++ column editor
texture #### { file "####.png" }
see https://wiki.eduke32...anguage#texture

Anyways post or pm me the link :)

This post has been edited by Photonic: 01 February 2019 - 03:21 PM

0

User is offline   MF_Kitten 

  • 3

#3

Fantastic! I'll PM you a .zip once I've gone through all of it :)
1

User is offline   MF_Kitten 

  • 3

#4

Oh, and should I convert it back to .BMP?
0

User is offline   Photonic 

  • 1,268

#5

png is good.
0

User is online   Trooper Dan 

  • Duke Plus Developer
  • 3,758

#6

For those doing this, I'm curious as to how you are handling palette swaps. Palette swaps are important and they won't work unless the images are (1) in Duke 3D palette, and (2) added to the game via the tilefromtexture command or directly placed in .art files. But since the upscaled tiles are much larger, this will impact the game with giant enemies etc., unless there is also code added to scale things appropriately. Or am I missing something? Does palette emulation make this unnecessary?
0

User is online   Phredreeke 

  • 391

#7

for Blood I did separate upscales for each palette swap. that wasn't too bad for enemies, but ended up really bloated for the background pack
0

User is online   Trooper Dan 

  • Duke Plus Developer
  • 3,758

#8

View PostPhredreeke, on 01 February 2019 - 05:14 PM, said:

for Blood I did separate upscales for each palette swap. that wasn't too bad for enemies, but ended up really bloated for the background pack


How many palette swaps does Blood have? In Duke 3D we have about 15 (technically we have more, but some of them aren't used). Then for projects such as AMC TC and Alien Armageddon which use Lezing's extended palette, we use way more than that.
0

User is online   Phredreeke 

  • 391

#9

Blood has 14 palswaps. The first is a bit special as it uses the default palette but fades towards white instead of black.

However for my enemy upscales I only did the palswaps used in the base game (4 for cultists, 2 for spiders, 1 for the seed pods and vines from Plasma Pak, for the stone gargoyle I simply took the already upscaled one and made it greyscale)
0

User is online   Trooper Dan 

  • Duke Plus Developer
  • 3,758

#10

Well for my projects that use a lot of palette swaps, I would definitely want the upscaled images to be in the correct palette already. Otherwise I will either lose a lot of pal swaps, or there will be a ton of bloat and wasted memory. Unless, like I said, palette emulation makes this all moot.
0

User is offline   MrFlibble 

  • 682

#11

Could you please show us your results, so that they could be compared to other neural network methods?

Recently I did some tests with ESRGAN here and here.
0

User is offline   Photonic 

  • 1,268

#12

View PostTrooper Dan, on 01 February 2019 - 04:54 PM, said:

For those doing this, I'm curious as to how you are handling palette swaps. Palette swaps are important and they won't work unless the images are (1) in Duke 3D palette, and (2) added to the game via the tilefromtexture command or directly placed in .art files. But since the upscaled tiles are much larger, this will impact the game with giant enemies etc., unless there is also code added to scale things appropriately. Or am I missing something? Does palette emulation make this unnecessary?

If it's needed (I haven't looked into the new palette emulation for polymost yet) then I will probably generate palswaps by hijacking the polymer highpalswap feature like this and load them each in the def file.

edit:
Palette emulation is a bit of a misleading term, it's about the distance shade tables from tables.dat not the lookup.dat palette swaps.

This post has been edited by Photonic: 02 February 2019 - 08:58 AM

0

User is offline   MF_Kitten 

  • 3

#13

I'll be honest, I don't understand this stuff well enough. I did enable the option to keep the colour palette/profile, so all sprites and textures will have the vanilla palette.

I will upload uncompressed images later. In the meantime, here's a thread I posted with a few examples: https://www.reddit.c...=reddit-android

I noticed after that the sharpening ruined the black background. I don't THINK the upscaling does that. If it does, it ought to be fairly easy to work around.
1

User is online   Phredreeke 

  • 391

#14

When doing side-by-side comparisons, use nearest neighbour scaling for the original. Also if it's ESRGAN with the manga model then yes, the upscaling does do that to black areas. Waifu2x does preserve them though
1

User is offline   MF_Kitten 

  • 3

#15

This is Topaz Gigapixel.

It's easy to just cut a mask from the original and put that onto the upscale, or if it's not identical to anything found in the actual sprite, just "clip" everything below a certain threshold to hard black.

It is possible to change the backgrounds into a color not used in the sprites, upscale, and convert it back to black
0

User is offline   MrFlibble 

  • 682

#16

For comparison, I grabbed a Cycloid Emperor sprite from the Spriters Resource and ran ESRGAN and SFTGAN tests, with results available here:
https://imgur.com/a/TeoH3L0

The source image in all cases was softened with xBRZ before feeding it to ESRGAN and waifu2x. The waifu2x result was used for SFTGAN processing.

The results are in the following order:
  • ESRGAN upscale with Manga109Attempt model
  • ESRGAN upscale w/ an interpolation of RRDB_ESRGAN_x4 and Manga109Attempt models (alpha = 0.5)
  • ESRGAN upscale w/ an interpolation of Manga109Attempt and RRDB_PSNR_x4 models (alpha = 0.5)
  • SFTGAN processing on an image scaled up with waifu2x (RGB model, TTA4)
  • blend of ESRGAN/Manga109 and SFTGAN
  • blend of Manga109/PSNR and SFTGAN

Blending was accomplished by using the Layers -> Blend [median] tool in G'MIC plugin for GIMP.
2

User is offline   MF_Kitten 

  • 3

#17

That's really good! It has a more cartoony look though, which isn't what I wanted from this personally. I feel like Gigapixel is trying to make it more photorealistic, and you start seeing that the creatures are made from clay.

I don't know how to streamline a way to fix the impure black backgrounds that results from the upscale. I could do it if I just tweak each sprite by hand. But that will be a lot of work!
0

User is online   Phredreeke 

  • 391

#18

I'd suggest making a separate mask for that and upscale that separately.

Hmmm... this gives me an idea, gotta look into it tomorrow
0

User is offline   MrFlibble 

  • 682

#19

View PostPhredreeke, on 03 February 2019 - 05:38 PM, said:

I'd suggest making a separate mask for that and upscale that separately.

That might not work with Gigapixel. It is my understanding that it uses some semantic networks to "hallucinate" additional detail, and pure masks have no data for such networks to work with.

View PostMF_Kitten, on 03 February 2019 - 12:36 PM, said:

I feel like Gigapixel is trying to make it more photorealistic, and you start seeing that the creatures are made from clay.

Duke3D sprites are not made from clay models, they're from high-res 3D renders. (sorry if I read your post wrong and you didn't imply anything about the original models)
0

User is offline   MF_Kitten 

  • 3

#20

That's pretty funny, it loojs like claymation!

I tried filling in the black background with purple. Gigapixel now shits itself and just resizes all the visible pixels. The model it uses just has no clue what to do with that.
0

User is offline   MF_Kitten 

  • 3

#21

OK, I've been reading up on the now-famous DooM upscale mod. They found some good methods to get around the issue of masks. However, I realized while messing with Half-Life textures, that Gigapixel actually does pretty well with transparency. So I'll be trying the simple method of cutting out the black background and saving it like that, and upscaling that result. If it goes well, I think we have a solution.
0

User is offline   MF_Kitten 

  • 3

#22

View PostMrFlibble, on 03 February 2019 - 08:10 AM, said:

For comparison, I grabbed a Cycloid Emperor sprite from the Spriters Resource and ran ESRGAN and SFTGAN tests, with results available here:
https://imgur.com/a/TeoH3L0

What's the idea behind blurring the source image first? I assume you resize first and then blur, right? So you kinda give it more space to interpret visual data, while also eliminating all the hard edges between pixels? Couldn't you just do a bicubic upscale to get a similar softening effect first? Also, ehat are you doing with the different background colors?

The source image in all cases was softened with xBRZ before feeding it to ESRGAN and waifu2x. The waifu2x result was used for SFTGAN processing.

The results are in the following order:
  • ESRGAN upscale with Manga109Attempt model
  • ESRGAN upscale w/ an interpolation of RRDB_ESRGAN_x4 and Manga109Attempt models (alpha = 0.5)
  • ESRGAN upscale w/ an interpolation of Manga109Attempt and RRDB_PSNR_x4 models (alpha = 0.5)
  • SFTGAN processing on an image scaled up with waifu2x (RGB model, TTA4)
  • blend of ESRGAN/Manga109 and SFTGAN
  • blend of Manga109/PSNR and SFTGAN

Blending was accomplished by using the Layers -> Blend [median] tool in G'MIC plugin for GIMP.

0

User is offline   Tea Monster 

  • Polymancer
  • 2,149

#23

Anyone tried this?

It will do 600%, which is probably overkill, but should get nice results. It dosen't seem to have the graphical artifacts of Waifu2x.

EDIT: I see someone mentioned Gigapixel above.

This post has been edited by Tea Monster: 17 February 2019 - 04:57 AM

0

User is online   Phredreeke 

  • 391

#24

For the last two weeks I've been working on upscaling DN3D sprites, and at this point I feel happy enough with them to present the results. (I hope no one feels I'm stepping on their toes here, I just didn't think the forum needs yet another topic on upscaling sprites)

Posted Image
7

User is offline   MusicallyInspired 

  • 3,390

#25

Quite nice!

Roland SC-55 Music Packs
Latest release: Heretic SC-55 Music Pack (12/12/18)
*new* Buy the Mage's Initiation Original Soundtrack on Bandcamp by me!
Buy the P&C Adventure/RPG game Mage's Initiation on Steam, GOG, and Humble
1

User is offline   Tea Monster 

  • Polymancer
  • 2,149

#26

Have you got a 'pipeline' down for converting images yet?

If so, do any of these apps have macro facility like Photoshop does? You can create a list of actions to perform on an image, and save that list of actions as a macro. Then you point PS at a folder with however many images you want to convert and tell it to run that macro on the folder. I used to work at a newspaper and it was great to process car dealership photos. You'd literally blow though hundreds of them in 20 minutes or so.

I was just thinking as if you have a method that worked, it would be cool to put a pack together.
0

User is online   Phredreeke 

  • 391

#27

Yes. ESRGAN explicitly works by putting the images to be upscaled in a folder called "LR", and the resulting files are saved in a folder called "results"

Waifu2X is also able to convert whole folders
0

User is offline   TerminX 

  • el fundador
  • 5,570

  #28

Dude, that looks amazing.

EDuke32 wiki svn builds bugs
Join us in #eduke32 on irc.freenode.net!
1

User is offline   Jimmy 100MPH 

  • Trooper
  • 4,429

#29

These are pretty sick, but I would highly recommend cleaning up the original tiles before upscaling them. Upscaling worked so well for Doom because the enemy art is so neat (95% of it anyway) but Duke's art is ridiculously messy and sometimes this causes the AI to misinterpret odd pixels or what have you.

Coke costs a lot of money you know. - oasiz
0

User is online   Phredreeke 

  • 391

#30

I do, albeit in an automated fashion.

Here's a frame of the octobrain upscaled, first without applying my AA process, then with it, both with no further processing. The second one is blurry, but looks good downscaled.

I'd say Redneck Rampage Rides Again is far worse in that regard, with many of the new enemies having seemingly random pixels on the edges.

Attached thumbnail(s)

  • Attached Image: upscale-noaa-aa.png

1

Share this topic:


  • 2 Pages +
  • 1
  • 2
  • You cannot start a new topic
  • You cannot reply to this topic


All copyrights and trademarks are property of their respective owners. Instead of reading this text, you could be playing Ion Maiden! ;) © 2018 Voidpoint, LLC

Enter your sign in name and password


Sign in options