[icecoldduke]

 Mblackwell, on 03 June 2016 - 11:00 AM, said:

No.

Not only is that likely impossible unless they were completely re-rigged and animated but it would be unfeasible. It would be much more feasible to just add vertex animation, and not offer bad suggestions like ONE FRAME OF ANIMATION PER TILE!

Spoiler

Seriously that's pretty bad.

The only support I'm going to offer for the first phase of PolymerNG for vertex animation is all frames on the video card at once. This would mean if you have a bunch of monsters in your maps you will take up at most 2gb of vram data of just mesh data. I am not uploading model data per frame. So if you want phase 1 to require a 3 - 4 gb video card(which I have) then that's fine, if not we have to go the skeletal animation route.

[Mblackwell]

 icecoldduke, on 03 June 2016 - 11:04 AM, said:

So if you want phase 1 to require a 3 - 4 gb video card(which I have) then that's fine

So first of all, that's a consequence of a design decision for a feature that no one is using at the moment or necessarily plans to use. (and even when in use the speed difference may not be drastic assuming it's done in tandem)
Second, the above statement is false because there is no option for that since you're disabling it.

[icecoldduke]

 Mblackwell, on 03 June 2016 - 11:17 AM, said:

So first of all, that's a consequence of a design decision for a feature that no one is using at the moment or necessarily plans to use. (and even when in use the speed difference may not be drastic assuming it's done in tandem)
Second, the above statement is false because there is no option for that since you're disabling it.

I won't disable it, if everyone is ok with me requiring a 3-4gb video card. That is a design decision I'm sticking with.

We have gone back and forth on this before. So right now the new questions are:

• Which project do we want to be the phase 1 project.
  
• Do we want skeletal animation or vertex animation for everything.

[Mblackwell]

Phase 1: Render everything in D3D correctly.
Phase 2: Render everything in the Polymost HRP correctly
Phase 3: Render everything in the Polymer HRP correctly
Phase 4: Create and render additional effects and shaders.

You're not even done with Phase 1 and you want to skip to Phase 4. Sounds to me like you made the classic mistake of getting distracted by shiny things.

[Kyanos]

 icecoldduke, on 03 June 2016 - 11:20 AM, said:

I won't disable it, if everyone is ok with me requiring a 3-4gb video card. That is a design decision I'm sticking with.

We have gone back and forth on this before. So right now the new questions are:

• Which project do we want to be the phase 1 project.
  
• Do we want skeletal animation or vertex animation for everything.

Do you have skeletal animations yet? How long? jk.

Make it run the HRP for fucks sake, then work forwards like a normal person would.

[Danukem]

 Mblackwell, on 03 June 2016 - 10:56 AM, said:

The original levels will have a ton of issues since the game uses textures in really odd ways since they were pixelated. The same material in two places in the same map might be two completely different things, and the way that things tile is a bit arbitrary depending on the circumstance. That's why some of the HRP (particularly polymer) content can look a bit weird.

Yep, there's going to be all kinds of problems if they go the map-hack route and the results will be disappointing. It's not like this is our first rodeo -- that's why Duke Nukem Eternity was created in the first place. DanM's first level featured a movie theater...coincidence? No, that was the whole point.

Just to drive the point home, let's talk about E1L1. All the furniture (the couch in the bathroom, and movie theater seats, the secret apartment furnishings) are all made of very simple level geometry. Furniture obviously needs to be models, and the sector-based stuff will look like shit if the level is presented as "next gen". Slapping new textures on them with gloss/detail maps etc. is just polishing turds. And as you pointed out, those 8-bit textures are multi-use. If you replace the simple brown texture that covers a couch with one that actually looks like a couch texture, then there are levels were that same tile number is used as dirt, and then the ground will look like a couch cushion. The answer is to make new levels. Again, this has long been known by anyone who has seriously thought about it and worked on it before.

[Kyanos]

Seriously, it sounds as if a simple bit of reworking the MD3s and defs so that you don't overlap and load the same thing many times and then the HRP as is (minus the HUGE newer models) would run fine.

[icecoldduke]

 Drek, on 03 June 2016 - 11:25 AM, said:

Do you have skeletal animations yet? How long? jk.

Make it run the HRP for fucks sake, then work forwards like a normal person would.

I'm designing PolymerNG to be fast, this is how a normal person would approach the problem . For now I will just load everything into vram, and we can replace vertex animation with skeletal animation as content guys have time.

[Mblackwell]

Btw, you keep saying "content guys" as if we're a company with a dedicated team of artists on staff. Just so you know there are no such guys. The closest thing would have been the original volunteers working on the HRP, but most of them have industry jobs now and have moved on.

[Danukem]

 icecoldduke, on 03 June 2016 - 11:04 AM, said:

The only support I'm going to offer for the first phase of PolymerNG for vertex animation is all frames on the video card at once. This would mean if you have a bunch of monsters in your maps you will take up at most 2gb of vram data of just mesh data. I am not uploading model data per frame. So if you want phase 1 to require a 3 - 4 gb video card(which I have) then that's fine, if not we have to go the skeletal animation route.

That's fine with me. In my current project I only use a couple of enemies that have vertex animation (but I really need them), so that shouldn't be an issue; I seriously doubt it will take even close to 3-4 gb of video memory.

EDIT: Also, I'd like to add, providing even limited support for existing vertex animated models is very useful because it allows us to test things. If phase 1 PolymerNG didn't have that support, then those of us who don't make models would have literally nothing we could use for models. Even if we are limited by video memory to using very small numbers of models, that would still allow us to test and develop stuff on small scales, then we can go to full-scale when you add full support at a later phase, or if we are lucky enough to find modelers who will make new content.

[TerminX]

Damn it Justin, just make it upload the data every frame like Polymer for now, so everything actually works and is testable in the meantime by people without 4 GB cards. The trick is to only upload vertex models per frame if the vertices have actually changed.

This way, skeletal stuff will still use the fast path you have planned, while static, un-animated md3 props will only upload once and sit in VRAM otherwise (like you wanted), and if uploading the animated vertices per frame for other stuff uses too much bandwidth between system RAM and VRAM later on, smack the content guys because they'll have ways to not do that that they should be using instead.

There is a way to make everyone happy here and I don't see why you aren't just committing to it.

[Hendricks266]

Has anyone explained how 60 MB of MD3s takes up 2000 MB of memory?

[MusicallyInspired]

This I guess?

 icecoldduke, on 02 June 2016 - 03:29 PM, said:

The model format isn't the issue. It's vertexdata x numvertexes x numframes = huge amount of data. Initially I wasn't going to ditch vertex animation, but your assets are too high poly to support vertex animation. Polymer and Polymost upload frame updates on the fly, which taxes the bus on your motherboard. This conflicts with me implementing virtual texturing for example, since I need to transfer data from main ram to vram, which uses the bus.

There is no magic Polymost fix for this. I don't want model data going from main ram -> vram, per animated asset in view, per frame change. So the only option I have is to leave each frame resident in vram, which means you will need a 3gb+ card to run PolymerNG with all the current vertex animated objects.

Your guys's stuff is too high poly.

 icecoldduke, on 02 June 2016 - 03:55 PM, said:

Vertex count for all models in the HRP.
Vertex counts for HRP models.

My favorite so far:
Fan sprite.

D:\BlackenmaceStudios\DukeNukem\Tools\dukenukem\highres\
sprites\props\0407_fansprite.md3: VertexCountPerFrame 9036 IndexCount 9036 NumFrames 16 Total Vertexes 144576

Stripper

D:\BlackenmaceStudios\DukeNukem\Tools\dukenukem\highres\
sprites\characters\1312_stripper.md3: VertexCountPerFrame 4080 IndexCount 4080 NumFrames 91 Total Vertexes 371280

Why does the fan have 9036 vertexes when the stripper only has 4080?

Quote

The MD3 format is not the issue its (Vertex * NumVertexs) * numFrames = too_much_data. I don't want to move all that data across the bus every frame, I want to leave the bus free for virtual texturing. So the only other options is to leave each frame in memory.

[Kyanos]

 Hendricks266, on 03 June 2016 - 12:42 PM, said:

Has anyone explained how 60 MB of MD3s takes up 2000 MB of memory?

I think he is loading the entire MD3 for every tile that uses it. So a simple 5 tile animation, that has 10 frames in MD3 loads itself 5 times for 50 frames.

take aplayer-tile 1405 for example.

the def calls for this

      // Idle (1405-1409)
      anim { frame0 "frame5" frame1 "frame10" fps 3 flags 0 }
      frame { name "frame5" tile0 1405 tile1 1409 smoothduration 0.3 }

but he loads the entire MD3 all 134 frames

then next this

      // Jetpack (1420-1424)
      anim { frame0 "frame15" frame1 "frame25" fps 5 flags 0 }
      frame { name "frame15" tile0 1420 tile1 1424 smoothduration 0.3 }

we ask for ten frames, but he loads 134 again?

I'm just speculating by what he said earlier about needing one MD3 per tile to do animations.

 icecoldduke, on 03 June 2016 - 10:03 AM, said:

 Drek, on 03 June 2016 - 09:55 AM, said:

I plan on defining 1 frame to 1 tile, from a MD3 with many frames (exactly how they sit in the HRP) then defining the next tile to a different frame in the same MD3.

Will that work? Does the engine only load the single frame I def in? Or will it load the entire model each time? You don't need a single model with a single frame per tile do you?

For this too work its one model, single frame, per tile. What might work instead is having a seperate text file that has each model that you want to be vertex animated. Send me over that file, I'll add functionality to the model cache build system, and we'll see how much vram this will take up.

e.g.

highres/blah/blah.md3
highres/blah/blah2.md3

and so on.

[icecoldduke]

 Hendricks266, on 03 June 2016 - 12:42 PM, said:

Has anyone explained how 60 MB of MD3s takes up 2000 MB of memory?

//
// Build3DVertex
//
struct Build3DVertex
{
	Build3DVertex()
	{
		memset(this, 0, sizeof(Build3DVertex));
	}

	Build3DVector3 position;
	Build3DVector2 uv;
	Build3DVector3 tangent;
	Build3DVector3 binormal;
	Build3DVector3 normal;
};

I can probably get rid of the binormal and calculate that in the shader, I have heard of being able to calculate tangent in the shader as well. Anyway thats how 60mb turns into 2gb, also doesn't the md3 format store vertexes as 16 bit values?

[Hendricks266]

There are commented lines in mdsprite.c involving "memoryusage" that print a tally to the log. I loaded the HRP and the grand total was 94,703,504 bytes. 94 MB. That's a factor of 20 less than your estimate. I don't see anything in that struct that would blow up that badly. Something must be wrong. And when you find out what it is, don't blank all your posts again.

[Kyanos]

 icecoldduke, on 03 June 2016 - 12:53 PM, said:

Anyway thats how 60mb turns into 2gb,

Just by cutting back the bosses and duke to polymost version for now that 2GB will plummit, at max the rest of the HRP will be around a gig. (if nothing else changes)

 icecoldduke, on 03 June 2016 - 12:53 PM, said:

also doesn't the md3 format store vertexes as 16 bit values?

Yes it's 16bit.

[Steveeeie]

 Drek, on 03 June 2016 - 01:26 PM, said:

Just by cutting back the bosses and duke to polymost version for now that 2GB will plummit, at max the rest of the HRP will be around a gig. (if nothing else changes)

Yes it's 16bit.

So we need to lose better assets to support the new renderer?

[Steveeeie]

 Drek, on 03 June 2016 - 01:26 PM, said:

Just by cutting back the bosses and duke to polymost version for now that 2GB will plummit, at max the rest of the HRP will be around a gig. (if nothing else changes)

Yes it's 16bit.

I think a gig is crazy, there are really next to no models in the HRP when compared to any game in the past 10 years and hardly any models being loaded in any given base game level.

This all an issue because there is no proper animation system like other modern engines.

[Kyanos]

 Steveeeie, on 03 June 2016 - 01:28 PM, said:

So we need to lose better assets to support the new renderer?

Quote

Something must be wrong.

It really should work just fine on a xbox1. The more I think about it, I'm almost positive now he was loading the same model over and over for each tile that used it.

[icecoldduke]

 Drek, on 03 June 2016 - 01:51 PM, said:

It really should work just fine on a xbox1. The more I think about it, I'm almost positive now he was loading the same model over and over for each tile that used it.

No I was not .

 Hendricks266, on 03 June 2016 - 01:18 PM, said:

There are commented lines in mdsprite.c involving "memoryusage" that print a tally to the log. I loaded the HRP and the grand total was 94,703,504 bytes. 94 MB. That's a factor of 20 less than your estimate. I don't see anything in that struct that would blow up that badly. Something must be wrong. And when you find out what it is, don't blank all your posts again.

I can't speak for how your counter works, but remember 16bit position in the model format, and 32bit position on the GPU so thats 2x more right there + 3 more 3 vector 32 bit channels, things add up fast. My stuff is measured with whats on the GPU if we upload all the frames to the GPU, and at that point those frames are vram ready, so its going to take up more space then whats on disk.

[Mblackwell]

The vertex count is the same even if you are using bones.

Each frame just throws data up showing where every vertex is. There's no need to save the entire mesh into memory multiple times, instead offload the animation data into your own deltas.

Edit:
http://icculus.org/~.../md3format.html

[icecoldduke]

 Mblackwell, on 03 June 2016 - 02:31 PM, said:

The vertex count is the same even if you are using bones.

Each frame just throws data up showing where every vertex is. There's no need to save the entire mesh into memory multiple times, instead offload the animation data into your own deltas.

Edit:
http://icculus.org/~.../md3format.html

I would either have to upload the data every frame, or store compressed data on the GPU and use a compute program to populate the vertex buffers. I don't want to do either if I can help it.

[Tea Monster]

For a lot of the models with just two frames (the trash can) we've used two models as it's only two frames. That's a lot of the stuff right there.

A lot of the animated models can be done with armatures if required.

[Steveeeie]

 Tea Monster, on 03 June 2016 - 04:07 PM, said:

For a lot of the models with just two frames (the trash can) we've used two models as it's only two frames. That's a lot of the stuff right there.

A lot of the animated models can be done with armatures if required.

Skeletal meshes are for characters and trees, this will not cater for props.

[LeoD]

 Mblackwell, on 02 June 2016 - 08:52 PM, said:

I'm sure if there were people to (re)do it someone like LeoD would want to issue style guidelines.

I know what I want, what I like, and what I like less. But since I personally have no idea how to make models or advanced textures, I'm not exactly qualified for that.

 Steveeeie, on 03 June 2016 - 10:35 AM, said:

I know its the wrong topic but do we currently run the pngs through any kind of compression? because I bet we could kill a tone of weight.

Yes, every single PNG has been processed by PNGOUT. Although I skipped some results if I suspected them to mess up normal or specular maps. (Read: they did not convert to the same bit-identical TGA like the uncompressed version did.)

 Trooper Dan, on 03 June 2016 - 10:41 AM, said:

If I had access to the new renderer, I would be interested in fixing up and augmenting Duke Nukem Eternity,

Good Idea! DNE 2,0 beta is more like alpha. Overall brightness, xscale and yscale of textures are kind of messed up, IIRC. The new level is great though. I started some DNE2-NixFix last year, but got interrupted and did not continue. I might dig that up if you're interested.

[icecoldduke]

 LeoD, on 03 June 2016 - 06:11 PM, said:

Good Idea! DNE 2,0 beta is more like alpha. Overall brightness, xscale and yscale of textures are kind of messed up, IIRC. The new level is great though. I started some DNE2-NixFix last year, but got interrupted and did not continue. I might dig that up if you're interested.

So are we going with DNE or the original maps?

[Hendricks266]

 icecoldduke, on 04 June 2016 - 09:01 PM, said:

So are we going with DNE or the original maps?

Depends on whether you want two projects (Cataclysm, HRP) or three (DNE), unless I'm misunderstanding something by virtue of it not being public knowledge.

[Spiker]

I've checked DNE not so long ago just out of curiosity. It may have been amazing back then but now it's pretty outdated. Huge amount of work would be necessary to make it look NG. I would go for classic maps first.

[Danukem]

 icecoldduke, on 04 June 2016 - 09:01 PM, said:

So are we going with DNE or the original maps?

DNE was created mostly by Danny Mason, who designed the levels and gathered/edited most of the resources specific to that episode. It uses the DukePlus mod, which is my project but has contributions from a lot of people (very notably Muelsa who made the Dukebike and super lizard trooper). Danny M. made some questionable choices and got into a spat with TerminX in the process of promoting a re-release of his episode a couple of years ago, and as a result isn't a member here at this time. However, I think the levels he made are fine and don't need much adjustment. What would need work are various models, textures, graphical effects, model definitions and scripting. The scripting is my problem since it comes down to DukePlus features needing improvement (for example I noticed that the on-screen ammo display was overlapping part of the Dukebike HUD). With models and textures, yeah it would be good to fix those up, although most of them look fine and they are almost all static meshes without animation (e.g. lots of cars that just sit still). The only thing I can think of to do on the maps would be to adjust some of the lighting, but they already have Polymer lights in them and Danny M did a great job with it. If performance were less of an issue we would want fancier lighting, since I know he had to compromise on that. As far as I am concerned this process is going to happen one way or another eventually, although I have too many other things on my plate right now and would not be prepared to work on it yet.