[Danukem]

I thought the plan all along was to allow direct CON control of Polymer lights, but so far that hasn't happened. It is possible to have some control over lights indirectly by spawning and manipulating SE sprites, but this is hackish and somewhat limiting.

On a related topic, there has been a lot of discussion lately about the need to optimize/overhaul Polymer so that it runs better. I've been thinking about this, and it seems to me that huge gains could be made if it were possible to make lights or surfaces temporarily ignore any changes, thereby saving a lot of calculation. Obviously, a lot of calculation is required with every frame, to figure out the effect of every light on every rendered surface. But in many (most?) cases, nothing is actually changing from frame to frame, so all that calculation is being wasted and the lights might as well be baked on. For example, if there is a static point light in a hallway, and nothing is moving near it, then it will have the same effect on the scene from one frame to the next and it would be very beneficial if there was a way to save and apply what it did the last time, instead of doing the calculations again. I know next to nothing about renderers and about how Polymer works, so what I'm suggesting might not even make sense. But if there is some kind of update function that does the calculations, and if that function could be suspended for certain lights and/or surfaces, then that would be all we need to make some big optimizations. Obviously, some thought has to go into when to suspend updating, but I would be willing to try my hand at that on the CON end of things, if that's possible. For example, I could code a flag that mappers put on point lights which would make the game treat them more like baked on light (they would seldom update or only update under certain conditions).

I know I'm probably making an ass of myself and revealing my ignorance of how the renderer works, but I'm willing to do that because I care a lot about EDuke32 and I want to see Polymer shine so we can continue to make great projects with it.

[Stabs]

sounds good to me, so i would basically be flagging point lights that would never have to operate around any moving geometry for one of those one of these fixes to work? assuming it can be done of course

[Danukem]

DanM, on 31 May 2011 - 05:10 PM, said:

sounds good to me, so i would basically be flagging point lights that would never have to operate around any moving geometry for one of those one of these fixes to work? assuming it can be done of course

Yeah but it probably can't be done. Regardless, it would still be good to have true CON access to the lights.

[Plagman]

You're right that what you said doesn't relate to anything factual; fortunately what needs to be done isn't a mystery at all. The problem isn't that we need to brainstorm for optimization ideas, it's that I have to break everything and put it together in a different fashion. It's been known from the start; I've done it like that at first because it's always easier to get something that works right than something that works right and well. Too bad that only applies to the short term.

There are several different problems; one aspect a map is made of a lot of planes that are all drawn separately from front to back; materials are switched between each plane, and each time a red wall is crossed a "portal" is drawn to determine if the sector that follows needs to be queued for rendering or not. This constant material switching and drawing only one plane at a time isn't really efficient. The former causes a lot of state validation and thrashing, while the latter doesn't utilize the GPU in an efficient fashion. It's constantly submitting tiny pieces of work and reaping it, causing a lot of idle time waiting for the results. This also causes big maps with lots of red walls to be insanely complex to draw.

Another problem is that the multi-pass drawing approach uses a lot of fillrate as soon as a fairly big plane gets a few too many lights on it.

Things that needs to be done:
- Move the diffuse modulation from the material to the vertex attribute. That would cause the amount of different materials in a map to be all the different tiles used, instead of the more complex tile X shade.
- Move plane vertex attributes from several unique vertex buffers to a single storage or several depending on the staticness of the data.
- With a more reasonable material count and stuff sharing buffers, we can start batching planes together. To leverage that, throw planes into material buckets instead of rendering them right away. When we're done building index buffers for the buckets, submit them in one big go (one per material).
- Change the HSR walking to do a first pass before doing the actual shaded drawing. This means that in the event that occlusion queries have to be reaped, we can get the result a lot faster since we're pushing a lot less pixels using a passthrough fragment program.
- For rendering the shadow maps, we only need materials for sprite silhouettes. The rest can be batched in a single draw call after building the right index buffer, same as above. This will be a lot faster.

(Even) bigger projects:
- Defer the lighting model to shade from a fat buffer instead of inline. This eliminates two very expensive steps, light culling and multi-pass drawing.
- Instance models together.

[Danukem]

Thank you for your detailed reply.

Plagman, on 31 May 2011 - 09:39 PM, said:

This also causes big maps with lots of red walls to be insanely complex to draw.

That's a very important fact that mappers need to consider. EDuke32 allows up to 16384 walls, and mappers have gotten used to using most or all of them in every map. The original maps run pretty well in Polymer, even with maphacks, compared with newer user maps. If I were a mapper making levels for Polymer, I would avoid going over the old limits. Also, I'm guessing that having models of objects in the map will get better results than creating fake objects out of nested red sectors. EDuke32 has a map caching system, and it's pretty easy to use (I have used it successfully in a couple of unreleased projects), so there's no excuse for making huge levels -- maps can be patched together like in the HL series.

[Micky C]

The question is probably on a lot of peoples' minds, so I'll go ahead and ask it; with all due respect, is the optimization going to happen any time soon? Everyone understands that it's a lot of work, and no one has the expectation that you'll go out of your way more than you already have for the sake of polymer, but is the optimization a higher priority or is it lower down than other features that you plan to add to the engine, such as the showview command, mirrored surfaces, or other things I wouldn't know much about?

[Jblade]

It would probably go by quicker if there were more coders in the community; from what I can tell plagman's the only one on the team who actually deals with the rendering stuff whilst TX handles the game stuff and Helixhorned tackles the mapster side of things. Maybe with the added publicity from DNF we might get some more able coders who can contribute to the project and speed things up a bit. Right now from what plagman's said that sounds like a shitload of work even for someone who would be working on it 24/7, let alone people who have other more important comitments.

[Mark]

Because of all the countless hours I have enjoyed in this Eduke community over the last 2 years I am willing to help bribe / incentivise someone to get it done. I have contributed before and I'm will to join in with others to do so again to advance the Polymer system. A little off topic from the post title, but I thought it would mention it.

[Danukem]

I think it's pretty clear that Plagman is going to be one to do it, since he has created Polymer from scratch all by himself, and he has a plan for exactly what needs to be done.

[Hank]

DeeperThought, on 01 June 2011 - 03:53 PM, said:

I think it's pretty clear that Plagman is going to be one to do it, since he has created Polymer from scratch all by himself, and he has a plan for exactly what needs to be done.

Still, if it is a matter of writing it, perhaps he could define which sections can be written by itself without affecting others.

You have those split those roles in OOP languages. One writes the class, and all others can use it. That way Plagman could duplicate himself, without spending hours writing all this. In C he would issue a 'Struct' and others can develop with it until it works. - Just mouthing off

[Hendricks266]

Hank, on 01 June 2011 - 04:14 PM, said:

Still, if it is a matter of writing it, perhaps he could define which sections can be written by itself without affecting others.

You have those split those roles in OOP languages. One writes the class, and all others can use it. That way Plagman could duplicate himself, without spending hours writing all this. In C he would issue a 'Struct' and others can develop with it until it works. - Just mouthing off

I think that even if we did have more coders willing to help optimize Polymer, Plagman would still want to do it himself. It's like translating a game: from what I've read, it's best to have one person do all the translation, so that everything is consistent and has a clear vision. Plagman is the man for the job.