[Doom64hunter]

Introducing: The EDuke32 Addon Manager

Download links for the test release are available below.



What is this about?

Over the past few months, we have been busy implementing a new feature for eduke32, which will hopefully find its way into the main builds of eduke32 soon.
This feature is an integrated addon manager, which will allow users to select and load custom content in a clean and intuitive manner.

What's the point of this feature?

The goal of this project was to define a specification to easily package and distribute addons without interfering with or locking out compatibility with older versions of the engine, or even DOS Duke3D.
It is intended to overcome limitations, such as the current lack of important metadata to determine any compatibility issues or dependencies, while also offering basic information to the end user that can be presented in-game.
Moreover, the specification allows loading of CON and DEF script files modularly, a feature that was previously restricted to command-line parameters only.

This means that with this addon manager, it will become possible to load mods cumulatively. For instance, with only minor changes, it will become possible to have Dzierzan's upscaled sounds and voxel packages active at the same time.
Gone are the days of having to dump everything in the base folder or of being required to use the autoload feature. This manager will provide a ways to separate data cleanly, and activate them all from inside the game.

But the feature doesn't end there. Assume you're the developer of a large Total Conversion like WGRealms 2, Alien Armageddon, or a gameplay mod like Brutal Duke, or Ion Fury: Anger Management, and you provide new features for people to create maps with.
A previous limitation was that people could not easily distribute maps that depend on these TCs and package them in a way to load them together cleanly. With this addon manager, such issues are eliminated, and mods will be able to build on top of each other.

(Also note that ART files can now be stacked cumulatively as well -- thanks to a recent commit by Striker, blank spaces in ART files will no longer override existing tiles.)

Addon Manager Test Build -- Downloads & Documentation

To provide you with the ins-and-outs on how to use the feature, we have prepared a separate documentation, which you find in the link above.

There you will also find downloads to the current test builds of the addon manager, as well as the source code. It also includes some sample addons that you can use for testing.

We are currently looking for testers to try and break the addon manager, report bugs, and provide feedback on the usability, or on any other issues you can find.
Whether you are a content creator or a user, we encourage you to package your mods to be used with this feature, and to test it in any way you can think of.
Try to load mods in any order, package any type of mod to use with the manager, combine mods in any way possible, etc, and discuss the results in this thread or on Discord.

Credits

• Doom64hunter - Design and Implementation, Documentation
  
• Oasiz - Hosting, Documentation, Feedback, Early Testing, General Support
  
• TerminX - Advice and Feedback, General Support
  
• Striker - Advice and Feedback, General Support
  
• dwtietz - Early Testing, Feedback
  
• Mblackwell - Feedback
  
• The respective authors of the sample addons, credited within each package.
  
• And of course, everyone else who will test the system and provide feedback.

[Jblade]

Excellent work!

[oasiz]

I have waited so long to have something like this in the game. Thanks for doing the hard work!

[DavoX]

Great Work! Ive been a spectator while this whole thing developed and I'm excited to see how the community uses it.

[NightFright]

Have you guys already found a solution for grpinfo requiring separate definitions for Atomic and World Tour? Currently I have to clone the Atomic grpinfo and replace dependencies with WT CRCs. It would be better if EDuke32 was able to load the addons with any of the two, without requiring two separate files.

[Doom64hunter]

NightFright, on 02 May 2022 - 02:52 AM, said:

Have you guys already found a solution for grpinfo requiring separate definitions for Atomic and World Tour? Currently I have to clone the Atomic grpinfo and replace dependencies with WT CRCs. It would be better if EDuke32 was able to load the addons with any of the two, without requiring two separate files.

Not yet implemented, but it should be doable. I wanted to add some new tokens to grpinfo anyways such that more information can be displayed in the addons menu.

[NightFright]

Yeah, basically what you currently have in the addons panel under "Author(s)" and "Description" (probably even "Version") could be in grpinfo, so it doesn't have to be read from any external file.

[Radar]

How do I define an addon that is meant to be a simple asset replacement? For example if I just want to replace a few VOC files, thus no need to define a path for a CON or DEF file?

[NightFright]

If replacement files have the same name as the originals, I would assume it works directly. How it works if several addons do that I can't imagine.

[oasiz]

I'm not 100% sure I remember but it should be enough to have the .json file there.
The "Nuclear winter" example has some in the flat dir.
Check the examples I guess?
You might be limited to loading it as a "TC".

[Doom64hunter]

NightFright, on 15 May 2022 - 02:25 AM, said:

If replacement files have the same name as the originals, I would assume it works directly. How it works if several addons do that I can't imagine.

Correct, if an addon is selected it will treat the contents of the folder/zip as if they were in the base folder. So you can just give sound and ART files the same names that you would usually give to replace files in Duke3D.

You could theoretically also replace GAME.CON and duke3d.def like this, but I recommend using the CON/DEF keys in the JSON instead to make it more clear.
Moreover, using the CON/DEF keys with the 'module' type is the only way to append CON or DEF code in this system without outright replacing the game's main scripts.

If two addons replace the same files, then the addon that's last in the load order will be the one that takes effect for those files.

[Jamesfff]

I gave this a test run this afternoon and had an issue with savegames. It seems that games saved while playing a mod are saved in the /useraddons directory, not the specific mod's directory, preventing the mod from recognizing them. They can still be loaded without loading the mod, but then it's impossible to progress through the mod's levels - you're sent to the next level of regular Duke3D To be able to play the mod properly (progress from one level to the next), the savegame must be moved to the mod's subfolder (and that mod must be loaded from the addon manager, of course). Savegames for a specific mod should be saved in that mod's subfolder so they'll be recognized when the mod is loaded.

[Doom64hunter]

Jamesfff, on 17 May 2022 - 05:14 PM, said:

I gave this a test run this afternoon and had an issue with savegames. It seems that games saved while playing a mod are saved in the /useraddons directory, not the specific mod's directory, preventing the mod from recognizing them. They can still be loaded without loading the mod, but then it's impossible to progress through the mod's levels - you're sent to the next level of regular Duke3D To be able to play the mod properly (progress from one level to the next), the savegame must be moved to the mod's subfolder (and that mod must be loaded from the addon manager, of course). Savegames for a specific mod should be saved in that mod's subfolder so they'll be recognized when the mod is loaded.

Savegames aren't stored in the /useraddons directory on my end, even having loaded an addon. Did you mean the game root, i.e. the same directory that contains the binary, or are they actually landing in the 'useraddons' subdir?

If the latter is happening, then it is unintentional and a bug. Can you provide details on what Operating System you are running the game on, whether you are using user profiles (appdata folder on Windows, .config on Linux) and what kind of addon you are running?

As for your suggestion, the package folders are intended to store game data, and hence should not contain user config data or savegames. The saves should remain in the root folder, or go in a dedicated directory. However, I do think that it should not be possible to select the save if the corresponding addons are not active. It should ideally show which addons are required, maybe even automatically activate them -- but this is currently not easy to accomplish due to restrictions on what can currently be done with the savegame header.

[Jamesfff]

They WERE actually landing in the /useraddons directory - because I had that directory selected in the "custom game content" window, it turns out (I'd had some issues getting the manager to work, but managed to iron them out); without that selected, saves are made to the main directory. However, they're still not recognized there, but only in the mod's directory. So, if that's how you want it, something needs to be done to make the addon recognize them there; otherwise it is, as I said in my last post, impossible to progress through the mod's levels from a savegame.

BTW I use Windows 10.0.19044.1706. I don't have user profiles for Duke3d and discovered this issue while playing the Steve Long Episodes.

[Doom64hunter]

Jamesfff, on 20 May 2022 - 05:53 PM, said:

They WERE actually landing in the /useraddons directory - because I had that directory selected in the "custom game content" window, it turns out (I'd had some issues getting the manager to work, but managed to iron them out); without that selected, saves are made to the main directory. However, they're still not recognized there, but only in the mod's directory. So, if that's how you want it, something needs to be done to make the addon recognize them there; otherwise it is, as I said in my last post, impossible to progress through the mod's levels from a savegame.

BTW I use Windows 10.0.19044.1706. I don't have user profiles for Duke3d and discovered this issue while playing the Steve Long Episodes.

Hmm yeah the custom content directory option is an old holdover and isn't actually related to the addon manager, but I see where the confusion could come from.
It is not necessary to change that from the default option, i.e. from root.

We'll try to figure out a way to have saves with a specific addon setup, using the json header format (previously only used for Ion Fury), so that the config may be reloaded automatically.

[NightFright]

Can this system handle multiple con/def files?
Asking because I've been trying to get Billy Boy's maps compiled into one package for quite a while. However, a couple of maps come with their own con files. While user.con stuff often can be merged, in the worst case by editing the map to bypass sound conflicts, different game.con files usually inevitably collide without any hope of having them coexist. So it would have to be possible to load a specific con file with one map, but not with another - not within an episode, of course, but from the user maps menu.

[Doom64hunter]

NightFright, on 01 June 2022 - 05:41 AM, said:

Can this system handle multiple con/def files?
Asking because I've been trying to get Billy Boy's maps compiled into one package for quite a while. However, a couple of maps come with their own con files. While user.con stuff often can be merged, in the worst case by editing the map to bypass sound conflicts, different game.con files usually inevitably collide without any hope of having them coexist. So it would have to be possible to load a specific con file with one map, but not with another - not within an episode, of course, but from the user maps menu.

You can load multiple CON and DEF files as modules, but it requires a soft-reboot for each map you load.
Per-map CON scripts that change depending on the map loaded are not supported.

[NightFright]

Alright, so if it's just about "Play this one map with these cons and defs, then return to menu, then play another map with other cons/defs" should be possible in theory.

[NightFright]

Alright, so if it's just about "Play this one map with these cons and defs, then return to menu, then play another map with other cons/defs" should be possible in theory.

[dethtoll]

After playing around with this for a while and getting repeated syntax error messages, I think it would be really worthwhile to have an explanation of what error messages mean. It took me ages before I realized I was putting a comma where there shouldn't be one.

I do appreciate how simple the manager actually is, though.

[Doom64hunter]

dethtoll, on 10 February 2023 - 03:14 AM, said:

After playing around with this for a while and getting repeated syntax error messages, I think it would be really worthwhile to have an explanation of what error messages mean. It took me ages before I realized I was putting a comma where there shouldn't be one.

I do appreciate how simple the manager actually is, though.

Noted.

Work on this feature will resume soon. I have returned from the dead.

[Nacho]

I just wanted to say that this feature is something I think could really help modernize e-duke. I've been clamoring for something like this to make Duke content more accessible to modern audiences, the more plug'n'play things get, the better. Overly ambitious me would love for this to get further updated (even if much later) to connect to a "store" of sorts to allow for easy picking and downloading of curated content.

I'm very impressed with the work this small, but passionate, community still puts out and I just would love to see it grow again.

[Striker]

It's still being worked on. I recently helped Doom64Hunter fix a memory leak, and if I recall, he's been testing stuff lately. Hopefully will be ready soon.