[NY00123]

This thread covers the topic of old Duke3D modifications from around 1996-2003 getting (re)packaged, for single-player and multiplayer altogether. I want to see if anyone has something to say here (or possibly elsewhere), but I suspect this might be not be far from another example of overthinking.

To explain, I've recently found myself trying older TCs in multiplayer games, as described in another thread: https://forums.duke4...tered-contents/

For technical reasons, whenever I use NetDuke32 and the NukemNet multiplayer launcher, I tend to not use the original ZIP file as-is; Rather, I repackage it as a new PK3 file that lacks any internal GRP file, while having the main CON file named GAME.CON. You can tell NukemNet to use another CON file, but since the CON file name isn't any information that NukemNet may trivially gather from the PK3 file itself, it's better to use GAME.CON in practice.

Naturally, it was suggested to me to make a compilation of the mods for easier use, with the PK3 files being one obvious option. I also know that similar packages were made for use with Raze. I don't know to what level does this hold to updated mods, like the ones in the EDuke32 Addon Compilation.

Question is, if there's a potential for using the original ZIP files from the ~1990s in any manner? Or at least, a subset of them? One reason is being sure you play an original version from that time period, without other modifications.

Classic Gaming Arena (CGA)'s approach is probably the closest to what I have in mind. This is a multiplayer launcher for DOS games, including Vanilla Duke3D, albeit you may currently also play solo after logging in with an account. CGA has a database of maps and mods to which registered users can upload. Currently, when you add a new map or mod, you need to upload a ZIP file, from which you can select specific files that CGA would know to appropriately use (say a MAP, CON or GRP file).

For certain mods, you don't need to make any change to the ZIP file from the ~1990s. Proper CGA-side configuration is all that's needed. In other cases (e.g., renamed .art files), you're expected to somewhat modify the contents first, even if just by the names of files. But otherwise, it should all work if properly defined.

What I thought about was something close, but not specific to the database of one service (e.g., CGA), and potentially also letting you download from other sources. Essentially, it's about having a standard for a textual "mod definition" file, identifying a specific original ZIP from the ~1990s by name, size and/or hash, and having machine-readable instructions for setting it up for gameplay. Such files would theoretically be usable for source ports, as well as external launchers (for Vanilla Duke3D and ports not supporting these).

In reality, I know the answer will probably be to simply use repackaged PK3 files as a standard (just like new or updated mods for NetDuke32), or alternatively, a service-specific database (like CGA's). But I'm still checking first before publishing any separate PK3 file.

[Radar]

One thing that would seem quite practical is a collection of zipped up mods for online multiplayer, sort of like how NukemNet already has the "NetDuke32 complete fun pack". Since multiplayer goes out of sync if people don't have identical files, this seems useful in that case. This could easily be distributed as a zipped folder, no need to complicate it beyond that. Other than that, for general use I don't think we need more ways to repackage and distribute old mods. It is not that hard to set up an old 90s mod, and there are multiple source ports to choose from if one doesn't work.

[NY00123]

I'll first add something not brought up earlier. Instead of ZIP/PK3, I can make new GRP files, for usability with source ports that don't support loading data from ZIP/PK3 files. CON files can be renamed by me to GAME.CON as usual. There are some caveats, though, and the 3rd of them might make this idea less relevant:

1. Vanilla Duke3D still expects to find the main CON file outside of any GRP. The same also holds to the last released version of Rancidmeat's duke3d_w32, i.e., b20.3.
2. Tested with versions 19.6 and 19.8, xDuke doesn't require an external CON file. It'll still complain if you ask for a different CON file name via the switch /x without having a matching file in the directory, but GAME.CON can be overridden from a repackaged GRP as described above.
3. The path to the GRP file as specified with the switch /g, including the file name itself, must not exceed 15 characters in length in duke3d_w32. This also holds to other ports derived from duke3d_w32, including xDuke. Using a too long path will lead to a buffer overflow, so chances are that you'll get a crash (if not other unexpected behaviors). The -game_dir switch cannot be used as a workaround, since the directory is also counted in the 15 characters.
4. Certain mods might lead to problems in other manners, if due to lack to testing or any other reason. One example is a crash stemming from any CON code bug, not manifested as a crash in Vanilla Duke3D.

Radar, on 22 November 2023 - 08:12 PM, said:

One thing that would seem quite practical is a collection of zipped up mods for online multiplayer, sort of like how NukemNet already has the "NetDuke32 complete fun pack". Since multiplayer goes out of sync if people don't have identical files, this seems useful in that case. This could easily be distributed as a zipped folder, no need to complicate it beyond that. Other than that, for general use I don't think we need more ways to repackage and distribute old mods. It is not that hard to set up an old 90s mod, and there are multiple source ports to choose from if one doesn't work.

Something like the "fun pack" makes sense if you want to start with various contents at once and you aren't yet familiar with setting up a program like NukemNet; Or even if you're familiar, but want to have a quick setup. There's a reason other packs exist.

Otherwise, at least for me, I usually prefer to setup source ports and user-made contents separately. NukemNet and CGA can both assist with the latter. They should also assist (at least partially) with the problem of games going out of sync, due to mismatches in user-made contents.

If we ignore for a moment the aforementioned buffer overflow problem, I can use longer names for the repackaged mods' files, including, e.g., the date of preparing them (in some time zone); Such a date can be used for identifying a revision of a file. NukemNet's file syncing feature, assuming no technical issue, should make sure people are using the same mod file in a game. This also holds to CGA, albeit by using its maps and mods database instead.

[Radar]

I was just throwing out multiplayer as a scenario because it's the only one I could think of that has some sort of practical use for this sort of thing. In general I don't really see a reason for mod packs. I think fixating on how a mod is packaged rather than the artistic qualities of the mod itself is really missing the forest for the trees. Anyway, I'm just one person. Maybe someone who "gets it" a bit more than me will chime in soon. As for me, I simply download zips from Dukeworld. It just werks.

[NY00123]

Radar, on 23 November 2023 - 01:48 PM, said:

I was just throwing out multiplayer as a scenario because it's the only one I could think of that has some sort of practical use for this sort of thing. In general I don't really see a reason for mod packs. I think fixating on how a mod is packaged rather than the artistic qualities of the mod itself is really missing the forest for the trees. Anyway, I'm just one person. Maybe someone who "gets it" a bit more than me will chime in soon. As for me, I simply download zips from Dukeworld. It just werks.

Multiplayer has been the main scenario for me as well, given the relative lack of use of mods (especially older ones) in multiplayer games over the years. Convenience of use does matter, of course.

It also matters in single-player mode. I found two solo play-throughs in which custom CON code wasn't used (the base game's GAME.CON was); Also, in one of them, ART files were supposed to be renamed but weren't. Then again, it's not impossible this was done on purpose for the latter example, given that the demonstrated map wasn't the first.

As I wrote earlier, packaging mods as GRP files for xDuke(-derived) ports currently has a clear limit on the length of the path to the file. So, unless fixed in at least one xDuke fork, it's probably less relevant. Additionally, Rednukem can still be used, supports ZIP/PK3 files and should behave more like Vanilla Duke3D v1.5; Albeit it might have its problems in multiplayer games due to reduced usage for this purpose, and it doesn't appear to support P2P connectivity. Chances are these points (at least most of them) also hold to DukeGDX. I'm not currently aware of a way to use DukeGDX from an external multiplayer launcher (e.g., via command-line arguments).

[William Gee]

Is there a community of MP players who play together on a specific port? I am not in the MP scene so I have no idea what people are playing these days.
The zip file holding a collection of easy to load mods via the mods folder in netduke32 for example is what seems best to me, that would be a group of GRP files or PK3 files depending on the mod.

[NY00123]

William Gee, on 23 November 2023 - 11:20 PM, said:

Is there a community of MP players who play together on a specific port? I am not in the MP scene so I have no idea what people are playing these days.
The zip file holding a collection of easy to load mods via the mods folder in netduke32 for example is what seems best to me, that would be a group of GRP files or PK3 files depending on the mod.

There are a few Discord servers created (at least originally) with Dukematching in mind, albeit chances are most of the average day isn't filled with online Duke3D games. This can vary greatly by various factors, like time the zones and the interest of specific players.

The Shotspark Studios Discord had a section added for NetDuke32, and the channels still technically exist; In practice, when there's NetDuke32 (or StrikerDM) dev discussion, it usually occurs in the Build BROS Discord as of the last few years.

In addition to NetDuke32, hDuke is also used. hDuke is closer to xDuke in behaviors. Like NetDuke32, it supports Team Dukematch. It should be noted that hDuke's sources are not available as of now, just like xDuke v19.8 or nDuke's.

NetDuke32 and hDuke are both supported by NukemNet, along with Rednukem, xDuke, nDuke and rDuke. Vanilla Duke3D is further covered. The NukemNet IRC-based service has a Discord server, with a channel synced to NukemNet's lobby IRC channel.

For Vanilla Duke3D, another option (in addition to NukemNet) is Classic Gaming Arena, i.e., CGA. That's a lanucher supporting DOS versions of games exclusively. There are people playing multiple DOS games with it, albeit the Duke3D ones are usually a few from the ones I'm aware of.

[Radar]

NY00123, on 23 November 2023 - 11:06 PM, said:

As I wrote earlier, packaging mods as GRP files for xDuke(-derived) ports currently has a clear limit on the length of the path to the file. So, unless fixed in at least one xDuke fork, it's probably less relevant.

You probably don't need to consider xduke or any of its forks. xduke is typically regarded as a "purity" port for multiplayer, and probably never used for singler player, so it is unlikely anyone uses it to play mods.

[NY00123]

Radar, on 24 November 2023 - 05:11 PM, said:

You probably don't need to consider xduke or any of its forks.

That's been the conclusion I've gotten to after locating the crash.

Quote

xduke is typically regarded as a "purity" port for multiplayer, and probably never used for singler player, so it is unlikely anyone uses it to play mods.

I don't know how often was xDuke historically used for playing solo, albeit chances are I did so, at least here and there. There's the old crash report (fixed in xDuke v19.7.1) that I had, related to roch3.map's use of a palookup entry not defined in LOOKUP.DAT. Most chances are it was a single-player test. I suspect that TURRICAN was also using xDuke for single-player games.

Regarding the idea of repackaging as GRP instead of PK3/ZIP, I've mainly had in my mind the case players might want to play a mod with a source port closer to Vanilla Duke3D in behaviors. Of course, if any xDuke derivative ever implements PK3/ZIP mod file support, then there's even less of a need for new GRP files.

For now the following options exist, albeit not supporting P2P in the same manner that duke3d_w32/xDuke and derivatives do:
- Rednukem, via NukemNet (PK3/ZIP mod file transfer). Might be less stable as it wasn't often tested in multiplayer.
- Vanilla Duke3D with dosbox-staging, via CGA (mods database). Note that even if Duke3D uses P2P mode, network traffic will go through a central dosbox IPX emulation instance.

btw, about Rancidmeat's duke3d_w32 and the -game_dir switch inherited by xDuke:

Spoiler

While it might be true that xDuke was used with multiplayer in mind for most, and EDuke32 was more commonly used for single-player and mods, the situation was maybe different beforehand (even if similar). You had Rancidmeat's duke3d_w32 (used as a base for xDuke) and JFDuke3D (used for EDuke32). Initially, duke3d_w32's in-game timing was less accurate, making the game slower than Vanilla Duke3D . But duke3d_w32 was working better over the internet, so it was more commonly used for multiplayer.

While mods weren't that common for multiplayer, duke3d_w32 did introduce the -game_dir option for mods, also inherited by xDuke. It was introduced in b18.2. The earliest build I could get and check was 19.1. The following example is mentioned in Rancidmeat's website for Duke L.A.:

duke3d_w32.exe -game_dir mods\dukela /xLAGame.DM /gdla.prg

Even then, I had problems with starting the mod in this manner, using b19.1 and other few versions. This was expected, given the buffer overflow bug that I mentioned earlier. The change log for duke3d_w32 still mentions fixes related to -game_dir, including the following one from b20.1:

 * Fixed an issue with -game_dir over running array boundries and causing a crash.

My guess is, people were using -game_dir for mods without GRP files; While in situations like CON+GRP file pairs, no sub-directory was created. Assuming the mods were made for DOS Duke3D, each of the GRPs' file names would fit under 12 characters, thus not reproducing the aforementioned buffer overflow.

[Aleks]

Radar, on 24 November 2023 - 05:11 PM, said:

You probably don't need to consider xduke or any of its forks. xduke is typically regarded as a "purity" port for multiplayer, and probably never used for singler player, so it is unlikely anyone uses it to play mods.

I even know of someone making a (single player) mod specifically designed for xDuke/DOS Box!

[NY00123]

So, I think that I'm ready to write what I've eventually done. Indeed, I've resorted to repackaging mods as single ZIP/PK3 files compatible with NetDuke32. There wasn't that much automation, outside of a script using a few programs for eventually creating the final output from a directory's contents, but I did write in separate txt files how I prepared the archives from the original files (without repeatedly mentioning the script in question).

https://ny.duke4.net...epackaged-mods/

More details are currently in the main txt file you should find over there. My original idea was grabbing mods compatible with the Atomic Edition and making new packages with minimal changes to behaviors. For instance, I didn't want to introduce new bug fixes. However, I did expand just a bit with the idea, as demonstrated with the PaintBall mod, originally made for Duke Nukem 3D v1.3d:

Spoiler

Its CON code is incompatible with the Atomic Edition. NetDuke32 and other few ports can actually load v1.3d's data and CON files, but the mod (well, the release under pbduke2.zip) replaced just USER.CON. So, it wouldn't work with such source ports while using the game data from the Atomic Edition or World Tour. Therefore, I created a new archive with the missing CON files from v1.3d. I also added v1.3d's LOOKUP.DAT in order to fix the title screen's palette, given that this should be closer to the experience of using the mod back in 1998.

I actually didn't introduce the title screen palette fix to "Duke, it's ZeroHour". Yes, I'm aware of the fix in Zoom Platform's offering (I don't know how it was implemented); But with the original files from 1997, you could choose between CON files for v1.3d and ones for the Atomic Edition. If you went with the Atomic Edition, you'd get v1.3d's title screen with the Atomic Edition's title palette. I could also use v1.3d's CONs and the LOOKUP.DAT fix as in PaintBall's case, but the Atomic Edition CONs have a renamed episode and are compatible with a few more source ports, say Rednukem. So, I left these behaviors as an example of not fixing an old bug, as reproduced in 1997 while using the Atomic Edition.

A few more details:

Spoiler

You can find mentions of kextract in multiple txt files written by me. As of today, in case kextract was used, it was actually not an original DOS executable from the 1990s. Rather, it was a Linux executable built from the JFBuild repository. As for creating each new archive file, that was done using multiple programs. A helper script was used in order to make combined use of Info-ZIP, 7-Zip and Ken Silverman's KZIP, along with his ZIPMIX program.

By the way, I wasn't the first to give such an endeavor a try. Back in 2009, William Gee was preparing CON+GRP file pairs for usage with my old launcher, YANG (Yet Another Netplay Guider):

Spoiler

https://forums.duke4...dpost__p__24812

Despite this, it still didn't become commonplace to play Duke3D mods online. I suspect the lack of automatic mod file transfer or a mods database in YANG was the main cause, along with the need to repackage files at times.