Duke4.net Forums: Restoration of a few games' EXEs versions - Duke4.net Forums

Jump to content

  • 3 Pages +
  • 1
  • 2
  • 3
  • You cannot start a new topic
  • You cannot reply to this topic

Restoration of a few games' EXEs versions  "Let's call it software archaeology?"

User is offline   NY00123 

#1

Update (Feb 29 2020): The repository has been converted to git, split to submodules and moved to a new location: https://bitbucket.or...ver-recreation/

The rest of this post remains unchanged as of writing this.

Original post contents:

Hey there,

I think it's a good (albeit belated) time to introduce you to the following, at least if you aren't yet aware of this: https://bitbucket.or...ver-recreation/

To summarize, the above repository covers modifications of open source games, with attempts to recreate original old EXEs byte-by-byte. Sometimes it's more successful, while other times it's less. A few examples:
- For most Wolfenstein 3D and Spear of Destiny EXEs, with the right tools, you may now faithfully recreate the original EXEs from the 90s, byte-by-byte. There may be this or that EXE for which only the very first few bytes (in the EXE header) differ.
- For Rise of the Triad, I had less luck, and there are also technical limitations due to the usage of Watcom 10.0b (from the 10.0(x) series), but I believe it's *very* close.

Now, let's get to the following. Note that I consider this code recreation *partial*, hence you'll find it in a "bonus" subdirectory. In particular:
- This covers game code only.
- This does *not* cover engine code, audio library code* or closed-source libraries. Original binary OBJ and LIB files are used for these.

Basically, I'm talking about the game code (and no other code piece) of Duke Nukem 3D!

These EXEs are now covered in the "duke3d" directory (bonus/duke3d) within my repository:
- Duke Nukem 3D: Atomic Edition v1.5, up to differences described below (possibly hidden under "Spoiler" tags).
- The following EXEs originally built by Matt (recreated byte-by-byte): NAM/NAPALM Full Version 1.0, WW2GI Full Version 1.0 and EDuke 2.00.23 (the original binary EDuke release from the year 2000 mostly known as "EDuke 2.0").

Note that for any comparison you might be making, there should *not* be any (originally) unofficial patch applied, like the removal of copy protection. Digital releases (e.g., Steam and GOG.COM) may be impacted, too.

For Duke3D v1.5, use Watcom C 10.0 and *not* any other version; Not even 10.0a or 10.0b.
For any of Matt's EXEs, use Watcom C 10.6a (and no other version).

For a little background behind the beginning of all of this, I'll quote myself (with any possible minor diff) from the RGB Classic Games forums (https://www.classicd....php?f=3&t=1410):

Spoiler


Some more details regarding Duke Nukem 3D, quoting from a recent RGB forums post with a few changes (https://www.classicd...hp?p=8406#p8406):

Spoiler


* In fact, this isn't entirely accurate. I've modified the Apogee Sound System sources so you can more-or-less recreate v1.1. However, you'll still prefer to use the existing AUDIO_WF.LIB file, if only due to unused gaps between C string literals and more being filled with contents differing from the original. As in the case of the game code, this might be related to the Watcom C v10.0(x) series.

This post has been edited by oasiz: 29 February 2020 - 10:55 AM

9

User is offline   MrFlibble 

#2

Probably not the focus of your endeavour, but is it possible to create a shareware v1.5 version of Duke3D using your findings?
0

User is offline   NY00123 

#3

I should probably begin with a "tl;dr" summary.

So, with the modified Duke3D codebase of mine, you may now create a Duke3D EXE that (with very few exceptions) behaves *exactly* like v1.5.
Among other things, this means:
- Identical file sizes (up to the presence/lack of the DOS4GW extender).
- Identical binary EXE file contents (up to minor diffs that don't matter + DOS4GW).
- Compatibility with demos recorded with v1.5 (at least the original 3 demos seem to be OK).
- Compatibility with v1.5 saved games (although surprisingly enough, these may work in v1.4 after a simple hex-edit of BYTEVERSION in the SAV files; Your mileage may vary, though).

3 more EXEs may further be recreated, and even perfectly. (NAM v1.0, WW2GI v1.0 and Enhanced Duke (EDuke) v2.00.23.)


View PostMrFlibble, on 12 May 2018 - 04:47 AM, said:

Probably not the focus of your endeavour, but is it possible to create a shareware v1.5 version of Duke3D using your findings?


I suppose that a "new" shareware EXE can be made. Out of curiously, I've even tried to do this on my own.
For reference, see the attached source code patch and EXE (which requires DOS4GW or compatible).

A few notes about my "shareware v1.5" code changes:
- VOLUMEALL and PLUTOPAK shall *not* be defined.
- On the other hand, VOLUMEONE should be. A definition of NOCOPYPROTECT is probably desired, too.
- There are a few instances in which the Atomic Edition logo (PLUTOPAKSPRITE and other few sprites) is shown. While there's code to exclude this, only the UK macro is checked for some reason, not PLUTOPAK. So, better fix these as well.

As for the data, I've used shareware v1.3d's. However, it looks like the new EXE is incompatible with v1.3d's CON, so I've used v1.5.

There may still be this and that issue. I've got this message of a missing sound file in-game. This might be related to at least one of the CON files.

Attached File  duke3d_v15_recreated_shareware_patch_20180512.7z (395.11K)
Number of downloads: 6

This post has been edited by NY00123: 12 May 2018 - 06:14 AM

2

User is offline   MrFlibble 

#4

View PostNY00123, on 12 May 2018 - 06:04 AM, said:

As for the data, I've used shareware v1.3d's. However, it looks like the new EXE is incompatible with v1.3d's CON, so I've used v1.5.

The GRP from the Macintosh demo of Duke3D includes the missing Atomic logos for the title screen & menu, and the CONs might be v1.4/1.5 compatible although I haven't checked this yet. I have always thought that the Mac demo GRP could actually be a stripped-down version of the post 1.3d shareware that never went public.
1

User is offline   NY00123 

#5

View PostMrFlibble, on 12 May 2018 - 12:39 PM, said:

The GRP from the Macintosh demo of Duke3D includes the missing Atomic logos for the title screen & menu, and the CONs might be v1.4/1.5 compatible although I haven't checked this yet. I have always thought that the Mac demo GRP could actually be a stripped-down version of the post 1.3d shareware that never went public.


This is a good idea. I've checked not just duke68kdemo.bin, but also duke3d.sit.hqx. I've used "The Unarchiver" (unar) for unpacking these.
These two demo archives seem to have the same CON and Group files. The CON files are exactly the same as in DOS v1.5, up to the differing new line formatting (CR for the old Macs, CR+LF for DOS).

While this seems to more-or-less work, there are still a few differences to mention. For instance:
- The Mac demo(s) lack MIDI files.
- There are less ART graphics when compared to DOS version. The missing graphics include the Atomic logos, too.

There are chances the Atomic logos are stored in a separate file. This may also apply to the graphics used for misc. Mac-exclusive cheat codes, like DN1984.

To conclude, though, then yeah, the Mac demo art is clearly based on DOS v1.5's, just with less of it.
As for comparison to shareware version 1.3d, all I'll say for now is the following:
- Shareware v1.3d and the Mac demo both have TILES000.ART-TILES012.ART and no more.
- The same non-shareware tiles were stripped from both versions i.e., there's no tile present in one of them and not in the other.
- Unsurprisingly, though, there are tens of tiles differing. Note that a few tiles didn't change between *registered* DOS versions 1.3d and 1.5, while there are earlier revisions in *shareware* version 1.3d.

This post has been edited by NY00123: 12 May 2018 - 02:43 PM

1

User is offline   TerminX 

  • el fundador

  #6

View PostNY00123, on 12 May 2018 - 02:36 PM, said:

Note that a few tiles didn't change between *registered* DOS versions 1.3d and 1.5, while there are earlier revisions in *shareware* version 1.3d.

That's correct. The art was not updated between shareware releases, so shareware 1.3d still contains things like the angled rocket sprites from 1.0, a switch sprite that exists in LameDuke, etc.
2

User is online   0815Jack 

#7

sorry for asking this dumb question, but what is the point of recreating the exe byte-by-byte ?
1

User is offline   Mark 

#8

I wondered the same thing. I kept quiet thinking I was the only dumb one who doesn't know. :(
1

User is offline   NY00123 

#9

This is actually a very good question! I'll give here two ways in which recreating original code may assist:

- Heard of the "Chocolate Doom" source port? It's the one that aims to accurately reproduce the experience of playing the original Doom games from the 90s using any of the original DOS EXEs, at least as of version 1.9.
Among other things, this includes the general feeling, as well as compatibility with original saved games, demo files and maps (while *not* supporting contents which is *not* compatible with any of the original DOS EXEs).

Thus, recreating an EXE byte-by-byte, even if just partially, can *very greatly* assist with such an endeavour.

Note that in the case of Chocolate Doom, they managed to do so without this so far - The Linux Doom sources were used, and there are some different behaviors from 1.9, so more work had to be done.
There may still be certain technically differing behaviors as a consequence.

- Secondly, and this mostly applies to code in my repository for other games, like Wolfenstein 3D and Blake Stone (sorry, not ETA for Duke Nukem 3D or any other game):
If anybody has ever been curious to know how do two versions of a game differ, not just in data but also in the code - this is more-or-less what recreated code allows you.
Note that there are a few possible pitfalls:
* You may assume that the various #if/#ifdef/#ifndef statements I've added are sufficient for locating *all* technical differences, but it isn't exactly accurate. In fact...
* Even if you change the compilation optimizations for just ONE compilation unit (e.g., a C source file), good chances are the EXE will behave somewhat different.
This can be anything, from speed-sensitive diffs that you won't notice, to undefined/unspecified behaviors that you *will* notice (even if not rarely).
* Finally, even if you change *just* the code, the compiler may add other technical differences you haven't caught (again related to undefined/unspecified behaviors).

This post has been edited by NY00123: 13 May 2018 - 11:36 AM

2

User is offline   Mark 

#10

I'm not familiar with Doom ports. But maybe my question has been answered. My uneducated guess was that maybe this new exe file might be an alternative to having to use Dosbox in windows to run games as they were originally meant to run in dos. In other words, playing a dos game without having to mount drives and configure this and that. But it doesn't look like it.

This post has been edited by Mark.: 13 May 2018 - 12:31 PM

1

User is offline   oasiz 

  • Dr. Effector

#11

Nah, this is more about accurately reconstructing the original DOS exe that you got back in the days.
Source releases that we usually get are not 1:1 of what they had when they pressed "compile". Like in D3D case it was said above that it's somewhere before 1.5.
Often they remove bits and pieces that they don't want to share like audio libraries, etc..

Cool aspect about this is actually getting the original files working and in this case we can finally re-create stuff like the shareware 1.5.
Sure it's a bit of a novelty for most, but it's really useful for comparison purposes and preservation.

Not to mention any source code studies and "chocolate" ports.
3

User is online   0815Jack 

#12

thx for the detailed explanation :(
0

User is offline   NY00123 

#13

oasiz is surely spot-on with his explanation, thanks!

View PostMark., on 13 May 2018 - 12:29 PM, said:

My uneducated guess was that maybe this new exe file might be an alternative to having to use Dosbox in windows to run games as they were originally meant to run in dos. In other words, playing a dos game without having to mount drives and configure this and that.


The modified codebase for DOS may still be a basis for some individual eventually adapting it to modern platforms, thus letting you do exactly that! (Basically the "Chocolate Doom" idea.)
1

User is offline   Mark 

#14

Thanks for explaining. I couldn't think of anything else other than it being a "nerd experiment". And I don't mean that in a bad way.
1

User is offline   NY00123 

#15

In fact, how could have I forgotten this!

The results of my work to recreate the NAM and WW2GI EXEs (EDUKE.EXE had been the first) were used to fix a few inaccuracies with the NAM and WW2GI support in EDuke32.

This post has been edited by NY00123: 14 May 2018 - 11:09 AM

2

User is offline   leilei 

#16

Different lineage, there's also some functionality loss in the source releases for Quake and Quake III Arena that existed in prior versions that are hard to recreate and have been omitted from the source release:

Quake:
- Menu fade tint (easily recreateable with the dead gelmap code scrap)
- Sound attenuation calculation is different
- DJGPP makefile is missing from source release

Quake III Arena:
- Dynamic lights updating the lightmap (Q3Test 1.01 to 1.07)
- Null sound effect sawtooth buzz (Q3Test 1.01 to 1.08)
- Fast 8-bit sample mixing (Q3Test 1.01 to Q3 1.27g - s_loadas8bit and s_compression)
- Earlier protocols/traps before the big mod regression (1.11 to 1.17)

Just mentioning it here because the q scene has 0 interest in anything that's not fucking promode aerowalk / brutal phase / bad hd packs. I don't really see interest in anything historical elsewhere.

This post has been edited by leilei: 14 May 2018 - 07:19 PM

2

#17

View PostNY00123, on 14 May 2018 - 10:57 AM, said:

In fact, how could have I forgotten this!

The results of my work to recreate the NAM and WW2GI EXEs (EDUKE.EXE had been the first) were used to fix a few inaccuracies with the NAM and WW2GI support in EDuke32.


I saw your name in the changelog of Eduke32 related to this. I must say, nice done!

This project has been proved to be useful already... :P

There still is a problem with the automatic weapons in WW2GI in Eduke32. When quickly pressing the fire button, the firing animation is played, although no shot is fired. If you have time and are willing to check why this is, that will be great! :(

Details are here:
https://forums.duke4...post__p__300746

This post has been edited by Little Tijn: 15 May 2018 - 11:14 AM

2

#18

Oops, hijacked this thread a bit with my request above, sorry about that.

More on-topic:
I would have expected that the released sources of games should be the latest code available, seeing that this code is used before to build the latest version. They had this code, to build the game. So why isn't it used, but instead some in-between version? Isn't it easier to release the latest code available? That are questions that I got from this thread.

I know that one of the most important fixed in the day for Duke Nukem 3D was a memory leak fixed in 1.5. Because with computer at that time with 8MB RAM etc, it would make a big different. Wondering is this is also included in the source release. Where there some other significant changes?

Speaking of released versions, we have some cases where the code to a later version that the releases binary is available. Off the top of my head, the code of Eduke 2.1 and the released code in dubious legal state of Witchaven II. If I recall correctly, the compressed file with the source of Witchaven II even included a unreleased patch for the offical release as a installable EXE that patched the binaries etc. Fixed a annoying bug with the strafing buttons not working. I guess the code also includes the changes of this patch.

BTW. How is the source of Shadow Warrior in comparison with the latest binary?
0

User is offline   NY00123 

#19

View PostLittle Tijn, on 15 May 2018 - 10:46 AM, said:

I saw your name in the changelog of Eduke32 related to this. I must say, nice done!

This project has been proved to be useful already... :(


Thanks!
As previously stated by me, it was used for fixing NAM & WW2GI behaviors EDuke32. Guess it's the first time such a thing has occurred, at least with regards to the Duke code.

View PostLittle Tijn, on 19 May 2018 - 11:27 PM, said:

I would have expected that the released sources of games should be the latest code available, seeing that this code is used before to build the latest version. They had this code, to build the game. So why isn't it used, but instead some in-between version? Isn't it easier to release the latest code available? That are questions that I got from this thread.


It's not necessarily as simple as may initially seem. The sources were released on April 1st of 2003, about 6.5 years after Duke3D v1.5 was out.
So while what you've just stated makes sense, chances are the people who were in the offices by that time weren't entirely sure how were the files organized. Maybe some of the files moved between different locations and/or got lost for good. At the least, the binary libraries' OBJ and LIB files were clearly the ones used in v1.5 (and NAM+WW2GI+EDuke).

Quote

I know that one of the most important fixed in the day for Duke Nukem 3D was a memory leak fixed in 1.5. Because with computer at that time with 8MB RAM etc, it would make a big different. Wondering is this is also included in the source release. Where there some other significant changes?


If you're comparing to v1.4, I don't know for sure. Don't think I saw a lot of differences in the Duke3D code between 3DR's released sources and v1.5, although there's that fixed music bug in MENUES.C.

Quote

BTW. How is the source of Shadow Warrior in comparison with the latest binary?


I don't know for sure about this. What I can tell, though, is that we're much less lucky with the SW sources, when compared to DN3D (in terms of recreating any original EXE). There are multiple reasons for this, but here are just a few examples I could spot.

Regarding Duke Nukem 3D:
- We have the EDuke 2.0 sources which more-or-less cover the v1.5 code. The MAKEFILE from the same EDuke archive also appears to have the original optimizations passed to Watcom C, for the creation of v1.5.
- We also have the correct libraries' OBJ and LIB files: Build Engine, Apogee Sound System, MACT and Total Entertainment Network (TEN).
- In the Duke3D sources prepared by Charlie Wiederhold for release by 3DR, actual changes to the code are clearly marked with "CTW" comments. There are just 22 pairs of these, and they aren't present in the EDuke sources at all.

As for Shadow Warrior:
- We have the sources released by 3D Realms and prepared by Charlie Wiederhold; No additional source archive like EDuke is available.
- When Charlie Wiederhold originally got the files, it wasn't as organized and ready for building as the Duke3D sources were. Certain files were even missing, so they were borrowed from the Duke3D sources.
- There are *way* more CTW changes; I think there are more than 100.
- We also don't have the exact binary library OBJ/LIB files, at least for certain libraries. While it might be possible to recreate the Build Engine and Apogee Sound System code, the TEN lib is closed-source and appears to have at least a few differences from the code in DUKE3D.EXE (from BUTWCD4.LIB). I'm also not sure about the closed-source MACT.
- On the contrary to Duke3D, it looks like the SW code has some more TEN-specific code integrated. Certain code pieces may even be missing. Duke3D didn't use as much, and we have for the latter the relevant code either way.
2

User is offline   Hendricks266 

  • Weaponized Autism

  #20

Fortunately, one of the things I did receive from the 3DR archives were full backups of the hard drives from Jim Norwood and Frank Maddin's workstations.
3

#21

That's some interesting information about Duke and SW. Thanks for that!

It's a bit pity that the source code of Shadow Warrior isn't in such a great state. Although we have some nice ports for it even with the code that was given.

Any idea what those changes by Charlie Wiederhood could have been and why they where needed?

View PostNY00123, on 20 May 2018 - 07:56 AM, said:

It's not necessarily as simple as may initially seem. The sources were released on April 1st of 2003, about 6.5 years after Duke3D v1.5 was out.
So while what you've just stated makes sense, chances are the people who were in the offices by that time weren't entirely sure how were the files organized. Maybe some of the files moved between different locations and/or got lost for good. At the least, the binary libraries' OBJ and LIB files were clearly the ones used in v1.5 (and NAM+WW2GI+EDuke).


That are good reasons why sources aren't always the latest. Seeing that source code has been lost for games before already. It reminds me of stories of lost sources of games that I read on the Internet, like System Shock 2.

View PostHendricks266, on 20 May 2018 - 09:14 AM, said:

Fortunately, one of the things I did receive from the 3DR archives were full backups of the hard drives from Jim Norwood and Frank Maddin's workstations.


That's interesting! Are the sources of Bio Menace on it by any change? :( (Just kidding). But on a serious note, it might provide some very good insight of some things, or even some missing part.

This post has been edited by Little Tijn: 20 May 2018 - 11:48 AM

0

User is offline   NY00123 

#22

Guess what? (Thanks to NukeYKT for the hint!)

Turns out the "extras" directory from the Duke3D sources, as released by 3DR, includes the code for v1.5!
I suppose that I've suspected this beforehand. It's just a bit weird this hasn't been confirmed until now. Then again, maybe this isn't very surprising.
- The libraries' OBJ and LIB files are missing, and should be brought from the "source" directory of 3DR's sources (or from the EDuke sources).
- BUILD.H is also missing. You shall copy the one from the EDuke sources, *not* from 3DR's Duke3D sources.
- MAKEFILE is more-or-less the same as in the EDuke sources, while MAKEFILE.LNK is based on the one from my repository.

As a side-note, there's a bit of concerning news regarding BUILD.H:
When I attempt to use BUILD.H from 3DR's sources (rather than the EDuke sources),
the output EXE's layout becomes somewhat different, even if not by much. Maybe certain global variables are positioned differently.
The only real difference in BUILD.H code is the addition of "extern" declarations of unused variables ("validmode*" vars),
as well as minor changes in comments (the addition of support for masked/translucent ceilings/floors via ceilingstat/floorstat bits).

View PostLittle Tijn, on 20 May 2018 - 11:17 AM, said:

Any idea what those changes by Charlie Wiederhood could have been and why they where needed?


It's probably the best to see the CTW comments in the code for yourself; In some cases, there's an explanation, but when there's not, chances are it's related to TEN.
Also, without some of his changes, it wouldn't even be possible to build an EXE at all. There were also a few obstacles on the way (like some issue with CD Audio playback).

View PostHendricks266, on 20 May 2018 - 09:14 AM, said:

Fortunately, one of the things I did receive from the 3DR archives were full backups of the hard drives from Jim Norwood and Frank Maddin's workstations.

View PostLittle Tijn, on 20 May 2018 - 11:17 AM, said:

That's interesting! Are the sources of Bio Menace on it by any change? :( (Just kidding). But on a serious note, it might provide some very good insight of some things, or even some missing part.


I should probably note here that I've referred only to sources which were already made available to the public in various forms.
1

User is offline   MrFlibble 

#23

View PostNY00123, on 12 May 2018 - 02:36 PM, said:

While this seems to more-or-less work, there are still a few differences to mention. For instance:
- The Mac demo(s) lack MIDI files.
- There are less ART graphics when compared to DOS version. The missing graphics include the Atomic logos, too.

There are chances the Atomic logos are stored in a separate file. This may also apply to the graphics used for misc. Mac-exclusive cheat codes, like DN1984.

Uhh, concerning the Atomic logo, I remembered that wrong, sorry. There's no Atomic logo in the Mac demo. I ran it through Executor and took some screenshots, here's the menu:
Posted Image
As for MIDIs, they're useless on Mac so were cut out, I guess if there is any music (Executor doesn't handle it so I can't tell) it's probably stored in the data part of the binary or whatever that's called on Macs.

View PostNY00123, on 12 May 2018 - 02:36 PM, said:

To conclude, though, then yeah, the Mac demo art is clearly based on DOS v1.5's, just with less of it.

Thanks for confirming that.

View PostNY00123, on 12 May 2018 - 02:36 PM, said:

- The same non-shareware tiles were stripped from both versions i.e., there's no tile present in one of them and not in the other.

That's what made me think that this GRP for the demo was based on a v1.5 shareware release with further cuts of MIDI files and levels beyond E1L3.

View PostNY00123, on 12 May 2018 - 02:36 PM, said:

- Unsurprisingly, though, there are tens of tiles differing. Note that a few tiles didn't change between *registered* DOS versions 1.3d and 1.5, while there are earlier revisions in *shareware* version 1.3d.

Not counting the Atomic Edition menu font, I only remember the yellow switch w/ red/greed lights which looks more like the LameDuke version in shareware :(

View PostTerminX, on 12 May 2018 - 03:36 PM, said:

a switch sprite that exists in LameDuke, etc.

IIRC it's slightly different although the design is clearly more LameDuke-ish than the registered version.
0

#24

View PostLittle Tijn, on 20 May 2018 - 11:17 AM, said:

Any idea what those changes by Charlie Wiederhood could have been and why they where needed?

The vast majority are split into three categories:
  • Moving initializations around due to mismatches between SW and Duke 3D assumptions.
  • Removing TEN dependencies/references.
  • Updating to current (at the time) compiler expectations in terms of variable definitions.

And if I recall correctly one item... just because.

This post has been edited by JumpJaneJump: 02 June 2018 - 07:21 PM

2

#25

View PostJumpJuneJump, on 02 June 2018 - 07:20 PM, said:

The vast majority are split into three categories:
  • Moving initializations around due to mismatches between SW and Duke 3D assumptions.
  • Removing TEN dependencies/references.
  • Updating to current (at the time) compiler expectations in terms of variable definitions.

And if I recall correctly one item... just because.


Thanks for the interesting information!

Now that the code has been recreated code from 1.2, we can know for use what the changes where. :)

With the release of NBlood and BloodGDX I started to check my original version of Blood 1.00 and reading the patch details with the patches that came after it.
I think it would be very interesting to restore (or reverse-engineer) the code of those version as well. Especially considering the changes (nerfing) to the weapons with the 1.10 patch.

From 1.00 till 1.02 the life leech replenish your health with use and the voodoo-doll alt fire was way more powerful. Also in the patch notes there is a reference that Tchernobog is not so weak anymore and Cerberus could be killed a bit too easy with the Tesla cannon. The bobbing of the screen is much more visible on 1.00 and 1.01. From 1.02 it is heavily reduced especially when crouching.

Maybe it is a nice challenge to try to recreate (or reverse-engineer) the code for these version as well. It would be very interesting to see what has been been changed exactly.

Also, it would be nice to have the original working of the Life leech and the voodoo-doll as a option in the Blood ports. :lol: Playing the game without the nerfed weapons is quite a bit of fun. Although I can understand the nerfing. The life leech is a bit unbalanced, not to speak of the voodoo-doll.

This post has been edited by Little Tijn: 17 February 2019 - 08:07 AM

0

User is offline   NY00123 

#26

I think that it's a good time for a new post <_<

Turning out to be quite lengthy (of course), I've decided to hide most of the details under spoiler tags, and keep a short/medium summary on the top.

I should probably begin by saying that the bonus/duke3d directory was collapsed into duke3d. You should see why soon. A few build scripts were also changed again.

Before getting to actual updates in the gamesrc-ver-recreation repository itself, I'll first link to the following threads, in case you aren't aware of at least one of the covered topics:

https://forums.duke4...de-re-released/ - Thanks to Devolver Digital, last February had a re-release of Shadow Warrior sources, this time confirmed to match versions 1.0, 1.1 and 1.2 in behaviors (some work will still be required for saved games compatibility); Additionally, thanks to Ken Silverman, the sources for the ENGINE.C/ENGINE.OBJ file as used in SW 1.1 and 1.2 are also covered, along with a renamed CACHE1D.C file from 1995 named CACHE1D.TXT.

https://forums.duke4...es-and-sources/ - Again thanks to Ken, this covers historical Build Engine revisions, mostly in the forms of object and header files. There's also a 95 revision of ENGINE.C which covers the abandoned groudraw feature.

It is now the time to list some additions to gamesrc-ver-recreation. As usual, I cannot guarantee that I'll do more than what's listed here in the future.

A summary is following:

- The repository has a new "kenbuild" directory. Among other things (a portion of which to be mentioned soon), you can use it to recreate the version of Ken-Build's GAME.EXE as originally released by Ken on June 2000, using Watcom C 11.0. Note that this should *not* be confused with minor edits done by him on Nov 2002.
- Using the ENGINE.C code which is matching SW 1.1/1.2, SW 1.0's revision could be recreated. Turns out it differs from 1.1/1.2 just by a few constants in drawmapview, clippoly and fillpolygon (unless I've missed something).
- Nuke.YKT came in and used the above SW 1.1-1.2 revision of ENGINE.C in order to recreate Duke3D 1.5's. In fact, originally he wanted to do something similar for Redneck Rampage, but eventually it turned out to pave the way for the Duke.
- A bit later, I came in and took the Ken-Build 2000 revision of A.ASM, in order to recreate the one common to Duke3D 1.5 and SW 1.0-1.2. Unlike the 2000 revision which should better be built with WASM, the older revision requires MASM 5.10.
- As for CACHE1D.C, both Nuke.YKT and me confirmed that the Duke3D 1.5 revision isn't that different from the 2000 one. The former is also the same revision as in SW 1.0-1.2.
- No change was required for MMULTI.C: It's the same as in the Ken-Build release, and also in a source archive of Ken's 2DRAW engine.
- Later, I got to take care of Duke Nukem 3D: Atomic Edition 1.4. This covers the game code and Build Engine code altogether, with a few bits of very important help from Nuke.YKT. I think that most of the changes were in the Build Engine. 1.4 and 1.5 don't differ as much, and the GPLed release's "source" dir is closer to 1.4 than 1.5.
- Finally, following the Shadow Warrior sources re-release, I faithfully recreated almost all of the Apogee Sound System revision used in SW 1.0-1.2 i.e., version 1.09 of AUDIO_WF.LIB. Only actual code difference after compiling, is that the recreated FX_SetupCard code has a bit different register read, although the behaviors should be equivalent.

More details for the interested:

Spoiler

5

User is offline   NY00123 

#27

Hey there,

I hope to get some feedback on the following details about a plan to migrate to git.

I don't have much to add in terms of contents, although I've recently been doing work on the BUILD editor in a separate "branch", which is actually a separate hosted copy of the repository.

As some of you should know by now, gamesrc-ver-recreation is currently a Mercurial repository, which is hosted on Bitbucket. Thing is, Mercurial support is being deprecated in Bitbucket. You can't create new Mercurial repositories over there as of this month, and existing repositories are expected to be removed by June: https://bitbucket.or...rt-in-bitbucket

Personally, I'll probably convert the repository to git, mostly since I'm way more used to git nowadays (and it generally seems to be way more supported).

Thing is, since the repository's history (including all commit hashes) will essentially be recreated, this is a chance to make one-time changes to its history.

So, it can assist if anybody has some feedback on any of the following points:

- Before creating gamesrc-ver-recreation as a Mercurial repository hosted on Bitbucket, I briefly had this GitHub-hosted git repository titled game-srccode-ver-recreation, covering just Catacomb Abyss shareware v1.13. After about a week of use, I removed the repository from GitHub and created gamesrc-ver-recreation, currently a Bitbucket-hosted Mercurial repository. At some later point, I re-uploaded the original git repository to Bitbucket, currently titled game-srccode-ver-recreation-git-legacy. So I think that I can basically kind-of merge the two, re-applying the Mercurial changesets as git commits on top of the original repository, possibly after adding a new transitional git commit with a short explanation.

- What about removing the commit/changeset renaming the wolf3d directory to w3d_plus? Originally I did this after adding recreated Super 3-D Noah's Ark DOS code to the Wolf3D/SOD tree. It still makes sense, given that it's a different game made by a different company; Not entirely sure about this as of now. Of course, if I do rename the directory, then other commits referencing w3d_plus (even if just in commit messages) should be updated accordingly.

- Alternatively, instead of changing the repository's history by removing the above commit, I can also add 1-2 new commits reverting the change.

- Rename gamesrc-ver-recreation? I know, this might seem a bit unexpected. Reason for suggesting this, is that this repository isn't exactly limited just to game code anymore; There are libraries, like the Build Engine, and I've also been working on a tool, the BUILD editor. Also, I occasionally think of this as a "reconstruction" rather than a "recreation", but the latter term is shorter and possibly considered more accurate. It's also the case that this repository is already known as "gamesrc-ver-recreation", so maybe its name should indeed remain unchanged.

- What to do with the older Mercurial repository, including Mercurial changeset IDs? Obviously, existing URLs directly linking to data in specific revisions of the repository will become broken, so I at least want to archive the changeset IDs in some way. I can re-upload the Mercurial repository's files (including hg-specific files) as a git repository, and/or upload a TXT file with a list of logs and IDs of the changesets.

Thanks again for any interest in a portion of the contents found in this repository.
3

User is offline   NY00123 

#28

Hi all,

I've gotten an important technical update:
- Following Bitbucket's planned removal of Mercurial support, I've converted the repository to git.
- Additionally, I've then split it into separate submodules, and set up a Bitbucket team for all of them (thanks to Blzut3 for the suggestions).

As a consequence, gamesrc-ver-recreation has just gotten a new URL: https://bitbucket.or...ver-recreation/

We'll see if the above migration has worked well, and especially if there's a difference in the convenience of using it, when compared to working with just one repository.

Notes for anybody trying to clone any of the submodules:

Spoiler


================

As for any added contents, there's now the addition of a tool, which is still gaming-related, but isn't exactly a game on its own. Basically, this is the Build Editor for Duke Nukem 3D.

This covers the editor as distributed with registered version 1.3d of Duke Nukem 3D, as well as the editor bundled with Duke Nukem 3D: Atomic Edition v1.4-1.5 (1.4's editor was reused for 1.5).
As a small bonus, the Build Engine objects as used in the editor for Shadow Warrior v1.2 are also covered.

================

What can I say about the code recreation process, for the two Duke3D editor versions? Well, it was quite different for each of them.

For the ones not aware, like the rest of the Build Engine, most of the Build editor code was originally offered to developers in a binary BUILD.OBJ file, with no source code.
However, they had the option to extend its functionality, with the assistance of hooks present in a file named BSTUB.C.

For Duke Nukem 3D, this file was renamed ASTUB.C. The Shadow Warrior sources have FMSTUB.C and JNSTUB.C, although it was eventually the latter which was used for the editor as distributed with SW 1.2.

So basically, there were two components to take care of: ASTUB.C, and the Build Engine code.
After the Duke3D sources were GPLed, a version of ASTUB.C was released under the same terms. I suspected that it matched v1.3d, but I first wanted to recreate 1.4/1.5's editor. As expected, there was a bunch of code to add, rather than delete (dealing with additional enemies introduced in 1.4 was just one example).

- At this point, only the ASTUB.C:Keys3d function is not 100% matching the original in terms of layout, but it should be equivalent in behaviors from what I recall.
- Fun fact: It actually turned out that certain bits of Duke3D's ASTUB.C code were later used in SW's JNSTUB.C. This clearly helped me, especially with the Keys3d function.
- In terms of Build Engine code, I initially assumed that it matched Duke3D v1.4. While A.OBJ and CACHE1D.OBJ are matching, it actually turned out that the editor uses a somewhat earlier revision of ENGINE.OBJ. I also had to update the code for BUILD.OBJ, which was expected (it was originally matching just Ken-Build as released in 2000, including a minor update from 2002).

I then went on to take care of v1.3d's editor. I'm reminding about the guess that the GPLed ASTUB.C file is matching it. As it eventually turned out, except for the version string being "DUKE NUKEM BUILD: V032696" instead of "DUKE NUKEM BUILD: V041996", as well as changes I assume that Charlie did for compatibility with Duke3D v1.5's engine for most, it was indeed perfectly matching v1.3d.

- Unlike v1.4, almost all work done for Duke3D v1.3d's editor was in the engine code, with very little done in ASTUB.C itself.
- Portions of code from a 1995 revision of ENGINE.C released on May 2019 were used (https://forums.duke4...es-and-sources/).
- Thanks again to Nuke.YKT, he already got a great deal (in fact, most?) of the required work on v1.3d's engine done. He was originally doing this for Redneck Rampage, with the initial assumption that its engine was at least earlier than Duke3D v1.4's in some way, before figuring out that it's actually matching 1.5 (also used by me in the same repository later).
- A few ENGINE.OBJ functions still don't perfectly match, but it should be quite close now, and the file mostly matches anyway (with a bit of hacks covering assembly pragmas, so a couple of functions are at least a bit closer in layout, and match in size).

Also, this time, there were instances in which I used a combination of a few files, like a script and a (variant of a) makefile. What for? Well, instead of manually guess, say, what should be the order in which the local variables of a certain function are defined, I could kinda brute-force this via scripting. A few more details:
Spoiler


Before this is forgotten, what about Shadow Warrior v1.2's Build Editor again? Well:
- All of the Build Engine objects are covered. In particular, A.OBJ is the same as used in Duke3D v1.4-1.5 and their editor, CACHE1D.OBJ is the same as in Duke3D v1.5, and BUILD.OBJ is currently identified as the 19961213 revision. The version found in the Ken-Build sources as released on 2000 seems to be virtually identical. It think that the couple of changes from the 10/4/97 entry of BUILD2.TXT are the only differences. This comes to increasing MAXTILES from 6144 to 9216 for Xatrix, as well as adding "#pragma push" and "#pragma pop" to BUILD.H, reason being the default struct packing was changed from 1 to 8 with the move to Watcom 11.0.
- For the SW-specific additions, see the JNSTUB.C file in the SW sources as re-released on Feb 2019 (REGCODE subdir): https://gitlab.com/N...EGCODE/JNSTUB.C

This post has been edited by NY00123: 26 February 2020 - 02:01 PM

5

User is offline   MrFlibble 

#29

I just had a random thought. A few days ago I learned about a project called FastDoom, a DOS source port of Doom that aims to optimise the game for maximum speed.

Without getting carried away too much, how realistic would it be to make optimisations to the restored code of later games, e.g. Hexen or Duke3D, to attain better performance on low-end machines/emulators (for example, browser implementation of DOSBox) without compromising quality in the same was as switching to low detail/decreasing screen size does? Is it at all possible?
0

User is offline   K1n9_Duk3 

#30

DOSBox is a bad example in this context, since it is in no way authentic when it comes to the timing of the CPU instructions. Code that's fast on real 386 or 486 CPUs might be slower in DOSBox, while code that's fast in DOSBox might be slower on real CPUs.
3

Share this topic:


  • 3 Pages +
  • 1
  • 2
  • 3
  • You cannot start a new topic
  • You cannot reply to this topic


All copyrights and trademarks not owned by Voidpoint, LLC are the sole property of their respective owners. Play Ion Fury! ;) © Voidpoint, LLC

Enter your sign in name and password


Sign in options