Request for Comment: Source Code Rearrangement

2 Pages
1
2
→

You cannot start a new topic
You cannot reply to this topic

Request for Comment: Source Code Rearrangement

Hendricks266

Weaponized Autism

#1 Posted 18 June 2016 - 10:15 AM

I am planning to rearrange the entire structure of source code in our Subversion repository. Through testing, I have confirmed that git blame and git log --follow understand commits where files have been moved with svn mv. Unfortunately the gitk GUI tool does not yet understand this, but the problem is strictly client-side.

With this in mind, I believe there would be no recurring cost to performing this clean up. There may be one-time costs, such as handling unversioned files and symlinks, and I would like to minimize these, but their existence does not dissuade me. Here are my plans.

The first change I would make is eliminating the two unnecessary folders in the root of the tree.
/polymer/eduke32/ --> /
If we ever have a serious need for maintaining multiple branches (like polymer-perf in the past), we should drop SVN and migrate to git on the server.

The rest is simple reorganizing:
/build/ --> /source/build/
/source/{*.[chm], android/, lunatic/} --> /source/duke3d/src/
/Makefile.deps --> /source/duke3d/
/source/gpl-2.0.txt --> /source/duke3d/
/rsrc/ --> /source/duke3d/rsrc/
/source/misc/ --> /source/duke3d/rsrc/
/source/jaudiolib/third-party/ --> /platform/
/source/jaudiolib/ --> /source/audiolib/
/source/jmact/ --> /source/mact/{src, include}/
/source/testgame/ --> /source/kenbuild/
/Makefile --> /GNUmakefile
/Makefile.common --> /Common.mak
/Makefile.msvc --> /platform/Windows/
/eduke32.{sln, vcxproj, vcxproj.filters} --> /platform/Windows/

Additionally, once I reconcile Lunatic with the C++ structure trackers and finish cleaning up Shadow Warrior, there is nothing preventing us from moving to C++, and C++11 at that. (C++11 will prevent building on old OS X versions, but it will still be possible to build on a modern Mac and run on as old an OS X version as we support, as long as we don't use the STL.)

This will cause the following changes in file extension:
.c --> .cpp
~~.h --> .hpp~~
.m --> .mm

I would like to request comment from anyone who works with the source code. I suspect icecoldduke, LeoD, and rhoenie may have input. End-users will not be affected, except in that happier developers make a better product.

This post has been edited by Hendricks266: 28 October 2016 - 02:53 PM

icecoldduke

#2 Posted 18 June 2016 - 10:46 AM

I'll have more time to go over this later on today, but going from c -> c++ is a great idea. I don't think you should do .h -> .hpp, .hpp isn't used any codebase I've ever seen.

Also is there a reason why you wouldn't use Perforce if your going to be moving from SVN? Perforce has a lot of great features that git or svn doesn't have. If you wan to host your own P4, its free if you have less then 20 users I think. If you want a offsite backup you can use assembla. Then you can have github be your public facing repository, since all you need it for is for the public to download code. I use assembla for PolymerNG and github for public facing code, works great. You can a bot migrate changelists as well for you.

This post has been edited by icecoldduke: 18 June 2016 - 10:50 AM

TerminX

el fundador

#3 Posted 18 June 2016 - 10:49 AM

We aren't moving away from svn.

Hendricks266

Weaponized Autism

#4 Posted 18 June 2016 - 10:51 AM

icecoldduke, on 18 June 2016 - 10:46 AM, said:

I don't think you should do .h -> .hpp, .hpp isn't used any codebase I've ever seen.

Sure it is. I support changing it because C++ code can include .h files that have corresponding .c files built as C, as long as the extern "C" is set up correctly. But a .c including a .hpp is a guaranteed compilation error.

icecoldduke, on 18 June 2016 - 10:46 AM, said:

Also is there a reason why you wouldn't use Perforce if your going to be moving from SVN?

We're not planning to move from SVN under current circumstances. But,

Quote

Perforce is a commercial, proprietary revision control system

That's all I need to hear.

Plagman

Former VP of Media Operations

#5 Posted 18 June 2016 - 10:55 AM

You'll have to walk me through the P4 features that git doesn't support sometime. I can't think of any, especially if we're only talking about the source tree.

icecoldduke

#6 Posted 18 June 2016 - 10:59 AM

Hendricks266, on 18 June 2016 - 10:51 AM, said:

Sure it is. I support changing it because C++ code can include .h files that have corresponding .c files built as C, as long as the extern "C" is set up correctly. But a .c including a .hpp is a guaranteed compilation error.

Sure but I have never seen that standard used in any codebase, which is why I would be against it.

Hendricks266, on 18 June 2016 - 10:51 AM, said:

That's all I need to hear.

I know you hate proprietary closed source solutions but...

Having a one click branch mapping that brings me up to date with whats in your SVN is fantastic. P4's branch mapping saves a whole bunch of time.

I'll have more opinions on the rest of it later on tonight.

This post has been edited by icecoldduke: 18 June 2016 - 11:01 AM

Hendricks266

Weaponized Autism

#7 Posted 18 June 2016 - 11:02 AM

icecoldduke, on 18 June 2016 - 10:59 AM, said:

Sure but I have never seen that standard used in any codebase, which is why I would be against it.

"I personally haven't used it so no one should use it."

https://www.google.com/search?q=".hpp"

icecoldduke, on 18 June 2016 - 10:59 AM, said:

Having a one click branch mapping that brings me up to date with whats in your SVN is fantastic. P4's branch mapping saves a whole bunch of time.

This isn't a reason to use Perforce on our upstream. Moreover, I don't know what that does that git rebase or git merge can't do, and "more easily" is subjective.

icecoldduke

#8 Posted 18 June 2016 - 11:19 AM

Plagman, on 18 June 2016 - 10:55 AM, said:

You'll have to walk me through the P4 features that git doesn't support sometime.

I like shelving better in P4 vs Git for example. I remember in Git it was a pain in the ass to view someone else's stash, then having another developer unshelve someone elses changes. I like having other developers on a project be able to pass off code/assets to other developers without actually committing anything. This also has the added benefit of keeping Perforce in the loop on local changes. I hate having people use reconcile. Perforce makes it really easy to shelve files, and have someone else unshelve those same files, I remember this wasn't a easy task in Git at one point.

That being said, I think Perforce now integrated git into Perforce so you don't really have to choose anymore, but this isn't the topic for "which source control program is better"

.

I'll have more critiques on the above post later on today, but other then the .h -> .hpp everything looks fine to me.

This post has been edited by icecoldduke: 18 June 2016 - 11:34 AM

Mblackwell

Evil Overlord

#9 Posted 18 June 2016 - 02:22 PM

I've used Git on large projects, and like many FLOSS tools it's easy to shoot yourself in the foot but if you know what you're doing it's quite powerful and flexible. I'd actually be okay with git over svn, but it really doesn't matter. Git is just faster and it's easier to test out changes on a separate and push out patch files. It's also easy to bisect when bugs crop in.

Perforce is just the new (old) hotness.

Edit: .h vs .hpp - if the project is migrating to C++ it might be a good idea to differentiate between C and C++ only headers until the conversion is complete at the very least.

icecoldduke

#10 Posted 18 June 2016 - 05:17 PM

Mblackwell, on 18 June 2016 - 02:22 PM, said:

Edit: .h vs .hpp - if the project is migrating to C++ it might be a good idea to differentiate between C and C++ only headers until the conversion is complete at the very least.

The code compiles just fine under C++, just the code itself is not coded with OOP yet. All PolymerNG code is all OOP.

This post has been edited by icecoldduke: 18 June 2016 - 05:18 PM

Mblackwell

Evil Overlord

#11 Posted 18 June 2016 - 05:54 PM

It's not about how it compiles, it's about knowing what is C++ only and what is C. The standard specifies that C++ only should use .hpp, and in this case it would make sense if the headers end up being modified so you can keep track of your progress (or can see the difference at a glance if things aren't modified) in addition to the adherence.

TheZombieKiller

#12 Posted 18 June 2016 - 07:12 PM

I definitely support the .hpp extension, I use it all the time in my projects. I often see .cc/.hh and .c++/.h++ used as well.
It may be worthwhile to note that the Boost library uses .hpp

This post has been edited by TheZombieKiller: 18 June 2016 - 07:23 PM

TerminX

el fundador

#13 Posted 18 June 2016 - 07:22 PM

Mblackwell, on 18 June 2016 - 05:54 PM, said:

The standard specifies that C++ only should use .hpp

There is no standard in this regard. More or less any string of valid ASCII characters can be used as the header filename.

I'm not sure whether I support renaming them all to .hpp or not. In cases where the header is unusable without C++ I think it makes sense, but in cases where the header defines a C interface (everything declared extern "C", etc) it's harder to agree with it.

Neither way is really "wrong" and this point probably merits further discussion before committing to doing it one way or the other.

icecoldduke

#14 Posted 18 June 2016 - 07:25 PM

TheZombieKiller, on 18 June 2016 - 07:12 PM, said:

It may be worthwhile to note that the Boost library uses .hpp

I will definitely be against boost in anyway shape or form going into eduke32.

TheZombieKiller, on 18 June 2016 - 07:12 PM, said:

I definitely support the .hpp extension, I use it all the time in my projects. I often see .cc/.hh and .c++/.h++ used as well.

I would also be very much against any use of .cc/.hh .c++ or .h++. That looks awful, and serves no practical purpose.

TerminX, on 18 June 2016 - 07:22 PM, said:

Neither way is really "wrong" and this point probably merits further discussion before committing to doing it one way or the other.

I would agree using .hpp or .h there is no right or wrong answer. I just think the only extension changing that has to be done is going from .c -> .cpp.

This post has been edited by icecoldduke: 18 June 2016 - 07:30 PM

TheZombieKiller

#15 Posted 18 June 2016 - 07:32 PM

icecoldduke, on 18 June 2016 - 07:25 PM, said:

I will definitely be against boost in anyway shape or form going into eduke32.
I would also be very much against any use of .cc/.hh .c++ or .h++. That looks awful, and serves no practical purpose.

I agree with 100% of the above, they were merely examples

(I have a strong hatred of .h++/.c++ in particular)

This post has been edited by TheZombieKiller: 18 June 2016 - 07:33 PM

TerminX

el fundador

#16 Posted 18 June 2016 - 07:35 PM

People really put the plus sign in their filenames like that? Barf.

TheZombieKiller

#17 Posted 18 June 2016 - 07:36 PM

Yes, and I cringe every time I see it
I have an aversion to seeing non-alphanumeric characters in filenames, especially source files

rhoenie

#18 Posted 18 June 2016 - 11:36 PM

Are you serious? The topic of this thread is "Source Code Rearrangement" and so far 90% of the thread are discussions about filename suffixes and the usual talk about which tool is the best.

Anyway, I really like the new directory structure - something that could have been done long ago.

What about those 3rd party platform libraries that ship as binaries within the source tree? I don't see much reason for including them - IMHO anyone that knows how to compile stuff also knows how to install dependancy packages.

Striker

Auramancer

#19 Posted 19 June 2016 - 01:02 AM

Why stick with SVN when GIT is vastly superior? (At least, from my experience)

This post has been edited by StrikerMan780: 19 June 2016 - 01:04 AM

The Mechanic

#20 Posted 19 June 2016 - 01:02 AM

rhoenie, on 18 June 2016 - 11:36 PM, said:

Because its darn useful to be able to download a project that compiles out-of-the-box without having to faff around searching for versions of libraries, or the right version, etc.

rhoenie, on 18 June 2016 - 11:36 PM, said:

Are you serious? The topic of this thread is "Source Code Rearrangement" and so far 90% of the thread are discussions about filename suffixes and the usual talk about which tool is the best.

Thats because in programming THE most contentious issues for a given language are naming conventions (and where to indent curly braces).

Must admt I only ever used .h but for no other reason than never really thought about it ... but can see the logic for .hpp (or even .h++ for those who are products of in-breading), particularly where c++ and c are getting mixed up in the same project.

TTFN,
Jon

icecoldduke

#21 Posted 19 June 2016 - 06:50 AM

rhoenie, on 18 June 2016 - 11:36 PM, said:

There is no reason to remove them either.

Hendricks266

Weaponized Autism

#22 Posted 19 June 2016 - 11:25 AM

TerminX, on 18 June 2016 - 07:22 PM, said:

I'm not sure whether I support renaming them all to .hpp or not. In cases where the header is unusable without C++ I think it makes sense, but in cases where the header defines a C interface (everything declared extern "C", etc) it's harder to agree with it.

Neither way is really "wrong" and this point probably merits further discussion before committing to doing it one way or the other.

As soon as I use constexpr or templates in compat.h, C support is over.

rhoenie, on 18 June 2016 - 11:36 PM, said:

Are you serious? The topic of this thread is "Source Code Rearrangement" and so far 90% of the thread are discussions about filename suffixes and the usual talk about which tool is the best.

Anyway, I really like the new directory structure - something that could have been done long ago.

Yeah I would have liked to get more feedback about the folder structure. You know, the reason I made this thread.

The prebuilt libs are Windows and Wii only since package managers are largely absent there.

StrikerMan780, on 19 June 2016 - 01:02 AM, said:

Why stick with SVN when GIT is vastly superior? (At least, from my experience)

We don't need git for our use cases yet. SVN is easier to grasp by the common man due to sequential revision numbering.

LeoD

Duke4.net topic/3513

#23 Posted 19 June 2016 - 05:57 PM

Hendricks266, on 18 June 2016 - 10:15 AM, said:

I would like to request comment from anyone who works with the source code. I suspect icecoldduke, LeoD, and rhoenie may have input.

I don't work with the code, I just stick my nose into other people's stuff.

But now that I've been named:

Hendricks266, on 18 June 2016 - 10:15 AM, said:

/Makefile.deps --> /source/duke3d/
/Makefile --> /GNUmakefile
/Makefile.common --> /Common.mak
/Makefile.msvc --> /platform/Windows/

Does GNU Make recognize GNUmakefile automatically? If yes, then OK. Common.mak looks weird Even if .mak may be a common extension (dunno), it's still not consistent with the other Makefile.* in the project.

Hendricks266, on 18 June 2016 - 10:15 AM, said:

The first change I would make is eliminating the two unnecessary folders in the root of the tree.
/polymer/eduke32/ --> /

Related side note: is it possible to rename the polymer_hrp repo to duke3d_hrp without forcing a re-download of its contents and thereby doubling the user's used diskspace for the local repo copy?

Hendricks266, on 18 June 2016 - 10:15 AM, said:

If we ever have a serious need for maintaining multiple branches (like polymer-perf in the past), we should drop SVN and migrate to git on the server.

Actually, we've been in serious need of stable/dev branches for more than two years.

But if you keep on fixing bugs like recently, this may become less urgent.

This post has been edited by LeoD: 19 June 2016 - 05:57 PM

Hendricks266

Weaponized Autism

#24 Posted 19 June 2016 - 06:27 PM

LeoD, on 19 June 2016 - 05:57 PM, said:

Does GNU Make recognize GNUmakefile automatically?

Yes, and in fact the point of this is so that GNU Make recognizes it and BSD Make does not. (BSD users must run gmake instead of make.)

LeoD, on 19 June 2016 - 05:57 PM, said:

Common.mak looks weird Even if .mak may be a common extension (dunno), it's still not consistent with the other Makefile.* in the project.

Good point, and consider all Makefile.deps --> Dependencies.mak added to the list. Probably Makefile.msvc --> msvc.mak too.

I like the look of the .mk extension better, as used by the Android NDK, but I think .mak has been around longer. Notepad++ currently only applies Makefile syntax highlighting to .mak, not .mk. That can be solved, of course.

LeoD, on 19 June 2016 - 05:57 PM, said:

It might be as simple as moving the repo and creating a symlink to the old location.

LeoD, on 19 June 2016 - 05:57 PM, said:

Actually, we've been in serious need of stable/dev branches for more than two years.

But if you keep on fixing bugs like recently, this may become less urgent.

Our development is not formal enough to warrant this, and I am a firm believer in continuous integration. (see: NW+/Vaca+ svn-only downloads) That way, improvements are distributed as fast as possible, and issues that arise are discovered similarly.

Unfortunately, it's on us to actually deal with the issues.

Player Lin

#25 Posted 21 June 2016 - 02:49 AM

Hendricks266, on 19 June 2016 - 11:25 AM, said:

We don't need git for our use cases yet. SVN is easier to grasp by the common man due to sequential revision numbering.

Unless eduke32 devs want to put their own sequential revision numbering for those eduke32 synthesis builds by theirself(or like this auto building site did). Otherwise with Git, only those ass long and no-meaningful of hash values can be distinguish the version number of commits. And for normal users, it can be confusing as hell.

This post has been edited by Player Lin: 21 June 2016 - 02:53 AM

icecoldduke

#26 Posted 23 June 2016 - 04:35 AM

Hendricks266, on 19 June 2016 - 11:25 AM, said:

We don't need git for our use cases yet. SVN is easier to grasp by the common man due to sequential revision numbering.

Perforce has sequential revision numbering

This post has been edited by icecoldduke: 23 June 2016 - 04:35 AM

Mblackwell

Evil Overlord

#27 Posted 23 June 2016 - 06:01 AM

You can do it with tagging in Git.

Vaporeon

#28 Posted 26 August 2016 - 07:31 AM

Hendricks266, on 19 June 2016 - 11:25 AM, said:

We don't need git for our use cases yet. SVN is easier to grasp by the common man due to sequential revision numbering.

If you are referring to the synthesis builds you can always do 'git rev-list --all --count'

This post has been edited by Vaporeon: 26 August 2016 - 07:31 AM

Striker

Auramancer

#29 Posted 03 September 2016 - 01:30 PM

Vaporeon, on 26 August 2016 - 07:31 AM, said:

If you are referring to the synthesis builds you can always do 'git rev-list --all --count'

This is what I do for my projects. It gives you a revision number identical to how SVN does it.

This is how I handle it in batch:

for /f "delims=" %%i in ('git rev-list HEAD --count') do set REVISIONNUMBER=%%i

If you know Bash scripts well, the same can be done in Linux.

Every commit bumps the number up by one.

This post has been edited by Striker: 03 September 2016 - 02:20 PM

Striker

Auramancer

#30 Posted 20 December 2016 - 06:20 PM

To add to my last point, here's an excellent SVN to Git migration guide, that'll keep all of the old commits and tie SVN users to the correct github / bitbucket / etc account e-mails.

https://gist.github....7313e0a4ad086c3

Moving to Github or Bitbucket would have the huge benefit of allowing people who want to contribute to EDuke32's development to create forks, which can be reviewed and merged into the main source tree with a simple pull request. Beats the hell out of .diff patches. It would also allow in-progress forks to easily synchronize with the main source tree. (Thus staying up to date with the latest EDuke32 version.)

This post has been edited by Striker: 20 December 2016 - 07:08 PM