Forming a GYP WG

[refack]

Follow up to #2

I suggest we form a GYP WG, and I volunteer to be the first member :).
The main goal of the WG will be to replace the GYP tool thus removing the python requirement from native modules (nodejs/node-gyp#629).
As an interim solution I suggest we fork GYP into the org, and minimally maintain it in a centralized, and managed manor (in #2 @rvagg suggests "We could consider taking over GYP from Chromium, either officially or by forking it" I'll try to talk with them about a hand off. If anyone can help as well?).

Reasoning

• It seems like the de-facto situation for the foreseeable future, is that we will continue to use .gyp files to scaffold our build system, and to build native add-ons.
• IMHO for the time being we are stuck with the GYP tool (until @indutny's gyp.js matures 🤞)
• Since GYP has been almost abandoned by chromium (at present there are 3 google stakeholders in GYP), pushing changes upstream is very slow.
• So we are floating two sets of patches in node-gyp and in node/tools (and a secret one in deps/npm)
• We don't explicitly regression test these patch sets:
  1. In node it just has to work, but sometimes regressions creep in use cases that are not covered by the CI (ninja build failure on master node#12448)
  2. node-gyp has a test-suite but it mainly tests the JS wrapper around GYP so again sometimes regressions creep in (Weird error MSB6006: "cmd.exe" exited with code 1 on some packages node-gyp#1151)
• We don't have a knowledgebase for how to write good .gyp files. Institutional knowledge is highly fragmented in comments and issue threads...

[refack]

For the last couple of months I had a chance to; dig in the actual GYP code, help @indutny with gyp.js, and try to improve some of our .gyp files. I've managed to push some changes upstream, and was able to get gyp.js to build node on windows.
I feel confident that with the help of the people of the org we can remove some of the obstacles GYP has put in our way.

[refack]

/cc @nodejs/collaborators

[vkurchatkin]

What's the point of forking if complete rewrite is the goal?

[refack]

Ok, you're right, I changed the title.
Forking GYP is an interim solution, since we still have multiple issues with it.

[refack]

P.S. we already have a fork. Actual two node-gyp and in node/tools (and a secret one in deps/npm).
But IMHO we don't manage them as well as we should.

[bnoordhuis]

Agreed. I've stated elsewhere that when (not if) we take over maintenance of GYP, we should strip it down to its core. The cmake generator, xcode emulation, old Visual Studio support, etc. - those can all probably go.

I don't think a separate Working Group is necessary (or desirable) but a separate repo and a team would make sense.

[jbergstroem]

Perhaps the working team can cover not just maintaining old gyp but also help push the gyp.js "compatibility layer" as well as thinking about a format that might be more suitable than Gypfile?

[refack]

Perhaps the working team can cover not just maintaining old gyp but also help push the gyp.js "compatibility layer" as well as thinking about a format that might be more suitable than Gypfile?

Exactly!

[refack]

Regarding a handoff, I got a general "we're open to it" from Chromium.

[refack]

@bnoordhuis I assume if we do a handoff they would be against stripping it down ;)
Besides IMHO gyp.js is a better (and python-free) option.
@indutny do you think it's time / willing to move gyp.js into the fold?

[bnoordhuis]

Can't find the comment but remember how I said (half in jest, half serious) that node-sass and node-iconv are the litmus test for gyp.js? Until it is able to build most popular add-ons, including ones with complex build scripts, it won't be a replacement for the python version.

[refack]

I remember that nodejs/node#7440 (comment)
To which I answered nodejs/node#7440 (comment)

[refack]

nodejs/node#7440 might be stage #2 of the WG

[refack]

So Dirk Pranke who's in charge of GYP suggested that as a first step he will invite members of the org to become maintainers. Anyone who has GYP experience and wants to pitch in is invited to submit CLs (PRs in Chromium), and get fast tracked to become maintainer.
/cc @nodejs/build @nodejs/ctc
Past patchers @shigeki @jbergstroem @bnoordhuis @danbev
@joaocgreis can you think of anyone else?

[refack]

/cc @dpranke

[dpranke]

I said nothing about "fast tracked", but otherwise, sure we'd be happy to have folks contributing to GYP :).

[jasnell]

This is likely something that we can make sure is discussed at the collaborator summit next week in Berlin and raised in various other discussions. One important thing to figure out, however, would be a clear delineation of what we actually need here.

[Trott]

This issue has been inactive for a while and this repository is now obsolete. I'm going to close this, but feel free to open another issue in a relevant active repository (TSC perhaps?) and include a link back to this issue if this is a subject that should receive continued attention.