ci(gh-actions): Add initial CI pipeline config

[PetarKirov]

No description provided.

[dukc]

I have absolutely no experience or knowledge from the CI yml so no insights on that.

Definitely wanted to have the CI though, it's maybe the most important thing that is missing from OldDDerivations.

[dukc]

I suspect you're trying to swallow more in one go than you want to. For example, I don't think it's a good idea to include LDC bootstrap update to 1.32.1 here. Instead, I'd just get the CI pipeline going, merge this, and then do those other tasks in separate direct commits or pull requests.

[dukc]

(reviewed a single commit)

Looks good in general, although I have no experience about CI and thus am not really qualified to stamp this.

[dukc]

(reviewing a single commit, "Define the package for more platforms")

All good here!

[dukc]

(Reviewing commit "Include dmd package only if the hostPlatform is x86")

LGTM.

[dukc]

(reviewing "Remove nixpkgs fallback")

I suspect that better not. Flakes still are not the default, not everybody is necessarily using them.

If this was the package itself, I'd agree regardless: the user should declare from the terminal what channel is to be used and if he/she forgets, better tell. However, my understanding is that shell.nix is supposed to be an easy-to-use, quick-and-dirty convenience tool rather than the scalable generic build description Nix files in general are. Thus an easy-to-use default makes sense in them even if it breaks the purity ideals we usually follow in Nix.

[dukc]

(reviewing commit "Use pkgs instance resulting from applying the flake overlay")

I'd think this makes no difference, with the new code working like the old one, but I have a feeling I'm missing something - if so, LGTM.

[dukc]

(Reviewing "Fix the dmd runnable_cxx parts of the checkPhase by using GCC11")

There is a commit made later on in DMD that changes the compatibility to 12 instead, so this will probably break with newer DMD versions. Hence I prefer my solution in #4 where I disable this test for versions where that change isn't yet included.

Of course we could pick GCC based on version, but IMO such complexity is an overkill to just enable one test for old versions.

[dukc]

(Reviewing "Expose {dmd,ldc}-binary packages")

I'm on fence about this. Exposing binary packages, in itself, is an idea I'm receptive to. However, this is conflating binary package with the bootstrap package. Yes, the bootstrap packages are binary packages, but the term "binary" doesn't indicate in any way what version you get, so I'd expect that:

• I can pick the version myself if I provide hashes, which would imply the binary.nix files instead.
• If there is a default, it'd be the same as the source-built version.

[dukc]

(Reviewing "Remove flake-utils")

I frankly don't know what flake-utils and flake-parts are about, so really I'm unqualified to do this review but I see nothing suspicious here.

[dukc]

It's getting late so I have to review the rest another day.

[PetarKirov]

(reviewing "Remove nixpkgs fallback")

I suspect that better not. Flakes still are not the default, not everybody is necessarily using them.

I've been living in an alternative universe where flakes are not only stable but the only way to use Nix 😄
Joke aside (well not really a joke, we've been using them in every "production" project at work for the past 2+ years) and so far we have no reason to consider going back to the pre-flake nix channel world; their "experimental" nature doesn't bother us in the slightest), I have two points:

• This change is only about shell.nix which useful only for developers working on this project, so this change is not user-facing. I did it because by mistake I entered the shell with nix-shell rather than nix develop and I got wildly different package versions, so this change is a precaution to ensure the dev environment is reproducible.
• Since I'm using Nix in a flake-only mode, for now I consider this project flake-only as well. This doesn't mean that I'm against non-flake usage, but it's simply an aspect that I haven't tested and I would have add additional CI configuration to guarantee it works. For now the CI only tests the flake interface. If non-flake usage is important to you, I suggest we dedicate a separate PR where we would specific tests and document the usage.

If this was the package itself, I'd agree regardless: the user should declare from the terminal what channel is to be used and if he/she forgets, better tell. However, my understanding is that shell.nix is supposed to be an easy-to-use, quick-and-dirty convenience tool rather than the scalable generic build description Nix files in general are. Thus an easy-to-use default makes sense in them even if it breaks the purity ideals we usually follow in Nix.

As mentioned above, I did this because I was bitten by using packages from a stale nix channel. I suggest you try out the nix develop + direnv combo - it works great!

[PetarKirov]

(reviewing commit "Use pkgs instance resulting from applying the flake overlay")

I'd think this makes no difference, with the new code working like the old one, but I have a feeling I'm missing something - if so, LGTM.

Before this change, the pkgs instance passed to the perSystem function for us by Flake Parts was an instance of nixpkgs without any overlays applied, so when using dmd, ldc, etc. in shell.nix they would be the exact same version as in nixpkgs upstream.

To understand the what's going on in the flake.nix outputs I suggest you check the docs of the Flake Parts library that we're using. Specifically, we're using its Easy Overlay feature, where it creates an overlay function out of the flake packages outputs and then it applies this overlay to nixpkgs and passes it to us as the final parameter in perSystem.

• https://flake.parts/overlays
• https://flake.parts/options/flake-parts-easyoverlay

This has the effect of making the pkgs defined in this repo directly accessible in its dev shell.

[PetarKirov]

(reviewing commit "Split packages list based on the platform")

This change in itself is good.

However, an urelated notice: for me it seems this default.nix is relying on flakes being enabled. Is this a good idea?

Yes, you are correct that pkgs/default.nix is usable only through flake.nix and this is intentional, as this repo so far is currently designed with flake-only usage in mind.
That said, this is not a blocker for non-flake usage, as we could easily support this scenario via https:/edolstra/flake-compat, which allows non-flake users to use all everything exposed by the flake through a top-level default.nix file (not to be confused with pkgs/default.nix), so we wouldn't need to make changes to structure.

[PetarKirov]

Reviewing "Expose {dmd,ldc}-binary packages")

I'm on fence about this. Exposing binary packages, in itself, is an idea I'm receptive to. However, this is conflating binary package with the bootstrap package. Yes, the bootstrap packages are binary packages, but the term "binary" doesn't indicate in any way what version you get [..]

Agreed. I plan to redo the flake output structure completely to allow the user to pick any binary or source version of the compiler packages and to allow the user manually specify hashes (e.g. I want dmd from this PR branch). For now the flake output structure is a placeholder just to show what works.

[PetarKirov]

It's getting late so I have to review the rest another day.

Thanks for the review so far! I myself am traveling this past week (without a computer nearby), so that's why I've been so slow. Although there are many things to improve, I feel that it's better for me to go ahead and merge this PR, so I can proceed with rebasing and merging yours.

I'd appreciate if you could finish reviewing this PR and if you find any issue that need addressing (even pure questions) file issue(s) for them, so they're not forgotten.

[dukc]

* This change is only about `shell.nix` which useful only for developers working on this project, so this change is not user-facing. I did it because by mistake I entered the shell with `nix-shell` rather than `nix develop` and I got wildly different package versions, so this change is a precaution to ensure the dev environment is reproducible.

Good point. Since I and anyone else who wants to develop this should figure out flakes anyway, it's reasonable to demand developers have flakes enabled.

* Since I'm using Nix in a flake-only mode, for now I consider this project flake-only as well. This doesn't mean that I'm against non-flake usage, but it's simply an aspect that I haven't tested and I would have add additional CI configuration to guarantee it works. For now the CI only tests the flake interface. If non-flake usage is important to you, I suggest we dedicate a separate PR where we would specific tests and document the usage.

It's enough that the packages themselves - dmd, ldc and dub - can be built without flakes. Our update scripts and the CI using them is okay.

As mentioned above, I did this because I was bitten by using packages from a stale nix channel. I suggest you try out the nix develop + direnv combo - it works great!

Sounds worth checking!

I personally have been still using channels, trying to avoid that problem by simply avoiding impurity. I don't import <anything> in the .nix files, instead I list dependencies just like in NixPkgs and build like nix-build -E 'with import <pick-your-channel> {}; callPackage ./the-package.nix {inherit (darwin.apple_sdk.frameworks) Foundation;}' I think this already works reasonably well, but there must be a reason why flakes were developed so probably they're even better.

[dukc]

Reviewing "Set CC env variable for both the build and the check phase".

Looks okay.

[dukc]

Reviewing "config(flake.nix): Add Cachix binary cache settings"

These prebuilds still go through hash checks, no? If so, the more binary mirror sites the merrier!

[dukc]

Reviewing "Allow skipping missing binary packages".

Makes sense. It's not the sort of thing that would cause bigger harm down the road if some binaries didn't get laoded.

[dukc]

Reviewing "Upgrade to 1.32.1"

If you were updating DMD bootstrapper, I'd protest since it'd be likely to break older DMD versions I haven't yet ported. But this is LDC, where we have no older versions to break, so no problem.

[dukc]

Reviewing "Add D-based implementation of fetch_binary script".

I'd prefer this file had @safe attributes added, but for a script it makes no big difference. Nothing worth blocking here.

[dukc]

We don't want to keep these around long, since they will be continuosly getting out of date. But I'm pretty sure you're planning on replacing them with something that reads the newest version we support from a flake, or something like that.

[dukc]

Maybe "Help texts not written yet, sorry."?

[dukc]

Reviewing "Disable failing macos tests".

#4 is removing cppa anyway for versions before 2.103.0 (and I think 2.102.2 is the newest one we currently support), so that file can probably be removed from here. Most likely it'll work on Darwing too from 2.103 on, and if not we can add that back.

Otherwise ok.

[dukc]

Reviewing "Upgrade to 1.34.0"

Same comment as with "Update to 1.32.1", in other words, ok! Plus FreeBSD hash is there again.

[dukc]

Reviewing "Allow ldc on macos to fail"

That you have to write two tables as opposed to one to turn "experimental" on for one complier/platform combination feels a bit kludgy for me, but this is a small configuration file so probably not worth it to hunt for a more scalable solution. Looks ok.

[dukc]

Every commit is reviewed unless I missed something.