Release process

Meta release process: put three dashes (---) at the very top of this file, then start copying instructions from below as you update and run them. (Try to always update them first in here and then only run code you copy from here in the terminal, not vice versa.) When the release is done, remove the section about the old release.

Release process for ceylon.formatter 1.3.3:

We start off on the version-1.3.3 that @FroMage already created for us (including the version bump) and cherry-pick some commits that happened on master but need to be in 1.3.3 as well (because the branch was created way before the release was ready – I wasn’t aware it was coming up). Add the bootstrap script:

ceylon bootstrap
git add ceylonb ceylonb.bat .ceylon/bootstrap/
git ci -m 'Add ceylon bootstrap'

Build the formatter, and test that the source code is formatted properly.

ant clean compile
ceylon run ceylon.formatter/1.3.3 source/
git status # check that there are no changes

In this case, it wasn’t, so I created a commit to fix the errors, rebased it below the ceylon bootstrap commit, and cherry-picked it to master.

Choose a timestamp: date +%s, round it a bit, check with date -d @<rounded timestamp> that it’s still plausible.

Copy the reproducible build script from ceylon.formatter’s 1.3.3 tag. Update version and timestamp.

Clone a new Debian Jessie container for all Ceylon 1.3.3 releases.

sudo machinectl clone Debian-Jessie-{Base,ceylondeps-1.3.3}

Do an APT update, because that’s not part of the build script and my Debian-Jessie-Base image is so old that it still relies on the now discontinued httpredir.debian.org. (This should also be a part of future releases. Always use up-to-date software.)

sudo systemd-nspawn -M Debian-Jessie-ceylondeps-1.3.3 -- apt-get update

Edit the build script to add an exit just before the “build project” step, and run it in the container.

sed -i '/^# build project$/ i exit' build
time sudo systemd-nspawn -M Debian-Jessie-ceylondeps-1.3.3 < build

That image took quite some time to create, so let’s make sure we don’t accidentally taint it.

sudo machinectl read-only Debian-Jessie-ceylondeps-1.3.3 true

Clone the container again into a persistent container for the release.

sudo machinectl clone Debian-Jessie-{ceylondeps,formatter}-1.3.3

Edit the build script to move the “exit” above the “compare checksums” step, and to add a version- qualifier to the git checkout line, since the real tag hasn’t been created yet. Run the script in the container.

sed -i '/git checkout/ s/$projectVersion/version-&/; /^exit$/d; /^# compare checksums$/ i exit' build
time sudo systemd-nspawn -M Debian-Jessie-formatter-1.3.3 /bin/bash < build

Enter the container.

sudo systemd-nspawn -M Debian-Jessie-formatter-1.3.3

Build the formatter documentation and the plugin.

cd /opt/ceylon.formatter/
ceylon doc --source-code ceylon.formatter/1.3.3
ceylon plugin pack ceylon.formatter
exit

Create a Herd upload. Upload everything with ceylon copy.

ceylon copy --out https://modules.ceylon-lang.org/uploads/105588/repo/ --user lucaswerkmeister --pass 'XXX' --all ceylon.formatter/1.3.3

Verify that the upload contains everything and has no warnings. Most importantly: do not forget the plugin script. Herd does not warn about that if it’s missing. (See ceylon/ceylon-herd#253.) To check for encoding issues, look at e. g. the InvariantToken documentation (expand the initializer parameters and search for “␣”).

Edit the build script to change the repo URL to the upload’s public URL, and remove the exit. Run it in an ephemeral version of the container.

sed -i $'/^exit$/d\n /repo=/ {\n s/^/# /\n a repo=https://modules.ceylon-lang.org/uploads/105588/public/repo/ \n }' build
time sudo systemd-nspawn --ephemeral -M Debian-Jessie-ceylondeps-1.3.3 /bin/bash < build

Rejoice that it prints two identical checksums at the end.

Enter the persistent container again.

sudo systemd-nspawn -M Debian-Jessie-formatter-1.3.3

Set up the locale.

apt install locales
dpkg-reconfigure locales # enable en_US.UTF-8 (as of this writing: 146 on Jessie, 152 on Stretch)
export LANG=en_US.UTF-8

Run the tests, using the uploaded module.

cd /opt/ceylon.formatter
rm -rf modules
ceylon compile --rep https://modules.ceylon-lang.org/uploads/105588/public/repo/ test.ceylon.formatter
ceylon test --rep https://modules.ceylon-lang.org/uploads/105588/public/repo/ --rep modules test.ceylon.formatter
ls modules # check that the only folder inside is test, i. e., the ceylon.formatter module really came from the Herd upload

Publish the upload.

Run the tests again, this time from Herd.

rm -rf ~/.ceylon/cache
ceylon test test.ceylon.formatter

Edit the build script to change the repo URL back to Herd. Run it again:

sed -i '/^repo=/d; s/^# repo=/repo=/' build
time sudo systemd-nspawn --ephemeral -M Debian-Jessie-ceylondeps-1.3.3 /bin/bash < build

Rejoice that the checksums are still identical.

Remove the version- qualifier from the git checkout line in the build script. Write and create the tag. (Make sure you’re on the right branch for the tag!)

sed -i '/git checkout/ s/version-//' build
{ curl https://raw.githubusercontent.com/wiki/ceylon/ceylon.formatter/CHANGES.md; printf '\n\nThe `.car` file for this release has been made reproducible\nusing a fork of Debian’s `strip-nondeterminism` program.\nThe following script, on a fresh Debian Jessie GNU/Linux system,\nshould produce the exact same binary:\n\n'; sed 's/^/    /' build; } >| CHANGES
git tag -s -F CHANGES 1.3.3

Verify one last time that the reproducible build works on a completely fresh Debian system. Since the tag hasn’t been pushed yet, we re-add the version- qualifier in sed. (We also add the apt-get update.)

{ printf 'apt-get update\n'; git show --no-patch --pretty=format:%m 1.3.3 | sed -n '/#!/,/exit 0/ { s/^    //; s/checkout /checkout version-/; p; }'; } | sudo systemd-nspawn --ephemeral -M Debian-Jessie-Base /bin/bash

Push the tag. Test the ultra-condensed “tweet” version of the reproducible build script:

mkdir deb
sudo debootstrap jessie deb
git show 1.3.3 | sed -n '/#!/,/exit 0/p' | sudo systemd-nspawn -D deb /bin/bash

Make sure you use the right Debian version here. I wasted half a day after getting a different hash because I did that very last build on Stretch instead of Jessie by accident (different Java version).

Go to the GitHub release page and edit the tag to turn the message into proper Markdown (mostly: undo line breaking).

Download the release files from Herd:

wget https://modules.ceylon-lang.org/repo/1/ceylon/formatter/1.3.3/ceylon.formatter-1.3.3.{src,car,scripts.zip}{,.sha1}

Verify that you have 6 files:

find -type f | wc -l

Add them all to the GitHub release, in groups of about 3 files. (If you upload them all at once, you get an error, and you’ll have to delete them all again one by one. Very annoying.)

Create a 1.3.3 folder under /keybase/public/lucaswerkmeister/ceylon.formatter/ and copy the files there.

Publish the GitHub release. Update the README. Tweet about the release once @ceylonlang has tweeted about the general release.