Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare a new release and document the process #4496

Closed
laarmen opened this issue Nov 26, 2018 · 68 comments
Closed

Prepare a new release and document the process #4496

laarmen opened this issue Nov 26, 2018 · 68 comments

Comments

@laarmen
Copy link
Contributor

laarmen commented Nov 26, 2018

It's been a while since we've had a new release, and that's a shame. However, our build system has evolved quite a bit, and I've basically been told that we don't know how to do releases anymore.

The process might simply be "tag a commit, wait for defined $people to build on their respective platforms, upload the builds, then do the whole website/twitter/stuff dance", but it still has to be documented.

Oh and I'm not volunteering.

@fluffyfreak
Copy link
Contributor

Few reasons we haven't done this, I've been waiting on #4427 to be finished but @pcercuei seems to have been away a lot. It's a real shame as I think it'll be a great addition to the build process.

The CMake stuff has taken a while to work but I think that's now ready. I haven't been following along too closely since I want it to work on Linux (and OSX?) before I even try to use it with Visual Studio.

I've got a branch of #4427 and have just tried to update it to master, so I'll give that a go and see if it creates a usable windows package after merging latest into it.

@pcercuei
Copy link
Contributor

I'm still here, just travelling around without a computer. I will be back next week.

@fluffyfreak
Copy link
Contributor

I have a fork of the branch with latest code, some hacking around in it but it does produce an installer now. Hopefully I'll test tomorrow.

Hope you're having fun @pcercuei, I look forward to you getting back us getting this integrated :)

@impaktor
Copy link
Member

Do remind me:

  1. OSX builds still depend on @salborrelli or @Philbywhizz building and uploading, right?

  2. And if we use appveyor for windows builds and github for GNU/Linux build, only OSX will be hosted on sourceforge?

@fluffyfreak
Copy link
Contributor

Sort of:

  1. OSX has been @Philbywhizz the last few times.
  2. Appveyor we'd only use for building not as host, it auto deletes after 6 months and there's a 1GB size limit. Each build is ~330MB for Windows.

@fluffyfreak
Copy link
Contributor

Damn. My installer doesn't work. It puts the data directory too deep inside another data directory. Also missing the icon for the executable for some reason.

@pcercuei
Copy link
Contributor

@fluffyfreak: this is with #4427 rebased on top of master?

@fluffyfreak
Copy link
Contributor

@pcercuei that's with my own branch which has some other hacking in it. So rebasing atop #4427 might be better

@abbottjord94
Copy link

Genuine noob question: If I'm able to build from source on a given platform and pack it into a .7z file (and document the whole process), is that acceptable?

I ask because I can build for Windows and Linux (without CMake though), and I'd be happy to provide both of them if that's what we're looking for.

@impaktor
Copy link
Member

@abbottjord94 Thanks for the offer, but no. We can also build on linux and windows, after all, that's how we do our development. The idea is to get it automated.

@pcercuei
Copy link
Contributor

pcercuei commented Dec 1, 2018

Making the Windows release should be a matter of publishing the setup.exe built by AppVeyor once PR #4427 is updated and merged.

For Linux, the JSON and appdata files at https:/flathub/net.pioneerspacesim.Pioneer should be updated, then the software appstores of (recent) Linux distributions will provide the update. Additionally a .flatpakref (~10-lines text) file can be provided for download on Pioneer's main page, which will open with the distribution's software appstore. The actual build would be hosted for free on Flathub.

For OSX... I don't know. I have experience with creating OSX bundles in CMake, maybe we could setup Travis to build Pioneer just like we will do on Windows with Appveyor. Then the release process would be the same as for Windows, just download and publish a build artifact.

@laarmen
Copy link
Contributor Author

laarmen commented Dec 1, 2018 via email

@pcercuei
Copy link
Contributor

pcercuei commented Dec 2, 2018

It's not out of the box on all distributions (yet), but the page on Flathub to download Pioneer (https://flathub.org/apps/details/net.pioneerspacesim.Pioneer) already links to a setup guide for your distribution.

@laarmen
Copy link
Contributor Author

laarmen commented Dec 2, 2018 via email

@pcercuei
Copy link
Contributor

pcercuei commented Dec 2, 2018

Actually you can install flatpaks as user, from the command line at least (from the GUIs I think they are installed system-wide). Note that we can very well release as a tarball as well, and on the website, just notify that a flatpak version is also available.

@impaktor
Copy link
Member

impaktor commented Dec 2, 2018

IMO this is actually a regression for our users

But won't flatpack make a package that's compatible with the linux distro the user is on? Our current problem is our tar-ball only works for (most common) ubuntu / debian distros, which I thought flatpack would solve?

@laarmen
Copy link
Contributor Author

laarmen commented Dec 2, 2018 via email

@Web-eWorks
Copy link
Member

Tarball (well, zipfile) Linux builds via Travis are already complete in master, the Travisfile just needs a repo key to upload the release binaries to the GH release. The rest of the builds are outside of my realm of expertise, so I can't comment on them.

@pcercuei
Copy link
Contributor

Should it be automatically uploaded to github then? We can also do that from appveyor.

@Web-eWorks
Copy link
Member

I'm in favor of uploading all binaries to the appropriate Github release, and mirroring full releases (e.g. no RCs or pre-alpha builds) on the website/sourceforge proper.

Travis can handle that for both our Linux and macOS builds (assuming we get the previous OSX maintainer to port their build process to the Travisfile), and Appveyor can handle the Windows side of things.

To move forward at this point, @pcercuei needs to finalize the Appveyor setup, @impaktor or @ecraven needs to generate a API token with write access to the repo's releases (see Travis/Github docs for the process), and I need to write this all down in PUBLISHING.txt or similar.

Once we have builds from Appveyor and Travis successfully uploading to github releases (aka tags), the PR manager is responsible for uploading the build artifacts to Sourceforge, writing up the new release announcement, and broadcasting the news on Twitter/Mastodon/wherever.

@impaktor
Copy link
Member

and I need to write this all down in PUBLISHING.txt or similar.

I'd vote to have the documentation on wiki or something. Not sure the documentation of how we, the devs, publish the builds need to be in the source tree.

writing up the new release announcement

Usually, Changelog.txt is the only thing we use + I publish the highlights on twitter.

needs to generate a API token with write access to the repo's releases

Yeah, I gave them a look but it wasn't obvious to me what needed to be done. I don't know much about euthentication keys, and whatnot, and I didn't follow the jargon of the docs, i.e. I need doc to understand the doc. I'm sure others understand it better.

@laarmen
Copy link
Contributor Author

laarmen commented Dec 12, 2018 via email

@impaktor
Copy link
Member

I'm rather partial to having this documented in the source tree.

OK, fine. your wish is my command...

@pcercuei
Copy link
Contributor

Do I need to support 32-bit Windows? (that would require more work)

@impaktor
Copy link
Member

I think we prioritize any solution that 1. works, 2. is completed soonish, rather than a utopian solution that stalls a release for more indefinite time. So, I'd say it's up to the person doing the job to decide how far they take it. So if we just get 64 bit support, that's OK with me.

@laarmen
Copy link
Contributor Author

laarmen commented Dec 12, 2018 via email

@Web-eWorks
Copy link
Member

@impaktor To generate an access token, you need to do two things. First, you need to go to https:/settings/tokens/new and generate a token with the public_repo scope (and only that scope). Copy the token string and keep it secret, as it's basically a password.

Next, you need to install the travis CLI via gem install travis. Once that's done, run travis encrypt -r pioneerspacesim/pioneer <TOKEN>, replacing <TOKEN> with the token you (hopefully) copied earlier. The command should output something like secure: "blahblahblah", in YAML format. Copy the output and replace this line with it:

secure: # ENCRYPTED API KEY HERE

Now travis jobs running on the main repo have authorization to deploy release binaries to our Github releases.

@impaktor
Copy link
Member

@Web-eWorks ☝️ done? So if we push a tag, it will now do a GNU/Linux build and upload to github, I assume?

@Web-eWorks
Copy link
Member

I have to fix the build script first, but yes.

@fluffyfreak
Copy link
Contributor

Figured it out! Not shutting things down on exit is what causes the access violation. So there's a fix incoming. For some reason it works fine if the modelcompiler is built using VS201X, or on Linux, or on OSX. But if it's built by mingw-g++ for Windows ... then it has the problem.

The next problem is that it doesn't actually build any sgm files, because it's looking in the wrong folder for them. @pcercuei any idea how to change the CMake stuff to make it look on folder higher for the data folder?

@fluffyfreak
Copy link
Contributor

Fixes for the access violation in #4518 but the modelcompiler searching the wrong location will need someone who understands the CMake stuff to figure out.

I tried changing the working directory to the one with data in it but without success. Not sure what I was doing wrong but ... there ya go.

@pcercuei
Copy link
Contributor

The modelcompiler is already called by CMake in the top-level folder, not in the build folder, so it should be able to find the data.

@impaktor
Copy link
Member

We now have an appveyor pioneer project page (thanks @pcercuei ); and I just added appveyor build badge to the readme.

I wonder if I need to invite pinoeer "owners" to pioneer's appveyor account, or do you guys have access throuhgh loggin in to it with your github account? If not so, bump me, and I'll send appveyor team invite.

@impaktor
Copy link
Member

impaktor commented Jan 29, 2019

So Appveyor does successfully build pioneer now, the only problem is the naming of the file. The script that uploads the output to github release page needs to know the name of the file that was buildt, and ideally we would like the name to be yyyy-mm-dd. So not sure what the sane way to do this is. Appveyor does allow setting versioning, but I don't quite understand it.

Maybe this post (from 2015) might be of interest, it uses powershell:
https://help.appveyor.com/discussions/questions/1180-date-based-versioning

@impaktor
Copy link
Member

impaktor commented Feb 1, 2019

HOLY MOLY, in two days it's been 1 year since last release!

ALL HANDS ON DECK!

RED ALERT! - ASCHTUNG!

I suggest we just manually make windows and linux upload, asap, and upload manually. I think sourceforge is the only pioneer-system I don't have loggin to, so... this likely falls on @fluffyfreak, really sorry to swamp you with stuff to do.

So basically rename this latest appveryor build, and upload, please (and can someone test it as well, just that game starts and ship takes off without total crash?):

(from https://ci.appveyor.com/project/pioneerspacesim/pioneer/builds/21861028/artifacts )

@bszlrd
Copy link
Contributor

bszlrd commented Feb 1, 2019

@bszlrd
Copy link
Contributor

bszlrd commented Feb 1, 2019

That windows installer throws an error after starting it:
errormsg
But if I OK it, the installer runs fine, and the game runs fine as well. (I only did a quick test).

@WKFO
Copy link
Contributor

WKFO commented Feb 1, 2019

The Windows installer didn't give me an error.

@bszlrd
Copy link
Contributor

bszlrd commented Feb 1, 2019

On a second run, the error did not pop up here either.

@fluffyfreak
Copy link
Contributor

I've removed the version string from the ISS naming so that appveyor does a successful build again. Just waiting for that to finish, then I'll test and upload to Sourceforge.

@fluffyfreak
Copy link
Contributor

@pcercuei
Copy link
Contributor

pcercuei commented Feb 3, 2019

The AppStream data (metadata/net.pioneerspacesim.Pioneer.appdata.xml) was not modified to add the release information, could you add it?

@fluffyfreak
Copy link
Contributor

fluffyfreak commented Feb 3, 2019

Is there something somewhere, like on the wiki, that tells me what needs to be added because I've never heard of this AppStream data before?

EDIT: I figured the releases bit is just the bit at the end with the tag and date so updated that, hope it was correct

@pcercuei
Copy link
Contributor

pcercuei commented Feb 3, 2019

Just add a new line below this one: https:/pioneerspacesim/pioneer/blob/master/metadata/net.pioneerspacesim.Pioneer.appdata.xml#L59

Put a new tag just like the already existing one (don't remove it), with this release's date.

@pcercuei
Copy link
Contributor

pcercuei commented Feb 3, 2019

@fluffyfreak, the <releases> tag is in double now ;)

@impaktor
Copy link
Member

impaktor commented Feb 3, 2019

  1. https://twitter.com/pioneerspacesim/status/1092098677961838594
  2. fixed the <releases> issue (also, the file is tab indented, so fixed that line as well)
  3. fluffy, since the new builds don't show on pioneer's download page, I assume the script is pesky about naming convention of the files? I think rob's script runs once every hour on the web page. Or was it once every 6 hours?
  4. I renamed them on the github release page, to clearer point out which is windows and which is linux bulld.
  5. @pcercuei so that file needs to be manually edited every time we tag a release?

Also, thanks @fluffyfreak for fixing this! (meet each other back here in a year?)

@pcercuei
Copy link
Contributor

pcercuei commented Feb 4, 2019

@impaktor, yes, a good practice would be to edit that file to add the new release just right before tagging the release.

Oh, and AppStream complains that the two <release> are swapped, the latest should come first. Sorry 😀

I updated the flatpak recipe. Now you can install Pioneer from here for 32-bit and 64-bit x86, as well as 32-bit and 64-bit ARM. Clicking the "Install" button actually downloads a simple text file, that app stores will parse. This text file can be hosted on the pioneerspacesim.net too.

@impaktor
Copy link
Member

impaktor commented Feb 4, 2019

Cool, I've linked to the flatpak from the release notes on github, we should link to it from our homepage as well, and maybe the wiki. I've fixed the <release> issue again. Let's see if this is bettter.

@impaktor
Copy link
Member

impaktor commented Feb 4, 2019

@fluffyfreak can I ask you to rename the sourceforge files you uploaded to the same format as previous, so maybe pioneer website script picks them up, assuming @robn's script/cron job is still working.

EDIT I realize the windows build now is *exe, so renaming that might be a problem... unless we zip the exe... "we"....

@fluffyfreak
Copy link
Contributor

I think that's gonna require the website to change to be honest. I don't even know how to rename them, it seems needlessly complicated to do anything with Sourceforge...

@impaktor
Copy link
Member

impaktor commented Feb 6, 2019

@fluffyfreak done. @ecraven just launched our new webpage, we have a new repo for it, please comment or PR, or whatever.

@pcercuei how would one best link the flatpak thingy? Does that build against current master, or against latest tag/release?

@pcercuei
Copy link
Contributor

pcercuei commented Feb 6, 2019

The direct download link for the flatpak thingy: https://flathub.org/repo/appstream/net.pioneerspacesim.Pioneer.flatpakref. It will always point to the latest version. The package itself must be updated manually at each release, but it's as simple as updating the source download link and MD5 sum in the recipe.

@PtrMan
Copy link

PtrMan commented Jul 22, 2020

Wow you guys have some problems :D .

I would suggest for a linux build to rent a 5$ box and write a script to

  • download requirements for build
  • pull git
  • build

Looks maybe to simple, idfk.

Same can be done with a windows box, no idea here.

@impaktor
Copy link
Member

impaktor commented Feb 5, 2022

four years later, and just having made a new release the other day, I've documented everything I know, and who's responsible for each thin to do here:
https://pioneerwiki.com/wiki/Makeing_a_release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants