Publish nightly releases

[imjasonh]

Changes

This will run at midnight EST and produce a nightly-YYYY-MM-DD.yaml (and -debug.yaml) attached to a new (draft, pre-release) GitHub Release each night.

Note: This creates new GitHub Releases each day, and I assume this will become unwieldy at some point. Ideas welcome.

Related to discussion in #599

Submitter Checklist

• [n] Includes tests if functionality changed/was added
• [n] Includes docs if changes are user-facing
• [y] Release notes block has been filled in, or marked NONE

See the contributor guide
for details on coding conventions, github and prow interactions, and the code review process.

Release Notes

NONE

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign qu1queee after the PR has been reviewed.
You can assign the PR to them by writing /assign @qu1queee in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

Hi @imjasonh. Thanks for your PR.

I'm waiting for a shipwright-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[gabemontero]

/ok-to-test

[imjasonh]

Tested this in my own fork. This run produced this release: https:/ImJasonH/build-1/releases/tag/untagged-aad4f80e82cadbb1c018

In my fork, --push=false so the image it produced isn't pushed, but the YAML was generated successfully. But if that wasn't the case you could:

kubectl apply -f https:/ImJasonH/build-1/releases/download/untagged-aad4f80e82cadbb1c018/nightly-2021-02-19.yaml

[zhangtbj]

Cool!

A quick question, will it generate a nightly tag or release every time?

Or where we store the nightly build or release?

[imjasonh]

A quick question, will it generate a nightly tag or release every time?

Or where we store the nightly build or release?

This will create a new untagged draft release every day (see my fork's release for an example), which includes an image in quay.io for each nightly build.

This is obviously somewhat sub-optimal, since it spams draft releases, and can make it harder to find real tagged releases.

Actually, I realized while writing this response that draft releases aren't visible outside of repo collaborators, this is what my fork's releases look like to me:

This might be fine for the short term, since it means nightly release are only visible to collaborators... 🤔

We should improve on this, by:

1. not generating a release when there's been no commits since the last successful nightly release
2. maybe not creating a new release each day, and instead appending a new release asset to a single running nightly release
3. maybe hosting nightly releases outside of GitHub Releases. For Tekton, nightly release YAMLs are available on Google Cloud Storage (https://tekton-nightly.storage.googleapis.com

[zhangtbj]

Thanks Jason,

I am not sure why it reports Page not found error when I accessing your link: https:/ImJasonH/build-1/releases/tag/untagged-aad4f80e82cadbb1c018 :(

And if the proposal of the nightly release is just for testing, should we just keep one nightly release? Or do we need to keep every nightly release somewhere?

[imjasonh]

I am not sure why it reports Page not found error when I accessing your link: https:/ImJasonH/build-1/releases/tag/untagged-aad4f80e82cadbb1c018 :(

That's because the release is still a Draft, which means only repo contributors (me) can see it.

And if the proposal of the nightly release is just for testing, should we just keep one nightly release? Or do we need to keep every nightly release somewhere?

I think it'd be useful to keep at least a few days' back, it can be really helpful in pinpointing when a regression was introduced.

[qu1queee]

@imjasonh thanks for this. I think the problem we want to solve with nightly releases is

provide users a very fresh recently cooked set of YAMLs and a container image for further usage

(sorry for the above quoting, just wanna ensure we are on the same line of thought).

I really like this idea, but to be honest, the spamming of draft releases is not very appealing. I think the git release is the right place to host our latest stable release assets, but not the right place for a nightly CI. Here is where I will say that we really need an s3 storage or similar. If we get an s3 storage we can use the github actions history as a way for users to look on the latest execution of the nightly release action and from there gather the s3 bucket endpoint.

[imjasonh]

I really like this idea, but to be honest, the spamming of draft releases is not very appealing.

Yeah, I completely agree 😞

I think we could instead have one Nightly release that we update with new nightly YAMLs, always appending new release assets, and maybe trimming off old ones as needed. This will be a bit more involved to write up, but I think overall worth it in the end. If that sounds good to you I can keep working on it.

It'd be really nice not to introduce a new service we depend on (S3, Google Cloud Storage, etc.) just to host YAMLs. This means one more moving piece in the CI, and one more set of creds to manage and share.

[qu1queee]

I think we could instead have one Nightly release that we update with new nightly YAMLs, always appending new release assets, and maybe trimming off old ones as needed. This will be a bit more involved to write up, but I think overall worth it in the end. If that sounds good to you I can keep working on it.

definitely. Pls ask for help if needed.

It'd be really nice not to introduce a new service we depend on (S3, Google Cloud Storage, etc.) just to host YAMLs. This means one more moving piece in the CI, and one more set of creds to manage and share.

I think credentials wise is not too much of a trouble, e.g. we do this now for quay, where the setup was very simple. Aiming to reduce dependencies for CI make sense.

[imjasonh]

Closing this for now while I rework it to append to an existing release.