Automate apmpackage publishing to package-storage (snapshot)

[simitt]

Whenever a PR is merged that introduces changes to the apmpackage, or when the APM Server version is bumped, a new PR for the current version of the package should automatically be created and auto-merged if all checks pass. After merging, a promotion to the package registry needs to be triggered.

1. There are two actions that can trigger to publish a new apm package:
   a. When identifying changes to the apmpackage after merging a PR to main or a release branch
   b. When bumping the version (usually happens after a stack release)
2. Run make build-package, which builds the apm integration under the apm-server's root directory at build/integrations/apm/<version>.
3. Run elastic-package check (https:/elastic/elastic-package#elastic-package-check) on the created package and only continue if successful.
4. Push the new apm package to the snapshot branch of the package-storage repo. If a package with this version already exists, overwrite it, otherwise update the existing one.
5. Once pushed, release the changes to the elastic package registry .

Existing tooling for creating the PR and releasing might be used, see https:/elastic/package-storage#package-promotion.

[simitt]

@kseniia-kolpakova is this something that the automation team could help out with?

[kseniia-kolpakova]

@v1v can you take a look?

[axw]

@simitt is this needed if we bundle the package with Kibana?

[simitt]

I am not certain how to packages will be bundled with Kibana. @joshdover do you have a plan already where the apm-package will be fetched from/ingested?

Either way, the packages most certainly won't be bundled before 8.2, therefore automation would make sense for versions 7.17, 8.0 and 8.1. But it is a good point, we should be mindful about how much effort it is to automate the process.

[axw]

@simitt OK, fair enough.

According to elastic/kibana#112095 (comment), the plan is to fetch packages from EPR or package-storage, which would be compatible with what you described (assuming the automation can be directed to pull them from staging.)

[simitt]

I left a comment to discuss pulling from staging elastic/kibana#112095 (comment).

[simitt]

Updated the description with some more details, that should also simplify the process a bit.

[simitt]

The package registry might not be pulling from the package-storage but from some cloud bucket in the future. Given that this is not currently on the way and that the current manual release process is slow and error prone I still think we should move forward with this proposal as is.

[simitt]

Let's hold off with this for a bit, due to internal conversations of how to evolve the package in the future.