[BUG] Checked in versions of job-scheduler-*-SNAPSHOT.zip

[dblock]

Is your feature request related to a problem? Please describe.
We commit the job-scheduler depenency into src/test/resources/job-scheduler/opendistro-job-scheduler-1.15.0.0-SNAPSHOT.zip and similar which is just some random version built on a developer desktop. This is not reliable.

Describe the solution you'd like
Replace with a dynamic maven dependency or build one on demand in CI.

Describe alternatives you've considered
This is what it takes to update this today.

cd job-scheduler
git checkout 1.2
git pull
./gradlew assemble
rm ../anomaly-detection/src/test/resources/job-scheduler/*
rm -rf ../anomaly-detection/src/test/resources/org/opensearch/ad/bwc/job-scheduler/*
cp ./build/distributions/opensearch-job-scheduler-1.2.3.0-SNAPSHOT.zip ../anomaly-detection/src/test/resources/job-scheduler/
mkdir -p ../anomaly-detection/src/test/resources/org/opensearch/ad/bwc/job-scheduler/1.2.3.0-SNAPSHOT
cp ./build/distributions/opensearch-job-scheduler-1.2.3.0-SNAPSHOT.zip ../anomaly-detection/src/test/resources/org/opensearch/ad/bwc/job-scheduler/1.2.3.0-SNAPSHOT

Additional context
Part of #15, right now integTest is failing because of version incompatibility.

| Output for ./bin/opensearch-plugin:-> Installing file:/home/ubuntu/source/opensearch-project/anomaly-detection/dblock/src/test/resources/job-scheduler/opendistro-job-scheduler-1.15.0.0-SNAPSHOT.zip
| -> Downloading file:/home/ubuntu/source/opensearch-project/anomaly-detection/dblock/src/test/resources/job-scheduler/opendistro-job-scheduler-1.15.0.0-SNAPSHOT.zip
| -> Failed installing file:/home/ubuntu/source/opensearch-project/anomaly-detection/dblock/src/test/resources/job-scheduler/opendistro-job-scheduler-1.15.0.0-SNAPSHOT.zip
| -> Rolling back file:/home/ubuntu/source/opensearch-project/anomaly-detection/dblock/src/test/resources/job-scheduler/opendistro-job-scheduler-1.15.0.0-SNAPSHOT.zip
| -> Rolled back file:/home/ubuntu/source/opensearch-project/anomaly-detection/dblock/src/test/resources/job-scheduler/opendistro-job-scheduler-1.15.0.0-SNAPSHOT.zip
| Exception in thread "main" java.lang.IllegalArgumentException: Plugin [opendistro-job-scheduler] was built for OpenSearch version 7.10.3 but version 1.0.0 is running
|       at org.opensearch.plugins.PluginsService.verifyCompatibility(PluginsService.java:365)
|       at org.opensearch.plugins.InstallPluginCommand.loadPluginInfo(InstallPluginCommand.java:801)
|       at org.opensearch.plugins.InstallPluginCommand.installPlugin(InstallPluginCommand.java:856)
|       at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:263)
|       at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:237)
|       at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:99)
|       at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:140)
|       at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
|       at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:140)
|       at org.opensearch.cli.Command.main(Command.java:103)
|       at org.opensearch.plugins.PluginCli.main(PluginCli.java:60)

[dblock]

Not sure why I closed this, this is a major pain in version increments, e.g. #344, and whatever was done isn't working.

[amitgalitz]

This seems to be overlapping with #352, do you have a preference to which issue I should close. Currently I opened this issue in opensearch-build (https:/opensearch-project/opensearch-build/issues/1441) in hopes to have job-scheduler snapshots always up to date in maven. In this issue are you suggesting a change to have the zips themselves uploaded to maven?

[dblock]

I closed #352 since it's newer.

[dblock]

Here's a full list of ZIPs.

./anomaly-detection/src/test/resources/org/opensearch/ad/bwc/anomaly-detection/1.13.0.0/opendistro-anomaly-detection-1.13.0.0.zip
./anomaly-detection/src/test/resources/org/opensearch/ad/bwc/anomaly-detection/1.1.0.0-SNAPSHOT/opensearch-anomaly-detection-1.1.0.0-SNAPSHOT.zip
./anomaly-detection/src/test/resources/org/opensearch/ad/bwc/job-scheduler/1.13.0.0/opendistro-job-scheduler-1.13.0.0.zip
./anomaly-detection/src/test/resources/org/opensearch/ad/bwc/job-scheduler/1.2.0.0-SNAPSHOT/opensearch-job-scheduler-1.2.0.0-SNAPSHOT.zip
./anomaly-detection/src/test/resources/org/opensearch/ad/bwc/job-scheduler/1.3.0.0-SNAPSHOT/opensearch-job-scheduler-1.3.0.0-SNAPSHOT.zip
./anomaly-detection/src/test/resources/job-scheduler/opensearch-job-scheduler-1.3.0.0-SNAPSHOT.zip

This is now a child of opensearch-project/opensearch-build#1750

[dblock]

I am still seeing these:

./anomaly-detection/src/test/resources/org/opensearch/ad/bwc/anomaly-detection/1.13.0.0/opendistro-anomaly-detection-1.13.0.0.zip
./anomaly-detection/src/test/resources/org/opensearch/ad/bwc/anomaly-detection/1.1.0.0-SNAPSHOT/opensearch-anomaly-detection-1.1.0.0-SNAPSHOT.zip
./anomaly-detection/src/test/resources/org/opensearch/ad/bwc/job-scheduler/1.13.0.0/opendistro-job-scheduler-1.13.0.0.zip
./anomaly-detection/src/test/resources/org/opensearch/ad/bwc/job-scheduler/1.2.0.0-SNAPSHOT/opensearch-job-scheduler-1.2.0.0-SNAPSHOT.zip
./anomaly-detection/src/test/resources/org/opensearch/ad/bwc/job-scheduler/1.3.0.0-SNAPSHOT/opensearch-job-scheduler-1.3.0.0-SNAPSHOT.zip

[amitgalitz]

@dblock The left over zips are for BWC. From a different issue I understand backfilling is a problem. I can either Implement code that parses manifests files for older zips or I can get the build numbers from the official docker releases. Is there a preferences between the two?

[dblock]

@amitgalitz I think you should do something like opensearch-project/cross-cluster-replication#366. Right now you're using "some" -SNAPSHOT builds, which is precisely the issue.