[BUG] Main branch does not pull 9.1.0-snapshot-ea989fe8f30 to m2 after deploying mavenlocal

[peterzhuamazon]

Run this on main branch of OpenSearch core as of b9f04405d6dfabcc2b8e87a86d4f83b003b3ffe7

./gradlew publishToMavenLocal -Dopensearch.version=2.0.0-SNAPSHOT -Dbuild.snapshot=true"

Check ~/.m2/repository/org/apache/lucene/lucene-core, 9.1.0-snapshot-ea989fe8f30 not exist.

This cause things after core such as common-utils to fail to these msg:

Logs

* What went wrong:
Execution failed for task ':compileKotlin'.
> Error while evaluating property 'filteredArgumentsMap' of task ':compileKotlin'
   > Could not resolve all files for configuration ':compileClasspath'.
      > Could not find org.apache.lucene:lucene-core:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-core/9.1.0-snapshot-ea989fe8f30/lucene-core-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-core/9.1.0-snapshot-ea989fe8f30/lucene-core-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-core/9.1.0-snapshot-ea989fe8f30/lucene-core-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-core/9.1.0-snapshot-ea989fe8f30/lucene-core-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-analysis-common:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-analysis-common/9.1.0-snapshot-ea989fe8f30/lucene-analysis-common-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-analysis-common/9.1.0-snapshot-ea989fe8f30/lucene-analysis-common-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-analysis-common/9.1.0-snapshot-ea989fe8f30/lucene-analysis-common-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-analysis-common/9.1.0-snapshot-ea989fe8f30/lucene-analysis-common-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-backward-codecs:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-backward-codecs/9.1.0-snapshot-ea989fe8f30/lucene-backward-codecs-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-backward-codecs/9.1.0-snapshot-ea989fe8f30/lucene-backward-codecs-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-backward-codecs/9.1.0-snapshot-ea989fe8f30/lucene-backward-codecs-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-backward-codecs/9.1.0-snapshot-ea989fe8f30/lucene-backward-codecs-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-grouping:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-grouping/9.1.0-snapshot-ea989fe8f30/lucene-grouping-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-grouping/9.1.0-snapshot-ea989fe8f30/lucene-grouping-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-grouping/9.1.0-snapshot-ea989fe8f30/lucene-grouping-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-grouping/9.1.0-snapshot-ea989fe8f30/lucene-grouping-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-highlighter:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-highlighter/9.1.0-snapshot-ea989fe8f30/lucene-highlighter-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-highlighter/9.1.0-snapshot-ea989fe8f30/lucene-highlighter-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-highlighter/9.1.0-snapshot-ea989fe8f30/lucene-highlighter-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-highlighter/9.1.0-snapshot-ea989fe8f30/lucene-highlighter-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-join:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-join/9.1.0-snapshot-ea989fe8f30/lucene-join-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-join/9.1.0-snapshot-ea989fe8f30/lucene-join-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-join/9.1.0-snapshot-ea989fe8f30/lucene-join-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-join/9.1.0-snapshot-ea989fe8f30/lucene-join-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-memory:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-memory/9.1.0-snapshot-ea989fe8f30/lucene-memory-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-memory/9.1.0-snapshot-ea989fe8f30/lucene-memory-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-memory/9.1.0-snapshot-ea989fe8f30/lucene-memory-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-memory/9.1.0-snapshot-ea989fe8f30/lucene-memory-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-misc:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-misc/9.1.0-snapshot-ea989fe8f30/lucene-misc-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-misc/9.1.0-snapshot-ea989fe8f30/lucene-misc-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-misc/9.1.0-snapshot-ea989fe8f30/lucene-misc-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-misc/9.1.0-snapshot-ea989fe8f30/lucene-misc-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-queries:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-queries/9.1.0-snapshot-ea989fe8f30/lucene-queries-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-queries/9.1.0-snapshot-ea989fe8f30/lucene-queries-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-queries/9.1.0-snapshot-ea989fe8f30/lucene-queries-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-queries/9.1.0-snapshot-ea989fe8f30/lucene-queries-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-queryparser:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-queryparser/9.1.0-snapshot-ea989fe8f30/lucene-queryparser-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-queryparser/9.1.0-snapshot-ea989fe8f30/lucene-queryparser-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-queryparser/9.1.0-snapshot-ea989fe8f30/lucene-queryparser-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-queryparser/9.1.0-snapshot-ea989fe8f30/lucene-queryparser-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-sandbox:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-sandbox/9.1.0-snapshot-ea989fe8f30/lucene-sandbox-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-sandbox/9.1.0-snapshot-ea989fe8f30/lucene-sandbox-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-sandbox/9.1.0-snapshot-ea989fe8f30/lucene-sandbox-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-sandbox/9.1.0-snapshot-ea989fe8f30/lucene-sandbox-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-spatial-extras:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-spatial-extras/9.1.0-snapshot-ea989fe8f30/lucene-spatial-extras-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-spatial-extras/9.1.0-snapshot-ea989fe8f30/lucene-spatial-extras-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-spatial-extras/9.1.0-snapshot-ea989fe8f30/lucene-spatial-extras-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-spatial-extras/9.1.0-snapshot-ea989fe8f30/lucene-spatial-extras-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-spatial3d:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-spatial3d/9.1.0-snapshot-ea989fe8f30/lucene-spatial3d-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-spatial3d/9.1.0-snapshot-ea989fe8f30/lucene-spatial3d-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-spatial3d/9.1.0-snapshot-ea989fe8f30/lucene-spatial3d-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-spatial3d/9.1.0-snapshot-ea989fe8f30/lucene-spatial3d-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT
      > Could not find org.apache.lucene:lucene-suggest:9.1.0-snapshot-ea989fe8f30.
        Searched in the following locations:
          - file:/home/zhujiaxi/.m2/repository/org/apache/lucene/lucene-suggest/9.1.0-snapshot-ea989fe8f30/lucene-suggest-9.1.0-snapshot-ea989fe8f30.pom
          - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-suggest/9.1.0-snapshot-ea989fe8f30/lucene-suggest-9.1.0-snapshot-ea989fe8f30.pom
          - https://plugins.gradle.org/m2/org/apache/lucene/lucene-suggest/9.1.0-snapshot-ea989fe8f30/lucene-suggest-9.1.0-snapshot-ea989fe8f30.pom
          - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-suggest/9.1.0-snapshot-ea989fe8f30/lucene-suggest-9.1.0-snapshot-ea989fe8f30.pom
        Required by:
            project : > org.opensearch.client:opensearch-rest-high-level-client:2.0.0-SNAPSHOT > org.opensearch:opensearch:2.0.0-SNAPSHOT

* Try:
> Run with --scan to get full insights.

[peterzhuamazon]

Seems like this is caused by 1st time we use anything that is not officially on lucene maven repo.
If we ought to test snapshots this way, repos like common-utils that utilize lucene needs to add https://artifacts.opensearch.org/snapshots/lucene/ as part of their repo in build.gradle.

@nknize @dblock please confirm.

Thanks.

[mch2]

Hi @peterzhuamazon,
Gradle does not store build dependencies to maven local during builds, only build artifacts.

We can fix this by updating common-utils to apply the opensearch.repositories plugin. This plugin is where the maven repo is configured. Most plugins are applying the plugin through the opensearch.java plugin that also includes the repositories plugin.

It would be nice to publish these lucene snapshots to https://aws.oss.sonatype.org/content/repositories/snapshots, but I'm not sure we are able to publish anything there thats not in the org.opensearch namespace.

[dblock]

Why aren't we using official Lucene builds/snapshots? https://lucene.apache.org/core/developer.html. Is that what opensearch.repositories does?

@mch2 Do you have time to look into this?

[mch2]

Learned this from @nknize - The official Lucene snapshots build each artifact separately after commit. So each artifact is built with a separate sha and at a different time.

Our build system right now locks to a single sha in version.properties 9.1.0-snapshot-ea989fe8f30. We can easily change this, but the original purpose of this was to decouple our builds from bugs that may creep in nightly lucene builds and lock to a specific "stable-ish" version of the snapshot. To do this we need to a top-down build of all components and host the artifacts ourselves.

Either way, we should update common-utils to add the repositories plugin, have made a PR over there.

[mch2]

@peterzhuamazon Did this resolve your issue? are there other repos with the same problem?

[peterzhuamazon]

@mch2 I have tested on 2.0.0 build this morning and it passed.
Thanks.