Updated OpenSearch Lucene snapshot location (opensearch-project#1832)

* Backport multiple PRs to main from 2.x (opensearch-project#1652)

* fix parameter name in preprocess function; fix remote model function … (opensearch-project#1362)

* fix parameter name in preprocess function; fix remote model function name

Signed-off-by: Yaliang Wu <[email protected]>

* fix failed unit test

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>

* throw exception when model group not found during update request (opensearch-project#1447)

Signed-off-by: Bhavana Ramaram <[email protected]>

* add status code to model tensor (opensearch-project#1443) (opensearch-project#1453)

Signed-off-by: Yaliang Wu <[email protected]>

* register new versions to a model group based on the name provided (opensearch-project#1452)

Signed-off-by: Bhavana Ramaram <[email protected]>

* fixing metrics correlation algorithm (opensearch-project#1448)

* fixing metrics correlation algorithm

Signed-off-by: Dhrubo Saha <[email protected]>

* if model version fails to register, update model group accordingly (opensearch-project#1463)

* if model version fails to register, update model group accordingly

Signed-off-by: Bhavana Ramaram <[email protected]>

* Update Model API (opensearch-project#1350)

* Update Model API POC

Signed-off-by: Sicheng Song <[email protected]>

* Using GetRequest to get model

Signed-off-by: Sicheng Song <[email protected]>

* Finalize model update API

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile

Signed-off-by: Sicheng Song <[email protected]>

* Fix compileTest

Signed-off-by: Sicheng Song <[email protected]>

* Add Unit Test Cases for Update Model API

Signed-off-by: Sicheng Song <[email protected]>

* Tune back test coverage thereshold

Signed-off-by: Sicheng Song <[email protected]>

* Add more unit tests on Update model API

Signed-off-by: Sicheng Song <[email protected]>

* Add unit test for TransportUpdateModelAction class

Signed-off-by: Sicheng Song <[email protected]>

* Fix a test error

Signed-off-by: Sicheng Song <[email protected]>

* Change exception thrown to failure response

Signed-off-by: Sicheng Song <[email protected]>

* Move the function judgement to the outter block

Signed-off-by: Sicheng Song <[email protected]>

* Check if model is undeployed before update model

Signed-off-by: Sicheng Song <[email protected]>

* Add more unit test for update model API

Signed-off-by: Sicheng Song <[email protected]>

* Fix unit test due to blocking java 11 CI workflow

Signed-off-by: Sicheng Song <[email protected]>

* Enabling auto bumping model version during registering to a new model group and address reviewers' other concern

Signed-off-by: Sicheng Song <[email protected]>

* Autobump new model groups' latest version when register to a new model

Signed-off-by: Sicheng Song <[email protected]>

* Change the REST API method from POST to PUT

Signed-off-by: Sicheng Song <[email protected]>

* Change the update REST API endpoint

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>

* Add a setting to control the update connector API (opensearch-project#1465)

* Add a setting to control the update connector API

Signed-off-by: Sicheng Song <[email protected]>

* Enabling the update connnector setting in unit test

Signed-off-by: Sicheng Song <[email protected]>

* Enabling the update connnector setting in corresponding unit test

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>

* fix update connector API (opensearch-project#1484)

* fix update connector API

Signed-off-by: Yaliang Wu <[email protected]>

* Performance enhacement for predict action by caching model info (opensearch-project#1472) (opensearch-project#1508)

* Performance enhacement for predict action by caching model info

Signed-off-by: zane-neo <[email protected]>

* Add context.restore() to avoid missing info

Signed-off-by: zane-neo <[email protected]>

---------

Signed-off-by: zane-neo <[email protected]>
(cherry picked from commit a985f6e)

Co-authored-by: zane-neo <[email protected]>

* fix failed ut from PR 1472 (opensearch-project#1479) (opensearch-project#1510)

* fix failed ut from PR 1472

Signed-off-by: Yaliang Wu <[email protected]>

* exclude class for low coverage

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
(cherry picked from commit da5d829)

Co-authored-by: Yaliang Wu <[email protected]>

* [Backport to 2.11] throw exception if remote model doesn't return 2xx status code; fix p… (opensearch-project#1477) (opensearch-project#1509)

* throw exception if remote model doesn't return 2xx status code; fix p… (opensearch-project#1473)

* throw exception if remote model doesn't return 2xx status code; fix predict runner

Signed-off-by: Yaliang Wu <[email protected]>

* fix kmeans model deploy bug

Signed-off-by: Yaliang Wu <[email protected]>

* support multiple docs for remote embedding model

Signed-off-by: Yaliang Wu <[email protected]>

* fix ut

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>

* fix wrong class

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
(cherry picked from commit 201c8a8)

Co-authored-by: Yaliang Wu <[email protected]>

* fix no worker node exception for remote embedding model (opensearch-project#1482) (opensearch-project#1511)

* fix no worker node exception for remote embedding model

Signed-off-by: Yaliang Wu <[email protected]>

* only add model info to cache if model cache exist

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
(cherry picked from commit 6f83b9f)

Co-authored-by: Yaliang Wu <[email protected]>

* fix for delete model group API throwing incorrect error when model index not created (opensearch-project#1485) (opensearch-project#1486) (opensearch-project#1512)

* fix for delete model group API throwing incorrect error when model index not created

Signed-off-by: Bhavana Ramaram <[email protected]>
(cherry picked from commit 60ef0fd)

Co-authored-by: Bhavana Ramaram <[email protected]>
(cherry picked from commit 5544681)

Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>

* fix no worker node error on multi-node cluster (opensearch-project#1487) (opensearch-project#1513)

Signed-off-by: Yaliang Wu <[email protected]>
(cherry picked from commit cea1cd6)

Co-authored-by: Yaliang Wu <[email protected]>

* add prefix to show the error is from remote service (opensearch-project#1499) (opensearch-project#1515)

Signed-off-by: Yaliang Wu <[email protected]>
(cherry picked from commit 3897ad1)

Co-authored-by: Yaliang Wu <[email protected]>

* fix multiple docs support (opensearch-project#1516)

Signed-off-by: Yaliang Wu <[email protected]>

* adding another fix issue to the release note (opensearch-project#1498) (opensearch-project#1514)

Signed-off-by: Dhrubo Saha <[email protected]>
(cherry picked from commit 440155c)

Co-authored-by: Dhrubo Saha <[email protected]>

* add bedrockURL to trusted connector regex list (opensearch-project#1461)

Signed-off-by: Bhavana Ramaram <[email protected]>

* return parsing exception 400 for parsing errors

Signed-off-by: Xun Zhang <[email protected]>

* add more ut in restupdateconnector

Signed-off-by: Xun Zhang <[email protected]>

* fix format violations

Signed-off-by: Bhavana Ramaram <[email protected]>

* Fix model/connector update API to address security concern (opensearch-project#1595)

* Fix model/connector update API to address appsec concern

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile and build failure

Signed-off-by: Sicheng Song <[email protected]>

* Improve unit test coverage

Signed-off-by: Sicheng Song <[email protected]>

* Fix spotless

Signed-off-by: Sicheng Song <[email protected]>

* Merge update connector feature flag to remote inference feature flag

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile

Signed-off-by: Sicheng Song <[email protected]>

* Fix exception status

Signed-off-by: Sicheng Song <[email protected]>

* Keep fixing exception status

Signed-off-by: Sicheng Song <[email protected]>

* Spotless fix

Signed-off-by: Sicheng Song <[email protected]>

* Add UT on parsing exception

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>

* change XContentFactory to MediaTypeRegistry builder in MLRegisterModelInputTest class

Signed-off-by: Bhavana Ramaram <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
Signed-off-by: Bhavana Ramaram <[email protected]>
Signed-off-by: Dhrubo Saha <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Co-authored-by: Yaliang Wu <[email protected]>
Co-authored-by: Dhrubo Saha <[email protected]>
Co-authored-by: Sicheng Song <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: zane-neo <[email protected]>
Co-authored-by: Xun Zhang <[email protected]>

* Updated OpenSearch Lucene snapshot location

Signed-off-by: Owais Kazi <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
Signed-off-by: Bhavana Ramaram <[email protected]>
Signed-off-by: Dhrubo Saha <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Owais Kazi <[email protected]>
Co-authored-by: Bhavana Ramaram <[email protected]>
Co-authored-by: Yaliang Wu <[email protected]>
Co-authored-by: Dhrubo Saha <[email protected]>
Co-authored-by: Sicheng Song <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: zane-neo <[email protected]>
Co-authored-by: Xun Zhang <[email protected]>

build-tools/repositories.gradle

@@ -8,5 +8,5 @@ repositories {
  maven { url "https://aws.oss.sonatype.org/content/repositories/snapshots" }
  mavenCentral()
  maven {url 'https://oss.sonatype.org/content/repositories/snapshots/'}
- maven { url "https://d1nvenhzbhpy0q.cloudfront.net/snapshots/lucene/" }
+ maven { url "https://artifacts.opensearch.org/snapshots/lucene/" }
 }

build.gradle

@@ -30,7 +30,7 @@ buildscript {
  maven { url "https://aws.oss.sonatype.org/content/repositories/snapshots" }
  mavenCentral()
  maven { url "https://plugins.gradle.org/m2/" }
- maven { url "https://d1nvenhzbhpy0q.cloudfront.net/snapshots/lucene/" }
+ maven { url "https://artifacts.opensearch.org/snapshots/lucene/" }
  }
 
  dependencies {