Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pre conditions check before updating weighted routing metadata #4955

Merged

Conversation

imRishN
Copy link
Member

@imRishN imRishN commented Oct 27, 2022

Description

PR scope -

  1. Earlier, we blocked weight updates if any attribute is undergoing decommission action. This PR tries to modify the behaviour and will allow weight updates as long as weights of a decommissioned attribute is not modified.
  2. Weight update should be done for all discovered zones and force zone values
  3. Relax two zones having zero weight check

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@imRishN imRishN requested review from a team and reta as code owners October 27, 2022 10:24
Signed-off-by: Rishab Nahata <[email protected]>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@codecov-commenter
Copy link

codecov-commenter commented Oct 27, 2022

Codecov Report

Merging #4955 (622fe6d) into main (31bcda6) will decrease coverage by 0.15%.
The diff coverage is 63.26%.

@@             Coverage Diff              @@
##               main    #4955      +/-   ##
============================================
- Coverage     71.01%   70.86%   -0.16%     
+ Complexity    58149    58107      -42     
============================================
  Files          4711     4712       +1     
  Lines        277573   277612      +39     
  Branches      40180    40184       +4     
============================================
- Hits         197122   196731     -391     
- Misses        64293    64719     +426     
- Partials      16158    16162       +4     
Impacted Files Coverage Δ
...ting/WeightedRoutingUnsupportedStateException.java 40.00% <40.00%> (ø)
...search/cluster/routing/WeightedRoutingService.java 77.08% <64.28%> (-10.85%) ⬇️
.../main/java/org/opensearch/OpenSearchException.java 92.37% <100.00%> (-0.20%) ⬇️
...weighted/put/ClusterPutWeightedRoutingRequest.java 62.29% <100.00%> (-2.86%) ⬇️
...g/opensearch/index/analysis/CharFilterFactory.java 0.00% <0.00%> (-100.00%) ⬇️
...java/org/opensearch/client/indices/DataStream.java 0.00% <0.00%> (-76.09%) ⬇️
...adonly/AddIndexBlockClusterStateUpdateRequest.java 0.00% <0.00%> (-75.00%) ⬇️
...pensearch/client/cluster/RemoteConnectionInfo.java 0.00% <0.00%> (-73.18%) ⬇️
...readonly/TransportVerifyShardIndexBlockAction.java 9.75% <0.00%> (-58.54%) ⬇️
...a/org/opensearch/client/cluster/ProxyModeInfo.java 0.00% <0.00%> (-55.00%) ⬇️
... and 490 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2022

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Rishab Nahata <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2022

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Rishab Nahata <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2022

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2022

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2022

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2022

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Rishab Nahata <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2022

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2022

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testIndexDeletionDuringSnapshotCreationInQueue

@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2022

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Rishab Nahata <[email protected]>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Rishab Nahata <[email protected]>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@Bukhtawar Bukhtawar merged commit aecd1ea into opensearch-project:main Dec 13, 2022
@imRishN imRishN added the backport 2.x Backport to 2.x branch label Dec 13, 2022
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-4955-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 aecd1ea3994d99f8e1394f1b2bffec0bc0c00c71
# Push it to GitHub
git push --set-upstream origin backport/backport-4955-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4955-to-2.x.

imRishN added a commit to imRishN/OpenSearch that referenced this pull request Dec 13, 2022
…earch-project#4955)

* Pre conditions check to allow weight updates for non decommissioned attribute

Signed-off-by: Rishab Nahata <[email protected]>
Bukhtawar pushed a commit that referenced this pull request Dec 13, 2022
#5539)

* [Backport 2.x]Pre conditions check to allow weight updates for non decommissioned attribute

Signed-off-by: Rishab Nahata <[email protected]>
ryanbogan pushed a commit that referenced this pull request Dec 14, 2022
* Pre conditions check to allow weight updates for non decommissioned attribute

Signed-off-by: Rishab Nahata <[email protected]>
saratvemulapalli pushed a commit that referenced this pull request Dec 15, 2022
…ature/identity (#5581)

* Fix flaky ShardIndexingPressureConcurrentExecutionTests (#5439)

Add conditional check on assertNull to fix flaky tests.

Signed-off-by: Rishikesh1159 <[email protected]>

* Fix bwc for cluster manager throttling settings (#5305)

Signed-off-by: Dhwanil Patel <[email protected]>

* Update ingest-attachment plugin dependencies: Apache Tika 3.6.0, Apache Mime4j 0.8.8, Apache Poi 5.2.3, Apache PdfBox 2.0.27 (#5448)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

* Enhance CheckpointState to support no-op replication (#5282)

* CheckpointState enhanced to support no-op replication

Signed-off-by: Ashish Singh <[email protected]>
Co-authored-by: Bukhtawar Khan<[email protected]>

* [BUG] org.opensearch.repositories.s3.RepositoryS3ClientYamlTestSuiteIT/test {yaml=repository_s3/20_repository_permanent_credentials/Snapshot and Restore with repository-s3 using permanent credentials} flaky: randomizing basePath (#5482)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

* [Bug] fix case sensitivity for wildcard queries (#5462)

Fixes the wildcard query to not normalize the pattern when case_insensitive is
set by the user. This is achieved by creating a new normalizedWildcardQuery
method so that query_string queries (which do not support case sensitivity) can
still normalize the pattern when the default analyzer is used; maintaining
existing behavior.

Signed-off-by: Nicholas Walter Knize <[email protected]>

* Support OpenSSL Provider with default Netty allocator (#5460)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

* Revert "build no-jdk distributions as part of release build (#4902)" (#5465)

This reverts commit 8c9ca4e.

It seems that this wasn't entirely the correct way and is currently
blocking us from removing the `build.sh` from the `opensearch-build`
repository (i.e. this `build.sh` here is not yet being used).
See the discussion in opensearch-project/opensearch-build#2835 for
further details.

Signed-off-by: Ralph Ursprung <[email protected]>

Signed-off-by: Ralph Ursprung <[email protected]>

* Add max_shard_size parameter for Shrink API (fix supported version after backport) (#5503)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

* Sync CODEOWNERS with MAINTAINERS. (#5501)

Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>

Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>

* Added jackson dependency to server (#5366)

* Added jackson dependency to server

Signed-off-by: Ryan Bogan <[email protected]>

* Updated CHANGELOG

Signed-off-by: Ryan Bogan <[email protected]>

* Update build.gradle files

Signed-off-by: Ryan Bogan <[email protected]>

* Add RuntimePermission to fix errors

Signed-off-by: Ryan Bogan <[email protected]>

Signed-off-by: Ryan Bogan <[email protected]>

* Fix flaky test BulkIntegrationIT.testDeleteIndexWhileIndexing (#5491)

Signed-off-by: Poojita Raj <[email protected]>

Signed-off-by: Poojita Raj <[email protected]>

* Add release notes for 2.4.1 (#5488)

Signed-off-by: Xue Zhou <[email protected]>

Signed-off-by: Xue Zhou <[email protected]>

* Properly skip OnDemandBlockSnapshotIndexInputTests.testVariousBlockSize on Windows. (#5511)

PR #5397 skipped this test in @before block but still
frequently throws a TestCouldNotBeSkippedException.  This is caused by the after block still executing and throwing  an exception
while cleaning the directory created at the path in @before.  Moving the assumption to the individual test prevents this exception by ensuring the path exists.

Signed-off-by: Marc Handalian <[email protected]>

Signed-off-by: Marc Handalian <[email protected]>

* Merge first batch of feature/extensions into main (#5347)

* Merge first batch of feature/extensions into main

Signed-off-by: Ryan Bogan <[email protected]>

* Fixed CHANGELOG

Signed-off-by: Ryan Bogan <[email protected]>

* Fixed newline errors

Signed-off-by: Ryan Bogan <[email protected]>

* Renaming and CHANGELOG fixes

Signed-off-by: Ryan Bogan <[email protected]>

* Refactor extension loading into private method

Signed-off-by: Ryan Bogan <[email protected]>

* Removed skipValidation and added connectToExtensionNode method

Signed-off-by: Ryan Bogan <[email protected]>

* Remove unnecessary feature flag calls

Signed-off-by: Ryan Bogan <[email protected]>

* Renaming and exception handling

Signed-off-by: Ryan Bogan <[email protected]>

* Change latches to CompletableFuture

Signed-off-by: Ryan Bogan <[email protected]>

* Removed unnecessary validateSettingKey call

Signed-off-by: Ryan Bogan <[email protected]>

* Fix azure-core dependency

Signed-off-by: Ryan Bogan <[email protected]>

* Update SHAs

Signed-off-by: Ryan Bogan <[email protected]>

* Remove unintended dependency changes

Signed-off-by: Ryan Bogan <[email protected]>

* Removed dynamic settings regitration, removed info() method, and added NoopExtensionsManager

Signed-off-by: Ryan Bogan <[email protected]>

* Add javadoc

Signed-off-by: Ryan Bogan <[email protected]>

* Fixed spotless failure

Signed-off-by: Ryan Bogan <[email protected]>

* Removed NoopExtensionsManager

Signed-off-by: Ryan Bogan <[email protected]>

* Added functioning NoopExtensionsManager

Signed-off-by: Ryan Bogan <[email protected]>

* Added missing javadoc

Signed-off-by: Ryan Bogan <[email protected]>

* Remove forbiddenAPI

Signed-off-by: Ryan Bogan <[email protected]>

* Fix spotless

Signed-off-by: Ryan Bogan <[email protected]>

* Change logger.info to logger.error in handleException

Signed-off-by: Ryan Bogan <[email protected]>

* Fix ExtensionsManagerTests

Signed-off-by: Ryan Bogan <[email protected]>

* Removing unrelated change

Signed-off-by: Ryan Bogan <[email protected]>

* Update SHAs

Signed-off-by: Ryan Bogan <[email protected]>

Signed-off-by: Ryan Bogan <[email protected]>

* Bump commons-compress from 1.21 to 1.22 (#5520)

Bumps commons-compress from 1.21 to 1.22.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-compress
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [Segment Replication] Trigger a round of replication for replica shards during peer recovery when segment replication is enabled (#5332)

* Fix new added replica shards falling behind primary.

Signed-off-by: Rishikesh1159 <[email protected]>

* Trigger a round of replication during peer recovery when segment replication is enabled.

Signed-off-by: Rishikesh1159 <[email protected]>

* Remove unnecessary start replication overloaded method.

Signed-off-by: Rishikesh1159 <[email protected]>

* Add test for failure case and refactor some code.

Signed-off-by: Rishikesh1159 <[email protected]>

* Apply spotless check.

Signed-off-by: Rishikesh1159 <[email protected]>

* Addressing comments on the PR.

Signed-off-by: Rishikesh1159 <[email protected]>

* Remove unnecessary condition check.

Signed-off-by: Rishikesh1159 <[email protected]>

* Apply spotless check.

Signed-off-by: Rishikesh1159 <[email protected]>

* Add step listeners to resolve forcing round of segment replication.

Signed-off-by: Rishikesh1159 <[email protected]>

Signed-off-by: Rishikesh1159 <[email protected]>

* Adding support to register settings dynamically (#5495)

* Adding support to register settings dynamically

Signed-off-by: Ryan Bogan <[email protected]>

* Update CHANGELOG

Signed-off-by: Ryan Bogan <[email protected]>

* Removed unnecessary registerSetting methods

Signed-off-by: Ryan Bogan <[email protected]>

* Change setting registration order

Signed-off-by: Ryan Bogan <[email protected]>

* Add unregisterSettings method

Signed-off-by: Ryan Bogan <[email protected]>

* Remove unnecessary feature flag

Signed-off-by: Ryan Bogan <[email protected]>

Signed-off-by: Ryan Bogan <[email protected]>

* Updated 1.3.7 release notes date (#5536)

Signed-off-by: owaiskazi19 <[email protected]>

Signed-off-by: owaiskazi19 <[email protected]>

* Pre conditions check before updating weighted routing metadata (#4955)

* Pre conditions check to allow weight updates for non decommissioned attribute

Signed-off-by: Rishab Nahata <[email protected]>

* Atomically update cluster state with decommission status and corresponding action (#5093)

* Atomically update the cluster state with decommission status and its corresponding action in the same execute call

Signed-off-by: Rishab Nahata <[email protected]>

* Update Netty to 4.1.86.Final (#5529)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

* Update release date in 2.4.1 release notes (#5549)

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Suraj Singh <[email protected]>

* Update 2.4.1 release notes (#5552)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

* Refactor fuzziness interface on query builders (#5433)

* Refactor Object to Fuzziness type for all query builders

Signed-off-by: noCharger <[email protected]>

* Revise on bwc

Signed-off-by: noCharger <[email protected]>

* Update change log

Signed-off-by: noCharger <[email protected]>

Signed-off-by: noCharger <[email protected]>
Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]>

* Upgrade lucene version (#5570)

* Added bwc version 2.4.2

Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>

* Added 2.4.2.

Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>

* Update Lucene snapshot to 9.5.0-snapshot-d5cef1c

Signed-off-by: Suraj Singh <[email protected]>

* Update changelog entry

Signed-off-by: Suraj Singh <[email protected]>

* Add 2.4.2 bwc version

Signed-off-by: Suraj Singh <[email protected]>

* Internal changes post lucene upgrade

Signed-off-by: Suraj Singh <[email protected]>

Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>
Signed-off-by: Suraj Singh <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]>

* Add CI bundle pattern to distribution download (#5348)

* Add CI bundle pattern for ivy repo

Signed-off-by: Zelin Hao <[email protected]>

* Gradle update

Signed-off-by: Zelin Hao <[email protected]>

* Extract path

Signed-off-by: Zelin Hao <[email protected]>

* Change with customDistributionDownloadType

Signed-off-by: Zelin Hao <[email protected]>

* Add default for exception handle

Signed-off-by: Zelin Hao <[email protected]>

* Add documentations

Signed-off-by: Zelin Hao <[email protected]>

Signed-off-by: Zelin Hao <[email protected]>

* Bump protobuf-java from 3.21.9 to 3.21.11 in /plugins/repository-hdfs (#5519)

* Bump protobuf-java from 3.21.9 to 3.21.11 in /plugins/repository-hdfs

Bumps [protobuf-java](https:/protocolbuffers/protobuf) from 3.21.9 to 3.21.11.
- [Release notes](https:/protocolbuffers/protobuf/releases)
- [Changelog](https:/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.21.9...v3.21.11)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Updated changelog

Signed-off-by: Owais Kazi <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Owais Kazi <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Owais Kazi <[email protected]>
Co-authored-by: Suraj Singh <[email protected]>

Signed-off-by: Rishikesh1159 <[email protected]>
Signed-off-by: Dhwanil Patel <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: Ashish Singh <[email protected]>
Signed-off-by: Nicholas Walter Knize <[email protected]>
Signed-off-by: Ralph Ursprung <[email protected]>
Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>
Signed-off-by: Ryan Bogan <[email protected]>
Signed-off-by: Poojita Raj <[email protected]>
Signed-off-by: Xue Zhou <[email protected]>
Signed-off-by: Marc Handalian <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: owaiskazi19 <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
Signed-off-by: Suraj Singh <[email protected]>
Signed-off-by: noCharger <[email protected]>
Signed-off-by: Zelin Hao <[email protected]>
Signed-off-by: Owais Kazi <[email protected]>
Co-authored-by: Rishikesh Pasham <[email protected]>
Co-authored-by: Dhwanil Patel <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
Co-authored-by: Ashish <[email protected]>
Co-authored-by: Nick Knize <[email protected]>
Co-authored-by: Ralph Ursprung <[email protected]>
Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]>
Co-authored-by: Ryan Bogan <[email protected]>
Co-authored-by: Poojita Raj <[email protected]>
Co-authored-by: Xue Zhou <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Owais Kazi <[email protected]>
Co-authored-by: Rishab Nahata <[email protected]>
Co-authored-by: Suraj Singh <[email protected]>
Co-authored-by: Louis Chu <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Zelin Hao <[email protected]>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
@rramachand21 rramachand21 added v2.5.0 'Issues and PRs related to version v2.5.0' roadmap labels Jan 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch v2.5.0 'Issues and PRs related to version v2.5.0'
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants