Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replication namespace #464

Draft
wants to merge 51 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
106fd00
replication added
Tokesh Aug 3, 2024
e912e13
fixing lint and trying test verbose
Tokesh Aug 3, 2024
bd877a8
changing docker compose
Tokesh Aug 3, 2024
6fd6292
Merge branch 'main' into replication-namespace
Tokesh Aug 3, 2024
addcd76
adding tests for replication namespace
Tokesh Aug 3, 2024
638d60e
fixing lint, validate specs ci
Tokesh Aug 3, 2024
a6e0a87
fixing docker-compose and validate specs ci
Tokesh Aug 3, 2024
b012e7b
fixing validate specs ci
Tokesh Aug 3, 2024
236fea7
trying to fix validate specs
Tokesh Aug 3, 2024
864ee68
trying to fix validate specs v2
Tokesh Aug 3, 2024
30fca51
Merge branch 'replication-namespace'
Tokesh Aug 14, 2024
1d051cd
Merge branch 'main' of https:/Tokesh/opensearch-api-speci…
Tokesh Aug 14, 2024
0cdec4d
Merge pull request #1 from Tokesh/main
Tokesh Aug 14, 2024
cacd32e
fixing tests which using cluster of OS
Tokesh Aug 14, 2024
2548bbf
add changelog changes, fixing ci issues
Tokesh Aug 14, 2024
eaab260
fixing empty volumes in docker-compose
Tokesh Aug 14, 2024
1fbca08
trying to fix docker compose
Tokesh Aug 14, 2024
833d3fd
adding supporting cluster containers
Tokesh Aug 14, 2024
b861f51
configuring docker compose
Tokesh Aug 14, 2024
e83589c
trying to fix docker-compose problems
Tokesh Aug 14, 2024
bc2800b
adding verbose to check
Tokesh Aug 14, 2024
87bdbdb
changing follower cluster role
Tokesh Aug 14, 2024
f075bd7
adding default follower role
Tokesh Aug 14, 2024
5cd2b95
trying to catch all possible roles
Tokesh Aug 14, 2024
d114038
deleting response
Tokesh Aug 14, 2024
ac1dc59
adding all access
Tokesh Aug 14, 2024
8c6ec73
checking all_access
Tokesh Aug 14, 2024
334b09d
adding admin to rolemapping
Tokesh Aug 14, 2024
018be31
checking current user info
Tokesh Aug 14, 2024
b4f6a3c
testing new docker-compose
Tokesh Aug 15, 2024
f609a97
adding opensearch_url
Tokesh Aug 15, 2024
b2d92e8
trying to change url
Tokesh Aug 15, 2024
bbc3ae1
trying to handle in https
Tokesh Aug 15, 2024
a1c8cc0
trying to handle in https
Tokesh Aug 15, 2024
f2a898d
checking role mapping
Tokesh Aug 15, 2024
bd92a0c
adding round at the end of synopsis
Tokesh Aug 15, 2024
7803383
adding getting users and user detail
Tokesh Aug 15, 2024
e04fec9
trying to create
Tokesh Aug 15, 2024
90e72da
fixing uri
Tokesh Aug 15, 2024
e79cb97
deleting not neccessary synopsis
Tokesh Aug 15, 2024
fd4349c
deleting not neccessary synopsis
Tokesh Aug 15, 2024
2b6dbdb
waiting 201 not 200
Tokesh Aug 15, 2024
56e1c56
adding permissions
Tokesh Aug 15, 2024
ba4dcea
formatting yaml
Tokesh Aug 15, 2024
4e20ee9
testing concept from documentation
Tokesh Sep 5, 2024
a603d74
Merge branch 'main' into replication-namespace
Tokesh Sep 5, 2024
03c4bc2
fixing tabling
Tokesh Sep 5, 2024
345c054
fixing tabling v2
Tokesh Sep 5, 2024
3f98811
adding index pattern
Tokesh Sep 5, 2024
8798e82
getting all role mappings
Tokesh Sep 5, 2024
5ed10fb
checking roles
Tokesh Sep 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .cspell
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ authc
authinfo
authtoken
autocut
autofollow
backendroles
backpressure
beider
Expand Down Expand Up @@ -67,6 +68,7 @@ hybridfs
integ
internalusers
Intertransport
ipam
Jelinek
kibanainfo
kibanaserver
Expand All @@ -81,6 +83,7 @@ lucene
Lucene
lycheeverse
marvinpinto
memlock
metaphone
mget
millis
Expand Down Expand Up @@ -180,6 +183,7 @@ tokenfilters
translog
Translog
tubone
ulimits
Undeploys
unigrams
Unmanaged
Expand Down
Empty file added .env
Empty file.
10 changes: 9 additions & 1 deletion .github/workflows/test-spec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ jobs:
tests: routing
- version: 2.16.0
tests: snapshot
- version: 2.16.0
tests: replication
- version: 2.17.0
hub: opensearchstaging
ref: '@sha256:1273489ebbedcb470ea13563dae4c6dc6b2ed431e87e686ed931ae0733034b25'
Expand Down Expand Up @@ -68,11 +70,17 @@ jobs:
- name: Run OpenSearch Cluster
working-directory: tests/${{ matrix.entry.tests || 'default' }}
run: docker compose up -d

- name: Get Container ID
id: container
run: |
CONTAINER_ID=$(docker ps -aqf "ancestor=opensearchproject/opensearch:${{ matrix.entry.version }}" | head -n 1)
echo "CONTAINER_ID=$CONTAINER_ID" >> $GITHUB_OUTPUT

- name: Run Tests
run: |
npm run test:spec -- \
--opensearch-insecure \
--opensearch-insecure --verbose\
--opensearch-version=${{ matrix.entry.version }} \
--coverage coverage/test-spec-coverage-${{ matrix.entry.version }}-${{ matrix.entry.tests || 'default' }}.json \
--tests=tests/${{ matrix.entry.tests || 'default' }}
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Added support for request headers in tests [#461](https:/opensearch-project/opensearch-api-specification/pull/461)
- Added metadata additionalProperties to `ErrorCause` ([#462](https:/opensearch-project/opensearch-api-specification/pull/462))
- Added `creation_date` field to `DanglingIndex` ([#462](https:/opensearch-project/opensearch-api-specification/pull/462))
- Added `replication` namespace API specifications ([#464](https:/opensearch-project/opensearch-api-specification/pull/464))
- Added doc on `cluster create-index blocked` workaround ([#465](https:/opensearch-project/opensearch-api-specification/pull/465))
- Added `observability` namespace API specifications ([#474](https:/opensearch-project/opensearch-api-specification/pull/474))
- Added support for reusing output variables as keys in payload expectations ([#471](https:/opensearch-project/opensearch-api-specification/pull/471))
Expand Down
322 changes: 322 additions & 0 deletions spec/namespaces/replication.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,322 @@
openapi: 3.1.0
info:
title: OpenSearch Replication API
description: OpenSearch Replication API.
version: 1.0.0
paths:
/_plugins/_replication/{index}/_start:
put:
operationId: replication.start.0
x-operation-group: replication.start
x-version-added: '1.1'
description: Initiate replication of an index from the leader cluster to the follower cluster.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#start-replication
parameters:
- $ref: '#/components/parameters/replication.start::path.index'
requestBody:
$ref: '#/components/requestBodies/replication.start'
responses:
'200':
$ref: '#/components/responses/replication.start@200'
/_plugins/_replication/{index}/_stop:
post:
operationId: replication.stop.0
x-operation-group: replication.stop
x-version-added: '1.1'
description: Terminates replication and converts the follower index to a standard index.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#stop-replication
parameters:
- $ref: '#/components/parameters/replication.stop::path.index'
requestBody:
$ref: '#/components/requestBodies/replication.stop'
responses:
'200':
$ref: '#/components/responses/replication.stop@200'
/_plugins/_replication/{index}/_pause:
post:
operationId: replication.pause.0
x-operation-group: replication.pause
x-version-added: '1.1'
description: Pauses replication of the leader index.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#pause-replication
parameters:
- $ref: '#/components/parameters/replication.pause::path.index'
requestBody:
$ref: '#/components/requestBodies/replication.pause'
responses:
'200':
$ref: '#/components/responses/replication.pause@200'
/_plugins/_replication/{index}/_resume:
post:
operationId: replication.resume.0
x-operation-group: replication.resume
x-version-added: '1.1'
description: Resumes replication of the leader index.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#resume-replication
parameters:
- $ref: '#/components/parameters/replication.resume::path.index'
requestBody:
$ref: '#/components/requestBodies/replication.resume'
responses:
'200':
$ref: '#/components/responses/replication.resume@200'
/_plugins/_replication/{index}/_status:
get:
operationId: replication.status.0
x-operation-group: replication.status
x-version-added: '1.1'
description: Gets the status of index replication.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#get-replication-status
parameters:
- $ref: '#/components/parameters/replication.status::path.index'
responses:
'200':
$ref: '#/components/responses/replication.status@200'
/_plugins/_replication/leader_stats:
get:
operationId: replication.leader_stats.0
x-operation-group: replication.leader_stats
x-version-added: '1.1'
description: Gets information about replicated leader indexes on a specified cluster.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#get-leader-cluster-stats
responses:
'200':
$ref: '#/components/responses/replication.leader_stats@200'
/_plugins/_replication/follower_stats:
get:
operationId: replication.follower_stats.0
x-operation-group: replication.follower_stats
x-version-added: '1.1'
description: Gets information about follower (syncing) indexes on a specified cluster.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#get-follower-cluster-stats
responses:
'200':
$ref: '#/components/responses/replication.follower_stats@200'
/_plugins/_replication/autofollow_stats:
get:
operationId: replication.autofollow_stats.0
x-operation-group: replication.autofollow_stats
x-version-added: '1.1'
description: Gets information about auto-follow activity and any replication rules configured on the specified cluster.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#get-auto-follow-stats
responses:
'200':
$ref: '#/components/responses/replication.autofollow_stats@200'
/_plugins/_replication/{index}/_update:
put:
operationId: replication.update_settings.0
x-operation-group: replication.update_settings
x-version-added: '1.1'
description: Updates settings on the follower index.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#update-settings
parameters:
- $ref: '#/components/parameters/replication.update_settings::path.index'
requestBody:
$ref: '#/components/requestBodies/replication.update_settings'
responses:
'200':
$ref: '#/components/responses/replication.update_settings@200'
/_plugins/_replication/_autofollow:
post:
operationId: replication.create_replication_rule.0
x-operation-group: replication.create_replication_rule
x-version-added: '1.1'
description: Automatically starts replication on indexes matching a specified pattern.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#create-replication-rule
requestBody:
$ref: '#/components/requestBodies/replication.create_replication_rule'
responses:
'200':
$ref: '#/components/responses/replication.create_replication_rule@200'
delete:
operationId: replication.delete_replication_rule.0
x-operation-group: replication.delete_replication_rule
x-version-added: '1.1'
description: Deletes the specified replication rule.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#delete-replication-rule
requestBody:
$ref: '#/components/requestBodies/replication.delete_replication_rule'
responses:
'200':
$ref: '#/components/responses/replication.delete_replication_rule@200'
components:
parameters:
replication.start::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
replication.stop::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
replication.pause::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
replication.resume::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
replication.status::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
replication.update_settings::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
requestBodies:
replication.start:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/Replication'
required: true
replication.stop:
content:
application/json:
schema:
type: object
required: true
replication.pause:
content:
application/json:
schema:
type: object
required: true
replication.resume:
content:
application/json:
schema:
type: object
required: true
replication.update_settings:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/UpdateSettings'
required: true
replication.create_replication_rule:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/CreateReplicationRule'
required: true
replication.delete_replication_rule:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/DeleteReplicationRule'
required: true
responses:
replication.start@200:
content:
application/json:
schema:
type: object
properties:
acknowledged:
type: boolean
replication.stop@200:
content:
application/json:
schema:
type: object
properties:
acknowledged:
type: boolean
replication.pause@200:
content:
application/json:
schema:
type: object
properties:
acknowledged:
type: boolean
replication.resume@200:
content:
application/json:
schema:
type: object
properties:
acknowledged:
type: boolean
replication.status@200:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/Status'
replication.leader_stats@200:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/LeaderStatus'
replication.follower_stats@200:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/FollowerStatus'
replication.autofollow_stats@200:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/AutoFollowStatus'
replication.update_settings@200:
content:
application/json:
schema:
type: object
properties:
acknowledged:
type: boolean
replication.create_replication_rule@200:
content:
application/json:
schema:
type: object
properties:
acknowledged:
type: boolean
replication.delete_replication_rule@200:
content:
application/json:
schema:
type: object
properties:
acknowledged:
type: boolean
Loading
Loading