[PROPOSAL] Align on meaning of a successfully initialized extension

[dbwiddis]

What are you proposing?

Initializing an extension requires several steps, including:

• Establishing a transport connection between the nodes
• Passing REST API from extension to orchestrator
• Registering the REST API with the RestHandler
• Passing NamedWriteables from extension to orchestrator
• Registering the NamedWriteables
• Future security capabilities such as passing tokens

Each of these events involves network transporation and ample possibility (expectation) of occasional IOExceptions. We need to align on:

• what are the minimum steps to consider an extension initialized?
• how should transport errors be handled? Should there be retries on failures?
• when and how can an extension's API or NamedWriteables be updated on-the-fly and how will the success of this update be communicated

How did you come up with this proposal?

Background: #74 (comment)

What is the user experience going to be?

Ideally, everything just works for the user on initialization. In the event things don't work, the user should have sufficient information in log files to quickly identify and correct any configuration errors.

If the user is updating API or NamedWriteables during operation, the success of the update should be promptly communicated to the user, with similar debugging information as above on failure.

Why should it be built? Any reason not to?

Users need to understand what the log message "Extension initialized" implies.

What will it take to execute?

We need a more detailed design document on extension lifecycle.

Presently extensions are assumed to be already in operation when the orchestrator starts. We need to handle extension node shutdown/restart or changing nodes, changing APIs, removing an extension mid-operation (and removing its registrations), etc.

What are remaining open questions?

What other exceptional situations need to be handled?

[owaiskazi19]

Thanks for writing this proposal @dbwiddis! It covers different aspects of initializing extension which is not present currently.

Users need to understand what the log message "Extension initialized" implies.

OpenSearch should be able to send metadata to extension consisting of the NamedXContent it has sent, it's host and port number, REST Handlers currently present and security credentials.

what are the minimum steps to consider an extension initialized?

We can add the above mentioned items in the checklist before saying that extension has been initialized.

how should transport errors be handled? Should there be retries on failures?

I feel like we should include attempt retry of 5 before throwing the failure.

when and how can an extension's API or NamedWriteables be updated on-the-fly and how will the success of this update be communicated

This is something we can take as a different issue to add dynamic support for extensions