Redesign Upgrade Assistant for minor upgrades

[cjcenizal]

The 7.16 Upgrade Assistant in 7.16 was designed specifically for upgrading to 8.x. It offers features that aren’t relevant beyond this specific major upgrade, such as automated system index migration, and was never designed for assistance with minor upgrades. We disabled it in 8.0 for these reasons.

Our immediate short-term goal is to identify the subset of functionality that’s pertinent to minor upgrades, remove anything that’s extraneous, redesign the landing page and copy to reflect this new purpose, and re-enable Upgrade Assistant in an 8.x minor as soon as possible. This will give us a temporary and localized solution to the problem users will face of identifying and remediating breaking changes between minor releases.

[elasticmachine]

Pinging @elastic/kibana-stack-management (Team:Stack Management)

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[lukeelmers]

Core team will be stepping in to help Deployment Management move this effort forward

[pgayvallet]

Just to open the discussion: What kind of info / warnings we may want to surface for a minor upgrade?

to the problem users will face of identifying and remediating breaking changes between minor releases

Our breaking change policy actually forbid us of introducing any breaking change between minors, so I'm not sure this is a valid concern?

The only related thing I could see is to surface deprecations in advance of the next major. E.g in version 8.x, we rename a config setting (with the proper rename deprecation) and flag is for removal in 9. If the user upgrade to a new minor, it will continue to work given we're internally managing the rename, however we could surface the info as a warning in UA, to let the customer perform the config change in advance.

Note though, that we would always have one upgrade of delay. E.g if a setting rename is performed in 8.7, we won't have this info when the customer tries to upgrade from 8.4 to 8.7, given the setting was not yet renamed in 8.4

Our immediate short-term goal is to identify the subset of functionality that’s pertinent to minor upgrades

One obvious feature that would be very valuable for all minor upgrades is to detect if the cluster is in a healthy state before performing the upgrade. This is specifically targeting the SO migration: we want to be able to detect if the ES cluster is healthy enough to initiate the SO migration (e.g low watermark, routing allocation and so on)

We recently introduced the migration.discardUnknownObjects (and soon another one) config setting. It could also be interesting to notify the user in UA if this config value is set (to an older version) to inform them that they should either remove it or set it to the version they're planning to upgrade to.

Aside from migration-related warnings / info though, I'm not sure what exactly we could / should put into UA regarding minor upgrades.

[cjcenizal]

Our breaking change policy actually forbid us of introducing any breaking change between minors, so I'm not sure this is a valid concern?

I think you're probably right. I wrote this when we were still iterating on our new policy of introducing breaking changes after a specified deprecation period, which I anticipated could mean we'd move away from semver to some other versioning scheme.

The only related thing I could see is to surface deprecations in advance of the next major.

Your analysis of this UX looks sound to me. 👍

One obvious feature that would be very valuable for all minor upgrades is to detect if the cluster is in a healthy state before performing the upgrade.

The new ES Health API becomes available in 8.3, and will provide this information. Note that it's internal, so there are no BWC guarantees between versions. The ES Data Management team is leading this work.

Aside from migration-related warnings / info though, I'm not sure what exactly we could / should put into UA regarding minor upgrades.

Long-term we might want to also use UA to notify users when entire systems have been replaced with new ones, e.g. Index Templates v1 -> Index Templates v2 + Component Templates, Beats -> Fleet, and Watcher -> Alerting. UA can indicate how much work you need to do to migrate (e.g. "You're using 81 deprecated index templates"), and redirect you to the appropriate UI for digging deeper and executing the migration (e.g. "Click here to migrate them in the Index Management UI").

[cjcenizal]

Here's the issue that captures the UX of migrating index templates: #97507.

[rudolf]

Just to open the discussion: What kind of info / warnings we may want to surface for a minor upgrade?

to the problem users will face of identifying and remediating breaking changes between minor releases

Our breaking change policy actually forbid us of introducing any breaking change between minors, so I'm not sure this is a valid concern?

The only related thing I could see is to surface deprecations in advance of the next major. E.g in version 8.x, we rename a config setting (with the proper rename deprecation) and flag is for removal in 9. If the user upgrade to a new minor, it will continue to work given we're internally managing the rename, however we could surface the info as a warning in UA, to let the customer perform the config change in advance.

Note though, that we would always have one upgrade of delay. E.g if a setting rename is performed in 8.7, we won't have this info when the customer tries to upgrade from 8.4 to 8.7, given the setting was not yet renamed in 8.4

Yes. The key benefit is users are aware of an upcoming breaking change long before the major is released. Ultimately that should improve the adoption of major versions since users are more likely to have already migrated to the new API by the time the major is released.

[Bamieh]

Closing this issue as resolved #135720 back in jul 2022