Overwriting a saved object clears the originId field

[jportner]

Kibana version: 8.0.0 - 8.3.0 (current)

Update: the bug that accidentally clears the originId field has been fixed in 8.3.1 via #135358
However, we still need a solution to repair objects that have already been affected.

Describe the bug:

In the 8.0 release, we converted many object types to be "share-capable" (#100489).
When we introduced "share-capable" and "shareable" object types, these have globally unique IDs, and we added an explicit "origin" that is set (1) when an object is upgraded and its ID is regenerated, and (2) when an object is copied or imported to a different space.

The originId field allows import/copy behavior to remain the same with the "Check for existing objects" option, e.g., you can import at most one copy of an object to a given space. If you do it multiple times, you will be prompted to overwrite the previous copy of the object.

Now for the bug: if you re-create an object using the overwrite: true option, you'll lose the originId field that existed on the old version of the object. If that happens, future import/copy attempts using the "Check for existing objects" option can break in two different ways:

1. If the object existed in 7.x (prior to upgrading to 8.0), and it was in a custom space: there is a legacy URL alias that points to the object to preserve its old URL, and that results in an "aliasConflict" error during the create attempt -- this effectively means you can't import/copy that object to that space anymore while the alias exists
2. If the object was created after 8,0: there is no legacy URL alias that points to the object to preserve its URL, so it won't run into an "aliasConflict" error. However, importing/copying will no longer work as expected, instead it will create a second copy of that object in that space.

Unfortunately, when you edit a dashboard, a lens visualization, or a legacy visualization, under the hood Kibana re-creates the object, unintentionally stripping the originId field. I haven't exhaustively tested all share-capable/shareable object types, but I imagine others are affected as well.

Steps to reproduce:

1. Start Kibana
2. Add sample data
3. Create "Another" space
4. Copy a dashboard to Another space using the "Check for existing objects" option
5. In Another space, make a change to the dashboard and save it
6. In the Default space, copy the dashboard again using the "Check for existing objects" option
7. Observe that there are now two copies of the dashboard in Another space -- one that has the correct originId (the second copy), and one that does not (the first copy)

Expected behavior:

The root-level originId field is considered to be Kibana-owned metadata, and it should be preserved even when the object is overwritten (similar to the namespaces field).

Any additional context:

Unfortunately, after an object has had its origin stripped in this manner, we may not have enough information to automatically repair it:

• If there is a legacy URL alias pointing to this object, we can infer that (1) it was originally created prior to 8.0, and (2) this object should have a corresponding originId. We could add a "post-start" migration to repair affected objects in this manner. However, a regular SO migration wouldn't work, because each SO migration is applied to an object in isolation and does not allow for querying during the migration.
• Any object that was freshly copied to another space after the 8.0 release has an originId but does not have a legacy URL alias. In that case, after its origin is stripped, there's no way to figure out what its origin used to be 😞

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[afharo]

@TinaHeiligers I get access denied 😢

[TinaHeiligers]

@TinaHeiligers I get access denied 😢

I deleted the comment with the link. I think there's a dev doc somewhere about this or an RFC. I'll try to find some public reference that summarizes the issue.

[TinaHeiligers]

@elastic/kibana-security It would be very useful if we had public documentation around shared saved objects to reference if and when problems arise.

Unfortunately, ATM, our knowledge articles are for internal use only and we can't share those.

Is it possible to create summaries of these and add them to the docs, considering the recent team changes? I'm thinking along the lines of something similar to how Core documents known issues with saved objects migrations.

[azasypkin]

@TinaHeiligers I'm on the fence here to be honest, I'm not sure if we should separately document one-time bugs (i.e. after specific patch-version upgrade this issue should no longer be relevant) like that in our main docs. For these we usually create knowledge-sharing articles to help our support team (we have one for this particular issue too) and document issue (and potentially workaround) in the PR/issue that users can get to via the link in our release notes.

But having "known issues" docs for the recurring issues related to the shareable SO is a great idea, we certainly should have it!

[TinaHeiligers]

To be clear, the 👍 is for documenting recurring issues related to the shareable SO.

[rayafratkina]

Stale issue. Closing