[SavedObjects] Ability to allow a type to be importable without it showing in the UI

[jonathan-buttner]

The Cases app leverages inbound references for a few of the saved objects. The Comments and User Actions saved objects within cases reference the parent Case saved object. On exporting of a Case, we've implemented the onExport hook to handle finding the inbound associations to that case and including them in the exported ndjson file.

We run into a problem when trying to import the file because the Comments and User Actions don't have valid types to be imported. We can get around this by setting the importableAndExportable flag on Comments and User Actions but it causes those saved objects to show up in the Saved Object Management UI

We can prevent Comments and User Actions from being exported by themselves using the isExportable hook but this means that the UI is going to list every user action and comment for all cases and might be confusing to the user as to which items need to be selected for export.

Ideally we'd have a way to only show the Cases saved objects and hide all the associated saved objects that point to that case from the UI.

WIP PR for exporting cases: #110148

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[elasticmachine]

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

[jonathan-buttner]

cc: @kobelb

[kobelb]

The ability to import/export Cases is a dependency of #82020. If there's any way to address this limitation by 7.16, it'd be greatly appreciated.

[jonathan-buttner]

I met with @XavierM @pgayvallet @rudolf to discuss how to move forward for 7.16. For now we're going to implement the export functionality using the available flags. This will result in the associated objects showing up in the saved objects management UI. The core team is going to try and get changes implemented for 7.16 to give us the ability to hide these associated objects. If that work is completed in time, Cases will include it for 7.16.

[jonathan-buttner]

@pgayvallet @rudolf initially I thought I could use isExportable: (_: SavedObject<unknown>) => false to prevent users from exporting specific saved objects directly in the UI while still allowing them to be exported when the parent case is exported. Unfortunately, using this setting excludes the object when the parent case is exported. So with the current setup, the user will be able to export user actions and comments directly.

Here's an example ndjson file:

{"attributes":{"closed_at":null,"closed_by":null,"connector":{"fields":[],"name":"none","type":".none"},"created_at":"2021-08-26T19:48:01.292Z","created_by":{"email":null,"full_name":null,"username":"elastic"},"description":"a description","external_service":null,"owner":"securitySolution","settings":{"syncAlerts":true},"status":"open","tags":["some tags"],"title":"A case to export","type":"individual","updated_at":"2021-08-26T19:48:30.151Z","updated_by":{"email":null,"full_name":null,"username":"elastic"}},"coreMigrationVersion":"8.0.0","id":"85541260-06a6-11ec-b3f9-3d05c48a7d46","migrationVersion":{"cases":"7.15.0"},"references":[],"type":"cases","updated_at":"2021-08-26T19:48:30.162Z","version":"WzE3NDQsMV0="}
{"excludedObjects":[{"id":"9687c220-06a6-11ec-b3f9-3d05c48a7d46","reason":"excluded","type":"cases-comments"},{"id":"8cb85070-06a6-11ec-b3f9-3d05c48a7d46","reason":"excluded","type":"cases-user-actions"},{"id":"9710c840-06a6-11ec-b3f9-3d05c48a7d46","reason":"excluded","type":"cases-user-actions"}],"excludedObjectsCount":3,"exportedCount":1,"missingRefCount":0,"missingReferences":[]}

[pgayvallet]

We went back and forth numerous times on if we should, or not, dissociate a type's exportability status of its visibility in the SOM section.

Until now, we didn't identify any concrete use case of this need. However the current issue seems to be a perfect example of why we need to allow type owners to dissociate these meta.

Currently, SavedObjectsType.management.importableAndExportable is doing both:

• it's required to set it to true to have import/export support, both from the UI and directly from the HTTP API
• setting it to true will also necessarily display the type as selectable and searchable in SOM

I should the easiest and obvious solution to split these two would be to introduce an additional property to add control on an exportable type's visibility in the SOM page.

For BWC, the new property should be optional, and default the old behavior if unspecified. For this reason, I think it would be more logical to add an opt-out flag rather than an opt-in.

I propose to add a new hiddenFromManagement property to SavedObjectsTypeManagementDefinition (SavedObjectType.management)

importableAndExportable	hiddenFromManagement	exportable and importable via the HTTP API or when included via hooks	listed and searchable in the SOM page
unspecified	unspecified	🔴	🔴
true	unspecified	🟢	🟢
true	true	🟢	🔴
true	false	🟢	🟢
false	unspecified	🔴	🔴
false	false	🔴	🔴
false	true	🔴	🔴

Note: hiddenFromManagement could be replaced by visibleInManagement (defaulting to true), if we think this makes more sense.

@rudolf @kobelb wdyt?

[rudolf]

I agree about splitting these two very different things. My vote goes for visibleInManagement because the double negative in hiddenFromManagement: false makes it slightly harder to understand the option.

[pgayvallet]

Just to be sure we're all on the same page:

We're all aware that the suggested approach will only allow to exclude exportable objects from the SO management UI by specifying importableAndExportable: true and visibleInManagement: false, and not to allow to display non-exportable objects in the SOM UI by configuring it's type with importableAndExportable: false and visibleInManagement: true, right (this would throw an error during the type registration)?

If allowing the latest would eventually make sense, we don't have any identified needs for such configuration and given the additional amount of work that this would require, it doesn't seem worth it, at least for now.

[jonathan-buttner]

We're all aware that the suggested approach will only allow to exclude exportable objects from the SO management UI by specifying importableAndExportable: true and visibleInManagement: false

Yeah that's fine for Cases.