[Security] [Maps] Network Map fails to create layers for Kibana Index Patterns containing multiple indices or exclusions

[spong]

As part of #52565 (and fix), support was added for pattern matching Kibana Index Patterns against securitySolution:defaultIndex when generating layers for the Network Map.

What this implementation doesn't take into account is that Kibana Index Patterns can contain multiple comma-separated Elasticsearch indices, e.g. filebeat-*,auditbeat-*, and with CCS + wildcards as *:filebeat-*,*:auditbeat-*, or even CCS + wildcards + exclusions as *:filebeat-*,-*:filebeat-7.6.0*.

This affects both the Elastic Security implementation as well as the Maps implementation (cc @nreese), since they use the non-split Kibana Index Pattern title as the pattern and the split securitySolution:defaultIndex as the path.

As example, for the given:

securitySolution:defaultIndex: cluster2:filebeat-*, cluser1:auditbeat-*
Kibana Index Pattern: *:filebeat-*,*:auditbeat-*

The matching sequence would be:

minimatch('cluster2:filebeat-*', '*:filebeat-*,*:auditbeat-*'); // false
minimatch('cluster1:auditbeat-*', '*:filebeat-*,*:auditbeat-*'); // false

which would result in no matches, and thus no generated map layers.

If we were to comma split the Kibana Index Pattern title, use each of those as the pattern and remove the duplicate matches we should then create layers for the correct Kibana Index Patterns that exist. Note: this would need to include logic for the exclusion case as well, which could be done using a filter on matched paths against patterns with a leading -.

This behavior is present since v7.6.0.

[elasticmachine]

Pinging @elastic/kibana-gis (Team:Geo)

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[stephmilovic]

@spong to confirm the results we want...

Case 1:
should "auditbeat-*" be included in the matching patterns??

// Kibana Index Patterns
["filebeat-*,auditbeat-*", "auditbeat-*"]

// SIEM indexes
["apm-*-transaction*",  "auditbeat-*", "endgame-*","filebeat-*","packetbeat-*","winlogbeat-*"]

// Matching Patterns
["filebeat-*,auditbeat-*", "auditbeat-*"]

Case 2:

// Kibana Index Patterns
["filebeat-*,-filebeat-7.6.0*", "auditbeat-*"]

// SIEM indexes
["apm-*-transaction*",  "auditbeat-*", "endgame-*","filebeat-*","packetbeat-*","winlogbeat-*"]

// Matching Patterns
["filebeat-*,-filebeat-7.6.0*", "auditbeat-*"]

Case 3:

// Kibana Index Patterns
["*:filebeat-*,*:auditbeat-*", "auditbeat-*"]

// SIEM indexes
['cluster2:filebeat-*', 'cluster1:auditbeat-*']

// Matching Patterns
["*:filebeat-*,*:auditbeat-*"]

[stephmilovic]

Fixed by this PR: #80208

[stephmilovic]

Merged, going to unassign myself and mark this as fixed 👍

[stephmilovic]

Assigning to @MadameSheema to confirm fix!

[MadameSheema]

@stephmilovic we need to continue working on this issue :)