Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce field count for SIEM saved object types #138726

Open
rudolf opened this issue Aug 12, 2022 · 1 comment
Open

Reduce field count for SIEM saved object types #138726

rudolf opened this issue Aug 12, 2022 · 1 comment

Comments

@rudolf
Copy link
Contributor

rudolf commented Aug 12, 2022

SIEM is currently using 128 fields across the following types:

  • siem-ui-timeline:96
  • siem-ui-timeline-note:7
  • siem-ui-timeline-pinned-event:6
  • siem-detection-engine-rule-actions:9
  • siem-detection-engine-rule-execution-info:10

Especially siem-ui-timeline with its 96 fields seems really high and from a rough search it seems like we might only be searching/aggregating over a handful of these fields. Could we reduce the field count of these types?

From https:/elastic/kibana/blob/main/dev_docs/tutorials/saved_objects.mdx#mappings

Do not use field mappings like you would use data types for the columns of a SQL database. Instead, field mappings are analogous to a SQL index. Only specify field mappings for the fields you wish to search on or query. By specifying dynamic: false in any level of your mappings, Elasticsearch will accept and store any other fields even if they are not specified in your mappings.
Since Elasticsearch has a default limit of 1000 fields per index, plugins should carefully consider the fields they add to the mappings. Similarly, Saved Object types should never use dynamic: true as this can cause an arbitrary amount of fields to be added to the .kibana index.

Related #43673

@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants