[UA] Filter out elastic products deprecation logs

[sebelga]

This PR adds the filters to not display deprecation logs coming from Elastic products in Upgrade Assistant.

How to test

• Make sure to run the latest ES snapshot (yarn es snapshot --license trial) with Kibana
• Navigate to Upgrade Assistant
• Under "Review deprecated settings and resolve issues" click on the "Elasticsearch deprecation logs" link
• On an empty cluster there shouldn't be any deprecation in the counter
• Navigate to both the Discover and Observability and make sure that the filters are applied to not include Elastic products

Test the filters

• Open Console in a new tab and add a deprecation with a product that does not exists

POST .logs-deprecation.elasticsearch-default/_doc/
{
  "event.dataset" : "deprecation.elasticsearch",
  "@timestamp" : "2021-12-25T13:51:40,618Z", // Date in the future
  "log.level" : "WARN",
  "log.logger" : "org.elasticsearch.deprecation.cluster.metadata.IndexNameExpressionResolver",
  "elasticsearch.cluster.name" : "elasticsearch",
  "elasticsearch.cluster.uuid" : "4q7OmaQkT-S0v-GJogb7yQ",
  "elasticsearch.node.id" : "qY6aIkdIQTiAOdKbqMiQCg",
  "elasticsearch.node.name" : "Elastic.lan",
  "trace.id" : "",
  "message" : "this request accesses system indices: [.foo], but in a future major version, direct access to system indices will be prevented by default",
  "data_stream.type" : "logs",
  "data_stream.dataset" : "deprecation.elasticsearch",
  "data_stream.namespace" : "default",
  "ecs.version" : "1.7",
  "elasticsearch.event.category" : "api",
  "event.code" : "open_system_index_access",
  "elasticsearch.http.request.x_opaque_id" : "foo",
  "elasticsearch.elastic_product_origin" : "other" // Give any unknown name here
}

• Back in Upgrade assistant, the counter should have incremented by 1
• Back in console add a new doc but this time set the "elasticsearch.elastic_product_origin" : "kibana"
• Back in Upgrade assistant the counter should stay the same
• From UA navigate to Discover and in Discover edit the filter. Remove the "kibana" app
• You should now see the last log created in Console

Known issue

During the testing I realised that simply by entering the Console app we are generating deprecation logs. It seems that we have decided not to add the product origin header for requests coming from console but I do think we should add the header there too.

@sabarasaba pointed out that under the hood console make the following requests for the autocomplete functionality

GET _aliases
GET _mappings
GET _template

which would be the cause of the deprecation being generated

Request made in console that are deprecated in 8.0 will throw an error if the user executes them after upgrading. So it seems that we don't need to display deprecation logs for those requests made by Console in 7.x.

Fixes #120152

[elasticmachine]

Pinging @elastic/kibana-stack-management (Team:Stack Management)

[sabarasaba]

Thanks for working no this @sebelga! Code changes lgtm, left a few comments in a few tests.

While I was testing this out I noticed a few things:

• On an empty cluster with today's snapshot there's one deprecation showing in the count:

• The data view that we use in external links does not have elasticsearch.elastic_product_origin so we end up with an empty filter on the generated url which leads to having no filters applied in the discovery app:

• When generating new deprecations I ended up with 4 in the count but only 3 being shown in the observability app:

Screen.Recording.2021-12-20.at.10.09.01.mov

Unsure if the last two are side effects of perhaps the same problem.

[sebelga]

Thanks for the review @sabarasaba ! I addressed your 2 suggestions from the CR.

On an empty cluster with today's snapshot there's one deprecation showing in the count

Have you looked if it contained the elastic-product origin in the log? If not that's the reason it still appears in UA and we should try to find the team responsible. It wouldn't be a UA bug then. Looking at your screenshot it looks like a deprecated "legacy index template", is it possible that it was Console that triggered it?

The data view that we use in external links does not have elasticsearch.elastic_product_origin so we end up with an empty filter on the generated url which leads to having no filters applied in the discovery app:

That is OK, if there are no log with that field defined then the Data view field does not exist and we don't pass a filter. There wouldn't be anything to filter if there aren't any log containing that field 😊

When generating new deprecations I ended up with 4 in the count but only 3 being shown in the observability app

That needs to be investigated as indeed the count should match the logs in Observability. Mabe it is a mismatch in the @timestamp position? I will try to reproduce it on my side.

[sebelga]

@elasticmachine merge upstream

[sabarasaba]

@sebelga The log was there without going to console, just open up kibana login and go directly to UA, I'll see if I can figure out what's causing it then! Also, let me know if you had any luck replicating the count mismatch, otherwise I'm happy to zoom see if we can find it together!

[sebelga]

@sabarasaba The initial deprecation log is generated by the security team when starting Kibana. They've opened an issue to fix it (#121311). Once that issue get resolved there shouldn't be any log when starting with an empty cluster.

[sebelga]

@elasticmachine merge upstream

[sabarasaba]

The mismatch in the deprecations count and what was shown in the Observability app was due to my mistake of putting a future date in the timestamp when generating the deprecation. Since the count API doesn't have an upper time boundary it grabs all deprecations since the last checkpoint forward, but the Observability app grabs all the deprecations from the last checkpoint until now.

[sebelga]

Thanks for looking into the issue @sabarasaba ! I've updated the count search to discard deprecation in the future (they should theoretically never happen). fe2de31

[cjcenizal]

they should theoretically never happen

Dumb question... what if a log is created in a timezone that's ahead of the one you're currently in? Are all timestamps normalized somehow, for example by storing all timestamps as UTC?

[sebelga]

Not a dumb question, it made me realise that I need to add timezone information to the search lterange query. 😊

From what I see in different Discuss yes ES saves all timestamp in UTC and Kibana expects @timestamp in UTC and then converts to the local timezone for display.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 06c8bb5
• Storybooks Preview

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
upgradeAssistant	145	204	+59

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
upgradeAssistant	138.8KB	168.5KB	+29.7KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
upgradeAssistant	18.3KB	18.5KB	+209.0B

History

• 💔 Build #14502 failed fe2de31
• 💔 Build #14324 failed a789543
• 💔 Build #14299 failed 4303981
• 💔 Build #14198 failed 616b6ac
• 💔 Build #14185 failed d6b4926

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream