Inject Beat specific default processors

[michalpristas]

What does this PR do?

Managed beats do not respect default processors configured in {beat}.yml file.
This results in events without host, k8s or docker metadata.

In this PR we take default processors from config file to callback responsible for handling configuration passed from agent and make sure they are included during generate step.

Why is it important?

Missing metadata may break dashboards

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Fixes: #34125

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-12-30T16:05:51.205+0000

• Duration: 85 min 38 sec

Test stats 🧪

Test	Results
Failed	0
Passed	7408
Skipped	338
Total	7746

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[michalpristas]

Reordered processors so default are first,

[cmacknz]

There is this above:

		// 1. global processors
		streamSource = injectGlobalProcesssors(raw, streamSource)

The metadata processors behave more like global processors don't they? Shouldn't they be added there?

Both prepend and append are used here to control the order so just reading this code it is a bit tricky to follow what ends up where.

Could you post an example beat-rendered-config if you have one from diagnostics with this change?

[michalpristas]

yes but if you check the final order of the processors it's

		// now the order of the processors on this input is as follows
		// 1. stream processors
		// 2. agentInfo processors
		// 3. global processors
		// 4. stream specific processors

so if we put it with global, they will be pushed to later stage by next 2 steps. this is why i went with stream

• it makes sure it's first
• it makes clear that it will be part of the stream

[cmacknz]

Got it, thanks.

[cmacknz]

This file was committed by mistake I think, I see this come in when changing branches occasionally.

[michalpristas]

yep probably changing branches

[cmacknz]

Thanks!

[fearful-symmetry]

Just a heads up, I'm running into some weird behaviors with the processors under elastic-agent. They seem to be putting the beat in some kind of wedged state where elastic-agent won't actually stop when it gets a sigint. I can't manually remove the processors either, but commenting out the defaultProcessors line seems to fix it. Will investigate more tomorrow.

[cmacknz]

Thanks. FYI the default processors aren't new, they would have been configured in a different way prior to 8.6 (reading the default beat config file at startup) but they would have been running regardless.