Missing a behavioral definition of routing_rules.yml

[andrewkroh]

The package-spec definition of routing rules doesn't explain what happens to these rules. As a developer I need to know how these affect the data stream behavior. IMO one of these sections should be expanded to include this information to developers.

package-spec/spec/integration/data_stream/routing_rules.spec.yml

Line 58 in 2758f3e

description: Routing rules set.

package-spec/spec/integration/data_stream/routing_rules.spec.yml

Line 8 in 2758f3e

description: Routing rule definition (technical preview)

My suggestion is to include this information:

Each rule generates an Elasticsearch reroute processor that is injected into the data stream's default ingest pipeline. The reroute processors are added after the injected "custom" pipeline processors (e.g. global@custom, <data_stream.type>@custom, <data_stream.type>-<package_name>.integration@custom, <data_stream.type>-<data_stream.dataset>@custom) such that the user's customizations run before the reroute rules.

References:

• https:/elastic/kibana/blob/a493e4075b7267d34715607492197fc49c77b081/x-pack/plugins/fleet/server/services/epm/elasticsearch/ingest_pipeline/helpers.ts#L131-L136