Kubernetes Source: Add namespace annotations to event metadata

[jszwedko]

Discussed in #21409

^{Originally posted by matt-simons October 3, 2024}
https://vector.dev/docs/reference/configuration/sources/kubernetes_logs/#namespace_annotation_fields

TLDR: please add namespace_annotation_fields.namespace_annotations because labels are restricted to 63 characters which makes it hard to store much data for VRL transformations for example.

---

We currently give users of our platform a lot of flexibility to configure their observability pipelines via namespace labels however we need to add a new label for every bit of config and in VRL individually parse and handle each one which is a bit onerous.

We would like to move from this

labels:
  splunk.mycompany/forwarding: enabled 
  splunk.mycompany/index: customer-dev-index
  splunk.mycompany/source: platform
  splunk.mycompany/sourcetype: _json
  scalyr.mycompany/forwarding: enabled
  scalyr.mycompany/token: <base64-encoded-token>
  format.mycompany/my-container-name: keyvalue3

to this

annotations:
  logging.mycompany/config: |
    {
      "transforms": {
        "my-container-name": {
          "format": "keyvalue3",
          "remove_fields": ["message"],
          "timestamp": {
            "field": "timestamp",
            "format": "%+"
          },
      },
      "sinks": {
        "splunk_uk_dc": {
          "index": "my-index-dev",
          "source": "platform,
          "sourcetype": "_json"
        },
        "scalyr": {
          "token": "dG9rZW4="
        }
      }
    }

So we can simply parse the whole config one time. It also makes it more extensible in the future.

We could do this today for pod annotations as they are added to the event metadata however we typically keep the application Deployments the same across environments and depending where it is deployed (e.g. in a performance testing namespace, prod namespace, dev namespace) we want different log forward configurations. Namespace annotations would be greatly preferred if possible.

Thanks for considering, I hope this is an easy change to implement, I would love to contribute however I don't know Rust unfortunately. 😅

[jszwedko]

I converted this to a feature request. Thanks for opening it @matt-simons!