Filebeat auditd ingest pipeline fails at kv processor when field value contains whitespaces

[simioa]

• Version: 7.9.2
• Operating System: RHEL 7.9

Some auditd messages contain a reason field which contains informations on why a specific event was logged. This field can contain whitespaces. This whitespaces are interpreted by the kv ingest processor as a field split which causes the kv processor to fail.

beats/filebeat/module/auditd/log/ingest/pipeline.yml

Lines 22 to 26 in b6896ee

	- kv:
	field: auditd.log.kv
	field_split: "\\s+"
	value_split: "="
	target_field: auditd.log

• Steps to Reproduce:

  • Issue the following request against the pipeline simulation api:

POST _ingest/pipeline/filebeat-7.9.2-auditd-log-pipeline/_simulate
{
  "docs": [
    {
      "_source": {
        "message": "type=ANOM_ABEND msg=audit(1605431420.026:123): auid=12345 uid=123 gid=123 ses=123456789 pid=1234 comm=\"extproc\" reason=\"memory violation\" sig=6"
      }
    }
  ]
}

• The Ingest pipeline fails because of the "reason" Field with the following error:

...
          "error" : {
            "message" : "field [auditd.log.kv] does not contain value_split [=]"
          }
...

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[simioa]

elastic/elasticsearch#31786 looks related.
Using field_split and value_split patterns from elastic/elasticsearch#31786 (comment) seems to work but I only tested it with the message provided in the issue description, I did not test if this works with every other combination of possible messages.

[gyterpena]

it fails because of space in "reason="memory violation""

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jaijhala]

I believe this is still an issue. The workaround mentioned here does seem to work but would be good to fix this as part of the auditd integration?

[jamiehynds]

@efd6 would you mind taking a look at this one when you get a chance? Curious is the workaround mentioned in elastic/elasticsearch#31786 (comment) could be applied to our Auditd pipelines or if the bug within the kb processor needs to be addressed first?

[efd6]

@jamiehynds Fixes for the beats pipeline and the integration pipeline have been sent.