Prevent Logcollector from producing errors on blank lines at audit log files

[vikman90]

Target version	Related issue	Related PR
4.4.0	wazuh/wazuh#13710	wazuh/wazuh#15476

Description

This PR aims to fix a double issue in Logcollector, whose audit log reader produces errors in these cases:

An empty line was found.
A log with a valid header but an empty message was found.

Proposed checks

• Insert an empty line into an audit log file. We expect a debug message.
• Insert a log with a valid head and no content. We expect Logcollector to accumulate the log.
• Insert a log with an invalid head. We expect a warning log.

Steps to reproduce

• Empty line:

echo >> /root/audit_test.log

• Valid header, empty message:

cat >> /root/audit_test.log <<\EOF
type=UNKNOWN[1421] msg=audit(1660868417.986:12082):
type=PATH msg=audit(1660868417.986:12082): item=0 name="/usr/bin/df" inode=2097761 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=? nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
type=UNKNOWN[1421] msg=audit(1660868417.986:12082):
EOF

Configuration and considerations

<localfile>
  <location>/root/audit_test.log</location>
  <log_format>audit</log_format>
</localfile>

[fedepacher]

Update - 24/11/2022

Perform proposed checks for Wazuh manager

[fedepacher]

Review data

Tester	PR commit
@fedepacher	wazuh/wazuh@f91353b

Testing environment

OS	OS version	Deployment	Notes
Ubuntu	Jammy	AWS-EC2	ami-003530de8839921c4

Tested packages

wazuh-manager	wazuh-agent
deb

Status

• In progress
• Pending Review
• QA team manager approved (@jmv74211)
• DEV team leader approved (@vikman90 )

Conclusion

At this point, it seems to work as expected

[fedepacher]

Testing results 🟢

Unfixed

Install unfixed package

1. Install wazuh-agent v4.3.10 package

2. configure wazuh-agent ossec.conf file:

   <localfile>
      <location>/root/test.log</location>
      <log_format>audit</log_format>
   </localfile>
   

3. configure wazuh-agent local_internal_options.conf file:

   logcollector.debug=2
   

4. Insert a white line at the beginning of test.log

   echo >> /root/test.log
   

5. Check ossec.log file for error, warning, and critical logs

   2022/11/25 12:14:09 wazuh-logcollector[44285] read_audit.c:120 at read_audit(): ERROR: Discarding audit message because of invalid syntax.
   2022/11/25 12:14:09 wazuh-logcollector[44285] read_audit.c:150 at read_audit(): DEBUG: Read 1 lines from /root/test.log
   2022/11/25 12:14:09 wazuh-logcollector[44285] read_audit.c:120 at read_audit(): ERROR: Discarding audit message because of invalid syntax.
   2022/11/25 12:14:09 wazuh-logcollector[44285] read_audit.c:150 at read_audit(): DEBUG: Read 1 lines from /root/test.log
   

6. Insert line with empty messages

   type=UNKNOWN[1421] msg=audit(1660868417.986:12082):
   type=PATH msg=audit(1660868417.986:12082): item=0 name="/usr/bin/df" inode=2097761 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=? nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
   type=UNKNOWN[1421] msg=audit(1660868417.986:12082):
   

7. Check ossec.log file for error, warning, and critical logs

   2022/11/25 13:22:27 wazuh-logcollector[44285] read_audit.c:120 at read_audit(): ERROR: Discarding audit message because of invalid syntax.
   2022/11/25 13:22:27 wazuh-logcollector[44285] read_audit.c:150 at read_audit(): DEBUG: Read 1 lines from /root/test.log
   2022/11/25 13:22:27 wazuh-logcollector[44285] read_audit.c:120 at read_audit(): ERROR: Discarding audit message because of invalid syntax.
   2022/11/25 13:22:27 wazuh-logcollector[44285] read_audit.c:150 at read_audit(): DEBUG: Read 2 lines from /root/test.log
   

Upgrade

Upgrade fixed package 🟢

1. Upgrate to wazuh-agent v4.4.0 package 🟢

2. Check ossec.log file

   2022/11/24 19:47:12 wazuh-monitord: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/24 19:47:12 wazuh-logcollector: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/24 19:47:12 wazuh-remoted: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/24 19:47:12 wazuh-syscheckd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit  Cleaning...
   2022/11/24 19:47:12 wazuh-analysisd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/24 19:47:13 wazuh-execd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/24 19:47:13 wazuh-db: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/24 19:47:13 wazuh-authd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   

3. Insert a white line at the beginning of test.log

   echo >> /root/test.log
   

4. Check ossec.log file for error, warning, and critical logs 🟢

   2022/11/25 13:34:47 wazuh-logcollector[83145] read_audit.c:120 at read_audit(): DEBUG: audit reader: empty line, skipping.
   2022/11/25 13:34:47 wazuh-logcollector[83145] read_audit.c:155 at read_audit(): DEBUG: Read 1 lines from /root/test.log
   

5. Insert a line with empty messages and a valid head

   cat >> /root/test.log <<\EOF
   type=UNKNOWN[1421] msg=audit(1660868417.986:12082):
   type=PATH msg=audit(1660868417.986:12082): item=0 name="/usr/bin/df" inode=2097761 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=? nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
   type=UNKNOWN[1421] msg=audit(1660868417.986:12082):
   EOF
   

6. Check ossec.log file for error, warning, and critical logs 🟢

   2022/11/25 13:39:41 wazuh-logcollector[83145] read_audit.c:155 at read_audit(): DEBUG: Read 3 lines from /root/test.log
   

7. Insert a line with an invalid head

   cat >> /root/test.log <<\EOF
   invalid_head
   EOF
   

8. Check ossec.log file for error, warning, and critical logs 🟢

   2022/11/25 13:43:27 wazuh-logcollector[83145] read_audit.c:125 at read_audit(): WARNING: Discarding audit message because of invalid syntax.
   2022/11/25 13:43:27 wazuh-logcollector[83145] read_audit.c:155 at read_audit(): DEBUG: Read 1 lines from /root/test.log
   

Fresh install

Fresh install firmed package 🟢

1. Install wazuh-agent v4.4.0 package 🟢

2. Check ossec.log file for error, warning, and critical logs 🟢

   No error or warning messages were found
   

3. configure wazuh-agent ossec.conf file:

   <localfile>
      <location>/root/test.log</location>
      <log_format>audit</log_format>
   </localfile>
   

4. configure wazuh-agent local_internal_options.conf file:

   logcollector.debug=2
   

5. Check ossec.log file

   2022/11/25 13:56:53 wazuh-monitord: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/25 13:56:53 wazuh-logcollector: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/25 13:56:54 wazuh-remoted: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/25 13:56:54 wazuh-syscheckd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/25 13:56:54 wazuh-analysisd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/25 13:56:54 wazuh-execd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/25 13:56:54 wazuh-db: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   2022/11/25 13:56:55 wazuh-authd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
   

6. Insert a white line at the beginning of test.log

   echo >> /root/test.log
   

7. Check ossec.log file for error, warning, and critical logs 🟢

   2022/11/25 14:04:25 wazuh-logcollector[125529] read_audit.c:120 at read_audit(): DEBUG: audit reader: empty line, skipping.
   2022/11/25 14:04:25 wazuh-logcollector[125529] read_audit.c:155 at read_audit(): DEBUG: Read 1 lines from /root/test.log
   
   

8. Insert a line with empty messages and a valid head

   cat >> /root/test.log <<\EOF
   type=UNKNOWN[1421] msg=audit(1660868417.986:12082):
   type=PATH msg=audit(1660868417.986:12082): item=0 name="/usr/bin/df" inode=2097761 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=? nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
   type=UNKNOWN[1421] msg=audit(1660868417.986:12082):
   EOF
   

9. Check ossec.log file for error, warning, and critical logs 🟢

   2022/11/25 14:06:55 wazuh-logcollector[125529] read_audit.c:155 at read_audit(): DEBUG: Read 3 lines from /root/test.log
   

10. Insert a line with an invalid head

    cat >> /root/test.log <<\EOF
    invalid_head
    EOF
    

11. Check ossec.log file for error, warning, and critical logs 🟢

    2022/11/25 14:07:41 wazuh-logcollector[125529] read_audit.c:125 at read_audit(): WARNING: Discarding audit message because of invalid syntax.
    2022/11/25 14:07:41 wazuh-logcollector[125529] read_audit.c:155 at read_audit(): DEBUG: Read 1 lines from /root/test.log
    

[jmv74211]

I have launched the integration tests for logcollector and the changes made cause the following test failure 🔴 :

FAILED test_logcollector/test_log_format/test_log_format_values.py::test_log_format[('audit', False)]

I will investigate if that change has caused a new expected behavior and update that test. If not, the bug will be reported to be fixed in this development.

[jmv74211]

The test error is caused due to the log level change of ERROR to WARNING in the log message:

ERROR: Discarding audit message because of invalid syntax.

The following PR #3641 has been opened to update this new log level.

[jmv74211]

Closing conclusion 👍🏼

Everything seems to be working properly.