Add IT support for FIM whodata buffer

[Deblintrake09]

Target version	Related issue	Related PR
master	wazuh/wazuh#13920	wazuh/wazuh#16200

Description

This Issue aims to add IT support for the new FIM option whodata_queue_size that handles a new queue added for handling the messages that FIM receives from audit.

Test Cases

Linux Test cases (Manager & Agent)

Configuration test cases

Scenario: Default value

 Given that `whodata_queue_size` is removed from ossec conf
 When manager/agent is restarted
 Then an INFO level message shows the configured value to be `16384`

Scenario: Valid values - whodata_queue_size is set to limit values

Given that `whodata_queue_size` set to `1` or `1048576`
When manager/agent is restarted
Then an INFO level message shows the correct configured value

Scenario: Invalid values - whodata_queue_size is set to limit values

Given that `whodata_queue_size` set to `-1` or `1048577`
When manager/agent is restarted
Then an ERROR level message appears 
And syscheck is not restarted.

Scenario: Invalid values - whodata_queue_size has characters

Given that `whodata_queue_size` set to `10a`
When manager/agent is restarted
Then an ERROR level message appears explaining the conf error
And syscheck is not restarted.

Behavior test cases

Scenario: Adding less files than queue does not fill queue

Given that `whodata_queue_size` is set to 5
When 5  files ar created
All files are processed in whodata mode and the queue is not overflown

Scenario: When queue is overflown, dropped files are scanned in scheduled mode

Given that `whodata_queue_size` is set to 5
When 10 files are created
Then 5+ files are detected in `whodata` mode and the rest of the files are detected in the following scheduled scan

Scenario: Queue is freed at max_eps rate - queue not overflown

Given that `whodata_queue_size` is set to 5 and  `max_eps` is set to  1
When 5 files are created 
Then after 3 seconds 3 new files are inserted
Then all files are monitored in whodata mode and no events are dropped

Scenario: Queue overflows when space has not been freed by max_eps

Given that `whodata_queue_size` is set to 5 and  `max_file_per_second` is set to  1
When 7 amount of files is created
Then after 3 seconds 3 additional files are added
Then files are processed some events from first batch are dropped before files from second batch appear.
Then the dropped events from first batch are detected in scheduled mode

NOTE: The X amount of files to be created in the behavior test cases needs to be determined through manual testing, as the actual amount of files that can occupy one slot in the queue is not known at the moment.

Window Test cases Agent

Configuration test cases

Scenario: Default value

 Given that `whodata_queue_size` is removed from ossec conf
 When manager/agent is restarted
 Then an INFO level message shows the configured value to be `16384`

Scenario: Valid values - whodata_queue_size is set to limit values

Given that `whodata_queue_size` set to `1` or `1048576`
When manager/agent is restarted
Then an INFO level message shows the correct configured value

Scenario: Invalid values - whodata_queue_size is set to limit values

Given that `whodata_queue_size` set to `-1` or `1048577`
When manager/agent is restarted
Then an ERROR level message appears 
And syscheck is not restarted.

Scenario: Invalid values - whodata_queue_size has characters

Given that `whodata_queue_size` set to `10a`
When manager/agent is restarted
Then an ERROR level message appears explaining the conf error
And syscheck is not restarted.

Behavior test cases

Scenario: Adding less files than queue does not fill queue

Given that `whodata_queue_size` is set to 5
When 5  files ar created
All files are processed in whodata mode and the queue is not overflown

Scenario: When queue is overflown, dropped files are scanned in scheduled mode

Given that `whodata_queue_size` is set to 5
When 10 files are created
Then 5+ files are detected in `whodata` mode and the rest of the files are detected in the following scheduled scan

Scenario: Queue does not overflow and drop events

Given that `whodata_queue_size` is set to 3 and  `max_file_per_second` is set to  5
When X amount of files is created
Then all files are processed after X/5 seconds and no message is dropped

Scenario: Queue overflows and drop events from single insertion

Given that `whodata_queue_size` is set to 3 and  `max_file_per_second` is set to  5
When X amount of files is created
Then as files are processed at rate X/5 seconds the last set of files is missing (was dropped out of queue)

Scenario: Queue overflows and drop events - 2 Insertions - Drop from first set

Given that `whodata_queue_size` is set to 3 and  `max_file_per_second` is set to  5
When X amount of files is created
And files are processed at rate X/5 seconds
And a second X amount of files is created before events have finished appearing
Then files are processed some events from first batch are dropped before files from second batch appear.

NOTE: The X amount of files to be created in the behavior test cases needs to be determined through manual testing, as the actual amount of files that can occupy one slot in the queue is not known at the moment.

Test Tiers

Test	Tier	Justification
Scenario: Default value	0	This is basic default behavior
Scenario: Valid values - whodata_queue_size is set to limit values	0	This is basic correct behavior
Scenario: Invalid values - whodata_queue_size is set to limit values	1
Scenario: Invalid values - whodata_queue_size has characters	1
Scenario: Queue does not overflow and drop events	0	This is basic expected behavior
Scenario: Queue overflows and drop events from single insertion	1
Scenario: Queue overflows and drop events - 2 Insertions - Drop from first set	1

[Deblintrake09]

Update 24/02/2023

• Continue researching Issue
• Created Test modules for configuration cases
• Added new callbacks and event checkers
• Development is being handled in branch https:/wazuh/wazuh-qa/tree/3963-whodata-buffer-support

[Deblintrake09]

Update 27/02/2023

• Continue developing test cases

Update 28/02/2023

• Continue research and test development
• Ask Core about found results

[Deblintrake09]

Update 01/03/2022

• Continue research of behavior and developing test cases
• Created new test cases

Updated 03/03/2023

• Fix test cases
• Update documentation
• Move to review

[Deblintrake09]

Update 06/03/2023

• Check failing Linux Cases
• Fix test_sync_max_eps.py
• Launch Jenkins Instances

[Deblintrake09]

Update 03/08/2023

• Change target to 4.7.0
• Resolve conflicts
• Launch IT executions - WIP

[Deblintrake09]

Update 04/08/2023

• Created new PR due to unsigned commits on old PR.
• Ready to review 🚀

[davidjiglesias]

LGTM!

[Deblintrake09]

New ETA since issue was reopened. because it was merged in wrong branch. This is ready to merge.