IT Wazuh-logtest: Ruleset reloading at runtime

deps/wazuh_testing/wazuh_testing/logcollector.py

@@ -17,10 +17,11 @@
 GENERIC_CALLBACK_ERROR_ANALYZING_MACOS = "The expected analyzing macos log has not been produced"
 GENERIC_CALLBACK_ERROR_TARGET_SOCKET = "The expected target socket log has not been produced"
 GENERIC_CALLBACK_ERROR_TARGET_SOCKET_NOT_FOUND = "The expected target socket not found error has not been produced"
-LOG_COLLECTOR_GLOBAL_TIMEOUT = 20
 GENERIC_CALLBACK_ERROR_READING_FILE = "The expected invalid content error log has not been produced"
 GENERIC_CALLBACK_ERROR = 'The expected error output has not been produced'
 
+LOG_COLLECTOR_GLOBAL_TIMEOUT = 20
+
 DEFAULT_AUTHD_REMOTED_SIMULATOR_CONFIGURATION = {
  'ip_address': 'localhost',
  'client_keys': os.path.join(WAZUH_PATH, 'etc', 'client.keys'),

docs/tests/integration/help.md

@@ -12,6 +12,7 @@ Our newest integration tests are located in `wazuh-qa/tests/integration/`. They
 - _test_sca_
 - _test_vulnerability_detector_
 - _test_wazuh_db_
+- _test_logtest_
 
 Every group will have the following structure:
 

docs/tests/integration/index.md

@@ -15,14 +15,15 @@ Our newest integration tests are located in `wazuh-qa/tests/integration/`. They
 - **[_test_vulnerability_detector_](test_vulnerability_detector#tests-vulnerability-detector)**
 - **[_test_wazuh_db_](test_wazuh_db#test_wazuh_db)**
 - **[_test_logcollector_](test_logcollector#test_logcollector)**
+- **[_test_logtest_](test_logtest/index.md#test_logtest)**
 
 ## How to setup the test environment
 
 To run the tests you need to have `python3 >= 3.6` installed along with a set of additional dependencies.
 
 You can see all the information about it **[here](set_up_environment.md#setting-up-a-test-environment)**
 
-##  About test structure
+## About test structure
 
 See **[here](help.md#integration-tests-structure)** more information about the testing files structure or about `pytest`
 testing framework.

docs/tests/integration/test_logtest/index.md

@@ -1 +1,84 @@
-# Overview
+# Test Logtest
+
+## Overview
+
+Wazuh-Logtest allows testing and verifying rules and decoders and it is based on
+the use of unique sessions where each session loads its own rules and decoders.
+These tests ensure that logtest works correctly under different scenarios and
+that every option available work as expected.
+
+## Tiers
+
+### Tier 0
+
+#### Test configuration
+
+- **[Test configuration file](test_configuration/test_configuration_file.md)**:
+Check if `wazuh-logtest` works as expected under different pre-defined
+configurations that either produce the logtest to correctly start; to be
+disabled or to log an error.
+
+- **[Test get configuration sock](test_configuration/test_get_configuration_sock.md)**:
+Check if `wazuh-analisysd` correctly retrieves the `rule_test` configuration.
+
+#### Test invalid socket input
+
+- **[Test invalid socket input](test_invalid_socket_input/test_invalid_socket_input.md)**:
+Check if `wazuh-logtest` correctly detects and handles errors when sending a
+message through the socket to `analysisd`.
+
+#### Test invalid token
+
+- **[Test invalid token](test_invalid_token/test_invalid_session_token.md)**:
+Check if `wazuh-logtest` correctly detects and handles errors when using a token.
+
+#### Test remove session
+
+- **[Test remove session](test_remove_session/test_remove_session.md)**:
+Check if `wazuh-logtest` correctly detects and removes the sessions under
+pre-defined scenarios.
+
+#### Test remove old sessions
+
+- **[Test remove old sessions](test_remove_old_sessions/test_remove_old_sessions.md)**:
+Check that `wazuh-logtest` correctly detects and handles the situation where trying
+to use more sessions than allowed and then the oldest session is released.
+
+- **[Test remove old session for inactivity](test_remove_old_sessions/test_remove_old_session_for_inactivity.md)**:
+Check that `wazuh-logtest` correctly detects and handles the situation where trying
+to use more sessions than allowed and then old sessions are released due to
+inactivity.
+
+#### Test rules decoders load
+
+- **[Test load rules decoders](test_rules_decoders_load/test_load_rules_decoders.md)**:
+Check if `wazuh-logtest` produce the correct rule/decoder matching.
+
+#### Test ruleset refresh
+
+- **[Test alert labels](test_ruleset_refresh/test_alert_labels.md)**:
+Check that after modifying the alert level it takes effect when opening a new
+logtest sessions, without having to reset the manager.
+#### Test ruleset refresh
+
+- **[Test cdb labels](test_ruleset_refresh/test_cdb_labels.md)**:
+Check that `wazuh-logtest` works as expected with the operation of
+loading new cdb list files without the need to restart the manager.
+
+#### Test ruleset refresh
+
+- **[Test rule labels](test_ruleset_refresh/test_rule_labels.md)**:
+Checks if modifying the configuration of the rules, by using its labels, takes
+effect when opening new logtest sessions, without having to reset the manager.
+- **[Test decoder labels](test_ruleset_refresh/test_decoder_labels.md)**:
+Checks if modifying the configuration of the decoder, by using its labels, takes
+effect when opening new logtest sessions without having to reset the manager.
+#### Test invalid rule decoders syntax
+
+- **[Test invalid rules syntax](test_invalid_rule_decoders_syntax/test_invalid_rules_syntax.md)**:
+Check that `wazuh-logtest` correctly detects and handles errors when processing a
+rules file.
+
+- **[Test invalid decoder syntax](test_invalid_rule_decoders_syntax/test_invalid_decoder_syntax.md)**:
+Check that `wazuh-logtest` correctly detects and handles errors when processing a
+decoders file.

docs/tests/integration/test_logtest/test_configuration/test_configuration_file.md

@@ -0,0 +1,27 @@
+# Test logtest - configuration file
+
+## Overview
+
+Check if `wazuh-logtest` works as expected under different pre-defined
+configurations that either produce `wazuh-logtest` to correctly start; to be
+disabled or to log an error.
+
+## Objective
+
+- Confirm that, under different sets of configurations, `wazuh-logtest`
+correctly handles the configuration and creates a log entry on the Wazuh log,
+reporting the result of loading it.
+
+## General info
+
+|Tier | Total | Time spent |
+| :--:| :--: | :--: |
+| 0 | 5 | 1m30s |
+
+## Expected behavior
+
+- Fail if the expected log entry is not found among the Wazuh logs.
+
+## Code documentation
+
+::: tests.integration.test_logtest.test_configuration.test_configuration_file

docs/tests/integration/test_logtest/test_configuration/test_get_configuration_sock.md

@@ -0,0 +1,28 @@
+# Test logtest - get configuration socket
+
+## Overview
+
+Check if `wazuh-analisysd` correctly retrieves the `rule_test` configuration.
+
+## Objective
+
+- Confirm that, under different sets of configurations, `wazuh-analisysd`
+returns the right information from the `rule_test` configuration block.
+
+## General info
+
+|Tier | Total | Time spent |
+| :--:| :--: | :--: |
+| 0 | 5 | 1m23s |
+
+## Expected behavior
+
+- Fail if `wazuh-analisysd` does not retrieve the information in the expected format.
+- Fail if `wazuh-analisysd` does not retrieve the expected value of the `enabled` field.
+- Fail if `wazuh-analisysd` does not retrieve the expected value of the `threads` field.
+- Fail if `wazuh-analisysd` does not retrieve the expected value of the `max_sessions` field.
+- Fail if `wazuh-analisysd` does not retrieve the expected value of the `session_timeout` field.
+
+## Code documentation
+
+::: tests.integration.test_logtest.test_configuration.test_get_configuration_sock

docs/tests/integration/test_logtest/test_invalid_rule_decoders_syntax/test_invalid_decoder_syntax.md

@@ -0,0 +1,23 @@
+# Test logtest - invalid decoder syntax
+
+## Overview
+
+Check if `wazuh-logtest` correctly detects and handles errors when processing a decoders file.
+
+## Objective
+
+- Confirm that `wazuh-logtest` retrieves errors when the loaded decoders are invalid.
+
+## General info
+
+|Tier | Total | Time spent |
+| :--:| :--: | :--: |
+| 0 | 11 | 1s |
+
+## Expected behavior
+
+- Fail if `wazuh-logtest` does not retrieve an error when it should.
+
+## Code documentation
+
+::: tests.integration.test_logtest.test_invalid_rule_decoders_syntax.test_invalid_decoder_syntax

docs/tests/integration/test_logtest/test_invalid_rule_decoders_syntax/test_invalid_rules_syntax.md

@@ -0,0 +1,23 @@
+# Test logtest - invalid rules syntax
+
+## Overview
+
+Check if `wazuh-logtest` correctly detects and handles errors when processing a rules file.
+
+## Objective
+
+- Confirm that `wazuh-logtest` retrieves errors when the loaded rules are invalid.
+
+## General info
+
+|Tier | Total | Time spent |
+| :--:| :--: | :--: |
+| 0 | 15 | 9s |
+
+## Expected behavior
+
+- Fail if `wazuh-logtest` does not retrieve an error when it should.
+
+## Code documentation
+
+::: tests.integration.test_logtest.test_invalid_rule_decoders_syntax.test_invalid_rules_syntax

docs/tests/integration/test_logtest/test_invalid_socket_input/test_invalid_socket_input.md

@@ -0,0 +1,25 @@
+# Test logtest - invalid socket input
+
+## Overview
+
+Check if `wazuh-logtest` correctly detects and handles errors when sending a
+message through the socket to `analysisd`.
+
+## Objective
+
+- Confirm that the comunication through the sockets works well by verifying that
+all the test cases produce the right output.
+
+## General info
+
+|Tier | Total | Time spent |
+| :--:| :--: | :--: |
+| 0 | 26 | 1s |
+
+## Expected behavior
+
+- Fail if the message received through the socket does not match the expected output.
+
+## Code documentation
+
+::: tests.integration.test_logtest.test_invalid_socket_input.test_invalid_socket_input

docs/tests/integration/test_logtest/test_invalid_token/test_invalid_session_token.md

@@ -0,0 +1,23 @@
+# Test logtest - invalid session token
+
+## Overview
+
+Check if `wazuh-logtest` correctly detects and handles errors when using a session token.
+
+## Objective
+
+- Confirm that `wazuh-logtest` detects invalid session tokens.
+
+## General info
+
+|Tier | Total | Time spent |
+| :--:| :--: | :--: |
+| 0 | 6 | 3s |
+
+## Expected behavior
+
+- Fail if `wazuh-logtest` does not produce an error when trying to use an invalid session token.
+
+## Code documentation
+
+::: tests.integration.test_logtest.test_invalid_token.test_invalid_session_token

docs/tests/integration/test_logtest/test_remove_old_sessions/test_remove_old_session_for_inactivity.md

@@ -0,0 +1,27 @@
+# Test logtest - remove old session for inactivity
+
+## Overview
+
+Check if `wazuh-logtest` correctly detects and handles the situation where trying
+to use more sessions than allowed and then old sessions are released due to
+inactivity.
+
+## Objective
+
+- Confirm that `wazuh-logtest` removes the inactive sessions after a certain time.
+
+## General info
+
+|Tier | Total | Time spent |
+| :--:| :--: | :--: |
+| 0 | 1 | 1m 5s |
+
+## Expected behavior
+
+- Fail if `wazuh-logtest` does not start.
+- Fail if `wazuh-logtest` can not create a new session.
+- Fail if `wazuh-logtest` old session is not removed.
+
+## Code documentation
+
+::: tests.integration.test_logtest.test_remove_old_sessions.test_remove_old_session_for_inactivity

docs/tests/integration/test_logtest/test_remove_old_sessions/test_remove_old_sessions.md

@@ -0,0 +1,28 @@
+# Test logtest - remove old sessions
+
+## Overview
+
+Check if `wazuh-logtest` correctly detects and handles the situation when trying
+to use more sessions than allowed and so, to make room, the oldest session is
+released.
+
+## Objective
+
+- Confirm that `wazuh-logtest` releases the oldest session when a new session is
+opened and the number of active sessions reached its limit.
+
+## General info
+
+|Tier | Total | Time spent |
+| :--:| :--: | :--: |
+| 0 | 1 | 1m |
+
+## Expected behavior
+
+- Fail if `wazuh-logtest` does not start.
+- Fail if `wazuh-logtest` can not create a new session.
+- Fail if `wazuh-logtest` oldest session is not removed.
+
+## Code documentation
+
+::: tests.integration.test_logtest.test_remove_old_sessions.test_remove_old_sessions

docs/tests/integration/test_logtest/test_remove_session/test_remove_session.md

@@ -0,0 +1,24 @@
+# Test logtest - remove session
+
+## Overview
+
+Check if `wazuh-logtest` correctly detects and removes the sessions under
+pre-defined scenarios.
+
+## Objective
+
+- Confirm that `wazuh-logtest` correctly handles the sessions removals.
+
+## General info
+
+|Tier | Total | Time spent |
+| :--:| :--: | :--: |
+| 0 | 9 | 1s |
+
+## Expected behavior
+
+- Fail if the session removal attempt does not produce the expected result message.
+
+## Code documentation
+
+::: tests.integration.test_logtest.test_remove_session.test_remove_session

docs/tests/integration/test_logtest/test_rules_decoders_load/test_load_rules_decoders.md

@@ -0,0 +1,24 @@
+# Test logtest - load rules decoders
+
+## Overview
+
+Check if `wazuh-logtest` produce the correct rule/decoder matching.
+
+## Objective
+
+- Confirm that `wazuh-logtest` does produce the right decoder/rule matching when
+processing a log under different sets of configurations.
+
+## General info
+
+|Tier | Total | Time spent |
+| :--:| :--: | :--: |
+| 0 | 6 | 7s |
+
+## Expected behavior
+
+- Fail if `wazuh-logtest` does not produce the expected output when processing a log.
+
+## Code documentation
+
+::: tests.integration.test_logtest.test_rules_decoders_load.test_load_rules_decoders