Migrate test_providers documentation to qa-docs #2377
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
11 tasks
roronoasins
reviewed
Dec 22, 2021
Comment on lines
156
to
159
| - r'OVAL feed .* from provider .* not correctly assigned' | ||
| - r'Feed .* from provider .* was not correctly assigned' | ||
| - r'Event 'Fetching or Downloading .*' was not received' | ||
| - r'Event 'Trying to download .*' was not received' |
There was a problem hiding this comment.
Add the following callbacks
Suggested change
| - r'OVAL feed .* from provider .* not correctly assigned' | |
| - r'Feed .* from provider .* was not correctly assigned' | |
| - r'Event 'Fetching or Downloading .*' was not received' | |
| - r'Event 'Trying to download .*' was not received' | |
| - r'(Path|Url): (.*'|'none').*' | |
| - r'Multi (path|url): .*' | |
| - r'(Fetching feed from|Downloading) .*' | |
| - r'((Fetching .* from|Downloading) .*|.*Trying to download).*' |
roronoasins
reviewed
Dec 22, 2021
Comment on lines
139
to
140
| - 'Error log Configuration error at /var/ossec/etc/ossec.conf not found' | ||
| - r'Could not find .* update starting log' |
There was a problem hiding this comment.
Add the following callbacks
Suggested change
| - 'Error log Configuration error at /var/ossec/etc/ossec.conf not found' | |
| - r'Could not find .* update starting log' | |
| - r'.*: Configuration error at.*' | |
| - r'Starting .* database update' |
roronoasins
reviewed
Dec 22, 2021
Comment on lines
150
to
151
| - r'Warning log 'Invalid option 'os' for .* provider not found' | ||
| - r'Could not find .* update starting log' |
There was a problem hiding this comment.
Add the following callbacks
Suggested change
| - r'Warning log 'Invalid option 'os' for .* provider not found' | |
| - r'Could not find .* update starting log' | |
| - r'.*Invalid option .* for .* provider.*' | |
| - r'Could not find .* update starting log' |
roronoasins
reviewed
Dec 22, 2021
| test. | ||
|
|
||
| expected_output: | ||
| - r'It has been downloaded a feed from an invalid year .* for .*' |
There was a problem hiding this comment.
Suggested change
| - r'It has been downloaded a feed from an invalid year .* for .*' | |
| - r'.*Downloading .*cve.json\?after=1999-01-01.*' | |
| - r'.*Downloading .*cve.json\?after=.*-01-01.*' | |
| - r'.*Downloading .*nvdcve-\d.\d-.*.meta.*' | |
| - r'ERROR: Invalid content for \'update_from_year\' option at module \'vulnerability-detector\'' |
roronoasins
reviewed
Dec 22, 2021
| test. | ||
|
|
||
| expected_output: | ||
| - r'Unexpected event .* database updating' |
There was a problem hiding this comment.
Add the following callbacks
- r'Starting .* database update'
58 tasks
1 task
snaow
approved these changes
Jan 7, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
As part of epic #1796, this PR adds the missing documentation and migrates the current documentation to the new format used by qa-docs.
The schema used is the one defined in issue #1694
Generated documentation
test_providers_enable.json
{ { "copyright": "Copyright (C) 2015-2021, Wazuh Inc.\nCreated by Wazuh, Inc. <[email protected]>.\nThis program is free software; you can redistribute it and/or modify it under the terms of GPLv2", "type": "integration", "brief": "Wazuh is able to detect vulnerabilities in the applications installed in agents using the Vulnerability Detector module. This software audit is performed through the integration of vulnerability feeds indexed by Redhat, Canonical, Debian, Amazon Linux and NVD Database.", "tier": 1, "modules": [ "vulnerability_detector" ], "components": [ "manager" ], "daemons": [ "wazuh-modulesd", "wazuh-db", "wazuh-analysisd" ], "os_platform": [ "linux" ], "os_version": [ "Arch Linux", "Amazon Linux 2", "Amazon Linux 1", "CentOS 8", "CentOS 7", "CentOS 6", "Ubuntu Focal", "Ubuntu Bionic", "Ubuntu Xenial", "Ubuntu Trusty", "Debian Buster", "Debian Stretch", "Debian Jessie", "Debian Wheezy", "Red Hat 8", "Red Hat 7", "Red Hat 6" ], "references": [ "https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html", "https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#provider" ], "tags": [ "settings", "vulnerability", "vulnerability_detector" ], "name": "test_providers_enabled.py", "id": 1, "group_id": 0, "path": "tests/integration/test_vulnerability_detector/test_providers/test_providers_enabled.py", "tests": [ { "description": "Check if modulesd downloads the feeds from different providers when enabled is set to yes. To do this, it identifies the log which, if activated, indicates the update of the feeds. If the feeds are not active in the configuration, check that the above-mentioned log does not occur.", "wazuh_min_version": "4.2.0", "parameters": [ { "get_configuration": { "type": "fixture", "brief": "Get configurations from the module." } }, { "configure_environment": { "type": "fixture", "brief": "Configure a custom environment for testing." } }, { "restart_modulesd": { "type": "fixture", "brief": "Reset the logs file and start a new monitor." } } ], "assertions": [ "Verify that when the `enabled` tag of the provider option is set to `yes`, provider database starts updating.", "Verify that when the `enabled` tag of the provider option is set to `no`, checks that provider database does not start updating." ], "input_description": [ "Test cases are found in the test module and include parameters for the `enabled` tag of the provider option (`yes` and `no`). The `wazuh_providers_enabled.yaml` file provides the configuration of this module for this test." ], "expected_output": [ "Starting.+database update", "r'Unexpected event .* database updating'", "r'Starting .* database update'", "r'Could not find .* update starting log'" ], "name": "test_enabled", "test_cases": [ "yes-alas0", "yes-alas1", "yes-canonical", "yes-debian", "yes-redhat", "yes-nvd", "yes-msu", "yes-arch", "no-alas0", "no-alas1", "no-canonical", "no-debian", "no-redhat", "no-nvd", "no-msu", "no-arch" ] } ] }test_providers_multiple_providers.json
{ "copyright": "Copyright (C) 2015-2021, Wazuh Inc.\nCreated by Wazuh, Inc. <[email protected]>.\nThis program is free software; you can redistribute it and/or modify it under the terms of GPLv2", "type": "integration", "brief": "Wazuh is able to detect vulnerabilities in the applications installed in agents using the Vulnerability Detector module. This software audit is performed through the integration of vulnerability feeds indexed by Redhat, Canonical, Debian, Amazon Linux and NVD Database.", "tier": 0, "modules": [ "vulnerability_detector" ], "components": [ "manager" ], "daemons": [ "wazuh-modulesd", "wazuh-db", "wazuh-analysisd" ], "os_platform": [ "linux" ], "os_version": [ "Arch Linux", "Amazon Linux 2", "Amazon Linux 1", "CentOS 8", "CentOS 7", "CentOS 6", "Ubuntu Focal", "Ubuntu Bionic", "Ubuntu Xenial", "Ubuntu Trusty", "Debian Buster", "Debian Stretch", "Debian Jessie", "Debian Wheezy", "Red Hat 8", "Red Hat 7", "Red Hat 6" ], "references": [ "https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html", "https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#provider" ], "tags": [ "settings", "vulnerability", "vulnerability_detector" ], "name": "test_providers_multiple_providers.py", "id": 1, "group_id": 0, "path": "tests/integration/test_vulnerability_detector/test_providers/test_providers_multiple_providers.py", "tests": [ { "description": "Check if the path/url and multipath/url options work properly according to the configuration and check there are no conflicts when downloading or reading the feeds. To do this, it checks that the OVAL feed has been correctly assigned in the log file. In the case of the RedHat provider, it verifies the correct assignment of the global information feeds. Finally, when the feed assignment is correct, it checks that the updates for both types of feeds start. From now on, Redhat and Debian providers use two feeds to fetch vulnerabilities and CVEs' metadata. <os path=...> option indicates the local path where the OVAL feed is whereas <path> indicates the local path where the another feed is. The same goes for <os url=...> and <url> options.", "wazuh_min_version": "4.2.0", "parameters": [ { "clean_vuln_tables": { "type": "fixture", "brief": "Clean vulnerabilities tables." } }, { "get_configuration": { "type": "fixture", "brief": "Get configurations from the module." } }, { "configure_environment": { "type": "fixture", "brief": "Configure a custom environment for testing." } }, { "restart_modulesd": { "type": "fixture", "brief": "Reset the logs file and start a new monitor." } } ], "assertions": [ "Verify that the OVAL feed has been correctly assigned in the log file.", "In the case of the RedHat provider, verify the correct assignment of the global information feeds.", "When the feed assignment is correct, verify that the updates for both types of feeds start." ], "input_description": [ "Test cases are found in the test module and include parameters for the `os url/path` tag of the provider option. The `wazuh_providers_multiple_providers.yaml` file provides the configuration of this module for this test. `custom_redhat_json_feed.json`, `custom_redhat_oval_feed.xml`, `custom_debian_json_feed.json`, and `custom_debian_oval_feed.xml` provide the specific feeds to be downloaded or read." ], "expected_output": [ { "r'(Path|Url)": "(.*'|'none').*'" }, { "r'Multi (path|url)": ".*'" }, "r'(Fetching feed from|Downloading) .*'", "r'((Fetching .* from|Downloading) .*|.*Trying to download).*'" ], "name": "test_multiple_providers", "test_cases": [ "test_redhat_default", "test_redhat_path", "test_redhat_multipath", "test_redhat_path_multipath", "test_redhat_url", "test_redhat_multiurl", "test_redhat_url_multiurl", "test_debian_default", "test_debian_path", "test_debian_multipath", "test_debian_path_multipath", "test_debian_url", "test_debian_multiurl", "test_debian_url_multiurl" ] } ] }test_providers_no_os.json
{ "copyright": "Copyright (C) 2015-2021, Wazuh Inc.\nCreated by Wazuh, Inc. <[email protected]>.\nThis program is free software; you can redistribute it and/or modify it under the terms of GPLv2", "type": "integration", "brief": "Wazuh is able to detect vulnerabilities in the applications installed in agents using the Vulnerability Detector module. This software audit is performed through the integration of vulnerability feeds indexed by Redhat, Canonical, Debian, Amazon Linux and NVD Database.", "tier": 1, "modules": [ "vulnerability_detector" ], "components": [ "manager" ], "daemons": [ "wazuh-modulesd", "wazuh-db", "wazuh-analysisd" ], "os_platform": [ "linux" ], "os_version": [ "Arch Linux", "Amazon Linux 2", "Amazon Linux 1", "CentOS 8", "CentOS 7", "CentOS 6", "Ubuntu Focal", "Ubuntu Bionic", "Ubuntu Xenial", "Ubuntu Trusty", "Debian Buster", "Debian Stretch", "Debian Jessie", "Debian Wheezy", "Red Hat 8", "Red Hat 7", "Red Hat 6" ], "references": [ "https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html", "https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#provider" ], "tags": [ "settings", "vulnerability", "vulnerability_detector" ], "name": "test_providers_no_os.py", "id": 1, "group_id": 0, "path": "tests/integration/test_vulnerability_detector/test_providers/test_providers_no_os.py", "tests": [ { "description": "Check if modulesd downloads the feeds without specifing the os version. To do this, it checks if errors occur when the <os> tag is omitted in the configuration in providers that should have it and, on the other hand, if the update of the feeds starts normally in providers that do not require this tag.", "wazuh_min_version": "4.2.0", "parameters": [ { "clean_vuln_tables": { "type": "fixture", "brief": "Clean the tables involved with vulnerability detector packages and feeds." } }, { "get_configuration": { "type": "fixture", "brief": "Get configurations from the module." } }, { "configure_environment": { "type": "fixture", "brief": "Configure a custom environment for testing." } } ], "assertions": [ "Verify that the provider os data update starts when `os` has not a determined value.", "Verify that an error message appears when using `error` tag in providers." ], "input_description": [ "Test cases are found in the test module and include parameters for the `os` tag of the provider option and `error` tag. The `wazuh_providers_os.yaml` file provides the configuration of this module for this test." ], "expected_output": [ { "r'.*": "Configuration error at.*'" }, "r'Starting .* database update'" ], "name": "test_providers_no_os", "test_cases": [ "alas", "canonical", "debian", "redhat", "nvd", "msu", "arch" ] } ] }test_providers_os.json
{ "copyright": "Copyright (C) 2015-2021, Wazuh Inc.\nCreated by Wazuh, Inc. <[email protected]>.\nThis program is free software; you can redistribute it and/or modify it under the terms of GPLv2", "type": "integration", "brief": "Wazuh is able to detect vulnerabilities in the applications installed in agents using the Vulnerability Detector module. This software audit is performed through the integration of vulnerability feeds indexed by Redhat, Canonical, Debian, Amazon Linux and NVD Database.", "tier": 1, "modules": [ "vulnerability_detector" ], "components": [ "manager" ], "daemons": [ "wazuh-modulesd", "wazuh-db", "wazuh-analysisd" ], "os_platform": [ "linux" ], "os_version": [ "Arch Linux", "Amazon Linux 2", "Amazon Linux 1", "CentOS 8", "CentOS 7", "CentOS 6", "Ubuntu Focal", "Ubuntu Bionic", "Ubuntu Xenial", "Ubuntu Trusty", "Debian Buster", "Debian Stretch", "Debian Jessie", "Debian Wheezy", "Red Hat 8", "Red Hat 7", "Red Hat 6" ], "references": [ "https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html", "https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#provider" ], "tags": [ "settings", "vulnerability", "vulnerability_detector" ], "name": "test_providers_os.py", "id": 1, "group_id": 0, "path": "tests/integration/test_vulnerability_detector/test_providers/test_providers_os.py", "tests": [ { "description": "Check if modulesd downloads the feeds for each os. To do this, it verifies that the update of the feeds is started by checking the log file. Additionally, it verifies that a warning message is generated regarding the improper use of the <os> tag in providers that do not use it.", "wazuh_min_version": "4.2.0", "parameters": [ { "get_configuration": { "type": "fixture", "brief": "Get configurations from the module." } }, { "configure_environment": { "type": "fixture", "brief": "Configure a custom environment for testing." } }, { "restart_modulesd": { "type": "fixture", "brief": "Reset the logs file and start a new monitor." } } ], "assertions": [ "Verify that the provider os data update starts when `os` has a determined value.", "Verify that a warning message appears when using `os` tag in Red Hat and NVD providers." ], "input_description": [ "Test cases are found in the test module and include parameters for the `os` tag of the provider option. The `wazuh_providers_os.yaml` file provides the configuration of this module for this test." ], "expected_output": [ "r'.*Invalid option .* for .* provider.*'", "r'Could not find .* update starting log'" ], "name": "test_providers", "test_cases": [ "alas-amazon-linux", "alas-amazon-linux-2", "canonical-trusty", "canonical-xenial", "canonical-bionic", "debian-stretch", "debian-buster", "redhat-5", "redhat-6", "redhat-7", "redhat-8", "nvd-", "msu-" ] } ] }test_providers_update_from_year.json
{ "copyright": "Copyright (C) 2015-2021, Wazuh Inc.\nCreated by Wazuh, Inc. <[email protected]>.\nThis program is free software; you can redistribute it and/or modify it under the terms of GPLv2", "type": "integration", "brief": "Wazuh is able to detect vulnerabilities in the applications installed in agents using the Vulnerability Detector module. This software audit is performed through the integration of vulnerability feeds indexed by Redhat, Canonical, Debian, Amazon Linux and NVD Database.", "tier": 1, "modules": [ "vulnerability_detector" ], "components": [ "manager" ], "daemons": [ "wazuh-modulesd", "wazuh-db", "wazuh-analysisd" ], "os_platform": [ "linux" ], "os_version": [ "Arch Linux", "Amazon Linux 2", "Amazon Linux 1", "CentOS 8", "CentOS 7", "CentOS 6", "Ubuntu Focal", "Ubuntu Bionic", "Ubuntu Xenial", "Ubuntu Trusty", "Debian Buster", "Debian Stretch", "Debian Jessie", "Debian Wheezy", "Red Hat 8", "Red Hat 7", "Red Hat 6" ], "references": [ "https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html", "https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#provider" ], "tags": [ "settings", "vulnerability", "vulnerability_detector" ], "name": "test_providers_update_from_year.py", "id": 1, "group_id": 0, "path": "tests/integration/test_vulnerability_detector/test_providers/test_providers_update_from_year.py", "tests": [ { "description": "Check if vulnerability detector download feeds from the correct year based on `update_from_year` option. To do this, it checks for a valid year of the providers and downloads the feed.", "wazuh_min_version": "4.2.0", "parameters": [ { "clean_vuln_tables": { "type": "fixture", "brief": "Clean the tables involved with vulnerability detector packages and feeds." } }, { "get_configuration": { "type": "fixture", "brief": "Get configurations from the module." } }, { "configure_environment": { "type": "fixture", "brief": "Configure a custom environment for testing." } }, { "restart_modulesd_catching_ossec_conf_error": { "type": "fixture", "brief": "Restart the `modulesd` daemon and catch the ossec configuration errors." } } ], "assertions": [ "Verify that the vulnerability detector behavior when it is set the `x` year for `y` provider, taking into an account that there is a minimum year for each provider.", "Verify that there is no download from one year before the update from year value", "Verify that their respective log will be found and none from a previous year in `ossec.log`." ], "input_description": [ "Test cases are found in the test module and include parameters for the `update_from_year` tag of the provider option. The `wazuh_providers_update_from_year.yaml` file provides the configuration of this module for this test." ], "expected_output": [ "r'.*Downloading .*cve.json after=1999-01-01.*'", "r'.*Downloading .*cve.json after=.*-01-01.*'", "r'.*Downloading .*nvdcve-.*.meta.*'", { "r'ERROR": "Invalid content for `update_from_year` option at module `vulnerability-detector`'" } ], "name": "test_update_from_year", "test_cases": [ "redhat_1998", "redhat_2001", "redhat_2002", "redhat_2005", "redhat_2010", "redhat_2015", "redhat_2020", "nvd_1998", "nvd_2001", "nvd_2002", "nvd_2005", "nvd_2010", "nvd_2015", "nvd_2020" ] } ] }test_providers_update_interval.json
{ "copyright": "Copyright (C) 2015-2021, Wazuh Inc.\nCreated by Wazuh, Inc. <[email protected]>.\nThis program is free software; you can redistribute it and/or modify it under the terms of GPLv2", "type": "integration", "brief": "Wazuh is able to detect vulnerabilities in the applications installed in agents using the Vulnerability Detector module. This software audit is performed through the integration of vulnerability feeds indexed by Redhat, Canonical, Debian, Amazon Linux and NVD Database.", "tier": 1, "modules": [ "vulnerability_detector" ], "components": [ "manager" ], "daemons": [ "wazuh-modulesd", "wazuh-db", "wazuh-analysisd" ], "os_platform": [ "linux" ], "os_version": [ "Arch Linux", "Amazon Linux 2", "Amazon Linux 1", "CentOS 8", "CentOS 7", "CentOS 6", "Ubuntu Focal", "Ubuntu Bionic", "Ubuntu Xenial", "Ubuntu Trusty", "Debian Buster", "Debian Stretch", "Debian Jessie", "Debian Wheezy", "Red Hat 8", "Red Hat 7", "Red Hat 6" ], "references": [ "https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html", "https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#provider" ], "tags": [ "settings", "vulnerability", "vulnerability_detector" ], "name": "test_providers_update_interval.py", "id": 1, "group_id": 0, "path": "tests/integration/test_vulnerability_detector/test_providers/test_providers_update_interval.py", "tests": [ { "description": "Check if the provider database update is triggered after the set interval time has passed. To do this, it checks that the feed update is not started prematurely by checking the log file. It then travels forward in time, specifically to five seconds after the expiration of the time set in the update_interval option. Finally, it checks through the log file that the update of the feeds is started after this period.", "wazuh_min_version": "4.2.0", "parameters": [ { "get_configuration": { "type": "fixture", "brief": "Get configurations from the module." } }, { "configure_environment": { "type": "fixture", "brief": "Configure a custom environment for testing." } }, { "restart_modulesd": { "type": "fixture", "brief": "Reset the logs file and start a new monitor." } } ], "assertions": [ "Verify that the update has not started prematurely.", "Verify that the feed is downloaded after the set update interval." ], "input_description": [ "Test cases are found in the test module and include parameters for the `update_interval` tag of the provider option. The `wazuh_providers_update_interval.yaml` file provides the configuration of this module for this test." ], "expected_output": [ "r'Unexpected event .* database updating'", "r'Could not find the provider .* updating feed log after the interval update'", "Starting .* database update" ], "name": "test_update_interval", "test_cases": [ "Amazon_60s", "Redhat_{item}0", "Canonical_60s", "Debian_60s", "NVD_60s", "MSU_60s", "Arch_60s", "Amazon_60m", "Redhat_{item}1", "Canonical_60m", "Debian_60m", "NVD_60m", "MSU_60m", "Arch_60m", "Amazon_1h", "Redhat_{item}2", "Canonical_1h", "Debian_1h", "NVD_1h", "MSU_1h", "Arch_1h", "Amazon_1d", "Redhat_{item}3", "Canonical_1d", "Debian_1d", "NVD_1d", "MSU_1d", "Arch_1d" ] } ] }Tests
pycodestyle --max-line-length=120 --show-source --show-pep8 file.py.qa-docstool does not raise any error.