Skip to content

Commit

Permalink
merge(#3860): '4.5' in 3591-fix-test-large-changes
Browse files Browse the repository at this point in the history
  • Loading branch information
Deblintrake09 committed May 24, 2023
2 parents 920fcd3 + 56666ee commit 9ee8732
Show file tree
Hide file tree
Showing 200 changed files with 31,780 additions and 21,494 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/changelog.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
name: Changelog
on:
pull_request:
types:
- opened
- ready_for_review
branches:
- master
- '[0-9]+.[0-9]+'
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/code_analysis.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
name: Code analysis
on:
pull_request
pull_request:
types:
- opened
- ready_for_review
jobs:
Linting:
if: ${{ !github.event.pull_request.draft }}
Expand Down
69 changes: 54 additions & 15 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,12 @@

All notable changes to this project will be documented in this file.

## [4.5.0] - Development (unreleased)

Wazuh commit: TBD \
Release report: TBD
## [4.5.0] - TBD

### Added

- Add integration tests for AWS module. ([#3911](https:/wazuh/wazuh-qa/pull/3911)) \- (Framework + Tests + Documentation)
- Add tests for msu patches with no associated CVE . ([#4009](https:/wazuh/wazuh-qa/pull/4009)) \- (Framework + Tests)
- Add tests with new options to avoid FIM synchronization overlapping. ([#3318](https:/wazuh/wazuh-qa/pull/3318)) \- (Framework + tests)
- Add Logcollector millisecond granularity support test case ([#3910](https:/wazuh/wazuh-qa/pull/3910)) \- (Tests)
- Add Windows System folders FIM monitoring tests ([#3720](https:/wazuh/wazuh-qa/pull/3720)) \- (Tests)
Expand All @@ -21,6 +20,11 @@ Release report: TBD
### Changed

- Fix FIM test_large_changes test suite ([#3948](https:/wazuh/wazuh-qa/pull/3948)) \- (Tests)
- Improve the way that environment data is managed ([#4059](https:/wazuh/wazuh-qa/pull/4059)) \- (Framework)
- Update FIM test_ambiguous_confs IT to new framework ([#4121](https:/wazuh/wazuh-qa/pull/4121)) \- (Tests + Framework)
- Update `test_logcollector` invalid configs log level ([#4094](https:/wazuh/wazuh-qa/pull/4094)) \- (Tests)
- Update `test_office365` to support the new tag `API_TYPE` ([#4065](https:/wazuh/wazuh-qa/pull/4065)) \- (Framework + Tests)
- Update `test_wazuh_db` & `test_enrollment` to support new column `status_code` and new value on the enrollment `payload`. ([#4021](https:/wazuh/wazuh-qa/pull/4021)) \- (Tests)
- Update FIM `test_audit` tests to new framework ([#3939](https:/wazuh/wazuh-qa/pull/3939)) \- (Framework + Tests)
- Update FIM test to new FIM DBSync process ([#2728](https:/wazuh/wazuh-qa/pull/2728)) \- (Framework + Tests)
- Update file_limit and registry_limit tests ([#3280](https:/wazuh/wazuh-qa/pull/3280)) \- (Tests)
Expand All @@ -33,22 +37,44 @@ Release report: TBD

### Fixed

- Fix boto3 version requirement for legacy OS ([#4150](https:/wazuh/wazuh-qa/pull/4150)) \- (Framework)
- Fix cases yaml of the analysisd windows registry IT ([#4149](https:/wazuh/wazuh-qa/pull/4149)) \- (Tests)
- Fix a bug in on Migration tool's library ([#4106](https:/wazuh/wazuh-qa/pull/4106)) \- (Framework)
- Fix imports and add windows support for test_report_changes_and_diff IT ([#3548](https:/wazuh/wazuh-qa/issues/3548)) \- (Framework + Tests)
- Fix a regex error in the FIM integration tests ([#3061](https:/wazuh/wazuh-qa/issues/3061)) \- (Framework + Tests)
- Fix an error in the cluster performance tests related to CSV parser ([#2999](https:/wazuh/wazuh-qa/pull/2999)) \- (Framework + Tests)
- Fix bug in the framework on migration tool ([#2999](https:/wazuh/wazuh-qa/pull/4027)) \- (Framework)

## [4.4.2] - TBD

### Added

- Add test to check the Syscollector configuration. ([#3584](https:/wazuh/wazuh-qa/pull/3584)) \- (Framework + Tests)
- Add system tests for groups deletion ([#4057](https:/wazuh/wazuh-qa/pull/4057)) \- (Tests)

### Fixed

- Limit urllib3 major required version ([#4162](https:/wazuh/wazuh-qa/pull/4162)) \- (Framework)
- Fix daemons_handler fixture (fix GCP IT) ([#4134](https:/wazuh/wazuh-qa/pull/4134)) \- (Tests)
- Fix wazuhdb IT. ([#3584](https:/wazuh/wazuh-qa/pull/3584)) \- (Framework + Tests)
- Fix agentd IT for python3.10 AMI ([#3973](https:/wazuh/wazuh-qa/pull/3973)) \- (Tests)
- Fix unstable system tests ([#4080](https:/wazuh/wazuh-qa/pull/4080)) \- (Tests)

## [4.4.0] - Development (unreleased)
## [4.4.1] - 12-04-2023

Wazuh commit: TBD \
Release report: TBD
Wazuh commit: https:/wazuh/wazuh/commit/63a0580562007c4ba9c117f4a232ce90160481ff \
Release report: https:/wazuh/wazuh/issues/16620

## [4.4.0] - 28-03-2023

Wazuh commit: https:/wazuh/wazuh/commit/2477e9fa50bc1424e834ac8401ce2450a5978e75 \
Release report: https:/wazuh/wazuh/issues/15504

### Added

- Add new integration test for `authd` to validate error when `authd.pass` is empty ([#3721](https:/wazuh/wazuh-qa/pull/3721)) \- (Framework + Tests)
- Add new test to check missing fields in `cpe_helper.json` file ([#3766](https:/wazuh/wazuh-qa/pull/3766)) \- (Framework + Tests)
- Add multigroups tests cases for `test_assign_groups_guess` ([#3979](https:/wazuh/wazuh-qa/pull/3979)) \- (Tests)
- Fix test_agent_groups system test ([#3955](https:/wazuh/wazuh-qa/pull/3964)) \- (Tests)
- Add new group_hash case and update the `without condition` case output in `wazuh_db/sync_agent_groups_get` ([#3959](https:/wazuh/wazuh-qa/pull/3959)) \- (Tests)
- Add markers for each system test environment ([#3961](https:/wazuh/wazuh-qa/pull/3961)) \- (Framework + Tests)
- Adapt binary performance module to wazuh-cluster script renaming ([#3944](https:/wazuh/wazuh-qa/pull/3944)) \- (Framework)
Expand All @@ -60,7 +86,6 @@ Release report: TBD
- Add new test to check vulnerable packages with triaged null([#3587](https:/wazuh/wazuh-qa/pull/3587)) \- (Framework + Tests)
- Add new tests analysid handling of invalid/empty rule signature IDs ([#3649](https:/wazuh/wazuh-qa/pull/3649)) \- (Framework + Tests)
- Add integration test to check agent database version ([#3768](https:/wazuh/wazuh-qa/pull/3768)) \- (Tests)
- Fix Yara and VirusTotal E2E basic usage tests ([#3660](https:/wazuh/wazuh-qa/pull/3660))
- Add new test to check if syslog message are parsed correctrly in the `archives.json` file ([#3609](https:/wazuh/wazuh-qa/pull/3609)) \- (Framework + Tests)
- Add new logging tests for analysisd EPS limitation ([#3509](https:/wazuh/wazuh-qa/pull/3509)) \- (Framework + Tests)
- New testing suite for checking analysisd EPS limitation ([#2947](https:/wazuh/wazuh-qa/pull/3181)) \- (Framework + Tests)
Expand All @@ -87,14 +112,9 @@ Release report: TBD

### Changed

- Fix `test_file_limit_delete_full` module ([#3990](https:/wazuh/wazuh-qa/pull/3990)) \- (Tests)
- Improve `test_agent_groups_new_cluster_node` ([#3971](https:/wazuh/wazuh-qa/pull/3971)) \- (Tests)
- Fix Solaris agent provision schema ([#3750](https:/wazuh/wazuh-qa/issues/3744)) \- (Framework)
- Fix wazuh-db integration tests for agent-groups ([#3926](https:/wazuh/wazuh-qa/pull/3926)) \- (Tests + Framework)
- Fix `test_set_agent_groups` ([#3920](https:/wazuh/wazuh-qa/pull/3920)) \- (Tests)
- Improve `test_assign_groups_guess` ([#3901](https:/wazuh/wazuh-qa/pull/3901)) \- (Tests)
- Update `test_cluster_worker_logs_order` test ([#3896](https:/wazuh/wazuh-qa/pull/3896)) \- (Tests)
- Fix `test_agent_groups` ([#3889](https:/wazuh/wazuh-qa/pull/3889)) \- (Tests + Framework)
- Increase NVE download feed test timeout([#3769](https:/wazuh/wazuh-qa/pull/3769)) \- (Tests)
- Adapt wazuhdb integration tests for auto-vacuum ([#3613](https:/wazuh/wazuh-qa/issues/3613)) \- (Tests)
- Update logcollector format test due to audit changes ([#3641](https:/wazuh/wazuh-qa/pull/3641)) \- (Framework)
Expand Down Expand Up @@ -135,8 +155,16 @@ Release report: TBD

### Fixed

- Fix `test_assign_agent_group_with_enrollment` ([#3956](https:/wazuh/wazuh-qa/pull/3956)) \- (Tests)
- Fix `test_file_limit_delete_full` module ([#3990](https:/wazuh/wazuh-qa/pull/3990)) \- (Tests)
- Fix test_agent_groups system test ([#3955](https:/wazuh/wazuh-qa/pull/3964)) \- (Tests)
- Fix Solaris agent provision schema ([#3750](https:/wazuh/wazuh-qa/issues/3744)) \- (Framework)
- Fix wazuh-db integration tests for agent-groups ([#3926](https:/wazuh/wazuh-qa/pull/3926)) \- (Tests + Framework)
- Fix `test_set_agent_groups` ([#3920](https:/wazuh/wazuh-qa/pull/3920)) \- (Tests)
- Fix test_sync_agent_groups_get, replace hardcoded hash to a dinamically calculated one ([#3895](https:/wazuh/wazuh-qa/pull/3895)) \- (Framework + Tests)
- Fix `test_agent_groups` ([#3889](https:/wazuh/wazuh-qa/pull/3889)) \- (Tests + Framework)
- Fix test_db_backup for Ubuntu OS ([#3802](https:/wazuh/wazuh-qa/pull/3802)) \- (Tests)
- Fix Yara and VirusTotal E2E basic usage tests ([#3660](https:/wazuh/wazuh-qa/pull/3660)) \- (Tests)
- Fix commit option of the scan module for master case ([#3157](https:/wazuh/wazuh-qa/pull/3157)) \- (Tests)
- Fix Vulnerability Detector IT: test_validate_feed_content yaml cases had wrong extension. ([#3299](https:/wazuh/wazuh-qa/pull/3299)) \- (Tests)
- Fix Analysisd IT: test_syscollector_events failure on wait_for_analysisd_startup. ([#3110](https:/wazuh/wazuh-qa/pull/3110)) \- (Tests)
Expand All @@ -149,12 +177,23 @@ Release report: TBD
- Fix error in requirements.txt ([#3689](https:/wazuh/wazuh-qa/pull/3689)) \- (Framework)
- Fix sleep time in `test_agent_default_group_added`. ([#3692](https:/wazuh/wazuh-qa/pull/3692)) \- (Tests)
- Fix syscollector deltas integration tests. ([#3695](https:/wazuh/wazuh-qa/pull/3695)) \- (Tests)
- Fix test_response_postprocessing: duplicated slash in API endpoints ([#4048](https:/wazuh/wazuh-qa/pull/4048)) \- (Tests)

### Removed

- Remove all FIM Integration skipped tests ([#2927](https:/wazuh/wazuh-qa/issues/2927)) \- (Framework + Tests)
- VDT ITs: Remove Debian Stretch test support. ([#3172](https:/wazuh/wazuh-qa/pull/3172)) \- (Tests)

## [4.3.11] - 20-04-2023

Wazuh commit: https:/wazuh/wazuh/commit/776fda906581a1e4ee170c3e7e73a58d69e41f95 \
Release report: https:/wazuh/wazuh/issues/16758

## [4.3.10] - 16-11-2022

Wazuh commit: https:/wazuh/wazuh/commit/89530f11c9e592cd2e551432209b0080f08ff8e5 \
Release report: https:/wazuh/wazuh/issues/15372

## [4.3.9] - 13-10-2022

Wazuh commit: https:/wazuh/wazuh-qa/commit/8af0a5083bd69765f4d7878df9d3b785bb239723 \
Expand All @@ -178,7 +217,7 @@ Release report: https:/wazuh/wazuh/issues/14827
Wazuh commit: https:/wazuh/wazuh/commit/e2b514bef3d148acd4bcae1a1c7fa8783b82ca3a \
Release report: https:/wazuh/wazuh/issues/14562

## Added
### Added
- Added IT test to verify Active Response works with overwritten rules. ([#2984](https:/wazuh/wazuh-qa/pull/2984)) \- (Framework + Tests)
- Add Integratord IT - new test_integratord suite ([#3125](https:/wazuh/wazuh-qa/pull/3125)) \- (Framework + Tests)
- Add system test to check synchronization status in the cluster ([#3180](https:/wazuh/wazuh-qa/pull/3180)) \- (Framework + Tests)
Expand Down
2 changes: 1 addition & 1 deletion conftest.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import pytest


def pytest_collectreport(report):
if report.failed:
pass

5 changes: 5 additions & 0 deletions deps/wazuh_testing/wazuh_testing/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,14 @@
if sys.platform == 'win32':
WAZUH_PATH = os.path.join("C:", os.sep, "Program Files (x86)", "ossec-agent")
LOG_FILE_PATH = os.path.join(WAZUH_PATH, 'ossec.log')
SYSCOLLECTOR_DB_PATH = os.path.join(WAZUH_PATH, 'queue', 'syscollector', 'db', 'local.db')
else:
if sys.platform == 'darwin':
WAZUH_PATH = os.path.join("/", "Library", "Ossec")
else:
WAZUH_PATH = os.path.join("/var", "ossec")
LOG_FILE_PATH = os.path.join(WAZUH_PATH, 'logs', 'ossec.log')
SYSCOLLECTOR_DB_PATH = os.path.join(WAZUH_PATH, 'queue', 'syscollector', 'db', 'local.db')


WAZUH_CONF_PATH = os.path.join(WAZUH_PATH, 'etc', 'ossec.conf')
Expand All @@ -40,6 +42,8 @@
WAZUH_TESTING_PATH = os.path.dirname(os.path.abspath(__file__))
WAZUH_TESTING_DATA_PATH = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'data')
DEFAULT_AUTHD_PASS_PATH = os.path.join(WAZUH_PATH, 'etc', 'authd.pass')
TEMPLATE_DIR = 'configuration_template'
TEST_CASES_DIR = 'test_cases'


# Daemons
Expand All @@ -66,6 +70,7 @@
ANALYSISD_STATE = os.path.join(WAZUH_PATH, 'var', 'run', 'wazuh-analysisd.state')

# Timeouts
T_2 = 2
T_5 = 5
T_10 = 10
T_20 = 20
Expand Down
24 changes: 24 additions & 0 deletions deps/wazuh_testing/wazuh_testing/db_interface/cve_db.py
Original file line number Diff line number Diff line change
Expand Up @@ -228,3 +228,27 @@ def get_nvd_metadata_timestamp(year):
return None

return result[0]


def get_rows_from_table(value, column, table, limit=None):
"""
Args:
value (str): value that user wants to find in query
column (str): Name of the column where the value will be searched for.
table (str): Name of the table where the value will be searched for.
limit (int) - Optional: Maximum amount of results to look for. Default None (No Limit used).
Returns:
List (str): List with each instance of the value found
"""

query_string = f"SELECT * FROM {table} WHERE {column} LIKE '{value}'"

if limit is not None:
query_string = query_string + f"LIMIT {limit}"

result = get_sqlite_query_result(CVE_DB_PATH, query_string)
if len(result) == 0:
return None

return result[0]
18 changes: 11 additions & 7 deletions deps/wazuh_testing/wazuh_testing/event_monitor.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,35 +5,39 @@

def make_callback(pattern, prefix=''):
"""Create a callback function from a text pattern.
Args:
pattern (str): String to match on the log.
pattern (str): string to match on the log.
prefix (str): regular expression used as prefix before the pattern.
Returns:
lambda: function that returns if there's a match in the file
"""
pattern = r'\s+'.join(pattern.split())
regex = re.compile(r'{}{}'.format(prefix, pattern))
regex = re.compile(r'{}{}'.format(prefix, pattern)) if prefix else re.compile(pattern)

return lambda line: regex.match(line)


def check_event(file_monitor=None, callback='', error_message=None, update_position=True, timeout=20,
accum_results=1, file_to_monitor=None):
"""Check if an API event occurs
accum_results=1, file_to_monitor=None, prefix=None):
"""Check if an event occurs.
Args:
file_monitor (FileMonitor): FileMonitor object to monitor the file content.
callback (str): log regex to check in the file
error_message (str): error message to show in case of expected event does not occur
update_position (boolean): filter configuration parameter to search in the file
accum_results (int): accumulation of matches.
file_to_monitor (str): path to the file to be monitored.
timeout (str): timeout to check the event in the file
prefix (str): log pattern regex
accum_results (int): Accumulation of matches.
prefix (str): prefix of the log message regex
"""
file_monitor = FileMonitor(file_to_monitor) if file_monitor is None else file_monitor
error_message = f"Could not find this event in {file_to_monitor}: {callback}" if error_message is None else \
error_message

result = file_monitor.start(timeout=timeout, update_position=update_position, accum_results=accum_results,
callback=make_callback(callback), error_message=error_message)
callback=make_callback(callback, prefix), error_message=error_message)

return result
12 changes: 5 additions & 7 deletions deps/wazuh_testing/wazuh_testing/modules/__init__.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
# Copyright (C) 2015-2023, Wazuh Inc.
# Created by Wazuh, Inc. <[email protected]>.
# This program is free software; you can redistribute it and/or modify it under the terms of GPLv2

'''
The purpose of this file is to contain all the variables necessary for Wazuh in order to be easier
to maintain if one of them changes in the future.
copyright: Copyright (C) 2015-2023, Wazuh Inc.
Created by Wazuh, Inc. <[email protected]>.
This program is free software; you can redistribute it and/or modify it under the terms of GPLv2
'''
import pytest

WAZUH_SERVICE_PREFIX = 'wazuh'
# Services Variables
WAZUH_SERVICES_STOPPED = 'stopped'
WAZUH_SERVICE_PREFIX = 'wazuh'
WAZUH_SERVICES_STOP = 'stop'
WAZUH_SERVICES_START = 'start'

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,10 @@ def check_api_event(file_monitor=None, callback='', error_message=None, update_p
callback (str): log regex to check in the file
error_message (str): error message to show in case of expected event does not occur
update_position (boolean): filter configuration parameter to search in the file
timeout (str): timeout to check the event in the file
timeout (int): timeout to check the event in the file
prefix (str): log pattern regex
accum_results (int): Accumulation of matches.
file_to_monitor (str): File to be monitored.
"""
file_monitor = FileMonitor(file_to_monitor) if file_monitor is None else file_monitor
error_message = f"Could not find this event in {file_to_monitor}: {callback}" if error_message is None else \
Expand Down
58 changes: 58 additions & 0 deletions deps/wazuh_testing/wazuh_testing/modules/aws/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
from pathlib import Path

from wazuh_testing import WAZUH_PATH

AWS_MODULE_PATH = Path(WAZUH_PATH, 'wodles', 'aws')
S3_CLOUDTRAIL_DB_PATH = Path(AWS_MODULE_PATH, 's3_cloudtrail.db')
AWS_SERVICES_DB_PATH = Path(AWS_MODULE_PATH, 'aws_services.db')

AWS_LOGS = 'AWSLogs'
RANDOM_ACCOUNT_ID = '819751203818'
CLOUDTRAIL = 'CloudTrail'
GUARDDUTY = 'GuardDuty'
VPC_FLOW_LOGS = 'vpcflowlogs'
FLOW_LOG_ID = 'fl-0754d951c16f517fa'
CONFIG = 'Config'
ELASTIC_LOAD_BALANCING = 'elasticloadbalancing'
SERVER_ACCESS_TABLE_NAME = 's3_server_access'
PERMANENT_CLOUDWATCH_LOG_GROUP = 'wazuh-cloudwatchlogs-integration-tests'
TEMPORARY_CLOUDWATCH_LOG_GROUP = 'temporary-log-group'
FAKE_CLOUDWATCH_LOG_GROUP = 'fake-log-group'

EVENT_TIME_FORMAT = '%Y-%m-%dT%H:%M:%SZ'
PATH_DATE_FORMAT = '%Y/%m/%d'
PATH_DATE_NO_PADED_FORMAT = '%Y/%-m/%-d'
FILENAME_DATE_FORMAT = '%Y%m%dT%H%MZ'
ALB_DATE_FORMAT = '%Y-%m-%dT%H:%M:%fZ'

US_EAST_1_REGION = 'us-east-1'

JSON_EXT = '.json'
LOG_EXT = '.log'
JSON_GZ_EXT = '.jsonl.gz'
CSV_EXT = '.csv'

# Bucket types
CLOUD_TRAIL_TYPE = 'cloudtrail'
VPC_FLOW_TYPE = 'vpcflow'
CONFIG_TYPE = 'config'
ALB_TYPE = 'alb'
CLB_TYPE = 'clb'
NLB_TYPE = 'nlb'
KMS_TYPE = 'kms'
MACIE_TYPE = 'macie'
KMS_TYPE = 'kms'
TRUSTED_ADVISOR_TYPE = 'trusted'
CUSTOM_TYPE = 'custom'
GUARD_DUTY_TYPE = 'guardduty'
NATIVE_GUARD_DUTY_TYPE = 'native-guardduty'
WAF_TYPE = 'waf'
SERVER_ACCESS = 'server_access'
CISCO_UMBRELLA_TYPE = 'cisco_umbrella'

# Params

ONLY_LOGS_AFTER_PARAM = '--only_logs_after'


local_internal_options = {'wazuh_modules.debug': '2', 'monitord.rotate_log': '0'}
Loading

0 comments on commit 9ee8732

Please sign in to comment.