Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add qa-docs docker, copyright and readme changes after 0.1 testing #2647

Merged
merged 7 commits into from
Mar 8, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 50 additions & 21 deletions deps/wazuh_testing/wazuh_testing/qa_docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,6 @@ documentation content into the App UI.
| ├── search_ui | search-ui module directory
| ├── __init__.py
| ├── CHANGELOG.md | Record of all notable changes made
| ├── deploy_qa_docs.sh | Script that builds the qa-docs image and runs it using a specific branch
| ├── doc_generator.py | The main module and the entry point of the tool execution
| ├── requirements.txt | Contains the modules that qa-docs needs
| ├── README.MD
Expand All @@ -96,7 +95,7 @@ documentation content into the App UI.
## Schema
The schema file of the tool is located at **qa-docs/schema.yaml**.

The schema fields are specified in the qa-docs documenting test [wiki](https:/wazuh/wazuh-qa/wiki/QADOCS-tool-How-to-document-a-test).
The schema fields are specified in the qa-docs documenting test [wiki](https:/wazuh/wazuh-qa/wiki/QA-Documentation---How-to-document-a-test-using-Schema-2.0).

## Installation

Expand Down Expand Up @@ -191,36 +190,46 @@ Also, you can visit the `qa-docs` tool [use guide](https:/wazuh/wazu
#### Complete run
qa-docs --tests-path /path-to-tests-to-parse/

Using just the `--tests-path` flag, the tool will load the schema file and run a complete parse of the paths in the
Using just the `-p, --tests-path` flag, the tool will load the schema file and run a complete parse of the paths in the
configuration to dump the content into the output folder located in the `qa-docs` build directory. e.g: `qa-docs --tests-path /wazuh-qa/tests/`

#### Parse specific type(s)
qa-docs --tests-path /path-to-tests-to-parse/ --types <TYPE1> <TYPE2>

Using `--type` flag you can parse only the tests inside the type(s) folder(s) you want.
Using `-t, --types` flag you can parse only the tests inside the type(s) folder(s) you want.

#### Parse specific module(s)
qa-docs --tests-path /path-to-tests-to-parse/ --types <TYPE1> <TYPE2> --modules <MODULE1> <MODULE2>
#### Parse specific component(s)
qa-docs --tests-path /path-to-tests-to-parse/ --types <TYPE> --components <COMPONENT1> <COMPONENT2>

Using `-c, --components` flag you can parse only the tests inside the component(s) folder(s) you want. It also needs the type of tests where the tests are located.

Using `--modules` flag you can parse only the tests inside the modules(s) folder(s) you want. It also needs the type of tests where the tests are located.
#### Parse specific suite(s)
qa-docs --tests-path /path-to-tests-to-parse/ --types <TYPE> --components <COMPONENT> --suites <SUITE1> <SUITE2>

#### Parse specific test(s)
qa-docs --tests-path /path-to-tests-to-parse/ -t(--test) TEST_NAME1 TEST_NAME2
Using `-s, --suites` flag you can parse only the tests inside the components(s) folder(s) you want.

Using `-t, --test` flag you can parse only the tests that you want. The documentation parsed will be printed, if you want to save it you have to use the `-o`
flag and specify the output directory. e.g: `qa-docs --tests-path /wazuh-qa/tests/ -t test_cache test_cors -o /tmp`.
#### Parse specific module(s)
qa-docs --tests-path /path-to-tests-to-parse/ --types <TYPE> --components <COMPONENT> --suites <SUITE> -m(--modules) MODULE_NAME1 MODULE_NAME2

Using `-m, --modules` flag you can parse only the modules that you want. The documentation parsed will be printed, if you want to save it you have to use the `-o`
flag and specify the output directory. To parse modules you need to specify their type, component and suite. e.g: `qa-docs -p /wazuh-qa/tests/ --types integration --components test_api -s test_config -t test_cache test_cors -o /tmp`.

This option is not compatible with API-related options, because the output is printed or saved with `-o` in a custom directory.

#### Check if test(s) exist
qa-docs --tests-path /path-to-tests-to-parse/ -e(--exist) TEST_NAME1 TEST_NAME2
qa-docs --tests-path /path-to-tests-to-parse/ --types <TYPE> --components <COMPONENT> --suites <SUITE> -e(--exist) MODULE_NAME1 MODULE_NAME2

With this option the tool prints if test(s) do(es) exist.

#### Check if module(s) are documented following qa-docs current schema
qa-docs -p /path-to-tests-to-parse/ -t <TYPE> -c <COMPONENT> -s <SUITE> -m TEST_NAME1 TEST_NAME2 --check-documentation

With this option the tool prints if test(s) has the expected documentation blocks following the qa-docs schema.

#### Sanity Check
qa-docs --tests-path /path-to-tests-to-parse/ -s
qa-docs --tests-path /path-to-tests-to-parse/ --sanity-check

Using `-s`, the tool will run a sanity check of the content in the output folder.
Using `--sanity-check`, the tool will run a sanity check of the content in the output folder.

It will check the coverage of the already parsed files in the **output path** comparing it with the tests found within
the **tests path**.
Expand Down Expand Up @@ -274,7 +283,7 @@ Using `-l` option, the tool launches the application with a previously generated
#### Index output data and launch the api
qa-docs -il <INDEX_NAME>

Using `-il` option, the tool indexes the content of each file output as a document into ElasticSearch and then launches the API. The name of the index must be provided as a parameter. e.g: `qa-docs -I /wazuh-qa/tests/ -il qa-tests`. A previous run must be performed, e.g.`qa-docs -I /wazuh-qa/tests/` so the output data is previously generated.
Using `-il` option, the tool indexes the content of each file output as a document into ElasticSearch and then launches the API. The name of the index must be provided as a parameter. e.g: `qa-docs -p /wazuh-qa/tests/ -il qa-tests`. A previous run must be performed, e.g.`qa-docs -p /wazuh-qa/tests/` so the output data is previously generated.

### Sample executions

Expand All @@ -283,9 +292,24 @@ Using `-il` option, the tool indexes the content of each file output as a docume
qa-docs --tests-path /path-to-tests/
```

- Parse `fim` module
- Parse `fim` component
```
qa-docs --tests-path /path-to-tests/ --type integration --components test_fim
```

- Parse api `config` and `rbac` suites
```
qa-docs --tests-path /path-to-tests/ --type integration --modules test_fim
qa-docs --tests-path /path-to-tests/ --type integration --components test_api --suites test_config test_rbac
```

- Parse some `logtest` modules
```
qa-docs --tests-path /path-to-tests/ --type integration --components test_logtest --suites test_invalid_token --modules test_invalid_session_token
```

- Check if some `logtest` modules has documentation blocks
```
qa-docs -p /path-to-tests/ --type integration --components test_logtest --suites test_configuration --modules test_configuration_file test_get_configuration_sock --check-documentation
```

- Index the parsed data
Expand All @@ -298,9 +322,9 @@ qa-docs -i my_index
qa-docs -l my_index
```

- Parse `vulnerability_detector` module, index the output data, and launch `search-ui`
- Parse `vulnerability_detector` component, index the output data, and launch `search-ui`
```
qa-docs --tests-path /path-to-tests/ --type integration --modules test_vulnerability_detector -il vd-index
qa-docs --tests-path /path-to-tests/ --type integration --components test_vulnerability_detector -il vd-index
```

## Docker deployment
Expand All @@ -311,10 +335,15 @@ Few examples:

- Parse `test_active_response` tests and generate the documentation in `/tmp/qa_docs`:
```bash
qa-docs --docker-run --types integration --modules test_active_response --qa-branch 1796-migrate-doc-active-response
qa-docs --docker-run --types integration --components test_active_response --qa-branch 1796-migrate-doc-active-response
```

- Parse `test_fim` tests and generate the documentation in custom folder:
```bash
qa-docs --docker-run --types integration --components test_fim -o /tmp/fim_docu --qa-branch 1796-migrate-doc-active-response
```

- Parse `test_active_response` and `test_agentd` tests and launch search-ui to visualize the documentation:
```bash
qa-docs --docker-run --types integration --modules test_active_response test_agentd -il qa-index --qa-branch 1796-migrate-doc-schema-2
qa-docs --docker-run --types integration --components test_active_response test_agentd -il qa-index --qa-branch 1796-migrate-doc-schema-2
```
2 changes: 1 addition & 1 deletion deps/wazuh_testing/wazuh_testing/qa_docs/doc_generator.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2015-2021, Wazuh Inc.
# Copyright (C) 2015-2022, Wazuh Inc.
# Created by Wazuh, Inc. <[email protected]>.
# This program is free software; you can redistribute it and/or modify it under the terms of GPLv2

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ then
exit 1
fi

/usr/local/bin/qa-docs --tests-path /tests/wazuh-qa/tests --validate-parameters ${CMD}
/usr/local/bin/qa-docs -p /tests/wazuh-qa/tests --validate-parameters ${CMD}

# get run status
status=$?
Expand All @@ -37,8 +37,8 @@ fi
service wazuh-manager start

# Run qa-docs with the given args
echo "Running /usr/local/bin/qa-docs -I /tests/wazuh-qa/tests ${CMD}"
/usr/local/bin/qa-docs --tests-path /tests/wazuh-qa/tests ${CMD}
echo "Running /usr/local/bin/qa-docs -p /tests/wazuh-qa/tests ${CMD}"
/usr/local/bin/qa-docs -p /tests/wazuh-qa/tests ${CMD}

# Move the documentation parsed to the shared dir
echo "Moving qa-docs output to shared directory: ${SHARED_VOL}/output"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2015-2021, Wazuh Inc.
# Copyright (C) 2015-2022, Wazuh Inc.
# Created by Wazuh, Inc. <[email protected]>.
# This program is free software; you can redistribute it and/or modify it under the terms of GPLv2

Expand Down
2 changes: 1 addition & 1 deletion deps/wazuh_testing/wazuh_testing/qa_docs/lib/config.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2015-2021, Wazuh Inc.
# Copyright (C) 2015-2022, Wazuh Inc.
# Created by Wazuh, Inc. <[email protected]>.
# This program is free software; you can redistribute it and/or modify it under the terms of GPLv2

Expand Down
2 changes: 1 addition & 1 deletion deps/wazuh_testing/wazuh_testing/qa_docs/lib/index_data.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2015-2021, Wazuh Inc.
# Copyright (C) 2015-2022, Wazuh Inc.
# Created by Wazuh, Inc. <[email protected]>.
# This program is free software; you can redistribute it and/or modify it under the terms of GPLv2

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2015-2021, Wazuh Inc.
# Copyright (C) 2015-2022, Wazuh Inc.
# Created by Wazuh, Inc. <[email protected]>.
# This program is free software; you can redistribute it and/or modify it under the terms of GPLv2

Expand Down
2 changes: 1 addition & 1 deletion deps/wazuh_testing/wazuh_testing/qa_docs/lib/sanity.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2015-2021, Wazuh Inc.
# Copyright (C) 2015-2022, Wazuh Inc.
# Created by Wazuh, Inc. <[email protected]>.
# This program is free software; you can redistribute it and/or modify it under the terms of GPLv2

Expand Down
12 changes: 5 additions & 7 deletions deps/wazuh_testing/wazuh_testing/qa_docs/lib/utils.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2015-2021, Wazuh Inc.
# Copyright (C) 2015-2022, Wazuh Inc.
# Created by Wazuh, Inc. <[email protected]>.
# This program is free software; you can redistribute it and/or modify it under the terms of GPLv2

Expand All @@ -7,10 +7,9 @@
import sys
import shutil

from tempfile import gettempdir

from wazuh_testing.qa_docs import QADOCS_LOGGER
from wazuh_testing.tools.logging import Logging
from wazuh_testing.tools.exceptions import QAValueError

utils_logger = Logging.get_logger(QADOCS_LOGGER)

Expand Down Expand Up @@ -263,9 +262,8 @@ def run_local_command(command):
result_code = run.returncode

if result_code != 0:
print("error")
# raise QAValueError(f"The command {command} returned {result_code} as result code.", utils_logger.LOGGER.error,
# QADOCS_LOGGER)
raise QAValueError(f"The command {command} returned {result_code} as result code.", utils_logger.error,
QADOCS_LOGGER)


def run_local_command_with_output(command):
Expand Down Expand Up @@ -296,7 +294,7 @@ def qa_docs_docker_run(qa_branch, command, output_path):
"""
docker_args = f"{qa_branch} {output_path} {command}"
docker_image_name = 'wazuh/qa-docs'
docker_image_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', 'dockerfiles')
docker_image_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'dockerfiles')

utils_logger.info(f"Building qa-docs docker image")
run_local_command_with_output(f"cd {docker_image_path} && docker build -q -t {docker_image_name} .")
Expand Down
22 changes: 4 additions & 18 deletions deps/wazuh_testing/wazuh_testing/scripts/qa_docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,16 +43,6 @@ def set_qadocs_logger_level(logging_level):
qadocs_logger.set_level(logging_level)


def set_parameters(args):
# Set the qa-docs logger level
if args.debug_level:
set_qadocs_logger_level('DEBUG')

# Deactivate the qa-docs logger if necessary.
if args.no_logging:
set_qadocs_logger_level(None)


def set_parameters(args):
"""Set the QADOCS parameters.

Expand All @@ -70,16 +60,12 @@ def set_parameters(args):
if args.no_logging:
set_qadocs_logger_level(None)

if args.run_with_docker:
if args.output_path:
global OUTPUT_PATH
OUTPUT_PATH = os.path.join(gettempdir(), 'qa_docs')

if args.output_path and not args.run_with_docker:
OUTPUT_PATH = os.path.join(args.output_path, 'output')

if args.output_format:
global OUTPUT_FORMAT
OUTPUT_FORMAT = args.output_format
if args.run_with_docker:
OUTPUT_PATH = args.output_path if args.output_path else os.path.join(gettempdir(), 'qa_docs')


def get_parameters():
Expand Down Expand Up @@ -332,7 +318,7 @@ def validate_parameters(parameters, parser):
if parameters.test_modules or parameters.test_exist:
# If at least one module is specified
if len(parameters.test_components) != 1:
raise QAValueError('The --modules option work swhen is only parsing a single test component. Use '
raise QAValueError('The --modules option works when is only parsing a single test component. Use '
'--components with just one component if you want to parse some modules within a '
'test component.', qadocs_logger.error)

Expand Down