Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qa-docs: Prepare v0.1 tool release #2075

Closed
42 tasks done
roronoasins opened this issue Oct 19, 2021 · 0 comments · Fixed by #2649
Closed
42 tasks done

qa-docs: Prepare v0.1 tool release #2075

roronoasins opened this issue Oct 19, 2021 · 0 comments · Fixed by #2649
Assignees

Comments

@roronoasins
Copy link
Contributor

roronoasins commented Oct 19, 2021

We want to release the first version of the qa-docs tool. For this purpose, it is proposed to make the last improvements to the tool and to proceed with the testing and the necessary checks to verify the correct operation. So we are using the last qa-docs changes made in #1864.

Improvements

  • Create auto fields in schema.yaml
  • Fix inputs Auto/Manual generation

Tasks

  • Update qa-docs tool entry in the wiki
  • Fix examples in How to document a test wiki entry
  • Update entrypoint
  • Add new tags to schema.yaml
  • Update how qa-ctl uses qa-docs
  • Prepare the tests input branch to perform the run checks

Additional tasks

  • Change default path to tempdir
  • Extend -o use to all parse options
  • Generate only one format file(add format selection within tool parameters)
  • Improve indexing data logging info
  • Select with -o the Docker and host shared volume
  • Deprecate -I option
  • Extend incompatibilites/validation check to new parameters.
  • qa-docs: Improve SearchUI style #2196

Checks

  • Update qa-docs ChangeLog
  • qa-docs installation in Linux following the installation guide
  • qa-docs installation in Windows following the installation guide
    • qa-docs tool
    • ElasticSearch
    • Docker
    • npm

Run Checks

To test the qa-docs tool, I am creating a new branch with the tests to use as input because there are no modules documented in master following the qa-docs schema 2.0.

Branch: 2075-tests-input

Linux

  • Check that a test exists
    qa-docs --tests-path /path/to/tests -e test_os_exec
  • Parse a single test
    qa-docs --tests-path /path/to/tests -t test_os_exec -o /tmp
  • Parse two integration modules
    qa-docs --tests-path /path/to/tests --types integration --modules test_active_response test_analysisd
  • Perform a sanity check
    qa-docs --tests-path /path/to/tests -s
  • Index that parsed data
    qa-docs -i index
  • Launch search-ui with the index previously created
    qa-docs -l index
  • Docker runs
    qa-docs --docker-run --types integration --modules test_active_response --qa-branch 2075-tests-input
    qa-docs --docker-run --types integration --modules test_active_response --qa-branch 2075-tests-input -d -o /tmp -i qa

Windows

  • Check that a test exists
    qa-docs --tests-path /path/to/tests -e test_os_exec
  • Parse a single test
    qa-docs --tests-path /path/to/tests -t test_os_exec -o %TEMP%
  • Parse two integration modules
    qa-docs --tests-path /path/to/tests --types integration --modules test_active_response
  • Perform a sanity check
    qa-docs --tests-path /path/to/tests -s
  • Index that parsed data
    qa-docs -i index
  • Launch search-ui with the index previously created
    qa-docs -l index
  • Docker runs
    qa-docs --docker-run --types integration --modules test_active_response --qa-branch 2075-tests-input
    qa-docs --docker-run --types integration --modules test_active_response --qa-branch 2075-tests-input -d -o /tmp -i qa

Invalid qa-docs script parameters

  • -v, --version must be run in isolation
  • -s, --sanity-check cannot be run with -t, --tests, --types, --modules, -e, --exist
  • -t, --tests, --types, --modules, -e, --exist, -s, --sanity-check must to be run with -I
  • -e, --exist, -t, --tests cannot be run with -i, -l, -il
  • --types, --modules cannot be run with -o, -t, --tests, -e, --exist
@roronoasins roronoasins self-assigned this Oct 19, 2021
roronoasins pushed a commit that referenced this issue Oct 20, 2021
Now `qa-docs` allows the autogenerated fields.
When the autogenerated path is specified in the module documentation, it will raise an error.
roronoasins pushed a commit that referenced this issue Oct 20, 2021
New FIM tags added to `schema.yaml` file.
roronoasins pushed a commit that referenced this issue Oct 20, 2021
- Useless `schema.yaml` field removed.
- Refac how test_cases are stored.
roronoasins pushed a commit that referenced this issue Oct 20, 2021
Now `if-in` with `dict.keys()` is used instead of `try-except`.
roronoasins pushed a commit that referenced this issue Oct 25, 2021
Also added `run_local_command` to `utils.py`.
roronoasins pushed a commit that referenced this issue Oct 26, 2021
Instead of log error for subset of incompatible options, now, each incompatible possibility has its error message.
roronoasins pushed a commit that referenced this issue Nov 3, 2021
roronoasins pushed a commit that referenced this issue Nov 3, 2021
Now, using `--format` you can select if you want the output data with JSON or YAML format.
@roronoasins roronoasins linked a pull request Mar 10, 2022 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant