Research Vulnerability Detector integration tests

[damarisg]

Related Issue
#2792

Description

As part of #2792, it is necessary to investigate the development of the new structure in the Vulnerability Detector tests.
In addition, we must learn about the coverage of Suites: test_feeds, test_providers, and test_scan_results that integration tests have.

To Do List

• Analyze how it works Vulnerability Detector
• Watch the video that explains the new structure applied.
• Check coverage of the suite test_feeds
• Check coverage of the suite test_providers
• Check coverage of the suite test_scan_results

[CamiRomero]

Task 1: Analyze how it works Vulnerability Detector

For this task, I watch a video that explains the basic Vulnerability Detector module and a video that explains how it works the status of vulnerabilities, and with that information and the official documentation, I write a personal documentation with the more important information about how vulnerability detector works.

[CamiRomero]

Task 2: Watch the video that explains the new structure applied.

For this task, I watch a video that explains the new structure applied to the Integration test.
Issue related: #2462
PR related: #2499

[CamiRomero]

Task 3: Check coverage of the suite test_feeds

In this task, I do a quick view of tests inside test_feed.

Suite test feeds:

• test_cpe_indexing: Check that the CPE helper info related to the packages/system data is indexed.

• test_dowload_feed: Check if modulesd downloads successfully the feeds from different providers and os, and check that
  the feed update date does not exceed a set limit.

• test_duplicate_feed: Check that the vulnerabilities are not repeated in the database when they are indexed from the feeds.

• test_import_invalid_feed_type: Check the vulnerability detector behavior when importing unexpected file types as feeds

• test_validate_feed_content: Check if the downloaded XML/JSON feeds have the expected format and are XML/JSON parseable

[CamiRomero]

Task 4: Check coverage of the suite test_providers

In this task, I do a quick view of tests inside test_providers

• test_enabled: Check if modulesd downloads/does not download the feeds from different providers when enabled is set to yes/no
• test_missing_os: Check if modulesd starts downloading the feeds without specifying the os version
• test_multiple_provider_feeds: Check if the and options work properly according to the configuration
  and check there are no conflicts when reading the feeds and inserting them in the database.
• test_os: Check if modulesd downloads the feeds for each os.
• test_update_from_year: Check if vulnerability detector download feeds from the correct year based on update_from_year option
• test_update_interval.py: Check if the provider feed starts updating the database after the interval time set in
  <interval_update> tag.

[CamiRomero]

Task 5: Check coverage of the suite test_scan_results

• test_scan_nvd_vulnerabilities: Check if the corresponding vulnerability alerts and logs are generated after scanning with an NVD feed and custom vulnerable packages
• test_scan_provider_and_nvd_vulnerabilities.py: Check if the vulnerability alerts are reported from NVD and OVAL feed sources
• test_scan_provider_vulnerabilities: Check if the vulnerability alerts are reported making vulnerabilities found only by OVAL and not NVD.
• test_scan_vulnerability_removal: Check that the Vulnerability Detector module generates an alert when a vulnerability is removed from the inventory when a package is updated to a non-vulnerable version.

[Deblintrake09]

Task 1: Analyze how it works Vulnerability Detector

I watch a video that explains the basic Vulnerability Detector functionality, how to configure it, and how it works. Also, watched another video explaining how alerts are generated based on during partial and full scans. Wrote some notes on how VD works with the data from the videos and the official documentation.

[Deblintrake09]

Task 2: Watch the video that explains the new structure applied.

Watched a video that explains the new structure applied to the Integration test.
Issue related: #2462
PR related: #2499

[Deblintrake09]

Task 3: Check coverage of the suite test_feeds

For this Task, I did a quick analysis of the files in the test_feeds suite, and tried to determine possible test cases that were missing.

Test feeds Suidt:

• test_cpe_indexing: Check that the CPE helper info related to the packages/system data is indexed.
• test_dowload_feed: Check if modulesd downloads successfully the feeds from different providers and os, and check thatthe feed update date does not exceed a set limit.
• test_duplicate_feed: Check that the vulnerabilities are not repeated in the database when they are indexed from the feeds.
• test_import_invalid_feed_type: Check the vulnerability detector behavior when importing unexpected file types as feeds
• test_validate_feed_content: Check if the downloaded XML/JSON feeds have the expected format and are XML/JSON parseable

Possible coverage to Add:

• At first I thought it might be a good idea to test custom downloaded feeds, as per the documentation but that is already tested

[Deblintrake09]

Task 4: Check coverage of the suite test_providers

This task consisted of a quick review of tests inside test_providers

• test_enabled: Check modulesd downloads the feeds from different providers according to the <enabled> tag being set to yes/no
• test_missing_os: Check modulesd starts downloading feeds without specifying the os version
• test_multiple_provider_feeds: Check if the <path> and <os path=...> options work properly according to the configuration
  and check there are no conflicts when reading the feeds and inserting them in the database.
• test_os: Check if modulesd downloads the feeds for each os.
• test_update_from_year: Check if vulnerability detector download feeds from the correct year based on update_from_year option
• test_update_interval.py: Check if the provider feed starts updating the database after the interval time set in
  <interval_update> tag.

[Deblintrake09]

Task 5: Check coverage of the suite test_scan_results

• test_scan_nvd_vulnerabilities/: Checks the corresponding vulnerability alerts and logs are generated after scanning with an NVD feed and custom vulnerable packages (Works for MacOS & Windows).
• test_scan_nvd_vulnerabilities/test_no_agent_data: Checks a Warning message is displayed when attempting to perform vulnerability scan using only NVD feed for Linux systems. (For Linux Systems)
• test_scan_provider_and_nvd_vulnerabilities.py: Check the vulnerability alerts are reported from NVD and OVAL feed sources
• test_scan_provider_vulnerabilities: Check the vulnerability alerts are reported making vulnerabilities found only by OVAL and not NVD.
• test_scan_vulnerability_removal: Check that the Vulnerability Detector module generates an alert when a vulnerability is removed from the inventory when a package is updated to a non-vulnerable version.

[damarisg]

Task 1: Analyze how it works Vulnerability Detector

• I watch a video that explains the Vulnerability Detector module
• I watch a video that explains how it works the status of vulnerabilities,
• I write a personal draft with the more important information of the videos.

[damarisg]

Task 2: Watch the video that explains the new structure applied.

• I watch a video that explains the new structure applied to the Integration test.
• I see changes of Refactor of the testing framework for vulnerability_detector #2499

[damarisg]

Task 3: Check coverage of the suite test_feeds

I do a quick view of tests inside test_feed.

• test_cpe_indexing.
• test_dowload_feed.
• test_duplicate_feed.
• test_import_invalid_feed_type.
• test_validate_feed_content.

[damarisg]

Task 4: Check coverage of the suite test_providers

I do a quick view of tests inside test_providers:

• test_enabled.
• test_missing_os.
• test_multiple_provider_feeds.
• test_os.
• test_update_from_year.
• test_update_interval.py.

[damarisg]

Task 5: Check coverage of the suite test_scan_results

I do a quick view of tests inside test_scan_results:

• test_scan_nvd_vulnerabilities.
• test_scan_provider_and_nvd_vulnerabilities.py.
• test_scan_provider_vulnerabilities.
• test_scan_vulnerability_removal.