new(test): add `test/vm` for localhost VM-based driver kernel compatibility tests #524

incertum · 2022-08-02T05:40:58Z

Signed-off-by: Melissa Kilby [email protected]

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area API-version

/area build

/area driver-kmod

/area driver-bpf

/area driver-modern-bpf

/area libscap-engine-bpf

/area libscap-engine-gvisor

/area libscap-engine-kmod

/area libscap-engine-modern-bpf

/area libscap-engine-nodriver

/area libscap-engine-noop

/area libscap-engine-source-plugin

/area libscap-engine-udig

/area libscap

/area libpman

/area libsinsp

/area tests

/area proposals

Does this PR require a change in the driver versions?

/version driver-API-version-major

/version driver-API-version-minor

/version driver-API-version-patch

/version driver-SCHEMA-version-major

/version driver-SCHEMA-version-minor

/version driver-SCHEMA-version-patch

What this PR does / why we need it:

Edited:

Add test/vm (localhost automated VM based kernel - compiler version grid-search driver tests) to test targets similar to e2e sinsp tests.

CI integration is tracked here #531.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

new(test): add `test/vm` for localhost VM-based driver kernel compatibility tests

FedeDP · 2022-08-02T05:51:26Z

Just WOW 😲
This is something we have been needing for a while!
Having something similar in the CI is super valuable indeed!
Perhaps a cmake target to run the tests would help a bit :)
I will give it a look asap, at least for some initial feedback!
Thank you very much for this contribution!

jasondellaluce · 2022-08-02T08:21:35Z

This is real good Melissa! We've been looking for something like this for a while now. Thank you a lot!

incertum · 2022-08-03T05:45:17Z

Just WOW 😲 This is something we have been needing for a while! Having something similar in the CI is super valuable indeed!

localhost obviously is more tractable to gain more confidence in your code changes and spot issues you care about in your own deployment :) which typically is only a few distros and a smaller grid of kernel releases ...

Proper CI needs to care about more which is such a daunting task, that's probably why we are where we are.
#531

Perhaps a cmake target to run the tests would help a bit :) I will give it a look asap, at least for some initial feedback! Thank you very much for this contribution!

Let me look into this more as this gets a bit more polished. Will also look more into #506 to converge to a good common style for such tests.

incertum · 2022-11-05T03:57:22Z

@FedeDP: Added cmake targets as I finally got to refactoring and polishing it a bit. Thanks again for the suggestion, it helped making the driver-sanity tests cleaner and easier to use.
@Molter73: Tried to make the cmake setup similar to e2e targets. Thanks to your work on e2e it was pretty straight forward as I used the e2e cmake setup as template.
@LucaGuerra: Thanks for your feedback in the CI integration discussion. Did not forget about making sure to reboot the VM after each kmod test to be resilient against buggy kernel modules.

Removed WIP as v1 would now be considered complete. The PR is now ready for review. Aware that it's a random collection of scripts, mostly bash scripts. Happy to address any style feedback throughout in an attempt to make the scripts easier to follow and maintain, I think there is improvement potential in this regard.

Outlook: Emulation support still missing and arm64 support as well ... let's first see if v1 is useful for x86_64 for now.

Few observations: Noticed that in general ubuntu distro appears to be the most difficult one. Re-ran these tests many times and a few ubuntu kmod tests sometimes randomly just did not work. centos7 stable in comparison.
Unrelated to this PR also noticed that the eBPF verifier on ubuntu let's minor bugs pass compared to other distros (same kernel versions), hence yum based distros appear to be the better choice for dev for sure. Also kmod never seemed to work for newer ubuntu kernels (e.g. 5.19 or 6.0), while centos7 was fine.

FedeDP · 2022-12-02T13:50:18Z

/milestone 0.11.0

poiana · 2023-03-02T15:54:30Z

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https:/falcosecurity/community.

/lifecycle stale

incertum · 2023-03-02T16:49:53Z

/remove-lifecycle stale

and highlight basic system dependencies Co-authored-by: Massimiliano Giovagnoli <[email protected]> Signed-off-by: Melissa Kilby <[email protected]>

Co-authored-by: Massimiliano Giovagnoli <[email protected]> Signed-off-by: Melissa Kilby <[email protected]>

poiana · 2023-06-30T10:32:10Z

LGTM label has been added.

Git tree hash: e769b66a9136d603c536cdd4838609ae7769456e

Molter73

Sorry for the delay on this @incertum!! I left a few more comments, but overall LGTM and unless someone else has objections I think we can go ahead and merge.

test/vm/CMakeLists.txt

test/vm/README.md

test/vm/kernels.jsonl

test/vm/scripts/compile_drivers.sh

test/vm/scripts/compile_scap_open.sh

test/vm/scripts/kernel_download.sh

Signed-off-by: Melissa Kilby <[email protected]>

incertum · 2023-07-03T18:19:53Z

Thank you @Molter73 for taking another look, appreciate it 🙏 !

Also re-worked the previous commit of adding more info to the readme here 3d96343 more clearly stating how this is different and separate from the approach we will take for the new kernel driver CI-powered tests. I thought it was important to highlight as there will be no overlap in setup, scripts and design choices, because of these very different use cases. CC @LucaGuerra @therealbobo @FedeDP

minor cleanup setting expectations for duration of tests Co-authored-by: Mauro Ezequiel Moltrasio <[email protected]> Signed-off-by: Melissa Kilby <[email protected]>

incertum · 2023-07-07T04:12:30Z

@Molter73 kindly asking if you have additional change requests for v1 or if we can proceed? Thanks!

incertum · 2023-07-11T15:03:56Z

Hi everyone, kind bump, thank you!

Molter73

/lgtm

Sorry for the delay on this, LGTM!

poiana · 2023-07-11T15:33:17Z

LGTM label has been added.

Git tree hash: 45f77f49c9b008e201597b72bfc6bc55a2bcc778

poiana · 2023-07-13T07:08:19Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: incertum, jasondellaluce, Molter73

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Molter73,incertum,jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

FedeDP · 2023-07-28T12:59:23Z

/milestone 5.1.0+driver

poiana added kind/documentation Improvements or additions to documentation release-note-none do-not-merge/work-in-progress dco-signoff: yes area/tests labels Aug 2, 2022

poiana requested review from Andreagit97 and leogr August 2, 2022 05:41

poiana added the size/XXL label Aug 2, 2022

incertum mentioned this pull request Aug 3, 2022

[Tracking] CI Integration for Driver Kernel Version Testing #531

Closed

11 tasks

incertum force-pushed the test-driver-sanity-test-suites branch from dc03820 to ede9fee Compare August 8, 2022 03:13

incertum force-pushed the test-driver-sanity-test-suites branch from ede9fee to ca88e0b Compare November 5, 2022 01:40

poiana added release-note and removed release-note-none labels Nov 5, 2022

incertum changed the title ~~[WIP] new(test): add driver_sanity_test_suites~~ new(test): add driver-sanity-tests Nov 5, 2022

poiana removed the do-not-merge/work-in-progress label Nov 5, 2022

incertum force-pushed the test-driver-sanity-test-suites branch from ca88e0b to 598755c Compare November 16, 2022 23:10

poiana added this to the 0.11.0 milestone Dec 2, 2022

poiana added the lifecycle/stale label Mar 2, 2023

poiana removed the lifecycle/stale label Mar 2, 2023

incertum mentioned this pull request Mar 15, 2023

[Tracking] localhost test/vm test suite - occasionally capture results tables #982

Closed

incertum force-pushed the test-driver-sanity-test-suites branch from 598755c to a184d1f Compare March 15, 2023 06:34

incertum and others added 2 commits June 29, 2023 15:18

fix(test/vm): replace el7.elrepo mirror

b80a60c

and highlight basic system dependencies Co-authored-by: Massimiliano Giovagnoli <[email protected]> Signed-off-by: Melissa Kilby <[email protected]>

cleanup(test/vm): add dependency checks

8cf5c97

Co-authored-by: Massimiliano Giovagnoli <[email protected]> Signed-off-by: Melissa Kilby <[email protected]>

incertum force-pushed the test-driver-sanity-test-suites branch from e7d1fcf to 2d29259 Compare June 29, 2023 15:19

jasondellaluce previously approved these changes Jun 30, 2023

View reviewed changes

poiana assigned jasondellaluce Jun 30, 2023

poiana added the lgtm label Jun 30, 2023

Molter73 reviewed Jul 3, 2023

View reviewed changes

cleanup(test/vm): add info to instructions

3d96343

Signed-off-by: Melissa Kilby <[email protected]>

incertum dismissed jasondellaluce’s stale review via 3ff9ece July 3, 2023 18:15

incertum force-pushed the test-driver-sanity-test-suites branch from 2d29259 to 3ff9ece Compare July 3, 2023 18:15

poiana removed the lgtm label Jul 3, 2023

poiana requested a review from jasondellaluce July 3, 2023 18:15

cleanup(test/vm): address reviewers comments

11f5fc7

minor cleanup setting expectations for duration of tests Co-authored-by: Mauro Ezequiel Moltrasio <[email protected]> Signed-off-by: Melissa Kilby <[email protected]>

incertum force-pushed the test-driver-sanity-test-suites branch from 3ff9ece to 11f5fc7 Compare July 3, 2023 18:25

leogr mentioned this pull request Jul 5, 2023

Falco Graduation Path: Progress Tracker falcosecurity/evolution#281

Closed

Molter73 approved these changes Jul 11, 2023

View reviewed changes

poiana assigned Molter73 Jul 11, 2023

poiana added the lgtm label Jul 11, 2023

jasondellaluce approved these changes Jul 13, 2023

View reviewed changes

poiana merged commit 121d918 into falcosecurity:master Jul 13, 2023
17 checks passed

poiana modified the milestones: driver-backlog, 5.1.0+driver Jul 28, 2023

incertum deleted the test-driver-sanity-test-suites branch December 8, 2023 20:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new(test): add `test/vm` for localhost VM-based driver kernel compatibility tests #524

new(test): add `test/vm` for localhost VM-based driver kernel compatibility tests #524

incertum commented Aug 2, 2022 •

edited

Loading

FedeDP commented Aug 2, 2022

jasondellaluce commented Aug 2, 2022

incertum commented Aug 3, 2022

incertum commented Nov 5, 2022

FedeDP commented Dec 2, 2022

poiana commented Mar 2, 2023

incertum commented Mar 2, 2023

poiana commented Jun 30, 2023

Molter73 left a comment

incertum commented Jul 3, 2023

incertum commented Jul 7, 2023

incertum commented Jul 11, 2023

Molter73 left a comment

poiana commented Jul 11, 2023

poiana commented Jul 13, 2023

FedeDP commented Jul 28, 2023

new(test): add test/vm for localhost VM-based driver kernel compatibility tests #524

new(test): add test/vm for localhost VM-based driver kernel compatibility tests #524

Conversation

incertum commented Aug 2, 2022 • edited Loading

FedeDP commented Aug 2, 2022

jasondellaluce commented Aug 2, 2022

incertum commented Aug 3, 2022

incertum commented Nov 5, 2022

FedeDP commented Dec 2, 2022

poiana commented Mar 2, 2023

incertum commented Mar 2, 2023

poiana commented Jun 30, 2023

Molter73 left a comment

Choose a reason for hiding this comment

incertum commented Jul 3, 2023

incertum commented Jul 7, 2023

incertum commented Jul 11, 2023

Molter73 left a comment

Choose a reason for hiding this comment

poiana commented Jul 11, 2023

poiana commented Jul 13, 2023

FedeDP commented Jul 28, 2023

new(test): add `test/vm` for localhost VM-based driver kernel compatibility tests #524

new(test): add `test/vm` for localhost VM-based driver kernel compatibility tests #524

incertum commented Aug 2, 2022 •

edited

Loading