twister: Add skip as error feature #33874
Assignees
Labels
Enhancement
Changes/Updates/Additions to existing features
Comments
|
@galak @nashif @carlescufi You might be interested in this. I will create a corresponding PR soon. |
PerMac
added a commit
to PerMac/zephyr
that referenced
this issue
Apr 8, 2021
All skips for platforms listed in the .yaml's integration_platforms
will be treated as errors in the integration mode ('-G/--integration').
This feature serves to guarantee a proper testing scope in CI as no
integration platform will be skipped silently.
Fixes zephyrproject-rtos#33874
Signed-off-by: Maciej Perkowski <[email protected]>
carlescufi
pushed a commit
that referenced
this issue
Apr 22, 2021
All skips for platforms listed in the .yaml's integration_platforms
will be treated as errors in the integration mode ('-G/--integration').
This feature serves to guarantee a proper testing scope in CI as no
integration platform will be skipped silently.
Fixes #33874
Signed-off-by: Maciej Perkowski <[email protected]>
jori-nordic
pushed a commit
to jori-nordic/zephyr
that referenced
this issue
May 4, 2021
...integration mode
All skips for platforms listed in the .yaml's integration_platforms
will be treated as errors in the integration mode ('-G/--integration').
This feature serves to guarantee a proper testing scope in CI as no
integration platform will be skipped silently.
Fixes zephyrproject-rtos#33874
Signed-off-by: Maciej Perkowski <[email protected]>
(cherry picked from commit d08ee92)
Signed-off-by: Trond Einar Snekvik <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your enhancement proposal related to a problem? Please describe.
We want to be sure that the tests which have the
integration_platformslist and are executed in the integration mode are not silently skipped for any reason on platforms in the integration list. (Quarantine will be the only accepted skip when this feature is ready).Describe the solution you'd like
If the integration mode is on and a given platform is on the
integration_platformslist every skip attempt will be treated as an errorDescribe alternatives you've considered
Some convoluted way of verifying the reports looking for skips on integration platforms can be think of.
Additional context
This feature is needed for sdk-nrf CI where we want to be sure that each sample is tested on supported platforms and not skipped for any reason. However, I think that this option can become a default also in zephyr's CI. This way we will make sure that the proper coverage is maintained.
The text was updated successfully, but these errors were encountered: