datadog-ci synthetics run-tests --failOnMissingTests does not fail on missing tests

[PSanetra]

Bug description

Specifying --failOnMissingTests on datadog-ci synthetics run-tests should result in a failed CI pipeline, but even if there are no tests found, the pipeline succeeds. 😕

Describe what you expected

datadog-ci synthetics run-tests should return a non 0 exit code if --failOnMissingTests and no tests were executed.

Steps to reproduce the issue

The following command was executed in a GitLab CI Job:

$ datadog-ci synthetics run-tests --failOnMissingTests --datadogSite datadoghq.eu --apiKey "$DATADOG_SYNTHETICS_API_KEY" --appKey "$DATADOG_SYNTHETICS_APP_KEY" -s "non_existing_tag:non_existing_value env:dev"
No test to run.
00:01
Job succeeded

Additional context

Docker Image: datadog/ci:v2.23.0

Command

synthetics

[Drarig29]

Hi @PSanetra!

Thanks for reporting! When we added this flag, we didn't want to make side effects on Datadog (e.g. a deleted test) turn a lot of CI pipelines red, because of tests that are not explicitly listed in your repository.

But we can hear that referencing tests via explicit public IDs or via a -s query should work the same.

We created a card to tackle this 🙇

[Drarig29]

We forgot to close this, this was released in v2.25.0.