Tests for disabled/non-existent models fail silently

[drewbanin]

Issue

Issue description

When running dbt test, tests which reference non-existent or disabled models will not run, and there is no associated info or debug message indicating why.

We made dbt skip these tests quietly on purpose here. This is to avoid error messages if you disable a model but still have tests for that model. This happens somewhat frequently with models inside of dbt packages.

Results

A custom data test which refs a nonexistent or disabled model does not run, and no warning or error is emitted.

System information

this bug is present in dbt at the time of the 0.11.0 release

Steps to reproduce

-- tests/some_test.sql

select * from {{ ref('this_model_does_not_exist') }}

$ dbt test

[elexisvenator]

Most test suites has the concept of a skipped/inconclusive test, with an option to treat skipped as errors. I think it would make sense to do the same thing here.

[drewbanin]

Good idea @elexisvenator.

Another place this can happen is in relationship tests with non-existent refs used in foreign keys

[drewbanin]

Update: schema tests for invalid models also fail silently. We should just add a warning:

WARNING: Tests exist for the model `{model name}` which was not found (or disabled)

[Cskyleryoung]

Is there a way to make these fail silently again? A manual flag or other config? We are running tons of DBT packages in Dagster, which is polling for tests actively like every 30 seconds while running, and it's clogging our logs up with DBT warnings. We have a large number of global tests where we explicitly don't expect all models to be present at all times. We're just verifying columns and a few other tests for tables that do exist.