Add "test" log level

[nathaniel-may]

Describe the feature

There are a lot of tests that test for the presence of specific lines in debug-level logs that really don't need to be part of the user-facing debug-level output. This pattern is by far the easiest method of testing many of the difficult-to-test deep internals of core, so it is expected that the debug log level will become even more diluted for users.

See the static parser for an example.

To fix this, we should introduce a new log-level test (open to alternative names like dev, automation, correctness, testing, tests, integration or whatever) that we can use use in our integration tests. This keeps the debug level free from clutter for users, and still allows core contributors and developers to add new tests with the same pattern we do today.

Describe alternatives you've considered

• Not doing this. Maybe debug isn't used by users very often and it's ok to keep cluttering it. It doesn't feel very professional, but it's certainly an option.
• Refactoring code to test it a more conventional way. This would take far too long to do all at once and may significantly slow down feature development if we do it for every feature. Some features may be easy to do this and we should when we can, but it's not going to be easy for much of the code base.

Who will this benefit?

dbt users who use the debug log level to debug their dbt runs. The dbtLabs support department who may ask customers to run dbt with the debug flag and interpret the results.

[jtcohen6]

Agree with both goals: it should be easy to test dbt-core, and easy to find useful information in debug-level logs to see what dbt is doing under the hood. We regularly need to use debug logs for (what else) debugging issues that crop up in community members' projects.

One complication: logbook (our current logging library) doesn't support custom log levels. Instead, it supports custom log processors, the idea being that we tag these log messages as "just for testing," and then filter them out depending on the environment in which dbt-core is run.

I also know we've talked about moving away from logbook, e.g. #3359 (comment)

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.