hooks for tests

[drewbanin]

Tests should begin/end with on-run-start and on-run-end

[norton120]

I'm trying to think of a schema test that would be sensitive to environmental variables.. I can't.

[drewbanin]

thanks @norton120 -- updated the description accordingly

[danielerapati]

pre-hooks or post-hooks could be really useful for data tests: often to fix a failing test it is not enough to know that it affects a certain number of rows, you also have to manually lookup which rows it affects.
For that a post-hook like SELECT <failing rows> INTO <errors table> FROM {{ref(<model>) }} WHERE <test condition> could be useful.

I believe this would be a change to the TestRunner somehow adding in the hooks capability from ModelRunner: looks like a big change for a newcomer like myself, any useful tips or things I should really know before trying? e.g. Why was class TestRunner(ModelRunner) discarded?

[drewbanin]

Hey @danielerapati - this is a great idea. I think that this would be an incredible feature, and I'd like to integrate it deeper into dbt than just as a post-hook. Presently, post-hooks are kind of a catch-all for things like grants, warehouse operations, etc. This feature is core enough to testing that it would make sense to just build the functionality right into dbt.

I made an issue for this over here: #903

Feel free to comment on that with any further thoughts! I'm also happy to discuss potential implementations in that thread :)

[drewbanin]

I'm going to close this, as hooks are intended for statements like grants, vacuums, etc. There's some merit to inserting audit records, but I imagine there's a better solution to that than inserting one record at a time... I'm really happy we were able to pull #903 out of this, as I think that will be a really great feature to add some day.

Happy to discuss if anyone feels inclined to re-open it!

[trevor-petach]

@drewbanin I know there has been a lot of discussion around audit logs for tests and the recent PR #3316 by @jtcohen6. However, 4 years have passed since this issue, and there is still no way to write an audit log of which tests were executed when and what the results were.

Is there any reason not to have pre- and post-hooks for tests?

[erenelci]

This is still highly needed, especially when tests require access to views that are only available through a secondary role, i.e. if "USE SECONDARY ROLES ALL;" is required as pre_hook.

[jtcohen6]

@erenelci Someone just opened a related issue over in dbt-snowflake: dbt-labs/dbt-snowflake#449