-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose more structured logging capabilities to {{ log() }}
#4184
Comments
I love the idea of exposing some of this new functionality via our jinja interface. I do want to ask if some of the common use cases for |
The context available to users is the same as that available to dbt-core's built-in materializations today. It's a good callout, we could think about supporting other types of logging/event functions in the Jinja context, or optional arguments to From my vantage, I was thinking that the simplest way would be to add an argument to @dataclass
class MacroEventInfo(InfoLevel, CliEventABC):
msg: str
properties: dict
def cli_msg(self) -> str:
return self.msg dict = {thing_a: x, thing_b: y}
message = "Could not do {{ thing_a }} because {{ thing_b }}"
{{ log(message, info = true, properties = dict) }} To achieve the equivalent of: class ThisSpecificCustomEvent(...):
thing_a: str
thing_b: str
def cli_msg(self) -> str:
return(
f"Could not {self.thing_a} because {self.thing_b}"
) And be able to use the additional structure of |
Yeah I can see how that would work, and it seems consistent to use Jinja here as opposed to python f-string syntax for instance. |
Another version of this that we discussed, following the same pseudo-code above: {% set thing_a = ... %}
{% set thing_b = ... %}
{{ fire_event(event_type='MyCoolEvent', thing_a=thing_a, thing_b=thing_b }} Wherein That could be especially useful for events that are (a) consistent, frequent, expected, and (b) firmly in Jinja-land. Materializations are the prime example. |
as someone who spends way too much time debugging jinja, I'd like to "yes, and..." this idea by saying it'd be great to have to log message return:
|
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days. |
Closing in favor of #5325. Since this is going to be a part of that effort. |
Picking up from #4164 (comment). This is a fairly half-baked idea, and I'm definitely open to suggestions on how to structure/implement.
By my unscientific estimate, the most critical logging is:
{{ log() }}
Why so much in the last category? This could be logging from:
log()
Based on the implementation of structured logging that will be included in v1, these log messages will be passed along as relatively unstructured events,
MacroEventInfo
orMacroEventDebug
, with just one string property,msg
.As structured logs become an increasingly important interface for interacting with / listening to dbt, we should seek to open up more capabilities to end users who want to hook into more powerful logging from the macro/materialization context. I could see us doing that by:
log()
macro that enables users to pass along any key-value pairs they want, and a templated message (Jinja expression or f-string) using those keys/valueslog()
messageThe text was updated successfully, but these errors were encountered: