-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Investigate performance of anonymous usage statistics requests #2008
Comments
hey @sairam - check out the docs for disabling anonymous usage statistics here: https://docs.getdbt.com/docs/configure-your-profile#section-usage-statistics I'm surprised to see this is ballooning your dbt runtime - we coded this such that a single failed event request would prevent future requests from being made. Let me take a deeper look here - the behavior you're seeing is not right. |
Thanks for the link, I initially searched for telemetry and analytics, but could not find the right config.
@drewbanin , feel free to close this issue. |
I wanted to share that this was run with 1 thread. Looks like a request to a remote server is happening after each view/action is done which explains close to 100second delay. Changing the location to edge endpoint may help instead of us-east-1. |
Ok - that's not right - 1 second is far, far too long for these individual requests. I'll leave this issue open to investigate adding edge endpoints, batching the requests, or making the requests async. They actually used to be async but we had issues with the thread not being cleaned up and it causing dbt to hang. Thanks again for bringing this one to our attention. |
would it possible to have a dedicated thread just for sending out the analytics? |
Yep, that's totally possible. Alternatively, we could try batching these events and sending more of them at once. The payloads are pretty tiny (timestamps, the command that was run, etc) - the really slow thing here is just the round-trip to us-east-1 I think. |
Fixed in #2089 |
Describe the bug
Telemetry data is being sent to
fishtownanalytics.sinter-collect.com
ondbt run
. The speed ofdbt run
improved to 17 sec from 111sec (~2min) for about when the domain was blackholed (pointing to 127.0.0.1)Expected behavior
When disabled telemetry:
When enabled telemetry:
Each request to the telemetry server is taking about 1second. For 100 view models, its adding additional 100 seconds or 1 second per run.
The output of
dbt --version
:Is there a way to disable telemetry via commandline or via env variable?
The text was updated successfully, but these errors were encountered: