Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add task owner to dbt operator #1082

Merged
merged 14 commits into from
Jul 23, 2024
Merged

Add task owner to dbt operator #1082

merged 14 commits into from
Jul 23, 2024

Conversation

wornjs
Copy link
Contributor

@wornjs wornjs commented Jul 7, 2024

Set dbt meta owner as task owner in all Cosmos DbtxxxOperators.

Closes: #1065

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jul 7, 2024
Copy link

netlify bot commented Jul 7, 2024

Deploy Preview for sunny-pastelito-5ecb04 canceled.

Name Link
🔨 Latest commit bb4e334
🔍 Latest deploy log https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/669ee62e4f7922000821cd6a

@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Jul 7, 2024
@wornjs
Copy link
Contributor Author

wornjs commented Jul 7, 2024

here is the result in airflow web ui
스크린샷 2024-07-07 오후 11 18 59

스크린샷 2024-07-07 오후 11 19 24

@wornjs wornjs marked this pull request as draft July 8, 2024 03:22
Copy link
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking great, @wornjs !

I left a minor comment (sorry for doing this while the PR is in draft) - we can finish reviewing once the checks are passing and the PR is ready for review.

cosmos/dbt/graph.py Outdated Show resolved Hide resolved
@wornjs wornjs marked this pull request as ready for review July 14, 2024 11:06
Copy link

codecov bot commented Jul 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.48%. Comparing base (e61f3a3) to head (f008ab9).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1082   +/-   ##
=======================================
  Coverage   96.47%   96.48%           
=======================================
  Files          64       64           
  Lines        3288     3297    +9     
=======================================
+ Hits         3172     3181    +9     
  Misses        116      116           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@wornjs
Copy link
Contributor Author

wornjs commented Jul 17, 2024

@tatiana
plz check this

@tatiana tatiana changed the title add task owner to dbt operator Add task owner to dbt operator Jul 17, 2024
cosmos/airflow/graph.py Outdated Show resolved Hide resolved
@tatiana tatiana added this to the Cosmos 1.6.0 milestone Jul 17, 2024
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jul 23, 2024
@tatiana tatiana merged commit d383792 into astronomer:main Jul 23, 2024
62 checks passed
@wornjs wornjs deleted the issue/1605 branch July 30, 2024 02:35
@pankajkoti pankajkoti mentioned this pull request Jul 31, 2024
dwreeves pushed a commit to dwreeves/astronomer-cosmos that referenced this pull request Jul 31, 2024
Set dbt meta owner as task owner in all Cosmos `DbtxxxOperator`s.

Closes: astronomer#1065

---------
Co-authored-by: Tatiana Al-Chueyr <[email protected]>
pankajkoti added a commit that referenced this pull request Aug 20, 2024
New Features

* Add support for loading manifest from cloud stores using Airflow
Object Storage by @pankajkoti in #1109
* Cache ``package-lock.yml`` file by @pankajastro in #1086
* Support persisting the ``LoadMode.VIRTUALENV`` directory by @tatiana
in #1079
* Add support to store and fetch ``dbt ls`` cache in remote stores by
@pankajkoti in #1147
* Add default source nodes rendering by @arojasb3 in #1107
* Add Teradata ``ProfileMapping`` by @sc250072 in #1077

Enhancements

* Add ``DatabricksOauthProfileMapping`` profile by @CorsettiS in #1091
* Use ``dbt ls`` as the default parser when ``profile_config`` is
provided by @pankajastro in #1101
* Add task owner to dbt operators by @wornjs in #1082
* Extend Cosmos custom selector to support + when using paths and tags
by @mvictoria in #1150
* Simplify logging by @dwreeves in #1108

Bug fixes

* Fix Teradata ``ProfileMapping`` target invalid issue by @sc250072 in
#1088
* Fix empty tag in case of custom parser by @pankajastro in #1100
* Fix ``dbt deps`` of ``LoadMode.DBT_LS`` should use
``ProjectConfig.dbt_vars`` by @tatiana in #1114
* Fix import handling by lazy loading hooks introduced in PR #1109 by
@dwreeves in #1132
* Fix Airflow 2.10 regression and add Airflow 2.10 in test matrix by
@pankajastro in #1162

Docs

* Fix typo in azure-container-instance docs by @pankajastro in #1106
* Use Airflow trademark as it has been registered by @pankajastro in
#1105

Others

* Run some example DAGs in Kubernetes execution mode in CI by
@pankajastro in #1127
* Install requirements.txt by default during dev env spin up by
@@CorsettiS in #1099
* Remove ``DbtGraph.current_version`` dead code by @tatiana in #1111
* Disable test for Airflow-2.5 and Python-3.11 combination in CI by
@pankajastro in #1124
* Pre-commit hook updates in #1074, #1113, #1125, #1144, #1154,  #1167

---------

Co-authored-by: Pankaj Koti <[email protected]>
Co-authored-by: Pankaj Singh <[email protected]>
tatiana pushed a commit that referenced this pull request Sep 11, 2024
`dag.owner` is a computed property that joins owners of existing tasks.
We should rely on airflow's existing owner fallback in
[airflow.models.baseoperator.BaseOperator](https:/apache/airflow/blob/9a5f27f6dc560dcf456cabb793a32a61eab98dea/airflow/models/baseoperator.py#L865),
as it was the case before #1082.

Fixes #1194
ags-de pushed a commit to ags-de/astronomer-cosmos that referenced this pull request Sep 24, 2024
`dag.owner` is a computed property that joins owners of existing tasks.
We should rely on airflow's existing owner fallback in
[airflow.models.baseoperator.BaseOperator](https:/apache/airflow/blob/9a5f27f6dc560dcf456cabb793a32a61eab98dea/airflow/models/baseoperator.py#L865),
as it was the case before astronomer#1082.

Fixes astronomer#1194
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm This PR has been approved by a maintainer size:S This PR changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] add task owner to dbt operator
2 participants