-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AIP-31] Add dependency relationship on operator when templated_field is XComArg #8054
Labels
Comments
turbaszek
added a commit
to databand-ai/airflow
that referenced
this issue
May 5, 2020
6 tasks
turbaszek
added a commit
to databand-ai/airflow
that referenced
this issue
May 10, 2020
6 tasks
turbaszek
added a commit
to databand-ai/airflow
that referenced
this issue
Jun 9, 2020
turbaszek
added a commit
that referenced
this issue
Jun 15, 2020
* Resolve upstream tasks when template field is XComArg closes: #8054 * fixup! Resolve upstream tasks when template field is XComArg * Resolve task relations in DagRun and DagBag * Add tests for serialized DAG * Set dependencies only in bag_dag, refactor tests * Traverse template_fields attribute * Use provide_test_dag_bag in all tests * fixup! Use provide_test_dag_bag in all tests * Use metaclass + setattr * Add prepare_for_execution method * Check signature of __init__ not class * Apply suggestions from code review Co-authored-by: Ash Berlin-Taylor <[email protected]> * Update airflow/models/baseoperator.py Co-authored-by: Ash Berlin-Taylor <[email protected]>
kaxil
pushed a commit
to kaxil/airflow
that referenced
this issue
Jun 27, 2020
* Resolve upstream tasks when template field is XComArg closes: apache#8054 * fixup! Resolve upstream tasks when template field is XComArg * Resolve task relations in DagRun and DagBag * Add tests for serialized DAG * Set dependencies only in bag_dag, refactor tests * Traverse template_fields attribute * Use provide_test_dag_bag in all tests * fixup! Use provide_test_dag_bag in all tests * Use metaclass + setattr * Add prepare_for_execution method * Check signature of __init__ not class * Apply suggestions from code review Co-authored-by: Ash Berlin-Taylor <[email protected]> * Update airflow/models/baseoperator.py Co-authored-by: Ash Berlin-Taylor <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
XComArg allow us to set explicit XCom messages between operators. Adding a dependency between Operators is needed as it does not make sense to execute an operator that requires an XCom from another operator.
Use case / motivation
As a DAG Author, I want Airflow to automatically detect that an operator that I initialize using an XComArg is set as a downstream dependency from the XCom origin operator.
Related Issues
Blocked by: #8052
The text was updated successfully, but these errors were encountered: