Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle Global configuration properties #134

Closed
potiuk opened this issue May 4, 2019 · 0 comments
Closed

Handle Global configuration properties #134

potiuk opened this issue May 4, 2019 · 0 comments
Labels
enhancement New feature or request nice-to-have Nice to have - not needed for the initial implementation

Comments

@potiuk
Copy link
Collaborator

potiuk commented May 4, 2019

It is related to #39 but it's about making sure that global configuration properties are passed to all tasks inside generated dags.

Most actions in a workflow need several of the same settings (e.g., the JobTracker URI, NameNode URI, Hadoop scheduler queue, etc.). Users can avoid repeating all these settings and job configuration values in every action of the workflow by defining them once in the workflow’s section. The properties defined in the section are available to all actions of the workflow.

Seems that might be probably somehow mapped to default_args in most cases, but then we have to make sure we map all the configuration properties from #39 to some arg:

@potiuk potiuk added the enhancement New feature or request label May 4, 2019
@sprzedwojski sprzedwojski added this to the Delta milestone May 17, 2019
@potiuk potiuk added the nice-to-have Nice to have - not needed for the initial implementation label Jun 28, 2019
@potiuk potiuk removed this from the Delta (nice-to-have time permitting) milestone Jul 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request nice-to-have Nice to have - not needed for the initial implementation
Projects
None yet
Development

No branches or pull requests

3 participants