Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proposal: Pipelines in Pipelines Custom Tasks #330

Closed
jerop opened this issue Feb 1, 2021 · 7 comments
Closed

Proposal: Pipelines in Pipelines Custom Tasks #330

jerop opened this issue Feb 1, 2021 · 7 comments
Labels
lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness.

Comments

@jerop
Copy link
Member

jerop commented Feb 1, 2021 •
edited
Loading

Proposal for creating tektoncd/experimental/pipelines-in-pipelines to incubate Pipelines in Pipelines project

Per process, this project was proposed with verbal support at the Tekton API WG on Feb 1st 2021

The problem the project will solve

The project would enable users to specify in Pipelines how to create and run PipelineRuns, similarly to TaskRuns, as further described in this issue.

One use case is providing a set of Tasks as a complete unit of execution, as demonstrated by an example @madbence shared in tektoncd/pipeline#2134 (comment):

"This feature would be really useful for us. eg. right now we have a task that can set the status on a given commit, and a task that can execute unit tests (eg. run npm test, etc.). We'd like to distribute a task/pipeline that can combine these (eg. set the commit to pending, run unit tests, set commit status to success/failure)".

image

"We'd like to provide to option to use set-status and run-unit-tests in a pipeline, and to be able to use a "higher-level" task, like run-unit-tests-with-commit-status".

Other use cases are discussed in doc and will be explored further in a TEP.

Prototype: https:/jerop/experimental/tree/piprun/pipelines-in-pipelines
Issue: tektoncd/pipeline#2134

Who will own it

/cc @abayer @vdemeester @bobcatfish @afrittoli @imjasonh (Governing Board)
@afrittoli
Copy link
Member

Thanks you for this proposal, I think this is a really good idea, it helps a lot breaking down pipelines and boosts the pipeline reusability.

The idea of a "unit of execution" in the use case above is inline with pipeline reusability.
In the diagram above, we could use the pipeline finally capability to ensure the last update is always executed.

@bobcatfish bobcatfish removed their assignment Feb 2, 2021
@vdemeester
Copy link
Member

Thanks for this proposal @jerop. I 💯 % vote for experimenting on this, thus I approve 😛

@imjasonh
Copy link
Member

imjasonh commented Feb 5, 2021

I approve of this effort. 👍

In fact I'm really excited about it! 🎉

@jerop jerop mentioned this issue Feb 16, 2021
Closed
jerop added a commit to jerop/experimental that referenced this issue Feb 22, 2021
@jerop
Pipelines In Pipelines Custom Tasks
80c2058 
This is an experimental project to provide support for `Pipelines`
in `Pipelines`. The functionality is provided by a controller that
implements the `Custom Task` interface. Its use cases include
expressing a group of `Tasks` as a unit of execution to improve
composability and composability of [Tekton Pipelines](https:/tektoncd/pipeline).

Experimental project was proposed in tektoncd/community#330
jerop added a commit to jerop/experimental that referenced this issue Feb 22, 2021
@jerop
Pipelines In Pipelines Custom Tasks
54fc0fc 
This is an experimental project to provide support for `Pipelines`
in `Pipelines`. The functionality is provided by a controller that
implements the `Custom Task` interface. Its use cases include
expressing a group of `Tasks` as a unit of execution to improve
composability and composability of [Tekton Pipelines](https:/tektoncd/pipeline).

Experimental project was proposed in tektoncd/community#330
@jerop jerop mentioned this issue Feb 22, 2021
Merged
3 tasks
jerop added a commit to jerop/experimental that referenced this issue Feb 25, 2021
@jerop
Pipelines In Pipelines
d396209 
This is an experimental project to provide support for `Pipelines` in
`Pipelines` to improve the composability and reusability of
[Tekton Pipelines](https:/tektoncd/pipeline).

The functionality is provided by a controller
that implements the `Custom Task` interface.

Its use cases include enabling defining and executing a set of `Tasks`
as a complete unit of execution and decoupling `Pipeline` branches
failures such that a `Task` failure in one branch does not stop
execution of unrelated `Pipeline` branches.

Experimental project is proposed in tektoncd/community#330
Issue is further described in tektoncd/pipeline#2134
jerop added a commit to jerop/experimental that referenced this issue Feb 26, 2021
@jerop
Pipelines In Pipelines
55e8725 
This is an experimental project to provide support for `Pipelines` in
`Pipelines` to improve the composability and reusability of
[Tekton Pipelines](https:/tektoncd/pipeline).

The functionality is provided by a controller
that implements the `Custom Task` interface.

Its use cases include enabling defining and executing a set of `Tasks`
as a complete unit of execution and decoupling `Pipeline` branches
failures such that a `Task` failure in one branch does not stop
execution of unrelated `Pipeline` branches.

Experimental project is proposed in tektoncd/community#330
Issue is further described in tektoncd/pipeline#2134
jerop added a commit to jerop/experimental that referenced this issue Feb 26, 2021
@jerop
Pipelines In Pipelines
a23d906 
This is an experimental project to provide support for `Pipelines` in
`Pipelines` to improve the composability and reusability of
[Tekton Pipelines](https:/tektoncd/pipeline).

The functionality is provided by a controller
that implements the `Custom Task` interface.

Its use cases include enabling defining and executing a set of `Tasks`
as a complete unit of execution and decoupling `Pipeline` branches
failures such that a `Task` failure in one branch does not stop
execution of unrelated `Pipeline` branches.

Experimental project is proposed in tektoncd/community#330
Issue is further described in tektoncd/pipeline#2134
tekton-robot pushed a commit to tektoncd/experimental that referenced this issue Feb 26, 2021
@jerop @tekton-robot
Pipelines In Pipelines
e5f5078 
This is an experimental project to provide support for `Pipelines` in
`Pipelines` to improve the composability and reusability of
[Tekton Pipelines](https:/tektoncd/pipeline).

The functionality is provided by a controller
that implements the `Custom Task` interface.

Its use cases include enabling defining and executing a set of `Tasks`
as a complete unit of execution and decoupling `Pipeline` branches
failures such that a `Task` failure in one branch does not stop
execution of unrelated `Pipeline` branches.

Experimental project is proposed in tektoncd/community#330
Issue is further described in tektoncd/pipeline#2134
@jerop jerop mentioned this issue Mar 8, 2021
Merged
@tekton-robot
Copy link
Contributor

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

@tekton-robot tekton-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 26, 2021
@jerop
Copy link
Member Author

jerop commented May 27, 2021

/lifecycle frozen

actively working on this project as described in TEP-0056: Pipelines in Pipelines

@tekton-robot tekton-robot added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels May 27, 2021
@bobcatfish
Copy link
Contributor

@jerop i feel like since we have the experimental project we could probably close this specific issue? or did you want to keep it open for tracking?

@jerop
Copy link
Member Author

jerop commented May 27, 2021

sure, we can close it - can track the ongoing work in the experimental repo

@jerop jerop closed this as completed May 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@vdemeester @imjasonh @bobcatfish @afrittoli @jerop @tekton-robot