Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TEP-0090: Include Matrix in Validation of Parameters in TaskRun Reconciler #4844

Merged
merged 1 commit into from
May 11, 2022

Conversation

jerop
Copy link
Member

@jerop jerop commented May 6, 2022

Changes

The TaskRun reconciler validates that the needed Parameters are provided,
no extra Parameters are provide and the types of Parameters are matching.

Prior to this change, the reconciler did the above validation considering the
params field in the PipelineTask only. In this change, we include matrix
field into the validation routine described above. This includes validating
that Parameters in the Matrix are substituting Parameters of type String
in the underlying Task.

Related:

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • Docs included if any changes are user facing
  • Tests included if any functionality added or changed
  • Follows the commit message standard
  • Meets the Tekton contributor standards (including functionality, content, code)
  • Release notes block below has been filled in (if there are no user facing changes, use release note "NONE")

Release Notes

NONE

@tekton-robot tekton-robot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label May 6, 2022
@tekton-robot tekton-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label May 6, 2022
@jerop jerop force-pushed the tep-0090-validate-params branch from 03c3c50 to cd72aed Compare May 6, 2022 20:43
@jerop jerop marked this pull request as draft May 6, 2022 20:49
@jerop jerop force-pushed the tep-0090-validate-params branch 2 times, most recently from c7738d9 to f0d1734 Compare May 6, 2022 21:08
@tekton-robot tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 6, 2022
@jerop jerop marked this pull request as ready for review May 6, 2022 21:08
@tekton-robot tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 6, 2022
@@ -141,12 +141,26 @@ func wrongTypeParamsNames(params []v1beta1.Param, neededParamsTypes map[string]v
wrongTypeParamNames = append(wrongTypeParamNames, param.Name)
}
}
for _, param := range matrix {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels a bit strange to have code in reconciler/taskrun validating that matrix params aren't array params-- ideally the taskrun reconciler shouldn't have knowledge of matrix at all. could this validation be done at the pipelinerun level, and treat the matrix params as normal params by the time they get to the taskrun level? Or is it necessary to ensure that params passed via the matrix aren't declared as array params in the task?

My mental model of how this would work (based on tep 90) is that the params are declared as arrays in the pipeline, passed as matrix params in the pipeline task, and then passed from the pipeline task to the taskrun as a list of string params. If the params were declared as array params in the task, the taskrun reconciler would recognize that the string param passed didn't match the array param declared, and fail.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added that check out of an abundance of caution, but it's not necessary as that's already covered in the pipelinerun reconciler - #4704 - so I've removed it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than pass in matrix params separately, any reason to not just include them in the existing slice of params to validate?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... a bit confused since the above comment mentions that the check for param types was removed but I still see it below. Do we still need to resolve that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than pass in matrix params separately, any reason to not just include them in the existing slice of params to validate?

@lbernick - The existing slice of params is checking if the type of params provided in the pipeline matches the type of params in taskspec (string to string, and array to array). However, we want the type of params provided in the matrix to be array and the type of params they are substituting in the taskspec to be string. Because the types do not match for matrix params, we can't pass it to the existing validation and that's why we have its own validation. Does that clarify?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... a bit confused since the above comment mentions that the check for param types was removed but I still see it below. Do we still need to resolve that?

@dibyom - Removed the validation that checks that the param in a matrix is of type array - this is already handled in the pipelinerun reconciler which was implemented in #4704. We still need to validate that the param that's being substituted by the param in a matrix is of type string in the taskrun reconciler - this is the type checking that we still have here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it thanks Jerop!

pkg/reconciler/pipelinerun/pipelinerun.go Show resolved Hide resolved
@jerop jerop force-pushed the tep-0090-validate-params branch from f0d1734 to abc96f5 Compare May 9, 2022 14:00
@jerop
Copy link
Member Author

jerop commented May 9, 2022

/retest

@jerop jerop force-pushed the tep-0090-validate-params branch from abc96f5 to 982324c Compare May 9, 2022 15:06
@jerop
Copy link
Member Author

jerop commented May 9, 2022

/retest

@jerop
Copy link
Member Author

jerop commented May 9, 2022

/test pull-tekton-pipeline-go-coverage

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/validate_resources.go 96.0% 94.3% -1.7

… Reconciler

The `TaskRun` reconciler validates that the needed `Parameters` are provided,
no extra `Parameters` are provide and the types of `Parameters` are matching.

Prior to this change, the reconciler did the above validation considering the
`params` field in the `PipelineTask` only. In this change, we include `matrix`
field into the validation routine described above. This includes validating
that `Parameters` in the `Matrix` are substituting `Parameters` of type `String`
in the underlying `Task`.

Related:
- tektoncd#4704
- tektoncd#4841
@jerop jerop force-pushed the tep-0090-validate-params branch from 982324c to f1a3998 Compare May 9, 2022 19:02
@jerop
Copy link
Member Author

jerop commented May 9, 2022

/test pull-tekton-pipeline-go-coverage

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/validate_resources.go 96.0% 95.3% -0.8

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dibyom

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 9, 2022
@dibyom
Copy link
Member

dibyom commented May 9, 2022

I unresolved a thread because I wasn't sure it was fully resolved but otherwise this looks good to me!

@jerop
Copy link
Member Author

jerop commented May 9, 2022

I unresolved a thread because I wasn't sure it was fully resolved but otherwise this looks good to me!

@dibyom - I'd not seen the follow up questions, addressed them, thank you!

@abayer
Copy link
Contributor

abayer commented May 10, 2022

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label May 10, 2022
@jerop jerop marked this pull request as draft May 10, 2022 14:26
@jerop
Copy link
Member Author

jerop commented May 10, 2022

/hold

@tekton-robot tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 10, 2022
@jerop jerop marked this pull request as ready for review May 10, 2022 14:26
@dibyom
Copy link
Member

dibyom commented May 10, 2022

/lgtm

@jerop
Copy link
Member Author

jerop commented May 10, 2022

/hold cancel

@tekton-robot tekton-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 10, 2022
@jerop
Copy link
Member Author

jerop commented May 10, 2022

/test pull-pipeline-kind-k8s-v1-21-e2e

@jerop
Copy link
Member Author

jerop commented May 10, 2022

/test pull-tekton-pipeline-alpha-integration-tests

2 similar comments
@jerop
Copy link
Member Author

jerop commented May 10, 2022

/test pull-tekton-pipeline-alpha-integration-tests

@jerop
Copy link
Member Author

jerop commented May 11, 2022

/test pull-tekton-pipeline-alpha-integration-tests

@tekton-robot tekton-robot merged commit 827e164 into tektoncd:main May 11, 2022
@jerop jerop added the kind/feature Categorizes issue or PR as related to a new feature. label Jun 6, 2022
@jerop jerop deleted the tep-0090-validate-params branch June 11, 2022 01:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants