Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix PipelineResource name handling. #2697

Merged
merged 1 commit into from
Jun 11, 2020

Conversation

dlorenc
Copy link
Contributor

@dlorenc dlorenc commented May 26, 2020

Changes

This commit changes the names used throughout PipelineResource handling.
PipelineResources have two names when used in a TaskRun:

  • The name of the PipelineResource entity ("external name")
  • The name given to the resource inside the Task definition ("internal name")

Task code is currently using the external names everywhere, but should be using
the internal names.

Ref: #2694

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

The TaskRun.Status.ResourceResults field now contains a new field called "ResourceName" that points to the name of the PipelineResourceBinding.

The TaskRun.Status.ResourcesResults.ResourceRef.Name field points to this same object. This field is now redundant and will be removed in a future release.

The ClusterResource has been updated to write it's kubeconfig to the correct location on disk. Previously it was written to a location under /workspace/input/<resource ref name>, instead of the correct /workspace/input/<resource binding name>. If you hardcode resource paths in your tasks, you will need to update these. If you refer to paths by interpolation ( $(resources.inputs.<resource name>) ), you do not need to make any changes.
Both paths will work for this release, but you will need to update your Task definitions by a future release.

@tekton-robot tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 26, 2020
@tekton-robot
Copy link
Collaborator

This PR cannot be merged: expecting exactly one kind/ label

Available kind/ labels are:

kind/bug: Categorizes issue or PR as related to a bug.
kind/flake: Categorizes issue or PR as related to a flakey test
kind/cleanup: Categorizes issue or PR as related to cleaning up code, process, or technical debt.
kind/design: Categorizes issue or PR as related to design.
kind/documentation: Categorizes issue or PR as related to documentation.
kind/feature: Categorizes issue or PR as related to a new feature.
kind/misc: Categorizes issue or PR as a miscellaneuous one.

1 similar comment
@tekton-robot
Copy link
Collaborator

This PR cannot be merged: expecting exactly one kind/ label

Available kind/ labels are:

kind/bug: Categorizes issue or PR as related to a bug.
kind/flake: Categorizes issue or PR as related to a flakey test
kind/cleanup: Categorizes issue or PR as related to cleaning up code, process, or technical debt.
kind/design: Categorizes issue or PR as related to design.
kind/documentation: Categorizes issue or PR as related to documentation.
kind/feature: Categorizes issue or PR as related to a new feature.
kind/misc: Categorizes issue or PR as a miscellaneuous one.

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/taskrun.go 75.2% 75.0% -0.2

@dlorenc
Copy link
Contributor Author

dlorenc commented May 26, 2020

/kind bug

@tekton-robot tekton-robot added the kind/bug Categorizes issue or PR as related to a bug. label May 26, 2020
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/taskrun.go 75.2% 75.0% -0.2

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbwsg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 27, 2020
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/taskrun.go 75.2% 75.0% -0.2

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/taskrun.go 75.2% 75.0% -0.2

@bobcatfish
Copy link
Collaborator

hi all! can we get the release notes field filled out? just cruising by this one and i want to understand the user facing impact (if any - it seems like at least a bug fix tho?)

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/taskrun.go 75.2% 75.0% -0.2

@dlorenc
Copy link
Contributor Author

dlorenc commented May 28, 2020

hi all! can we get the release notes field filled out? just cruising by this one and i want to understand the user facing impact (if any - it seems like at least a bug fix tho?)

It's a bug fix that sort of requires an API change. Release Notes should be filled in now, we might want to discuss this in next week's meeting though before merging.

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/taskrun.go 76.1% 75.9% -0.2

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/taskrun.go 76.1% 75.9% -0.2

@vdemeester
Copy link
Member

@dlorenc needs a rebase 😅

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/taskrun.go 76.0% 75.8% -0.2

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/taskrun.go 76.0% 75.8% -0.2

Copy link
Collaborator

@bobcatfish bobcatfish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple quick requests (probably):

  • Could you update some docs with the new field? probably in https:/tektoncd/pipeline/blob/master/docs/taskruns.md#monitoring-execution-status (apologies that there isnt much there to start with)
  • If we want to deprecate the old field, could you add it to https:/tektoncd/pipeline/blob/master/docs/deprecations.md - i also want to double check, do we definitely want to remove it? it might make sense to have both? (or maybe it ultimately would make more sense to embed the resourcespecs in the status in the same way as we do the task spec? in that case i think deprecating the field makes sense - though you still might want to be able to map the task's resource name to the actual resource name used? ... i guess that's already in the spec?)

My one concern is that based on how the cluster end to end test changed, is this a backwards incompatible change for the behavior of the cluster pipelineresource, or was it just a quirk of how the test was setup? If the latter then I think we're all good, but if it is a backwards incompatible change we might want to do it a bit more slowly.

go.sum Show resolved Hide resolved
test/v1alpha1/cluster_resource_test.go Show resolved Hide resolved
pkg/reconciler/taskrun/taskrun.go Show resolved Hide resolved
@dlorenc
Copy link
Contributor Author

dlorenc commented Jun 5, 2020

  • If we want to deprecate the old field, could you add it to https:/tektoncd/pipeline/blob/master/docs/deprecations.md - i also want to double check, do we definitely want to remove it? it might make sense to have both? (or maybe it ultimately would make more sense to embed the resourcespecs in the status in the same way as we do the task spec? in that case i think deprecating the field makes sense - though you still might want to be able to map the task's resource name to the actual resource name used? ... i guess that's already in the spec?)

Yes, we definitely want to delete this. It doesn't make semantic sense for the ResourceResult to contain a ResourceRef for a few reasons:

  • The Ref could change later
  • The Ref may never exist - the user could have specified it inline

Instead it should probably contain a fully dereferenced copy of the resource (like you've suggested), but that should wait until we figure out the plan for resources.

@dlorenc
Copy link
Contributor Author

dlorenc commented Jun 5, 2020

My one concern is that based on how the cluster end to end test changed, is this a backwards incompatible change for the behavior of the cluster pipelineresource, or was it just a quirk of how the test was setup? If the latter then I think we're all good, but if it is a backwards incompatible change we might want to do it a bit more slowly.

And yes, this is slightly backwards incompatible. I just changed the test to be more clear. The old behavior for ClusterResources was just plain wrong - they were writing to the wrong location on disk. Referring to the location via $(resources.inputs.) still works and provides stable behavior.

I've explained this in the notes. I don't think there's a lot more we can do here...

@dlorenc
Copy link
Contributor Author

dlorenc commented Jun 5, 2020

OK, this has now been changed (and the release notes are updated) to keep both paths in place for now. We can remove the old path in the next release.

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/taskrun/taskrun.go 76.0% 75.8% -0.2

@vdemeester vdemeester added this to the Pipelines v0.14 milestone Jun 8, 2020
Copy link
Collaborator

@bobcatfish bobcatfish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dlorenc ! A couple minor requests around the deprecation doc, also want to make sure we have a test or something that ensures the deprecated version continues to work

docs/deprecations.md Outdated Show resolved Hide resolved
pkg/apis/resource/v1alpha1/cluster/cluster_resource.go Outdated Show resolved Hide resolved
test/cluster_resource_test.go Show resolved Hide resolved
pkg/apis/resource/v1alpha1/cluster/cluster_resource.go Outdated Show resolved Hide resolved
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/resource/v1alpha1/cluster/cluster_resource.go 79.1% 80.4% 1.4
pkg/reconciler/taskrun/taskrun.go 75.5% 75.2% -0.2

dlorenc added a commit to dlorenc/build-pipeline that referenced this pull request Jun 10, 2020
In discussion around tektoncd#2697, we realized the existing API compatiblity
policy is a little unclear, leading to confusion.
We discussed and clarified the intentions of this policy in tektoncd#2790.
This change adds an example to the policy, and tweaks the deprecations
table to make a bit more sense.

Fixes tektoncd#2790
tekton-robot pushed a commit that referenced this pull request Jun 10, 2020
In discussion around #2697, we realized the existing API compatiblity
policy is a little unclear, leading to confusion.
We discussed and clarified the intentions of this policy in #2790.
This change adds an example to the policy, and tweaks the deprecations
table to make a bit more sense.

Fixes #2790
@dlorenc
Copy link
Contributor Author

dlorenc commented Jun 10, 2020

Thanks @dlorenc ! A couple minor requests around the deprecation doc, also want to make sure we have a test or something that ensures the deprecated version continues to work

OK, this should be ready now.

This commit changes the names used throughout PipelineResource handling.
PipelineResources have two names when used in a TaskRun:
- The name of the PipelineResource entity (external name)
- The name given to the resource inside the Task definition (internal name)

Task code is currently using the external names everywhere, but should be using
the internal names.

This adds a new field for the internal name to PipelineResourceResult, and deprecates
the existing PipelineResourceResult.ResourceRef field.

Ref: tektoncd#2694
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/resource/v1alpha1/cluster/cluster_resource.go 79.1% 80.4% 1.4
pkg/reconciler/taskrun/taskrun.go 77.7% 77.4% -0.2

@bobcatfish
Copy link
Collaborator

/lgtm
/meow

@tekton-robot
Copy link
Collaborator

@bobcatfish: cat image

In response to this:

/lgtm
/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 11, 2020
@tekton-robot tekton-robot merged commit bd885f8 into tektoncd:master Jun 11, 2020
dlorenc added a commit to dlorenc/build-pipeline that referenced this pull request Jun 11, 2020
…release.

This is a cleanup commit that should go in after 0.14.0 is cut. It was introduced
in tektoncd#2697, to fix tektoncd#2694.
@dlorenc dlorenc deleted the fixresources branch June 12, 2020 12:51
tekton-robot pushed a commit that referenced this pull request Jul 6, 2020
…release.

This is a cleanup commit that should go in after 0.14.0 is cut. It was introduced
in #2697, to fix #2694.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants