Add explicit observed artifact content configurations in status

[darkowlzz]

GitRepositoryReconciler and OCIRepositoryReconcilers support no-op reconciliation which makes these reconcilers efficient in their operations. The no-op reconciliation is determined by taking into considerations all the variables that affect the source artifact content. In #724 and #917, the values of these variables were computed as a checksum and stored in the status in a field called the content config checksum. Although it worked, the checksum of multiple variables wasn't friendly to debug or see the observations of the controller. This change replaces the content config checksum with explicit status fields for the observed variables that contribute to the source artifact content. The content config checksum remains in the status API for now, but is not used and unset for any existing resources. It'll be removed in the next API version.

In GitRepository source, three new status fields observedIgnore, observedRecurseSubmodules and observedInclude are introduced. https://gist.github.com/darkowlzz/3ffc1000754580a0fac6f9068e06643e shows the status with these new fields under some relevant scenarios.

In OCIRepository source, two new status fields observedIgnore and observedLayerSelector are introduced. https://gist.github.com/darkowlzz/ece8718c5ed195948d1c8e2f4ea2f941 shows the status with these new fields under some relevant scenarios.

Spec docs and tests have been updated for these changes.

[stefanprodan]

Can you please add observedIgnore to the Bucket status also, it would be good to have consistency across all source APIs.

[darkowlzz]

Can you please add observedIgnore to the Bucket status also, it would be good to have consistency across all source APIs.

Added status.observedIgnore in bucket. Looks like the same as others.
Just for reference:

status:
  artifact:
    checksum: 1dbc7991adb507aaf62cb8ac0d973b2bd5929c0642632d66ec8b594ed277064a
    lastUpdateTime: "2022-10-10T14:01:44Z"
    path: bucket/default/foo/e271cae427bae7c073844cacafe6ef05ec2e064c69f9e7e06abab1fe849bf994.tar.gz
    revision: e271cae427bae7c073844cacafe6ef05ec2e064c69f9e7e06abab1fe849bf994
    size: 173
    url: http://localhost:9090/bucket/default/foo/e271cae427bae7c073844cacafe6ef05ec2e064c69f9e7e06abab1fe849bf994.tar.gz
  conditions:
  - lastTransitionTime: "2022-10-10T14:01:44Z"
    message: stored artifact for revision 'e271cae427bae7c073844cacafe6ef05ec2e064c69f9e7e06abab1fe849bf994'
    observedGeneration: 3
    reason: Succeeded
    status: "True"
    type: Ready
  - lastTransitionTime: "2022-10-10T14:01:44Z"
    message: stored artifact for revision 'e271cae427bae7c073844cacafe6ef05ec2e064c69f9e7e06abab1fe849bf994'
    observedGeneration: 3
    reason: Succeeded
    status: "True"
    type: ArtifactInStorage
  observedGeneration: 3
  observedIgnore: |
    /deploy/**/*.md
    /deploy/**/*.txt
  url: http://localhost:9090/bucket/default/foo/latest.tar.gz

[stefanprodan]

@darkowlzz with the Bucket changes in this PR we can close #712 right?

[darkowlzz]

with the Bucket changes in this PR we can close #712 right?

@stefanprodan IIRC, I think I heard that PR was abandoned because the contributor later found out that what they were trying was working fine for buckets. But based on my experience and understanding of it, I don't think it'll rebuild the artifact. We need to do similar artifact content config checks that we do in this PR for GitRepo and OCIRepo.
That PR can be closed, but the issue remains.

[pjbgf]

Small nit, otherwise LGTM.

[stefanprodan]

We need to do similar artifact content config checks that we do in this PR for GitRepo and OCIRepo.

Let's do it in this PR.

[darkowlzz]

Let's do it in this PR.

I tested it and looks like it does work but in a different way. Unlike other sources where we download all the content and then apply the ignore rules, in case of bucket, the ignore rule is applied in the etag fetcher itself and an index is calculated based on the etag values of the files. Files that are ignored are not included in the index computation, resulting in a different index value due to ignore value. Due to this, an ineffective ignore rule doesn't result in rebuilding of the artifact, but an effective rule does. Seems like the correct behavior.
With that #712 can be closed. That issue originated from GitRepo not respecting ignore change but that's been fixed already.

[stefanprodan]

LGTM

Thanks @darkowlzz 🏅

[pjbgf]

LGTM

Thanks @darkowlzz ☀️!

[hiddeco]

Looks great to me as well! 🙇