Issue with recurseSubmodules on 0.37.1

[cdm-arm]

Since update from 0.36 to 0.37.1 the below git repo definitions fail with this error:

{"level":"error","ts":"2022-11-25T17:36:54.324Z","msg":"Reconciler error","controller":"gitrepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"GitRepository","GitRepository":{"name":"abc-atest","namespace":"flux-system"},"namespace":"flux-system","name":"abc-atest","reconcileID":"d42e4151-0c5a-471f-bac5-ecee1927183d","error":"failed to checkout and determine revision: unable to clone 'https://bitbucket.example.com/bitbucket/scm/DEPLOYKUB/abc-atest.git': feature not supported"}

The source definition:

apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
  name: abc-atest
  namespace: flux-system
spec:
  recurseSubmodules: true
  interval: 1m2s
  ref:
    branch: master
  secretRef:
    name: bitbucket
  url: https://bitbucket.example.com/bitbucket/scm/DEPLOYKUB/abc-atest.git

The moment I remove the recurseSubmodules: true it get's fetched.

[pjbgf]

@aek-arm we have created a rc image to test the fix for this: ghcr.io/fluxcd/source-controller:rc-f50e968f. Changes are on branch fix-chroot.

Can you please give it a try and confirm whether it resolves your issue?

[pjbgf]

There was an issue with the previous rc release build. A new image has been generated (still from the same branch). The image above has been updated accordingly.

[cdm-arm]

Hi @pjbgf - We'll try that one and give feedback.

[cdm-arm]

Hi @pjbgf

Sorry for late reply, I had to define and setup a cluster first.

I changed gotk-components.yaml like below, committed and pushed:
#image: ghcr.io/fluxcd/source-controller:v0.32.1
image: ghcr.io/fluxcd/source-controller:rc-f50e968f

~ flux reconcile kustomization flux-system --with-source
► annotating GitRepository flux-system in flux-system namespace
✔ GitRepository annotated
◎ waiting for GitRepository reconciliation
✔ fetched revision develop/d63761677e8c507911ce525a18b4b6c92d6cdf97
► annotating Kustomization flux-system in flux-system namespace
✔ Kustomization annotated
◎ waiting for Kustomization reconciliation
✔ applied revision develop/d63761677e8c507911ce525a18b4b6c92d6cdf97

It proceed but then is failing with this errors on "mkdir modules: read-only file" - while the helm charts before get fetched and stored.

{"level":"error","ts":"2022-12-07T17:12:37.522Z","msg":"failed to checkout and determine revision: unable to clone 'https://git.example.com/bitbucket/scm/KUBDEP/cdm-mm-config.git': mkdir modules: read-only file system","controller":"gitrepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"GitRepository","GitRepository":{"name":"cdm-mm-config","namespace":"flux-system"},"namespace":"flux-system","name":"cdm-mm-config","reconcileID":"cd8ddcb5-a547-4a7a-89b1-6c1c8a6aff3c","error":"failed to checkout and determine revision: unable to clone 'https://git.example.com/bitbucket/scm/KUBDEP/cdm-mm-config.git': mkdir modules: read-only file system"}

{"level":"error","ts":"2022-12-07T17:12:37.566Z","msg":"Reconciler error","controller":"gitrepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"GitRepository","GitRepository":{"name":"cdm-mm-config","namespace":"flux-system"},"namespace":"flux-system","name":"cdm-mm-config","reconcileID":"cd8ddcb5-a547-4a7a-89b1-6c1c8a6aff3c","error":"failed to checkout and determine revision: unable to clone 'https://git.example.com/bitbucket/scm/KUBDEP/cdm-mm-config.git': mkdir modules: read-only file system"}

Happy to try any other bug fix

[pjbgf]

Thank you @aek-arm, we will try to reproduce and come up with another fix. (@aryan9600)

[pjbgf]

@aek-arm we have another fix for you. Do you mind retrying with ghcr.io/fluxcd/source-controller:rc-d24afcf2 please?

[cdm-arm]

@pjbgf - can confirm it works!

{"level":"info","ts":"2022-12-12T13:51:55.329Z","msg":"stored artifact for commit 'mycommit k8s-base-tenants-master submodule updg...'","controller":"gitrepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"GitRepository","GitRepository":{"name":"my-kustomization","namespace":"flux-system"},"namespace":"flux-system","name":"my-kustomization","reconcileID":"5b4b9fae-9b50-4e98-bc9f-e722325d0d30"}

[pjbgf]

@aek-arm excellent, thank you very much for confirming. 🙇

[cdm-arm]

For those looking how to test, just patch from the kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-components.yaml
- gotk-sync.yaml
patches:
  - patch: | 
      - op: replace
        path: /spec/template/spec/containers/0/image
        value: ghcr.io/fluxcd/source-controller:rc-d24afcf2
    target:
      kind: Deployment
      name: source-controller

[cdm-arm]

@aek-arm excellent, thank you very much for confirming. 🙇

@pjbgf any chance that will make it into 0.37.1?

[pjbgf]

@aek-arm that's the plan. Once #977 is merged, #975 is next.