BUG: deploy & run with kubernetes-module type do not copy / prepare secrets

[stefreak]

Bug

Current Behavior

Deploy and running tasks/tests on a kubernetes-module type ignores copySecrets and imagePullSecrets settings.

Expected behavior

When deploying a kubernetes-module type module, or running tasks/tests on it, we should copy the secrets as specified in copySecrets and imagePullSecrets.

Either we should copy all the secrets defined in imagePullSecrets, or we should introspect the manifests provided to the kubernetes module and only copy the ones that are referenced there.

Reproducible example

Take the kubernetes-secrets example, and rewrite the backend and frontend modules using the kubernetes-module type, instead of the container type.

Workaround

No known workaround, other than using the container Module type.

Suggested solution(s)

The container module type runs the following code in the handler that runs tasks or deploys the service:

  if (provider.config.imagePullSecrets.length > 0) {
    // add any configured imagePullSecrets.
    const imagePullSecrets = await prepareSecrets({ api, namespace, secrets: provider.config.imagePullSecrets, log })
    workload.spec.template.spec!.imagePullSecrets = imagePullSecrets
  }
  await prepareSecrets({ api, namespace, secrets: provider.config.copySecrets, log })

We need to add similar code to the handlers that deploy / run the kubernetes-module type.

The kubernetes-module type should probably not add the imagePullSecrets field automatically to the workload spec, but it might instead detect referenced secrets in the kubernetes module spec and emit warnings if they're not specified in imagePullSecrets, and copy the referenced secrets automatically.

Your environment

• OS: macOS
• How I'm running Kubernetes: GKE, local

garden version 0.12.51-edge-5015f6278

[stefreak]

This bug surfaced in the discord help thread/ discussion https://discord.com/channels/817392104711651328/1077630057712992377/1077630057712992377 – Thank you Th3FalleN for bringing this to our attention!

[stefreak]

#1774 and #1775 might be slightly related; users might expect that copySecrets always copies secrets, and not just when deploying services or running tasks.

[worldofgeese]

@stefreak is this still an issue?

[Dunemask]

Not entirely sure if this is related, but I'm noticing the same/similar issue on 0.13.13 onwards. When deploying actions to a kubernetes provider, the imagePullSecrets and copySecrets aren't used. This is most noticeable when creating CI namespaces. Workaround solutions involve using secret duplication pods, or a specific secrets provider. The logs don't even record an attempt to copy secrets like I normally using 0.13.12 or earlier, so perhaps the order of events changed when promises changed in 0.13.13 with regards to #4959

From my limited testing this occurs with build and deploy actions deploying to kubernetes provider. I'm also not able to find much documentation on where/when secrets are copied for each stage, but I'm also notorious for struggling to see things hidden in plain sight. 😂

[Dunemask]

@worldofgeese Yes this is still an issue with all versions after 0.13.12