Add Kubernetes client auth exec config support

The implementation is similar to other Kubernetes Terraform providers such as the kubectl or helm providers and allows the Flux provider to authenticate against EKS clusters (or others supporting this method) by configuring the client authentication exec plugin. Example configuration for EKS: ``` provider "flux" { ... kubernetes = { ... exec = { api_version = "client.authentication.k8s.io/v1beta1" env = { AWS_PROFILE = "example-profile" } command = "aws" args = [ "--region", "us-east-1", "eks", "get-token", "--cluster-name", "example-cluster", ] } } } ``` Signed-off-by: Aurel Canciu <[email protected]>
fluxcd · Jul 5, 2023 · 8189b59 · 8189b59
1 parent 24e727c
commit 8189b59
Show file tree

Hide file tree

Showing 4 changed files with 275 additions and 20 deletions.
diff --git a/docs/index.md b/docs/index.md
@@ -26,6 +26,89 @@ provider "flux" {
 }
 ```
 
+## Kubernetes Authentication
+
+The Flux provider can be configured to authenticate against Kubernetes using
+either of these methods:
+
+* [Using a kubeconfig file](#file-config)
+* [Supplying credentials](#credentials-config)
+* [Exec plugins](#exec-plugins)
+
+For a full list of supported provider authentication arguments, see the [argument reference](#nestedatt--kubernetes) below.
+
+### File config
+
+You can provide a path to a kubeconfig file using the `config_path` attribute or
+using the `KUBE_CONFIG_PATH` environment variable.
+A kubeconfig file can have multiple contexts, specify the desired one using the
+`config_context` attribute, otherwise, the `default` context will be used.
+
+```hcl
+provider "flux" {
+ kubernetes = {
+ config_path = "~/.kube/config"
+ }
+}
+```
+
+Similar to kubectl, the provider can also support multiple config paths using
+the `config_paths` attribute or setting the `KUBE_CONFIG_PATHS` environment
+variable.
+
+```hcl
+provider "flux" {
+ kubernetes = {
+ config_paths = [
+ "/path/a/kubeconfig",
+ "/path/b/kubeconfig"
+ ]
+ }
+}
+```
+
+
+### Credentials config
+
+The basic configuration attributes can also be explicitly specified using the
+respective attributes:
+
+```hcl
+provider "flux" {
+ kubernetes = {
+ host = "https://cluster-api-hostname:port"
+
+ client_certificate = file("~/.kube/client-cert.pem")
+ client_key = file("~/.kube/client-key.pem")
+ cluster_ca_certificate = file("~/.kube/cluster-ca-cert.pem")
+ }
+}
+```
+
+
+### Exec plugins
+
+For Kubernetes cluster providers using short-lived authentication tokens the
+exec client authentication plugin can be used to fetch a new token using a CLI
+tool before initializing the provider.
+
+One good example of such a scenario is on EKS:
+
+```hcl
+provider "flux" {
+ kubernetes = {
+ host = var.cluster_endpoint
+ cluster_ca_certificate = base64decode(var.cluster_ca_cert)
+ exec = {
+ api_version = "client.authentication.k8s.io/v1beta1"
+ args = ["eks", "get-token", "--cluster-name", var.cluster_name]
+ command = "aws"
+ }
+ }
+}
+```
+
+
 <!-- schema generated by tfplugindocs -->
 ## Schema
 
@@ -88,9 +171,23 @@ Optional:
 - `config_context_cluster` (String)
 - `config_path` (String) Path to the kube config file. Can be set with KUBE_CONFIG_PATH.
 - `config_paths` (Set of String) A list of paths to kube config files. Can be set with KUBE_CONFIG_PATHS environment variable.
+- `exec` (Attributes) Kubernetes client authentication exec plugin configuration (see [below for nested schema](#nestedatt--kubernetes--exec))
 - `host` (String) The hostname (in form of URI) of Kubernetes master.
 - `insecure` (Boolean) Whether server should be accessed without verifying the TLS certificate.
 - `password` (String) The password to use for HTTP basic authentication when accessing the Kubernetes master endpoint.
 - `proxy_url` (String) URL to the proxy to be used for all API requests
 - `token` (String) Token to authenticate an service account
-- `username` (String) The username to use for HTTP basic authentication when accessing the Kubernetes master endpoint.
+- `username` (String) The username to use for HTTP basic authentication when accessing the Kubernetes master endpoint.
+
+<a id="nestedatt--kubernetes--exec"></a>
+### Nested Schema for `kubernetes.exec`
+
+Required:
+
+- `api_version` (String) Kubernetes client authentication API Version
+- `command` (String) Client authentication exec command
+
+Optional:
+
+- `args` (List of String) Client authentication exec command arguments
+- `env` (Map of String) Client authentication exec environment variables
diff --git a/internal/provider/provider.go b/internal/provider/provider.go
@@ -19,7 +19,10 @@ package provider
 import (
  "context"
  "fmt"
+ "os"
+ "path/filepath"
 
+ "github.com/hashicorp/terraform-plugin-framework/attr"
  "github.com/hashicorp/terraform-plugin-framework/datasource"
  "github.com/hashicorp/terraform-plugin-framework/path"
  "github.com/hashicorp/terraform-plugin-framework/provider"
@@ -63,21 +66,29 @@ type Git struct {
  Http *Http `tfsdk:"http"`
 }
 
+type KubernetesExec struct {
+ APIVersion types.String `tfsdk:"api_version"`
+ Command types.String `tfsdk:"command"`
+ Env types.Map `tfsdk:"env"`
+ Args types.List `tfsdk:"args"`
+}
+
 type Kubernetes struct {
- Host types.String `tfsdk:"host"`
- Username types.String `tfsdk:"username"`
- Password types.String `tfsdk:"password"`
- Insecure types.Bool `tfsdk:"insecure"`
- ClientCertificate types.String `tfsdk:"client_certificate"`
- ClientKey types.String `tfsdk:"client_key"`
- ClusterCACertificate types.String `tfsdk:"cluster_ca_certificate"`
- ConfigPaths types.Set `tfsdk:"config_paths"`
- ConfigPath types.String `tfsdk:"config_path"`
- ConfigContext types.String `tfsdk:"config_context"`
- ConfigContextAuthInfo types.String `tfsdk:"config_context_auth_info"`
- ConfigContextCluster types.String `tfsdk:"config_context_cluster"`
- Token types.String `tfsdk:"token"`
- ProxyURL types.String `tfsdk:"proxy_url"`
+ Host types.String `tfsdk:"host"`
+ Username types.String `tfsdk:"username"`
+ Password types.String `tfsdk:"password"`
+ Insecure types.Bool `tfsdk:"insecure"`
+ ClientCertificate types.String `tfsdk:"client_certificate"`
+ ClientKey types.String `tfsdk:"client_key"`
+ ClusterCACertificate types.String `tfsdk:"cluster_ca_certificate"`
+ ConfigPaths types.Set `tfsdk:"config_paths"`
+ ConfigPath types.String `tfsdk:"config_path"`
+ ConfigContext types.String `tfsdk:"config_context"`
+ ConfigContextAuthInfo types.String `tfsdk:"config_context_auth_info"`
+ ConfigContextCluster types.String `tfsdk:"config_context_cluster"`
+ Token types.String `tfsdk:"token"`
+ ProxyURL types.String `tfsdk:"proxy_url"`
+ Exec *KubernetesExec `tfsdk:"exec"`
 }
 
 type ProviderModel struct {
@@ -167,6 +178,30 @@ func (p *fluxProvider) Schema(ctx context.Context, req provider.SchemaRequest, r
  Optional: true,
  Description: "URL to the proxy to be used for all API requests",
  },
+ "exec": schema.SingleNestedAttribute{
+ Attributes: map[string]schema.Attribute{
+ "api_version": schema.StringAttribute{
+ Description: "Kubernetes client authentication API Version",
+ Required: true,
+ },
+ "command": schema.StringAttribute{
+ Description: "Client authentication exec command",
+ Required: true,
+ },
+ "env": schema.MapAttribute{
+ ElementType: types.StringType,
+ Description: "Client authentication exec environment variables",
+ Optional: true,
+ },
+ "args": schema.ListAttribute{
+ ElementType: types.StringType,
+ Description: "Client authentication exec command arguments",
+ Optional: true,
+ },
+ },
+ Optional: true,
+ Description: "Kubernetes client authentication exec plugin configuration",
+ },
  },
  Optional: true,
  },
@@ -321,6 +356,20 @@ func (p *fluxProvider) Configure(ctx context.Context, req provider.ConfigureRequ
  if data.Git.AuthorName.IsNull() {
  data.Git.AuthorName = types.StringValue(defaultAuthor)
  }
+ if data.Kubernetes.ConfigPath.IsNull() {
+ if v, ok := os.LookupEnv("KUBE_CONFIG_PATH"); ok {
+ data.Kubernetes.ConfigPath = types.StringValue(v)
+ }
+ }
+ if data.Kubernetes.ConfigPaths.IsNull() {
+ if v, ok := os.LookupEnv("KUBE_CONFIG_PATHS"); ok {
+ var paths []attr.Value
+ for _, p := range filepath.SplitList(v) {
+ paths = append(paths, types.StringValue(p))
+ }
+ data.Kubernetes.ConfigPaths = types.SetValueMust(types.StringType, paths)
+ }
+ }
 
  prd, err := NewProviderResourceData(ctx, data)
  if err != nil {

diff --git a/internal/provider/provider_resource_data.go b/internal/provider/provider_resource_data.go
@@ -36,6 +36,7 @@ import (
  apimachineryschema "k8s.io/apimachinery/pkg/runtime/schema"
  restclient "k8s.io/client-go/rest"
  "k8s.io/client-go/tools/clientcmd"
+ "k8s.io/client-go/tools/clientcmd/api"
  clientcmdapi "k8s.io/client-go/tools/clientcmd/api"
  "sigs.k8s.io/controller-runtime/pkg/client"
 
@@ -50,7 +51,7 @@ type providerResourceData struct {
 func NewProviderResourceData(ctx context.Context, data ProviderModel) (*providerResourceData, error) {
  clientCfg, err := getClientConfiguration(ctx, data.Kubernetes)
  if err != nil {
- return nil, fmt.Errorf("invalid Kubernetes configuration: %w", err)
+ return nil, fmt.Errorf("Invalid Kubernetes configuration: %w", err)
  }
  rcg := utils.NewRestClientGetter(clientCfg)
  return &providerResourceData{
@@ -189,7 +190,7 @@ func (prd *providerResourceData) GetEntityList() (openpgp.EntityList, error) {
  var err error
  entityList, err = openpgp.ReadKeyRing(strings.NewReader(prd.git.GpgKeyRing.ValueString()))
  if err != nil {
- return nil, fmt.Errorf("failed to read GPG key ring: %w", err)
+ return nil, fmt.Errorf("Failed to read GPG key ring: %w", err)
  }
  }
  return entityList, nil
@@ -290,9 +291,7 @@ func getClientConfiguration(ctx context.Context, kubernetes *Kubernetes) (client
  if diag.HasError() {
  return nil, fmt.Errorf("%s", diag)
  }
- for _, p := range pp {
- configPaths = append(configPaths, p)
- }
+ configPaths = append(configPaths, pp...)
  }
  if len(configPaths) > 0 {
  expandedPaths := []string{}
@@ -356,6 +355,33 @@ func getClientConfiguration(ctx context.Context, kubernetes *Kubernetes) (client
  }
  overrides.ClusterDefaults.ProxyURL = kubernetes.ProxyURL.ValueString()
 
+ if kubernetes.Exec != nil {
+ var args []string
+ if diag := kubernetes.Exec.Args.ElementsAs(ctx, &args, false); diag.HasError() {
+ return nil, fmt.Errorf("%s", diag)
+ }
+ var envMap map[string]string
+ if diag := kubernetes.Exec.Env.ElementsAs(ctx, &envMap, false); diag.HasError() {
+ return nil, fmt.Errorf("%s", diag)
+ }
+ var env []api.ExecEnvVar
+ for k, v := range envMap {
+ env = append(env, api.ExecEnvVar{
+ Name: k,
+ Value: v,
+ })
+ }
+
+ exec := &clientcmdapi.ExecConfig{
+ InteractiveMode: clientcmdapi.IfAvailableExecInteractiveMode,
+ APIVersion: kubernetes.Exec.APIVersion.ValueString(),
+ Command: kubernetes.Exec.Command.ValueString(),
+ Args: args,
+ Env: env,
+ }
+ overrides.AuthInfo.Exec = exec
+ }
+
  cc := clientcmd.NewNonInteractiveDeferredLoadingClientConfig(loader, overrides)
  return cc, nil
 }
diff --git a/templates/index.md.tmpl b/templates/index.md.tmpl
@@ -17,4 +17,87 @@ Get Kubernetes credentials from a kubeconfig file. The current context set in th
 
 {{ tffile .ExampleFile }}
 
+## Kubernetes Authentication
+
+The Flux provider can be configured to authenticate against Kubernetes using
+either of these methods:
+
+* [Using a kubeconfig file](#file-config)
+* [Supplying credentials](#credentials-config)
+* [Exec plugins](#exec-plugins)
+
+For a full list of supported provider authentication arguments, see the [argument reference](#nestedatt--kubernetes) below.
+
+### File config
+
+You can provide a path to a kubeconfig file using the `config_path` attribute or
+using the `KUBE_CONFIG_PATH` environment variable.
+A kubeconfig file can have multiple contexts, specify the desired one using the
+`config_context` attribute, otherwise, the `default` context will be used.
+
+```hcl
+provider "flux" {
+ kubernetes = {
+ config_path = "~/.kube/config"
+ }
+}
+```
+
+Similar to kubectl, the provider can also support multiple config paths using
+the `config_paths` attribute or setting the `KUBE_CONFIG_PATHS` environment
+variable.
+
+```hcl
+provider "flux" {
+ kubernetes = {
+ config_paths = [
+ "/path/a/kubeconfig",
+ "/path/b/kubeconfig"
+ ]
+ }
+}
+```
+
+
+### Credentials config
+
+The basic configuration attributes can also be explicitly specified using the
+respective attributes:
+
+```hcl
+provider "flux" {
+ kubernetes = {
+ host = "https://cluster-api-hostname:port"
+
+ client_certificate = file("~/.kube/client-cert.pem")
+ client_key = file("~/.kube/client-key.pem")
+ cluster_ca_certificate = file("~/.kube/cluster-ca-cert.pem")
+ }
+}
+```
+
+
+### Exec plugins
+
+For Kubernetes cluster providers using short-lived authentication tokens the
+exec client authentication plugin can be used to fetch a new token using a CLI
+tool before initializing the provider.
+
+One good example of such a scenario is on EKS:
+
+```hcl
+provider "flux" {
+ kubernetes = {
+ host = var.cluster_endpoint
+ cluster_ca_certificate = base64decode(var.cluster_ca_cert)
+ exec = {
+ api_version = "client.authentication.k8s.io/v1beta1"
+ args = ["eks", "get-token", "--cluster-name", var.cluster_name]
+ command = "aws"
+ }
+ }
+}
+```
+
+
 {{ .SchemaMarkdown | trimspace }}