[Bug]: Consider handling non-existent `~/.docker/config.json` #2767

mmalenic · 2024-09-03T23:25:24Z

Testcontainers version

v0.33.0

Using the latest Testcontainers version?

Yes

Host OS

Ubuntu 24.04.1 LTS

Host arch

x86

Go version

1.22

Docker version

Client: Docker Engine - Community
 Version:           27.2.0
 API version:       1.46 (downgraded from 1.47)
 Go version:        go1.21.13
 Git commit:        3ab4256
 Built:             Tue Aug 27 14:15:15 2024
 OS/Arch:           linux/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.33.0 (160616)
 Engine:
  Version:          27.1.1
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.12
  Git commit:       cc13f95
  Built:            Tue Jul 23 19:57:19 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.19
  GitCommit:        2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc:
  Version:          1.7.19
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

Client: Docker Engine - Community
 Version:    27.2.0
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.1-desktop.1
    Path:     /usr/lib/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1-desktop.1
    Path:     /usr/lib/docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.34
    Path:     /usr/lib/docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.14
    Path:     /usr/lib/docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /usr/lib/docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     /usr/lib/docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /usr/lib/docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /usr/lib/docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /usr/lib/docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.11.0
    Path:     /usr/lib/docker/cli-plugins/docker-scout

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 11
 Server Version: 27.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.0-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 14.88GiB
 Name: docker-desktop
 ID: b8d41dab-9cfc-4d99-a8ba-66a0a767d1b1
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

What happened?

Similar to the Java testcontainers issue: testcontainers/testcontainers-java#4847, it could be useful to accept non-existent ~/.docker/config.json when finding docker auth configs.

Currently, when starting a container by building a Dockerfile using testcontainers.FromDockerfile, a ~/.docker/config.json is required, even if no credentials are needed to build the Dockerfile. This is an issue in environments that don't have a ~/.docker/config.json by default, such as on AWS CodeBuild, as it causes tests to fail. For example, an output of a failed test when running without a ~/.docker/config.json:

--- FAIL: TestHandler (10.81s)
    test.go:60: 
        	Error Trace:	/codebuild/output/src3804529192/src/actions-runner/_work/orcabus/orcabus/lib/workload/stateless/stacks/fmannotator/internal/test/test.go:60
        	            				/codebuild/output/src3804529192/src/actions-runner/_work/orcabus/orcabus/lib/workload/stateless/stacks/fmannotator/internal/test/test.go:86
        	            				/codebuild/output/src3804529192/src/actions-runner/_work/orcabus/orcabus/lib/workload/stateless/stacks/fmannotator/handlers_test.go:70
        	Error:      	Received unexpected error:
        	            	create container: build options: auth configs from Dockerfile: open /root/.docker/config.json: no such file or directory
        	Test:       	TestHandler
FAIL

A workaround is to create an empty ~/.docker/config.json or set the DOCKER_CONFIG environment variable.

I think this could be fixed by changing how the errors are handled in getDockerAuthConfigs by returning an empty map[string]registry.AuthConfig instead of an error if ~/.docker/config.json could not be found.

Relevant log output

No response

Additional information

No response

The text was updated successfully, but these errors were encountered:

mdelapenya · 2024-09-05T10:47:02Z

@stevenh you have this part of the code fresh in your mind. Do you think that the proposal of returning an empty map of credentials when the file does not exist is appropriate? At first sight, I see no harm on it

mdelapenya · 2024-09-05T10:48:53Z

BTW this test demonstrates the current error:

t.Run("docker config file does not exist", func(t *testing.T) {
	t.Setenv("DOCKER_CONFIG", "")
	tmp := t.TempDir()
	t.Setenv("HOME", tmp)
	t.Setenv("USERPROFILE", tmp) // Windows

	cfg, err := getDockerConfig()
	require.Error(t, err)
	require.Empty(t, cfg)
})

daniel-meyer-dme · 2024-09-05T15:00:17Z

We encounter the same issue but apparently even setting the environment variable DOCKER_AUTH_CONFIG errors. The reason here is that the config file is checked even though there is already a loaded configuration.
According to https://golang.testcontainers.org/features/docker_auth/ DOCKER_AUTH_CONFIG should have precedence over the files which might not exist.
But apart from that I would support to accept having no configuration required.

- allow missing config file -- replace registry cache key with a key generated by any defined config - allow no provided config

Don't error when looking up auth if no docker configuration file exists, instead treat it as no authentication was provided. Fixes testcontainers#2767

Handle file not exist error in getDockerAuthConfigs, treating it as if no authentication was provided. Use config directly for cache instead of loading the file a second time which may be the wrong file if loaded from the environment. Correctly handle json decode errors in getDockerConfig instead of falling back to the default config, which would result in unexpected behaviour. Fixes testcontainers#2767

Handle file not exist error in getDockerAuthConfigs, treating it as if no authentication was provided. Use config directly for cache instead of loading the file a second time which may be the wrong file if loaded from the environment. Correctly handle json decode errors in getDockerConfig instead of falling back to the default config, which would result in unexpected behaviour. Tests refactored to ensure all edge cases for getDockerConfig and getDockerAuthConfigs are handled. Fixes testcontainers#2767

stevenh · 2024-09-07T21:37:57Z

Did some digging and there was other config error handling edge cases which are all fixed by #2772

mmalenic added the bug An issue with the library label Sep 3, 2024

mmalenic mentioned this issue Sep 3, 2024

filemanager: annotate portal run id umccr/orcabus#529

Merged

daniel-meyer-dme mentioned this issue Sep 6, 2024

fix #2767 allow missing repository config #2771

Closed

stevenh added a commit to stevenh/testcontainers-go that referenced this issue Sep 7, 2024

fix: auth with no docker config

1aad304

Don't error when looking up auth if no docker configuration file exists, instead treat it as no authentication was provided. Fixes testcontainers#2767

stevenh mentioned this issue Sep 7, 2024

fix: docker config error handling when config file does not exist #2772

Merged

mdelapenya closed this as completed in #2772 Sep 9, 2024

mdelapenya closed this as completed in 6a947dc Sep 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Consider handling non-existent `~/.docker/config.json` #2767

[Bug]: Consider handling non-existent `~/.docker/config.json` #2767

mmalenic commented Sep 3, 2024 •

edited

Loading

mdelapenya commented Sep 5, 2024

mdelapenya commented Sep 5, 2024

daniel-meyer-dme commented Sep 5, 2024

stevenh commented Sep 7, 2024

[Bug]: Consider handling non-existent ~/.docker/config.json #2767

[Bug]: Consider handling non-existent ~/.docker/config.json #2767

Comments

mmalenic commented Sep 3, 2024 • edited Loading

Testcontainers version

Using the latest Testcontainers version?

Host OS

Host arch

Go version

Docker version

Docker info

What happened?

Relevant log output

Additional information

mdelapenya commented Sep 5, 2024

mdelapenya commented Sep 5, 2024

daniel-meyer-dme commented Sep 5, 2024

stevenh commented Sep 7, 2024

[Bug]: Consider handling non-existent `~/.docker/config.json` #2767

[Bug]: Consider handling non-existent `~/.docker/config.json` #2767

mmalenic commented Sep 3, 2024 •

edited

Loading