maven:3.3 no longer useable, as-is, in docker.inside from jenkins docker pipeline

[dweomer]

Sample Code:

timestamps { -> node('build && docker') {
    stage 'MAVEN 3.2 (win)'
        getDockerImage('maven:3.2').inside {
            sh 'mvn -v'
        }

    stage 'MAVEN 3.3 (fail)'
        getDockerImage('maven:3.3').inside {
            sh 'mvn -v'
        }
}}

def getDockerImage(imageName) {
    return { ->
        def img = docker.image(imageName)
        /* make sure we have the up-to-date image */
        img.pull()
        /* dance around https://issues.jenkins-ci.org/browse/JENKINS-34276 */
        return docker.image(img.imageName())
    }()
}

The 3.2 stage runs just fine but the 3.3 stage fails with:

[Pipeline] stage (MAVEN 3.3 (fail))
17:23:27 Entering stage MAVEN 3.3 (fail)
17:23:27 Proceeding
[Pipeline] sh
17:23:27 [maven-3-point-3] Running shell script
17:23:27 + docker pull maven:3.3
17:23:29 3.3: Pulling from library/maven
17:23:29 Digest: sha256:f6becf58234086fa8cbfa89a38917ed3bec81baaac4465c3275d51ec22b60461
17:23:29 Status: Downloaded newer image for maven:3.3
[Pipeline] sh
17:23:29 [maven-3-point-3] Running shell script
17:23:30 + docker inspect -f . maven:3.3
17:23:30 .
[Pipeline] withDockerContainer
17:23:30 $ docker run -t -d -u 1000:1000 -w /var/lib/jenkins/workspace/JCHRISTEN/maven-3-point-3 -v /var/lib/jenkins/workspace/JCHRISTEN/maven-3-point-3:/var/lib/jenkins/workspace/JCHRISTEN/maven-3-point-3:rw -v /var/lib/jenkins/workspace/JCHRISTEN/maven-3-point-3@tmp:/var/lib/jenkins/workspace/JCHRISTEN/maven-3-point-3@tmp:rw -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** maven:3.3 cat
[Pipeline] {
[Pipeline] sh
17:23:31 [maven-3-point-3] Running shell script
17:23:31 Error response from daemon: Container c744a6e4b7ecfc61ceb9d9c88c9af070309ea44d47e38f5b4ece06ecce6fdb17 is not running
[Pipeline] }
17:23:49 $ docker stop c744a6e4b7ecfc61ceb9d9c88c9af070309ea44d47e38f5b4ece06ecce6fdb17
17:23:49 $ docker rm -f c744a6e4b7ecfc61ceb9d9c88c9af070309ea44d47e38f5b4ece06ecce6fdb17
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] End of Pipeline
ERROR: script returned exit code -2
Finished: FAILURE

This used to work. In my opinion this kind of breakage is unacceptable and is the direct result of this image being far too opinionated (doing too much at startup).

This problem was originally identified in #14 which seems to have been summarily dismissed. Please reconsider.

[dweomer]

Running this in a Jenkins 2 instance I am able to pause the run and see this from the container logs:

touch: cannot touch ‘/root/.m2/copy_reference_file.log’: Permission denied
Can not write to /root/.m2/copy_reference_file.log. Wrong volume permissions?

This relates to the ill-fated decision to make /root/.m2 a volume, see #11, as well as this base image simply trying to do too much at startup.

[dweomer]

This fails in my live Jenkins 1.x LTS series instance as well in the latest, as of today, Jenkins 2.x LTS series.

[carlossg]

Isn't this caused by jenkinsci/docker-workflow-plugin#25

In the meantime you can use a explicit version of the image, getting it with

docker inspect --format='{{.RepoDigests}}'

i.e.

docker pull maven@sha256:e8b86eefc2a429f063b5ef3175984cbaa7dbccaa2978205f9a4123ccee93cac5

[dweomer]

@carlossg the thing with tags is they are expressive whereas image digests are not. I realize that this allows for some movement underneath in the same way that a branch in git is merely a human-readable label representing the tip of a development stream. I accept that a tag is not a static thing.

That being said, the behavior of an official image should not change so drastically as to introduce backwards-incompatible breakage such as this one has. The baseline example of using docker.inside + Maven from Cloudbees very own documentation on the subject is now non-functional if you use the 3 or 3.3 image instead of 3.3.3:

docker.image('maven:3.3.3-jdk-8').inside {
  git '…your-sources…'
  sh 'mvn -B clean install'
}

As for jenkinsci/docker-workflow-plugin#25 and jenkinsci/docker-workflow-plugin#57 which purports to fix it: I see more value in the build output (from the workspace injected into the container) being r/w by the account that initiated the in-container "build" than in accommodating images/containers that only function correctly when run as root. I am perplexed as to why the latter seems to be favored over the former.

[dweomer]

@carlossg hoping that you are back from JenkinsConf and can review/apply my PR to mitigate this issue without negating the new-ish functionality for the /root/.m2 / MAVEN_CONFIG additions: #18

[carlossg]

#18 is merged

[jglick]

jenkinsci/docker-workflow-plugin#63 should have solved this without an image change.