Demo of doing an equivalent build process without special plugins

[jglick]

For discussion, not merge. CC @HRMPW

[kuops]

*************** Installing a local Docker Registry Service for the demo ***************
***************            Please sit tight for a minute                ***************
Unable to find image 'registry:docker-workflow-demo' locally
docker: Error response from daemon: manifest for registry:docker-workflow-demo not found.
See 'docker run --help'.
***************         Docker Registry Service running now             ***************
"docker rmi" requires at least 1 argument(s).
See 'docker rmi --help'.

Usage:  docker rmi [OPTIONS] IMAGE [IMAGE...]

Remove one or more images

[jglick]

Just a warning which can be ignored.

[jglick]

This part is a bit awkward—you would like archiveArtifacts to be able to grab the file straight out of the container. Theoretically there could be a wrapper step which sets a contextual FilePath “workspace” to the container’s / (or some other directory of your choice), allowing nested steps to transparently operate in the container’s filesystem.

Unfortunately the current design of FilePath makes this impossible: it is not an extensible VFS, but rather a final class with two implementations basically selected by a switch: local File, or remote File using Channel calls. There is no place for a plugin to create a completely alternate implementation. And you cannot create a Channel into the container without running slave.jar inside it, which would require it have some JRE installed.

The Kubernetes plugin support for Pipeline, and IIRC the docker-slaves plugin, try to work around this by running a little pod of containers using shared volumes, and running slave.jar inside a designated one. That is a bit heavier weight, though, since it requires running an extra JVM process, and you have to be prepared to designate interesting parts of your “real” container as volumes.

Another trick would be to implement VirtualChannel from scratch to handle selected FilePath callables using completely different implementations, but this would be extremely fragile without a lot of work in core.

[jglick]

This is now easier using --output in Buildkit.

[ndeloof]

Things are even simpler now with multi-staged docker builds, and this all script can be handled by a single Dockerfile.

[jglick]

As I wrote elsewhere in the PR, I did actually try to do this with a multistage build, but soon found that I could not mount the Maven repository cache as a volume, which kills performance. Would be possible to prepopulate a cache as a base layer, though this does not easily allow for sharing caches across images. If you have any better ideas, let me know.

[ndeloof]

My experience is that it's way simpler to setup a maven proxy, which will offer high-speed downloads from localhost

[jglick]

Yes, that is an option, though you would need to do some work to make the proxy URL be defined as an ENV. For purposes of a demo like this, you would need to set up such a proxy as part of the demo I guess.

[jglick]

I could not mount the Maven repository cache as a volume

This has since been implemented in Buildkit.

[jglick]

Also, Dagger would now be a reasonable choice: dagger/dagger#1677 (comment)