[Docker build] add all files of type

[robert2411]

Tell us about your request
It would be good for example for maven projects that use docker multistage build to have to option to add all files of a type to the docker build keeping the folder structure so in case of a maven multi module project all the pom.xml files so you don't have to add them one by one like:

copy pom.xml pom.xml
copy module1/pom.xml module1/pom.xml

This way the docker build cache can be used more efficient

Which service(s) is this request for?
Docker build

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Simplify the dockerfile and make it less error prone for big projects

Are you currently working around the issue?
by adding a line per pom.xml file in the dockerfile

Additional context
Add any other context or screenshots about the feature request here.

[hrajchert]

Same feature would be nice for node.js projects. Have the ability to copy all package.json of a project respecting the structure.

[PavelSosin-320]

Maven has it's own very good and very popular Docker plugin. If pom.xml will be added to the project the question is where .m2 folder will be created. This is not a part of the workspace. Actually, Maven build system has been dockerized a long time ago as a separate official, maintained docker images with different java versions, etc.

[PavelSosin-320]

I'm not sure that robert241 really wants it because Maven build system requires 2 configuration files, local cash, proxy configuration, etc. Next, what about CMake, NMP, MSVS build systems? In the DockerHub it is possible to find official Docker build-system images providing comprehensive solutions backed by official Build system supporters for all these cases. I suppose that ability to invoke the corresponding Docker container with proper parameters is the better option overall. All this stuff was developed for the DevOps automation,i.e. the invocation is always one-liner. Invocation of Docker container from Dev-container applies only one restriction - privileged mode for Dev-container.

[Fydar]

This would be incredibly powerful for .csproj files when building a .NET application.

The typical .NET Dockerfile will attempt to restore dependencies via NuGet in a separate layer to the build. This created a cached layer that speeds up builds when no .csproj files change.

Unfortunately, the auto-generated template from Visual Studio has to manually specify EVERY .csproj file and this required ugly and manual maintenance. This feature request would seriously improve the experience when working with .NET applications.

There are workarounds. What I do is I copy all of the .csproj files and then run a script which computes the path that they should have been using information from the .sln file.

I would love to see this implemented as an option to the COPY command.

COPY --preserve-structure **/*.csproj /src

[thaJeztah]

Given that this (potentially) involves changes in the Dockerfile specification, perhaps it'd be good to open a ticket for this in the BuildKit repository for discussion (if not ticket exists for this yet); @robert2411 perhaps you're interested in opening a ticket in that project as well? https:/moby/buildkit

let me also /cc @docker/build (perhaps they know if there's a ticket for this)

[Hronom]

+1, same needed for proper caching of maven dependencies e.g. pom.xml

[thaJeztah]

The --parents option was merged in BuildKit (see moby/buildkit#3001). The --parents option allows copying files and preserve their location. Missing parent directories will be created in the destination (in the dockerfile).

⚠️ This feature has not yet shipped in a stable release of the dockerfile syntax, and will be in the "labs" channel first (before being added to the stable syntax), but with that feature, this will be possible.

The example below uses the docker/dockerfile-upstream:master-labs syntax;that's a build from "master" (so use at your own discretion). There's no need to update Docker or BuildKit (any current version will work as long as you use the custom # syntax directive), Here's an example;

First creating a "project" (various pom.xml files in (sub)directories);

mkdir -p example && cd example
mkdir -p module1/sub module2/sub module3/sub
touch pom.xml module1/pom.xml module2/pom.xml module3/pom.xml module1/sub/pom.xml module2/sub/pom.xml module3/sub/pom.xml

Directory should now look like this;

tree
.
├── module1
│   ├── pom.xml
│   └── sub
│       └── pom.xml
├── module2
│   ├── pom.xml
│   └── sub
│       └── pom.xml
├── module3
│   ├── pom.xml
│   └── sub
│       └── pom.xml
└── pom.xml

7 directories, 7 files

Using a Dockerfile with the docker/dockerfile-upstream:master-labs syntax (note that this feature will ultimately move into the regular (docker/dockerfile:1 stable syntax));

# syntax=docker/dockerfile-upstream:master-labs

FROM alpine
RUN apk add --no-cache tree
WORKDIR example
COPY --parents *.xml **/*.xml .
RUN tree

Building the dockerfile (I'm using --progress=plain so that the output is visible);

docker build --progress=plain .
#0 building with "desktop-linux" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 185B done
#1 DONE 0.0s

#2 resolve image config for docker.io/docker/dockerfile-upstream:master-labs
#2 DONE 0.5s

...

#10 [4/5] COPY --parents *.xml **/*.xml .
#10 DONE 0.0s

#11 [5/5] RUN tree
#11 0.105 .
#11 0.107 ├── module1
#11 0.107 │   ├── pom.xml
#11 0.107 │   └── sub
#11 0.107 │       └── pom.xml
#11 0.107 ├── module2
#11 0.107 │   ├── pom.xml
#11 0.107 │   └── sub
#11 0.107 │       └── pom.xml
#11 0.107 ├── module3
#11 0.107 │   ├── pom.xml
#11 0.107 │   └── sub
#11 0.107 │       └── pom.xml
#11 0.107 └── pom.xml
#11 0.107
#11 0.107 7 directories, 7 files
#11 DONE 0.1s