Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Push multi-platform images together in one go (i.e., push manifest list instead of manifest) #2523

Closed
tqh opened this issue Jun 11, 2020 · 8 comments · Fixed by #2734
Closed
Milestone

Comments

@tqh
Copy link

tqh commented Jun 11, 2020

docker buildx allows building for multiple archs (arm64, amd64) in one go. It would be nice if jib could do the same.

See https:/docker/buildx#building-multi-platform-images for more info.

@loosebazooka
Copy link
Member

So they way we generally see it is that multiplatform images make sense more for base image developers than they do for application devs (<-- jib is geared towards creating containers for application devs at the moment). Can you explain why you would want multiple platform images for the same application, it will help us rationalize this?

Anyway, we don't have great support for picking images from a manifest list based on architecture at the moment except for explicitly chosing based on digest (https:/GoogleContainerTools/jib/blob/master/docs/faq.md#how-do-i-specify-a-platform-in-the-manifest-list-or-oci-index-of-a-base-image).

I think @jonjohnsonjr or @mattmoor had some ideas for doing this with https:/google/ko, where if they encounter a manifest list, they just build the application for all platforms in the list? but I'm not sure where they got with it.

@chanseokoh
Copy link
Member

chanseokoh commented Jun 11, 2020

I've seen a use-case like this, where the user normally develops on an amd64 k8s while deploying the app onto a Raspberry PI cluster as a final target platform.

Because Java is platform-independent, I think it's a good candidate language for those who want to target multiple architectures/OSes; all you have to do is to pick a different base image containing a JVM.

As a workaround, @tgh can run Jib multiple times to build and push multiple arch images (which I think is a good alternative, as consecutive Jib builds should be super fast), but the problem is that these images that only differ in architectures/OSes should use different tags. (Not to mention that it's cumbersome to specify digests until we fix #1567.) That is, if Jib doesn't push multiple arch images as a bundle in one go as @tqh suggested, the second image push (say, arm64) with the same tag will just overwrite the first amd64 image. (Precisely speaking, the first amd64 image will still be in the registry of course, but the tag will be changed to point to arm64.) OTOH, when buildx pushes multiple arch images in one go, the tag in the registry becomes a manifest list, containing both amd64 and arm64.

I think @jonjohnsonjr or @mattmoor had some ideas for doing this with https:/google/ko, where if they encounter a manifest list, they just build the application for all platforms in the list? but I'm not sure where they got with it.

Perhaps our configuration could accept multiple arch/OS values for the base image when fixing #1567? That said, I now think perhaps we should accept a list of arch/OS in the Build Plan and the Build File specs?

@chanseokoh chanseokoh changed the title Multi-platform builds Push multi-platform images together in one go Jun 11, 2020
@chanseokoh chanseokoh changed the title Push multi-platform images together in one go Push multi-platform images together in one go (push manifest list instead of manifest) Jun 11, 2020
@chanseokoh chanseokoh changed the title Push multi-platform images together in one go (push manifest list instead of manifest) Push multi-platform images together in one go (i.e., manifest list instead of manifest) Jun 11, 2020
@tqh
Copy link
Author

tqh commented Jun 12, 2020

Our motivation is pretty much running on Arm64 and/or AMD64 Servers, but developers having AMD64 hw.

JIB is amazing, and since Debian and AdoptOpenJDK images already are multi-platform, this is the last step to have a very nice developer workflow.

@chanseokoh chanseokoh changed the title Push multi-platform images together in one go (i.e., manifest list instead of manifest) Push multi-platform images together in one go (i.e., push manifest list instead of manifest) Jul 23, 2020
@chanseokoh
Copy link
Member

chanseokoh commented Aug 11, 2020

FYI, #1567 is fixed, so now you can configure a desired platform. However, it only supports selecting a single platform as an incubating feature, and we are continuing to work on this issue to push a manifest list with multiple platforms in one go.

@chanseokoh
Copy link
Member

#2734 removed the feature lock and enabled the multi-platform (multi-arch) support. When multiple platforms are specified, Jib builds and pushes all the images from the specified platforms and then finally creates and pushes a manifest list (also known as a fat manifest).

@mpeddada1
Copy link
Contributor

@tqh We've released Jib 2.6.0, which now enables multi-platform image building. Please take a look at the FAQ for more info on this feature.

@tqh
Copy link
Author

tqh commented Oct 8, 2020

Thank you, I started using 2.6.0 already, but have not tried the <platforms> yet. I am very impressed that it got implemented so quickly, and JIB is awesome!

@barthanssens
Copy link

Currently experimenting with 2.6 for eclipse-rdf4j/rdf4j#2654, multi-platform is a seriously cool feature ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants