Implement the bootc provision plugin
#3161
Conversation
ckyrouac
requested review from
psss,
lukaszachy,
happz,
thrix and
janhavlin
as code owners
|
@cgwalters fyi |
|
There was a problem hiding this comment.
I'm still trying to speedrun reading/learning all things bootc (and tmt provisioning plugins), but fwiw, looks cool to me.
@happz about the container plugin support - Fedora docs says:
for fully-fledged tests it is not recommended to run a bootable container via, for instance, podman-run. One reason among others is that the filesystem is writable when being executed as an OCI container while most of the filesystem is mounted read-only on a deployed bootc system. That means the running container behaves differently than a deployed system. Yet, if you desire to run some quick tests it is recommended to run the container in detached mode.
From what I understand, podman-bootc could be pretty cool to use, once available.
I agree, it's not a perfect 1:1 substitution, but, exactly: for quick tests or basic test development, it may give me results faster than VM. I for one work on binutils and C/C++ toolchain in general, and my area of focus is fairly simple - compile this, run |
Thank you for the insight in your development process. I hope I can see it in more detail one day. |
ckyrouac
force-pushed
the
bootc-provision
branch
from
641a411
to
78e86f3
Compare
I think the existing container plugin will handle this case without any additional code. The bootc image is just another container that can be run like a typical image. |
ckyrouac
force-pushed
the
bootc-provision
branch
2 times, most recently
from
e3058f9
to
44ff728
Compare
|
@happz This is ready for a review. I added some docs, tests, and code to cleanup the container images. |
ckyrouac
force-pushed
the
bootc-provision
branch
from
44ff728
to
8cb1c10
Compare
|
@happz thanks for the review! I believe I addressed all your suggestions. |
Prep for LBI install tests. This is temporary until the plugin is released upstream by tmt. teemtee/tmt#3161 Signed-off-by: Chris Kyrouac <[email protected]>
This creates a new provision plugin that is built on top of the existing TestCloud (virtual) plugin. It adds new parameters to pass a Containerfile or container image. The plugin will then build a container image (if necessary) then build a bootc disk image from the container image using bootc image builder. Currently, bootc requires podman to be run as root when building a disk image. This is typically handled by running a podman machine as root. An additional parameter "add-deps" toggles building a derived container image with the tmt test requirements. Signed-off-by: Chris Kyrouac <[email protected]>
ckyrouac
force-pushed
the
bootc-provision
branch
from
707f5a8
to
28e37f6
Compare
|
@happz let's rebase and run the pipeline here? |
thrix
added
the
priority | must
|
We agreed to focus on getting this in 1.38, the due date for finishing is 24th October. |
|
thanks for looking at this and planning for 1.38. Is there anything I can do to help move it along? |
psss
changed the title
bootc provision plugin
@ckyrouac not really, the team is overhelmed with reviews. I am getting validation errors, I assume the jsonschema needs an update for the new provision plugin: |
|
/packit build |
| """ Build the bootc disk from a container image using bootc image builder """ | ||
| self._logger.debug("Building bootc disk image") | ||
| tmt.utils.Command( | ||
| "podman", "run", "--rm", "--privileged", |
There was a problem hiding this comment.
Whould this not run under root user for it to work? I get this error on my laptop when trying to run it:
❯ pwd
/var/home/mvadkert/git/github.com/teemtee/tmt/tests/provision/bootc/data
❯ tmt run -vvv -e TMT_BOOTC_CONTAINERFILE_RUNDIR=.
/var/home/mvadkert/.local/share/tmt/run-002
warn: /containerfile_includes_deps:provision - {'how': 'bootc', 'add-deps': False, 'containerfile': '$TMT_BOOTC_CONTAINERFILE_RUNDIR/includes_deps.containerfile', 'containerfile-workdir': '.', 'disk': 20} is not valid under any of the given schemas
warn: /containerfile_needs_deps:provision - {'how': 'bootc', 'add-deps': True, 'containerfile': '$TMT_BOOTC_CONTAINERFILE_RUNDIR/needs_deps.containerfile', 'containerfile-workdir': '.', 'disk': 20} is not valid under any of the given schemas
warn: /image_includes_deps:provision - {'how': 'bootc', 'add-deps': False, 'containerimage': 'localhost/tmt-bootc-includes-deps', 'disk': 20} is not valid under any of the given schemas
warn: /image_needs_deps:provision - {'how': 'bootc', 'add-deps': True, 'containerimage': 'localhost/tmt-bootc-needs-deps', 'disk': 20} is not valid under any of the given schemas
Found 4 plans.
/containerfile_includes_deps
discover
how: shell
order: 50
summary: 1 test selected
/booted image
provision
queued provision.provision task #1: default-0
provision.provision task #1: default-0
how: bootc
order: 50
name: default-0
how: bootc
image: file:///var/home/mvadkert/.local/share/tmt/run-002/containerfile_includes_deps/provision/default-0/qcow2/disk.qcow2
containerfile: ./includes_deps.containerfile
add deps: False
memory: 2048 MB
disk: 20 GB
fail: Command 'podman run --rm --privileged -v /var/lib/containers/storage:/var/lib/containers/storage --security-opt label=type:unconfined_t -v /var/home/mvadkert/.local/share/tmt/run-002/containerfile_includes_deps/provision/default-0:/output quay.io/centos-bootc/bootc-image-builder:latest build --type qcow2 --local localhost/tmtbase-1729527982' returned 1.
finish
warn: Nothing to finish, no guests provisioned.
plan failed
The exception was caused by 1 earlier exceptions
Cause number 1:
provision step failed
The exception was caused by 1 earlier exceptions
Cause number 1:
Command 'podman run --rm --privileged -v /var/lib/containers/storage:/var/lib/containers/storage --security-opt label=type:unconfined_t -v /var/home/mvadkert/.local/share/tmt/run-002/containerfile_includes_deps/provision/default-0:/output quay.io/centos-bootc/bootc-image-builder:latest build --type qcow2 --local localhost/tmtbase-1729527982' returned 1.
stderr (8/8 lines)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Trying to pull quay.io/centos-bootc/bootc-image-builder:latest...
Getting image source signatures
Copying blob sha256:311336741819859e92064b81199a47397788d736d764f561a89cba9583600387
Copying blob sha256:35056b112ca2a75be2e91b1076785d9712b1790aea49ebfd61f1799d090d9057
Copying blob sha256:2edf720f10afc7137e2387bdb3ba800eec2602ac16da698ec19642a4d6fbc4dd
Copying config sha256:767a10b2f63a72d479d9b102c2016790c63b045f6b9533c639fed605c299ee14
Writing manifest to image destination
2024/10/21 16:27:47 error: cannot validate the setup: this command must be run in rootful (not rootless) podman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
per:
https:/osbuild/bootc-image-builder?tab=readme-ov-file#building
And we will need to put it under the --feeling-safe option.
Personally I feel this might a bit limit the usability here, I am wondering if it would not make more sense to spin a VM and run this command in it, but yeah, that will make this a lot harder to do, but avoid the root requirement of this feature.
There was a problem hiding this comment.
We can do that later, any idea if this privileged root requirement will ever go away?
There was a problem hiding this comment.
currently, creating the bootc disk requires a loopback mount which is why root is required. I don't think that requirement will go away anytime soon, although it's a common pain point so it might happen sooner rather than later.
Personally I feel this might a bit limit the usability here, I am wondering if it would not make more sense to spin a VM and run this command in it
A common pattern is to run podman-machine (a VM) to enable rootful podman. Then default all podman commands to use the machine, e.g.
podman machine init --rootful --disk-size 200 --memory 8192 --cpus 8 -v /var/tmp/tmt:/var/tmp/tmt -v $HOME:$HOME
export CONTAINER_CONNECTION=podman-machine-default-root
podman run ...
There was a problem hiding this comment.
this is documented in the PR here: https:/teemtee/tmt/pull/3161/files#diff-f05fc41435a3de82fd67c56c9bb94d0b781ddaf4182fdf907b4d02ce3531476eR129
This creates a new provision plugin that is built on top of the existing TestCloud (virtual) plugin. It adds new parameters to pass a Containerfile or container image. The plugin will then build a container image (if necessary) then build a bootc disk image from the container image using bootc image builder. Currently, bootc requires podman to be run as root when building a disk image. This is typically handled by running a podman machine as root.
An additional parameter "add-deps" toggles building a derived container image with the tmt test requirements.
Resolves #3013
Pull Request Checklist
This is a work in progress. I'm opening this PR early to get feedback on the high level design. I will add tests, docs, etc. after we solidify the higher level design.
If you want to try running the code here is an example fmf plan: