Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI flake: podman image trust show: output is in random order #6764

Closed
edsantiago opened this issue Jun 24, 2020 · 3 comments · Fixed by #7338
Closed

CI flake: podman image trust show: output is in random order #6764

edsantiago opened this issue Jun 24, 2020 · 3 comments · Fixed by #7338
Assignees
@QiWang19
Labels
flakes Flakes from Continuous Integration kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@edsantiago
Copy link
Member

cirrus-flake-xref is reporting a lot of flakes in the podman image trust show test (see below).

Investigation shows that the output of podman image trust show comes out in random order. The test is expecting the following output, in this order:

default                        accept                                              
docker.io/library/hello-world  reject                                              
registry.access.redhat.com     signedBy  [email protected], [email protected]  https://access.redhat.com/webassets/docker/content/sigstore

...and is expecting accept, reject, and signed in lines 1, 2, and 3 respectively. However, often the output lines are in different order. This causes the test to fail, and if you get unlucky three times the entire CI run will fail.

One solution might be to fix podman image trust show to output lines in consistent order. Another might be to fix the test so it doesn't rely on order. I don't know enough to suggest which might be preferable.

Podman trust [It] podman image trust show
@github-actions
Copy link

A friendly reminder that this issue had no activity for 30 days.

@edsantiago
Copy link
Member Author

Still happening, mostly just singletons but every so often triplets causing CI flakes:

@edsantiago edsantiago added flakes Flakes from Continuous Integration kind/bug Categorizes issue or PR as related to a bug. and removed stale-issue labels Aug 14, 2020
@edsantiago
Copy link
Member Author

Removed stale-issue label, this is not stale. Triggered on #7317 : https://cirrus-ci.com/task/4997918144856064

edsantiago added a commit to edsantiago/libpod that referenced this issue Aug 17, 2020
@edsantiago
flake fix: podman image trust
8dc6b5c 
The output of 'podman image trust' is in random order; but
its e2e test was assuming a specific one. This caused flakes.

Fixes: containers#6764

Signed-off-by: Ed Santiago <[email protected]>
@edsantiago edsantiago mentioned this issue Aug 17, 2020
Merged
edsantiago added a commit to edsantiago/libpod that referenced this issue Aug 17, 2020
@edsantiago
flake fix: podman image trust
2f6bacf 
The output of 'podman image trust' is in random order; but
its e2e test was assuming a specific one. This caused flakes.

Fixes: containers#6764

Signed-off-by: Ed Santiago <[email protected]>
edsantiago added a commit to edsantiago/libpod that referenced this issue Aug 18, 2020
@edsantiago
flake fix: podman image trust
318336a 
The output of 'podman image trust' is in random order; but
its e2e test was assuming a specific one. This caused flakes.

Fixes: containers#6764

Signed-off-by: Ed Santiago <[email protected]>
edsantiago added a commit to edsantiago/libpod that referenced this issue Sep 14, 2020
@edsantiago
flake fix: podman image trust
ca23a3c 
The output of 'podman image trust' is in random order; but
its e2e test was assuming a specific one. This caused flakes.

Fixes: containers#6764

Signed-off-by: Ed Santiago <[email protected]>
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 22, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@QiWang19 QiWang19

Labels
flakes Flakes from Continuous Integration kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

flake fix: podman image trust edsantiago/libpod
2 participants
@edsantiago @QiWang19