Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to configure MaxConcurrentReconciles for our controllers #623

Conversation

SaschaSchwarze0
Copy link
Member

Changes

With the Git validation being added to the build reconciliation, we are observing increases in the time it takes to reconcile a build. This time is critical at the startup of the controller when all builds are reconciled. New builds that are getting created will then not be reconciled within a reasonable time. Example:

With 1000 builds in the system that do not use Git validation, we see the controller reconciling all of them in a little more than a minute. With Git validation enabled, it takes more than six minutes.

In this PR, I am exposing the MaxConcurrentReconciles controller setting as environment variable for our four controllers. The default is still 1.

Submitter Checklist

  • Includes tests if functionality changed/was added
  • Includes docs if changes are user-facing
  • Set a kind label on this PR
  • Release notes block has been filled in, or marked NONE

Release Notes

Exposing MaxConcurrentReconciles configuration of controllers to allow increasing them from the default (1) to be able to handle a larger amount of objects in the system

@SaschaSchwarze0 SaschaSchwarze0 added the kind/feature Categorizes issue or PR as related to a new feature. label Feb 26, 2021
@openshift-ci-robot openshift-ci-robot added the release-note Label for when a PR has specified a release note label Feb 26, 2021
@SaschaSchwarze0
Copy link
Member Author

Hm, the sanity check failed in files I did not change. Correcting this in a separate commit.

@SaschaSchwarze0 SaschaSchwarze0 force-pushed the sascha-configure-max-concurrent-reconciles branch from 060e617 to cce44a2 Compare February 26, 2021 12:38
@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 28, 2021
docs/configuration.md Outdated Show resolved Hide resolved
@zhangtbj
Copy link
Contributor

zhangtbj commented Mar 1, 2021

Hi Sascha,

Please fix the code conflict.

@SaschaSchwarze0 SaschaSchwarze0 force-pushed the sascha-configure-max-concurrent-reconciles branch from cce44a2 to fddc792 Compare March 1, 2021 08:39
@openshift-ci-robot openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 1, 2021
@gabemontero
Copy link
Member

/approve

@zhangtbj feel free to put a slash lgtm in this PR if you happy with @SaschaSchwarze0 's responses to your comments and we can merge this one

fwiw LGTM

@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gabemontero

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 1, 2021
Comment on lines 106 to 119
Controllers: Controllers{
Build: ControllerOptions{
MaxConcurrentReconciles: 1,
},
BuildRun: ControllerOptions{
MaxConcurrentReconciles: 1,
},
BuildStrategy: ControllerOptions{
MaxConcurrentReconciles: 1,
},
ClusterBuildStrategy: ControllerOptions{
MaxConcurrentReconciles: 1,
},
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A quick question, is that possible to skip this default setting here? And let framework to set default values for us, like before.

We manually set default to 1 for all, but I am not sure if the framework change the default value in future, then they will be mismatch.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks better to have a check by using zero.

Thanks Sascha!

@SaschaSchwarze0 SaschaSchwarze0 force-pushed the sascha-configure-max-concurrent-reconciles branch from fddc792 to 9389ea5 Compare March 2, 2021 09:27
@zhangtbj
Copy link
Contributor

zhangtbj commented Mar 2, 2021

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Mar 2, 2021
@openshift-merge-robot openshift-merge-robot merged commit 9b1cc8a into shipwright-io:master Mar 2, 2021
@SaschaSchwarze0 SaschaSchwarze0 deleted the sascha-configure-max-concurrent-reconciles branch March 2, 2021 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. release-note Label for when a PR has specified a release note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants