Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add byobject filter on nodes #2888

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Add byobject filter on nodes #2888

wants to merge 4 commits into from

Conversation

GnatorX
Copy link

@GnatorX GnatorX commented Apr 18, 2024

What type of PR is this?

improvement

Which issue does this PR fix?:

#2887

What does this PR do / Why do we need it?:
This PR adds a filter to reduce the number of node object VPC CNI watches for to just the node it is managing.

Testing done on this change:

Tested on our cluster of 3-4k node by @dl-stripe

Will this PR introduce any new dependencies?:

No
Will this break upgrades or downgrades? Has updating a running cluster been tested?:
No.

Does this change require updates to the CNI daemonset config files to work?:

No

Does this PR introduce any user-facing change?:

No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@GnatorX GnatorX closed this Apr 29, 2024
@GnatorX GnatorX reopened this Apr 29, 2024
@orsenthil
Copy link
Member

Closing per the discussion here - #2887

@dl-stripe
Copy link

A couple more datapoints in favor of this change. We have a few large node count clusters with a higher rate of churn for these nodes. The number of Node events pushed to the daemonset was causing excesive bandwidth usage. Adding the filter drastically reduced the number of events distributed to the system. Similarly the number of bytes pushed fell drastically (sharing a screenshot without absolute numbers, happy to share in private, but you can see the relative drop in bytes processed).

sum by (cluster, kind) (rate(apiserver_watch_events_total{kind="Node"}[5m]))

image
image

@jayanthvn jayanthvn reopened this Oct 11, 2024
@jayanthvn
Copy link
Contributor

@GnatorX - The PR is still in draft mode. Please feel free to move to review. Also can you run make format ?

@GnatorX GnatorX marked this pull request as ready for review October 11, 2024 03:01
@GnatorX GnatorX requested a review from a team as a code owner October 11, 2024 03:01
@GnatorX
Copy link
Author

GnatorX commented Oct 11, 2024

ya i can do that

@GnatorX
Copy link
Author

GnatorX commented Oct 11, 2024

ran make format

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants