-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x-pack/filebeat/input/entityanalytics/provider/okta: Rate limiting hangs #40106
Labels
Comments
chrisberkhout
added
bug
Team:Security-Service Integrations
Security Service Integrations Team
labels
Jul 4, 2024
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
Setting a negative rate in the distant past avoids generating new tokens for the period from now until diff --git a/x-pack/filebeat/input/entityanalytics/provider/okta/internal/okta/okta.go b/x-pack/filebeat/input/entityanalytics/provider/okta/internal/okta/okta.go
index 58495cbcd6..b33c7be7e6 100644
--- a/x-pack/filebeat/input/entityanalytics/provider/okta/internal/okta/okta.go
+++ b/x-pack/filebeat/input/entityanalytics/provider/okta/internal/okta/okta.go
@@ -444,6 +444,7 @@ func oktaRateLimit(h http.Header, window time.Duration, limiter *rate.Limiter) e
// estimate will be overwritten when we make the next
// permissible API request.
next := rate.Limit(lim / window.Seconds())
+ limiter.SetLimitAt(time.Time{}, rate.Limit(-1))
limiter.SetLimitAt(waitUntil, next)
limiter.SetBurstAt(waitUntil, burst)
return nil |
6 tasks
6 tasks
This was referenced Jul 24, 2024
Merged
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Around the time of a rate limit reset, the rate limiting code may set a negative target rate and wait forever before the next request.
The rate comes out negative if current time is after the reset time returned in response headers. If a negative rate is set at a time when no request budget has accumulated, it will not recover. How previous events and timing affect the outcome can be seen in this example code.
We can avoid setting a negative rate by changing
== 0
to<= 0
here.There are some other corrections that can be made to the rate limiting logic, listed below. Beyond correctness, there are improvements that could be made for better operability, fault tolerance and user feedback.
A similar set of changes should be considered in the CEL input and related Mito code (in OktaRateLimit and in DraftRateLimit).
Fix rate limiting logic
Improve operability
Improve fault tolerance and user feedback
The text was updated successfully, but these errors were encountered: