-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Single Dispatchers.IO is not enough - threads bulkheads is required #1273
Comments
@swarmshine take a look here #261 |
Thank you, @fvasco ! |
FWIW I think you can do this today:
It uses an internal coroutines API which may change when you upgrade, but I believe it does work! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem
Suppose that our application works with two databases that provides third-party blocking API:
And our application wants to publish two REST endpoints:
Suppose our second method starts to block for long time due to a bug.
After 64 invocation of second method there will be no free thread left in Dispatchers.IO.
As a result first method will start to block too.
Another case: suppose that our second database querying function not blocks forever.
Instead due to a performance problem in second database it starts to execute slowly.
This will increase latency not only for second REST method but also for first REST method too. And whole system performance will suffer due to one buggy second method.
One solution that I can think of is to explicitly separate thread pools for different methods:
This way problems in second method will not affect first method.
But we are loosing nice property of Dispatchers.IO: it prevents unnecessary context-switch.
In this example if there are no overloading of dispatchers queues the context switch will not occur.
With our fix with explicit dispatchers built on top of different executors context-switch will always occur:
Current state
Context switch reducing approach for IO and Deafult dispatchers internally implemented via
kotlinx.coroutines.scheduling.LimitingDispatcher
. Actually Dispatchers.IO and Dispatchers.Default both sits on top of a single dispatcher and simply limit amount of blocking and cpu intensive tasks.Proposal
Provide api to enable users to build lightweight limiting dispatchers that doesn't own any resources (its own threads) and simply provides a view of the original dispatchers.
or if we decided to use default IO initialized by max threads count of 64:
The text was updated successfully, but these errors were encountered: