Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coroutines Dispatchers should have parameter to limit threads count for several usage scenarios #1961

Closed
sandijs-private opened this issue Apr 27, 2020 · 4 comments

Comments

@sandijs-private
Copy link

Right now kotlinx.coroutines.io.parallelism limits globally how many threads will be used.
Imagine the app which heavily auto processes taken pictures and can do that on "2 threads only" to not get out of the memory crashes.
And there are some other app parts, which use neural networks and would want to use 8 threads - each for one core.
Or there are data sync with 8 API calls which could happen in parallel.
Internally you already have such parameters - they simply are not accessible from the outside.
How about some Dispatchers.LimitedThreadPool(2) option besides already existing IO, Default, Main, Unconfined?
Then there would be several Dispatchers instances: Dispatchers.LimitedThreadPool(2) and Dispatchers.LimitedThreadPool(8)

@fvasco
Copy link
Contributor

fvasco commented Apr 27, 2020

Are you considered newFixedThreadPoolContext?

@sandijs-private
Copy link
Author

sandijs-private commented Apr 27, 2020

Good software design is when all has some consistency - one does not have to look at several places to find Dispatchers Pool to use. That same should be reflected into the documentation as well. Strategically, you should target newbies to get mass of the followers. Easy quality.

Looking more into the Future with Kotlin Native, it would be cool to have something like in Kubernetes - option to limit resources for one Coroutine, and ensure that if it dies, main process and remaining coroutines survive.

Memory limits
CPU limits
Threads count limits
etc

How to make Kotlin Native better than Go? (Golang)

What would make it superior?

If the Go value is Goroutines, then first focus of the attack should be there by providing things which they can't.

Coroutine resources limitations and isolation from the main process would be great place where to start! How about multi processes coroutines on the UNIXes?

That would allow jumping in gaming, banking, and other demanding systems/servers development by providing EASY tools to develop multi processes systems.

JVM Server paradigm versus UNIX Server paradigm?

Julia programming language parallelism stuff inspires even more: https://docs.julialang.org/en/v1/manual/parallel-computing/

@zach-klippenstein
Copy link
Contributor

It seems like what you really are trying to do here is limit the number of operations being performed concurrently. That can be done with higher-level coroutines tools (e.g. a worker pool that has a limited number of workers) and doesn't need to be done at the dispatcher or thread level.

@elizarov
Copy link
Contributor

This is a duplicate of #261

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants