Consistent resource management across actions and resource types

[AlessandroPatti]

Description of the feature request:

There are several way of defining and allocating resources for bazel:

• --local_cpu_resources and --local_ram_resources can be used to define the amount of cpu and ram available for bazel to consume. They both accept decimal values (float)
• --local_extra_resources can be used to model arbitrary abstract resources, also as floats
• The cpu:<n> tag (or execution info) can be used to request more cpus for an action (defaults to 1)
  • Only works for tests
  • Only allows integer values, but it is modelled internally as a double
• The resources:<resource>:<amount> can be used to request any amount of an arbitrary resource
  • Only applied to tests
  • Accepts float values
• As far as I understand, there is no way of requesting more memory (ram), even thou it is modelled internally as number of MB (as a double value)

I would like have fine-grained controll on all the resources types and all the actions, with an unified api, which stores all the values as doubles, something like:

• resources:<resource>:<amount> allows allocating any resource, including cpu and memory
• --local_resources=<resource>=<amount> can be used to specify the amount of any resource available for bazel to use. This would include cpu and memory
• --default_test_resources=<resource>=<small>,<medium>,<large>,<enormous> allows setting the default allocation for test based on the test size.
• --default_resources=<resource>=<amount> can be used to change the default allocation for all actions
• resources:<resource>:<amount> can be applied to any action (not test only)

Which category does this issue belong to?

Local Execution

What underlying problem are you trying to solve with this feature?

In large, multi language repositories, the default values for resource allocation might not be suitable for all targets/actions. When that happens, bazel ends up over (or under) scheduling tasks, which affects performance and stability. With fine grained controll over the resource allocation, the build can be tweaked according to its needs.

Which operating system are you running Bazel on?

N/A

What is the output of bazel info release?

release 6.3.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[jin]

Related: #10443

[DavidANeil]

This is not strictly related to this task, but just musing:

I wonder if assigned resources ought to be available to an action/test as an environment variable.
That way, guard rails can be imposed in user-land that if a resource is not assigned to the target, then it should not be used while executing.

As an example, I am imagining us using something like --local_resources=clipboard=1 and then resources:clipboard:1 on tests that access the clipboard.
Then we could add a hook in where we are accessing the system clipboard and fail fast if the test has not been assigned that resource.