Crash when building objc_library

[thii]

Description of the problem / feature request:

2021/08/17 20:46:00 Using unreleased version at commit cfc2d1d98f827787b80d9c211fc17f97b03c558b
Starting local Bazel server and connecting to it...
Analyzing: target //some:objc_target (40 packages loaded, 537 targets configured)
    currently loading: @bazel_tools//src/tools/launcher
FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'ConfiguredTargetKey{label=//some:objc_target, config=BuildConfigurationValue.Key[759ce837fa28c3e3aaa73b2dc2f899ac4be3297e9cf4724d9361269d6cba6b4b]}' (requested by nodes )
	at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:644)
	at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:382)
	at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: net.starlark.java.eval.Starlark$UncheckedEvalException: NullPointerException thrown during Starlark evaluation (//some:objc_target)
	at <starlark>.create_compilation_attributes(<builtin>:0)
	at <starlark>._build_common_variables(/virtual_builtins_bzl/common/objc/compilation_support.bzl:79)
	at <starlark>._objc_library_impl(/virtual_builtins_bzl/common/objc/objc_library.bzl:114)
Caused by: java.lang.NullPointerException
	at java.base/java.lang.String.replace(Unknown Source)
	at com.google.devtools.build.lib.rules.objc.ObjcStarlarkInternal.expandFlag(ObjcStarlarkInternal.java:142)
	at com.google.devtools.build.lib.rules.objc.ObjcStarlarkInternal.expandToolchainAndRuleContextVariables(ObjcStarlarkInternal.java:116)
	at com.google.devtools.build.lib.rules.objc.ObjcStarlarkInternal.createCompilationAttributes(ObjcStarlarkInternal.java:82)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at net.starlark.java.eval.MethodDescriptor.call(MethodDescriptor.java:162)
	at net.starlark.java.eval.BuiltinFunction.fastcall(BuiltinFunction.java:77)
	at net.starlark.java.eval.Starlark.fastcall(Starlark.java:619)
	at net.starlark.java.eval.Eval.evalCall(Eval.java:672)
	at net.starlark.java.eval.Eval.eval(Eval.java:489)
	at net.starlark.java.eval.Eval.execAssignment(Eval.java:109)
	at net.starlark.java.eval.Eval.exec(Eval.java:268)
	at net.starlark.java.eval.Eval.execStatements(Eval.java:82)
	at net.starlark.java.eval.Eval.execFunctionBody(Eval.java:66)
	at net.starlark.java.eval.StarlarkFunction.fastcall(StarlarkFunction.java:191)
	at net.starlark.java.eval.Starlark.fastcall(Starlark.java:619)
	at net.starlark.java.eval.Eval.evalCall(Eval.java:672)
	at net.starlark.java.eval.Eval.eval(Eval.java:489)
	at net.starlark.java.eval.Eval.execAssignment(Eval.java:109)
	at net.starlark.java.eval.Eval.exec(Eval.java:268)
	at net.starlark.java.eval.Eval.execStatements(Eval.java:82)
	at net.starlark.java.eval.Eval.execFunctionBody(Eval.java:66)
	at net.starlark.java.eval.StarlarkFunction.fastcall(StarlarkFunction.java:191)
	at net.starlark.java.eval.Starlark.fastcall(Starlark.java:619)
	at com.google.devtools.build.lib.analysis.starlark.StarlarkRuleConfiguredTargetUtil.buildRule(StarlarkRuleConfiguredTargetUtil.java:98)
	at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createRule(ConfiguredTargetFactory.java:368)
	at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createConfiguredTarget(ConfiguredTargetFactory.java:188)
	at com.google.devtools.build.lib.skyframe.SkyframeBuildView.createConfiguredTarget(SkyframeBuildView.java:1040)
	at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.createConfiguredTarget(ConfiguredTargetFunction.java:981)
	at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.compute(ConfiguredTargetFunction.java:368)
	at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:560)
	at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:382)
	at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

I'll submit an example later.

What operating system are you running Bazel on?

macOS

What's the output of bazel info release?

I used last_green Bazel.

[thii]

cc @oquenchil

[oquenchil]

Can you include the target declaration that causes the crash?

[thii]

The crash was with a custom rule (from https:/bazel-ios/rules_ios) that generates a header map, you can see an example with the rule source code here: https:/thii/objc_library_crash, and reproduce the crash with:

bazelisk build lib

[thii]

Okay, it wasn't related to the custom rule, but seems related to the location expansion in copts. (the copts in the example below should be a no-op.

objc_library(
    name = "lib",
    srcs = ["lib.m"],
    copts = ["-I$(rootpath lib.m)"],
)

[rsahara]

Do you still have this problem on the latest builds?
I remember that I had a similar problem with 5.0.0-pre.20211011.2, but it disappeared with 6.0.0-pre.20211025.1, I'm not sure why.

EDIT: sorry it's a mistake. We still have the same error. (I'm trying to workaround this by using an old bazel + some patches.)

[thii]

Hmm, we're still seeing the same crash with 6.0.0-pre.20211025.1.

[tinder-maxwellelliott]

@thii how are you able to get around this? Do you have a custom Bazel fork with the necessary apple silicon and cross tool changes?

This seems like a significant issue since Bazel 5 is more or less tied to getting native apple silicon builds working (at least how it currently appears to me)

[thii]

@tinder-maxwellelliott We only have a handful of objc_library targets, so we just skip using header maps for now to work around this. If you just want to support building for the simulator on Apple Silicon, you can try to patch Bazel 4.2.1 with the necessary changes:

• Enable Apple Silicon iOS simulator support via use of new CPU type. c1ea2d4
• Add ios_sim_arm64 crosstool support 101e526 (or use a custom crosstool)

[thii]

@oquenchil This seems to affect a lot of users especially who need to build mixed Obj-C & Swift modules. What do you think about adding the native implementation of Obj-C rules back and adding a flag to allow switching between the native and the Starlark implementation of them until the Starlark one becomes stable?

[brentleyjones]

@Wyverald I think this would count as a 5.0 release blocker.

[keith]

Is #14173 the same?

[jerrymarino]

Yeah it looks like #14173 is the same problem 👍 . cc @amberdixon who I believe has context on an in-progress fix / added test coverage to prevent this from regressing

[Wyverald]

@oquenchil also has a fix in flight already. Should be submitted tomorrow

[brentleyjones]

Can we re-open this until it's in the 5.0 rc branch?

[Wyverald]

Cherrypicked. (Tried out GitHub desktop, hope it didn't backfire o,o)

[thii]

The fix was reverted. Can we re-open this?