ndk-stack does not work with 64-bit binaries

[DoDoENT]

This was also issue in r10.

When using 64-bit ARM target on 64-bit device (e.g. Samsung Galaxy S6) and triggering segfault, while attempting to catch stack trace with adb logcat | ndk-stack -sym ./obj/local/arm64-v8a/, I get "File format not recognized" error:

$ adb logcat | ndk-stack -sym ./obj/local/arm64-v8a/
********** Crash dump: **********
Build fingerprint: 'samsung/zerofltexx/zeroflte:6.0.1/MMB29K/G920FXXU3DPB8:user/release-keys'
pid: 10952, tid: 11448, name: Recognition  >>> com.microblink.test <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x18
Stack frame #00 pc 00000000000752cc  /data/app/com.microblink.test-1/lib/arm64/libBlinkRecognizer.so (Java_com_microblink_recognition_NativeRecognizerWrapper_initNativeRecognizers+144): Routine ndk-stack: ./obj/local/arm64-v8a//libBlinkRecognizer.so: File format not recognized

When forcing the same crash on arm7 target on same device, I get much more information (note the source file and line numbers):

$ adb logcat | ndk-stack -sym ./obj/local/armeabi-v7a/
********** Crash dump: **********
Build fingerprint: 'samsung/zerofltexx/zeroflte:6.0.1/MMB29K/G920FXXU3DPB8:user/release-keys'
pid: 12211, tid: 12459, name: Recognition  >>> com.microblink.test <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc
Stack frame #00 pc 000371cc  /data/app/com.microblink.test-2/lib/arm/libBlinkRecognizer.so (Java_com_microblink_recognition_NativeRecognizerWrapper_initNativeRecognizers+91): Routine Ref<protection::LicenseToken>::operator==(protection::LicenseToken const*) at /Users/dodo/Work/Microblink/android-core/core/CoreUtils/Source/Counted.hpp:191
Stack frame #01 pc 006f9e19  /data/app/com.microblink.test-2/oat/arm/base.odex (offset 0x4f1000) (long com.microblink.recognition.NativeRecognizerWrapper.initNativeRecognizers(long, long[], boolean, long)+140)
Stack frame #02 pc 006f9fbd  /data/app/com.microblink.test-2/oat/arm/base.odex (offset 0x4f1000) (long com.microblink.recognition.NativeRecognizerWrapper.llIIlIlIIl(long, long[], boolean, long)+88)
Stack frame #03 pc 006f7981  /data/app/com.microblink.test-2/oat/arm/base.odex (offset 0x4f1000) (void com.microblink.recognition.NativeRecognizerWrapper$1.run()+1636)
Stack frame #04 pc 0373da47  /system/framework/arm/boot.oat (offset 0x2f1e000)

I would be nice if ndk-stack could work with 64-bit binaries, since in development I only build native code to match ABI target of connected device (i.e. when I plug in SGS6, my gradle script builds native code only for AMR64, thus not wasting time on building native code for all supported platforms).

[slavasav]

Still the same problem with r13

[silentnuke]

@DanAlbert is there any updates on that?

[DanAlbert]

No, but we should prioritize this. We have a new hire coming soon, and this would make a good starter project.

[silentnuke]

[enh]

i suspect the best way forward here is to kill the existing ndk-stack and switch to the "stack" script we use internally (suitably adjusted to work with the NDK rather than the platform tools). anyone good at python can probably take stack, stack_core.py, and symbol.py from https://android.googlesource.com/platform/development.git/+/master/scripts/ and send us the diff.

obviously if you're on Windows there's probably a bit more work involved, .

other than this regular expression (which doesn't seem like it would lead to the error folks are seeing)...

static const char _frame_header[] = "\\#[0-9]+[ |\t]+[pc|eip]+:*[ |\t]+([0-9a-f]{8})*";

https://cs.corp.google.com/android/ndk/sources/host-tools/ndk-stack/ndk-stack-parser.c?l=74
...i took another look at ndk-stack and still don't see any obvious problem (unless somehow the libbfd it's linking against doesn't contain the LP64 arches, but i also didn't see where anything like that was configured).

anyway, anyone with a bit of time and some python ought to be able to switch us over to the platform tool (which even has tests!).

[silentnuke]

@DanAlbert is there any updates on that?

[DanAlbert]

No, unfortunately. We're still working on actually filling that new hire slot. It might happen in the r16 time frame, but almost certainly not for beta 1.

[DoDoENT]

Any updates/progress on that? Have you filled a new hire slot? Are there any workarounds I could use until official fix comes? I currently have a stack trace reported by a client on 64-bit samsung device and I am unable to symbolificate it using ndk-stack. Any help would be appreciated!

[enh]

see "anyone with a bit of time and some python ought to be able to switch us over to the platform tool (which even has tests!)" above. we should have someone starting next month, and what better introduction to the life than deleting ancient ndk-specific cruft and replacing it with the thing we use in the platform?

[DanAlbert]

Yep, should be able to make this happen for r16.

[rprichard]

FWIW: On my 64-bit Linux machine, ndk-stack already handles ARM64.

I see these lines in build-ndk-stack.sh:

if [ "$HOST_OS" != "darwin" -a "$DARWIN" != "yes" ]; then
    LDFLAGS=$LDFLAGS" -Wl,-gc-sections"
else
    # In darwin libbfd has to be built with some *linux* target or it won't understand ELF
    # Disable -Werror because binutils uses deprecated functions (e.g. sbrk).
    EXTRA_CONFIG="-target=arm-linux-androideabi --disable-werror"
fi

I think the Linux ndk-stack is being configured to work with host binaries, which are 64-bit. Maybe the 32-bit triple arm-linux-androideabi configures the macOS ndk-stack to handle only 32-bit binaries, so changing the target (to aarch64-linux-android?) might fix this issue on macOS?

I wonder about the Windows ndk-stack.

I'm still looking at replacing ndk-stack with the platform stack script, though.

[DanAlbert]

Yes.

[luoqii]

cool!!!! r16 has fixed this.

[hyUrbanManager]

r16b can do it