Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Uncaught exception java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider in dd-profiler-http-dispatcher #7643

Open
caramcc opened this issue Sep 18, 2024 · 1 comment
Open

Uncaught exception java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider in dd-profiler-http-dispatcher #7643

caramcc opened this issue Sep 18, 2024 · 1 comment

Comments

@caramcc
Copy link

caramcc commented Sep 18, 2024

I'm trying to get dd-trace-java set up for my JVM applications and ran into this error with FFI, hoping someone can help point me in the right direction:

[dd.trace 2024-09-18 19:33:24:713 +0000] [dd-profiler-http-dispatcher] ERROR java.util.concurrent.ThreadPoolExecutor$Worker - Uncaught exception java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider in dd-profiler-http-dispatcher
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
	at jnr.ffi.provider.InvalidProvider$1.loadLibrary(InvalidProvider.java:49)
	at jnr.ffi.LibraryLoader.load(LibraryLoader.java:420)
	at jnr.unixsocket.Native.<clinit>(Native.java:80)
	at jnr.unixsocket.UnixSocketChannel.<init>(UnixSocketChannel.java:101)
	at jnr.unixsocket.UnixSocketChannel.open(UnixSocketChannel.java:60)
	at datadog.common.socket.UnixDomainSocketFactory.createSocket(UnixDomainSocketFactory.java:27)
	at datadog.okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:241)
	at datadog.okhttp3.internal.connection.RealConnection.connect(RealConnection.java:167)
	at datadog.okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:258)
	at datadog.okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
	at datadog.okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
	at datadog.okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
	at datadog.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at datadog.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at datadog.okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
	at datadog.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at datadog.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at datadog.okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at datadog.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at datadog.okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)
	at datadog.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at datadog.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at datadog.okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
	at datadog.okhttp3.RealCall.execute(RealCall.java:93)
	at datadog.communication.ddagent.DDAgentFeaturesDiscovery.doDiscovery(DDAgentFeaturesDiscovery.java:150)
	at datadog.communication.ddagent.DDAgentFeaturesDiscovery.discoverIfOutdated(DDAgentFeaturesDiscovery.java:135)
	at datadog.communication.ddagent.DDAgentFeaturesDiscovery.discover(DDAgentFeaturesDiscovery.java:119)
	at datadog.communication.ddagent.SharedCommunicationObjects.featuresDiscovery(SharedCommunicationObjects.java:102)
	at datadog.trace.agent.common.writer.WriterFactory.createWriter(WriterFactory.java:81)
	at datadog.trace.agent.common.writer.WriterFactory.createWriter(WriterFactory.java:42)
	at datadog.trace.agent.core.CoreTracer.<init>(CoreTracer.java:662)
	at datadog.trace.agent.core.CoreTracer.<init>(CoreTracer.java:122)
	at datadog.trace.agent.core.CoreTracer$CoreTracerBuilder.build(CoreTracer.java:483)
	at datadog.trace.agent.tooling.TracerInstaller.installGlobalTracer(TracerInstaller.java:26)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at datadog.trace.bootstrap.Agent.installDatadogTracer(Agent.java:638)
	at datadog.trace.bootstrap.Agent.access$400(Agent.java:68)
	at datadog.trace.bootstrap.Agent$InstallDatadogTracerCallback.execute(Agent.java:521)
	at datadog.trace.bootstrap.Agent.start(Agent.java:347)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at datadog.trace.bootstrap.AgentBootstrap.agentmainImpl(AgentBootstrap.java:155)
	at datadog.trace.bootstrap.AgentBootstrap.agentmain(AgentBootstrap.java:72)
	at datadog.trace.bootstrap.AgentBootstrap.premain(AgentBootstrap.java:60)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:491)
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:503)
Caused by: java.lang.UnsatisfiedLinkError: could not get native definition for type `POINTER`, original error message follows: cannot determine CPU
	at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:253)
	at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
	at com.kenai.jffi.Type.resolveSize(Type.java:155)
	at com.kenai.jffi.Type.size(Type.java:138)
	at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:198)
	at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
	at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:77)
	at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:49)
	at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:73)
	at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:60)
	at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
	at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128)
	at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:341)
	at java.base/java.lang.Class.newInstance(Class.java:677)
	at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68)
	at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:57)
	at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
	at jnr.ffi.LibraryLoader.create(LibraryLoader.java:89)
	at jnr.unixsocket.Native.<clinit>(Native.java:76)
	... 46 more

Context:

  • Scala application
  • We configure the Datadog trace agent by setting JAVA_OPTS env variable that includes -javaagent:/usr/share/java/dd-java-agent.jar -Ddd.profiling.enabled=true -XX:FlightRecorderOptions=stackdepth=256 -Ddd.logs.injection=true (i.e. not using the sbt-datadog project I saw mentioned in a similar resolved issue, Datadog agent error to find FFI provider causes an application to crash #4083)
  • Application runs in pods in a k8s cluster, using Datadog agent configured via the Helm chart. We have DD APM working properly for Ruby and Python applications
  • arm64 architecture (I noticed this mentioned in a similar issue, Trace agent fails to start on arm64 nodes #6635, but that issue has been resolved.
  • OpenJDK 18 (from openjdk:18-slim-bullseye Docker image)
  • dd-java-agent version 1.39.0
  • I haven't explicitly installed libffi so I'm unsure what my Docker container is using (if there's an underlying libffi version requirement not mentioned in the docs)

Any advice about getting this working or how I can troubleshoot further is greatly appreciated!
@abhi92102
Copy link

abhi92102 commented Oct 11, 2024 •
edited
Loading

I am also facing the same issue in one of my Weblogic application and have set variables using JAVA_OPTIONS. It was working 3 weeks ago suddenly stopped working now and I see below error

[OkHttp http://localhost:8126/...] WARN com.datadog.profiling.uploader.ProfileUploader - Failed to upload profile to http://localhost:8126/profiling/v1/input java.io.IOException: canceled due to java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider (Will not log warnings for 5 minutes)
[dd.trace 2024-10-11 20:06:09:869 +0000] [dd-profiler-http-dispatcher] ERROR java.util.concurrent.ThreadPoolExecutor$Worker - Uncaught exception java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider in dd-profiler-http-dispatcher
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@caramcc @abhi92102