Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop event bus message spans #3238

Merged
merged 2 commits into from
May 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
/**
* Copyright (c) 2022 Contributors to the Eclipse Foundation
*
* See the NOTICE file(s) distributed with this work for additional
* information regarding copyright ownership.
*
* This program and the accompanying materials are made available under the
* terms of the Eclipse Public License 2.0 which is available at
* http://www.eclipse.org/legal/epl-2.0
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.eclipse.hono.service.quarkus;

import java.util.List;

import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.context.Context;
import io.opentelemetry.sdk.trace.data.LinkData;
import io.opentelemetry.sdk.trace.samplers.Sampler;
import io.opentelemetry.sdk.trace.samplers.SamplingResult;

/**
* Sampler that drops spans based on a given list of span name prefixes.
*/
public class DropBySpanNamePrefixSampler implements Sampler {

private final Sampler sampler;
private final List<String> spanNamePrefixList;

/**
* Creates a new DropBySpanNamePrefixSampler.
*
* @param sampler Sampler to use if the span name didn't match the given prefixes.
* @param spanNamePrefixList The list of prefixes to match the span name against for the decision whether to drop
* the span.
*/
public DropBySpanNamePrefixSampler(final Sampler sampler, final List<String> spanNamePrefixList) {
this.sampler = sampler;
this.spanNamePrefixList = spanNamePrefixList;
}

@Override
public SamplingResult shouldSample(final Context parentContext, final String traceId, final String spanName, final SpanKind spanKind,
final Attributes attributes, final List<LinkData> parentLinks) {

for (final String spanNamePrefix : spanNamePrefixList) {
if (spanName.startsWith(spanNamePrefix)) {
return SamplingResult.drop();
}
}
return sampler.shouldSample(parentContext, traceId, spanName, spanKind, attributes, parentLinks);
}

@Override
public String getDescription() {
return sampler.getDescription();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@
*/
package org.eclipse.hono.service.quarkus;

import java.net.URI;
import java.util.List;

import io.opentelemetry.api.common.Attributes;
Expand Down Expand Up @@ -43,25 +42,14 @@ public DropHttpRequestSpansSampler(final Sampler sampler) {
public SamplingResult shouldSample(final Context parentContext, final String traceId, final String spanName, final SpanKind spanKind,
final Attributes attributes, final List<LinkData> parentLinks) {
if (spanKind.equals(SpanKind.SERVER)) {
final String httpTargetPath = getHttpTargetPath(attributes.get(SemanticAttributes.HTTP_TARGET));
if (httpTargetPath != null && !httpTargetPath.isBlank() && httpTargetPath.equals(spanName)) {
final String httpTarget = attributes.get(SemanticAttributes.HTTP_TARGET);
if (httpTarget != null && httpTarget.equals(spanName)) {
return SamplingResult.drop();
}
}
return sampler.shouldSample(parentContext, traceId, spanName, spanKind, attributes, parentLinks);
}

private String getHttpTargetPath(final String httpTarget) {
if (httpTarget != null) {
try {
return URI.create(httpTarget).getPath();
} catch (final IllegalArgumentException e) {
// ignore
}
}
return null;
}

@Override
public String getDescription() {
return sampler.getDescription();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,15 @@
*/
package org.eclipse.hono.service.quarkus;

import java.util.List;
import java.util.Optional;

import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.inject.Produces;
import javax.inject.Singleton;

import org.eclipse.hono.util.AuthenticationConstants;
import org.eclipse.hono.util.Constants;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand Down Expand Up @@ -52,7 +55,12 @@ Sampler sampler(final TracerRuntimeConfig tracerRuntimeConfig) {
// Dropping all requests spans also makes sure there are no liveness/readiness request spans created.
// Suppression of these spans via "quarkus.opentelemetry.tracer.suppress-non-application-uris" doesn't work
// if the non-application-root-path is the same as the overall root path.
return new DropHttpRequestSpansSampler(sampler);
sampler = new DropHttpRequestSpansSampler(sampler);
// drop spans for the given event bus message prefixes
Comment on lines 56 to +59
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we might want to actually use the non-application root path then. Given that we already decided that the 2.0.0 chart will not be backward compatible with Hono 1.x, I think that we could as well take advantage of the existing features then ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, we could do that. (The DropHttpRequestSpansSampler would still be needed though for now, as not only the non-application-root-path spans shall get dropped. Issue for the removal of that sampler is #3213.)
Changing the non-application-root-path and the healthcheck endpoints should be done in a separate PR then.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as not only the non-application-root-path spans shall get dropped.

Couldn't we simply remove the TracingHandler in favor of the standard functionality provided by the quarkus-opentelemetry extension?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is basically the goal of #3213 (see also #3205 (comment)).

It's a bit more than just removing the TracingHandler class. You mean including this in 2.0.0?
I began working on this but then stopped, waiting for the Quarkus update. I can have another look.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, if we want to use the non-application-root path for probes and metrics, then I think we should do it with 2.0.0. Not sure how this is related to getting rid of the TracingHandler, but if we can kill two birds with one stone, then I think we should do it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case, this PR here is mostly about the event bus spans. I've only included a small fix for the DropHttpRequestSpansSampler regarding the Quarkus update (and the changed span names there).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the non-application-root-path and the healthcheck endpoints should be done in a separate PR then.

I take it that we indeed want to change these paths with Hono 2.0.0 so that they include a non-empty non-application-root-path (e.g. q), right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With regards to the dropping of the healthcheck endpoint spans, the current Hono code is independent from the used non-application-root-path. As a custom sampler is needed in Hono (for using the SamplingPrioritySampler), dropping of these spans needs to be done manually anyway (until a solution for quarkusio/quarkus#25525 is implemented). So, mandating usage of the /q healthcheck endpoint prefix would only enable us to simplify the span-dropping logic a bit here.

In that sense, I have no strong view about switching to use the /q prefix. But I guess, as Quarkus is now used for all Hono components and we don't need Helm-chart backwards-compatibility here, we might as well stick to the Quarkus default paths.

return new DropBySpanNamePrefixSampler(
sampler,
List.of(Constants.EVENT_BUS_ADDRESS_NOTIFICATION_PREFIX, Constants.EVENT_BUS_ADDRESS_TENANT_TIMED_OUT,
AuthenticationConstants.EVENT_BUS_ADDRESS_AUTHENTICATION_IN));
}

private static Sampler getBaseSampler(final String samplerName, final Optional<Double> ratio) {
Expand Down