Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jaeger http thrift #3081

Merged
merged 10 commits into from
Dec 20, 2019
Merged

Jaeger http thrift #3081

merged 10 commits into from
Dec 20, 2019

Conversation

axw
Copy link
Member

@axw axw commented Dec 19, 2019

Add an HTTP handler, muxer, and server, in beater/jaeger for accepting Thrift-encoded trace data over HTTP. Refactor beater/jaeger.GRPCServer into Server, which now encapsulates both gRPC and HTTP servers.

Move beater/api/jaeger code into beater/jaeger, which is the only user of GRPCCollector. If the beater/jaeger code grows significantly, we might consider having subpackages like beater/jaeger/grpc, beater/jaeger/http, etc.

Config has been updated to accommodate both gRPC and HTTP. Config now looks like this:

jaeger:
  grpc:
    # Set to true to enable the Jaeger gRPC collector service.
    enabled: false
    # Defines the gRPC host and port the server is listening on.
    # Defaults to the standard Jaeger gRPC collector port 14250.
    host: "localhost:14250"
  http:
    # Set to true to enable the Jaeger HTTP collector endpoint.
    enabled: false
    # Defines the HTTP host and port the server is listening on.
    # Defaults to the standard Jaeger HTTP collector port 14268.
    host: "localhost:14268"

Tests have been simplified to use mock TraceConsumers, rather than relying on the concrete processor/otel.Consumer. They're integrating less now; we can test the whole process in system tests.

I regenerated the TLS certificates under testdata, as the CA certificate did not appear to be related to the certificate. TLS wasn't being enabled correctly before - that's been fixed now, with tests added that prove it's enabled.

Closes #3008

@axw axw added the jaeger label Dec 19, 2019
@axw axw requested a review from simitt December 19, 2019 08:35
@codecov-io
Copy link

codecov-io commented Dec 19, 2019

Codecov Report

Merging #3081 into jaeger will increase coverage by 0.18%.
The diff coverage is 84.24%.

@@            Coverage Diff             @@
##           jaeger    #3081      +/-   ##
==========================================
+ Coverage   79.27%   79.45%   +0.18%     
==========================================
  Files         101      103       +2     
  Lines        5279     5365      +86     
==========================================
+ Hits         4185     4263      +78     
- Misses       1094     1102       +8
Impacted Files Coverage Δ
beater/jaeger/grpc.go 100% <100%> (+17.24%) ⬆️
beater/api/mux.go 81.96% <100%> (ø) ⬆️
beater/request/context_pool.go 100% <100%> (ø)
beater/server.go 73.91% <33.33%> (+3.07%) ⬆️
beater/jaeger/server.go 80% <80%> (ø)
beater/jaeger/common.go 80% <80%> (ø)
beater/config/jaeger.go 89.47% <84.61%> (+3.75%) ⬆️
beater/jaeger/http.go 91.07% <91.07%> (ø)
... and 1 more

Copy link
Contributor

@simitt simitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks great!

Only found some typo, for which it is concerning that data are properly decoded with the typo.

beater/config/jaeger.go Outdated Show resolved Hide resolved
beater/config/jaeger.go Outdated Show resolved Hide resolved
The certificates weren't being exercised in the
tests, and when I changed the tests to use them
the CA wouldn't verify the certificate. Regenerated
using mkcert.
Shuffle existing Jaeger config around a little
bit to make room for HTTP config.
Add an HTTP handler/muxer, for accepting
Thrift-encoded trace data over HTTP.

Refactor GRPCServer into Server, which
encapsulates both gRPC and HTTP servers.

Move beater/api/jaeger code into beater/jaeger,
which is the only user of GRPCCollector.

Simplify tests to use a mock TraceConsumer,
rather than sending through the concrete
processor/otel.Consumer.

Fix TLS-related tests to actually use TLS,
and add tests that prove it's enabled.
And add approvals for Thrift-over-HTTP.
@axw
Copy link
Member Author

axw commented Dec 20, 2019

@simitt I moved the approvals test into beater/jaeger as well, inside server_test.go, in ad31536. The test now also runs through the HTTP/Thrift endpoint.

@axw axw merged commit 77d2e38 into elastic:jaeger Dec 20, 2019
@axw axw deleted the jaeger-http-thrift branch December 20, 2019 08:13
axw added a commit that referenced this pull request Jan 9, 2020
* Integrate Jaeger gRPC collector (#2976)

Add support for Jaeger gRPC Trace Intake Collector. The gRPC endpoint collects monitoring metrics and supports TLS communication, by reusing the `apm-server.ssl.*` configuration. By default the gRPC endpoint is disabled.

closes #2962 

Co-Authored-By: Andrew Wilkins <[email protected]>

* [Jaeger] Add otel consumer converting batches to Elastic APM events (#3066)

Add consumer converting incoming otel batches to Elastic APM format.
Add integration tests covering incoming gRPC requests being transformed to beat events.

partially implements #3307

* Jaeger http thrift (#3081)

Add an HTTP handler, muxer, and server, in beater/jaeger for accepting Thrift-encoded trace data over HTTP. Refactor beater/jaeger.GRPCServer into Server, which now encapsulates both gRPC and HTTP servers.

Move beater/api/jaeger code into beater/jaeger, which is the only user of GRPCCollector. If the beater/jaeger code grows significantly, we might consider having subpackages like beater/jaeger/grpc, beater/jaeger/http, etc.

* [jaeger] Convert Timeevents to errors (#3085)

* [jaeger] Convert Timeevents to errors

Parse Timeevents from Jaeger spans and convert to elastic error events
if they describe an error.

Fixes #3007

* Add experimental flag to Jaeger integration (#3121)

* tests/system: system test for Jaeger Thrift/HTTP (#3114)

* tests/system: system test for Jaeger Thrift/HTTP

* tests/system: system test for Jaeger gRPC

* processor/otel: update approvals

Co-authored-by: Silvia Mitter <[email protected]>
simitt pushed a commit to simitt/apm-server that referenced this pull request Jan 14, 2020
Add an HTTP handler, muxer, and server, in beater/jaeger for accepting Thrift-encoded trace data over HTTP. Refactor beater/jaeger.GRPCServer into Server, which now encapsulates both gRPC and HTTP servers.

Move beater/api/jaeger code into beater/jaeger, which is the only user of GRPCCollector. If the beater/jaeger code grows significantly, we might consider having subpackages like beater/jaeger/grpc, beater/jaeger/http, etc.
simitt pushed a commit to simitt/apm-server that referenced this pull request Jan 14, 2020
Add an HTTP handler, muxer, and server, in beater/jaeger for accepting Thrift-encoded trace data over HTTP. Refactor beater/jaeger.GRPCServer into Server, which now encapsulates both gRPC and HTTP servers.

Move beater/api/jaeger code into beater/jaeger, which is the only user of GRPCCollector. If the beater/jaeger code grows significantly, we might consider having subpackages like beater/jaeger/grpc, beater/jaeger/http, etc.
simitt added a commit that referenced this pull request Jan 14, 2020
* Integrate Jaeger gRPC collector (#2976)

Add support for Jaeger gRPC Trace Intake Collector. The gRPC endpoint collects monitoring metrics and supports TLS communication, by reusing the `apm-server.ssl.*` configuration. By default the gRPC endpoint is disabled.

closes #2962

Co-Authored-By: Andrew Wilkins <[email protected]>

* [Jaeger] Add otel consumer converting batches to Elastic APM events (#3066)

Add consumer converting incoming otel batches to Elastic APM format.
Add integration tests covering incoming gRPC requests being transformed to beat events.

partially implements #3307

* Jaeger http thrift (#3081)

Add an HTTP handler, muxer, and server, in beater/jaeger for accepting Thrift-encoded trace data over HTTP. Refactor beater/jaeger.GRPCServer into Server, which now encapsulates both gRPC and HTTP servers.

Move beater/api/jaeger code into beater/jaeger, which is the only user of GRPCCollector. If the beater/jaeger code grows significantly, we might consider having subpackages like beater/jaeger/grpc, beater/jaeger/http, etc.

* [jaeger] Convert Timeevents to errors (#3085)

* [jaeger] Convert Timeevents to errors

Parse Timeevents from Jaeger spans and convert to elastic error events
if they describe an error.

Fixes #3007

* Add experimental flag to Jaeger integration (#3121)

* tests/system: system test for Jaeger Thrift/HTTP (#3114)

* tests/system: system test for Jaeger Thrift/HTTP

* tests/system: system test for Jaeger gRPC

Co-authored-by: Andrew Wilkins <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants