WIP Add runtime/metrics source option to instrumentation/runtime metrics #2643
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #2624
The code in
instrumentation/metrics
was contributed a long time ago. The Go team has since introduced runtime metrics in an official way. This PR adds optional support for the new metrics. There is a problem in the old implementation because it was written before the current SDK specification; it is no longer correct to use synchronous instruments from async callbacks--where there are more than one reader concerned. This leaves the old code as-is and focuses on the desired future outcome without changing current default behavior.I propose that we deprecate the old and adopt the new eventually, however there are a number of gotchas.
There are four histogram instruments, but we lack a way to record them asynchronously. Moreover, one is a Gauge Histogram which OTel has not specified.
open-telemetry/opentelemetry-specification#2713
open-telemetry/opentelemetry-specification#2714
There are questions about what is a UpDownCounter and what is a Gauge. They are all currently UpDownCounters, but that is no guarantee. Presently, if a non-cumulative metric is Uint64, it becomes UpDownCounter, and if the value is Float64 it becomes a Gauge. We have no way to test this Gauge path presently because the go-1.19 runtime currently produces all Uint64 metrics and Float64Hist metrics, no Float64 metrics.
Lastly, there are two cases in 1.19 where there are duplicate metric names, after removing units. This creates a technical problem for OTLP -> Prometheus -> OTLP; currently, the only way to do this that satisfies both groups' specifications is to use an empty unit string when the unit is in the metric name itself. Thus, when there is a conflict of this nature, the units are kept in the name and the OTel unit is empty. This is an area for development between OpenTelemetry and OpenMetrics/Prometheus.