Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Histogram crashes with more than the default number of boundaries #1665

Closed
supeterposition opened this issue Oct 7, 2022 · 0 comments · Fixed by #1685
Closed

Histogram crashes with more than the default number of boundaries #1665

supeterposition opened this issue Oct 7, 2022 · 0 comments · Fixed by #1685
Assignees
@esigo
Labels
bug Something isn't working priority:p1 Issues that are blocking
Milestone
Metrics v1.0.0 (GA)

Comments

@supeterposition
Copy link

Describe your environment Debian GNU/Linux 10 (buster), bazel 5.1.1, gcc 8.3.0

Steps to reproduce

  1. Modify examples/metrics_simple/metrics_ostream.cc to have more than the default (10) number of histogram boundaries—for example:
--- a/examples/metrics_simple/metrics_ostream.cc
+++ b/examples/metrics_simple/metrics_ostream.cc
@@ -77,7 +77,7 @@ void initMetrics(const std::string &name)
   static_cast<opentelemetry::sdk::metrics::HistogramAggregationConfig<double> *>(
       aggregation_config.get())
       ->boundaries_ =
-      std::list<double>{0.0, 50.0, 100.0, 250.0, 500.0, 750.0, 1000.0, 2500.0, 5000.0, 10000.0};
+      std::list<double>{0.0, 50.0, 100.0, 250.0, 500.0, 750.0, 1000.0, 2500.0, 5000.0, 10000.0, 20000.0};
   std::unique_ptr<metric_sdk::View> histogram_view{new metric_sdk::View{
       name, "description", metric_sdk::AggregationType::kHistogram, aggregation_config}};
   p->AddView(std::move(histogram_instrument_selector), std::move(histogram_meter_selector),
  1. Run the example, for instance with bazel run //examples/metrics_simple:metrics_ostream_example

What is the expected behavior?

The example should run and use the modified histogram boundaries.

What is the actual behavior?

The program crashes with error

metrics_ostream_example: malloc.c:2379: sysmalloc: Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed.
Aborted (core dumped)

Additional context

Running with --config=asan indicates a heap buffer overflow here:

=================================================================
==7421==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x608000014278 at pc 0x55c625e724b1 bp 0x7f377e6f8010 sp 0x7f377e6f8008
READ of size 8 at 0x608000014278 thread T6
    #0 0x55c625e724b0 in void opentelemetry::v1::sdk::metrics::HistogramMerge<double>(opentelemetry::v1::sdk::metrics::HistogramPointData&, opentelemetry::v1::sdk::metrics::HistogramPointData&, opentelemetry::v1::sdk::metrics::HistogramPointData&) bazel-out/k8-asan-dbg/bin/sdk/_virtual_includes/headers/opentelemetry/sdk/metrics/aggregation/histogram_aggregation.h:86
    #1 0x55c625e6f08b in opentelemetry::v1::sdk::metrics::DoubleHistogramAggregation::Merge(opentelemetry::v1::sdk::metrics::Aggregation const&) const sdk/src/metrics/aggregation/histogram_aggregation.cc:163
    #2 0x55c625e520e5 in operator() sdk/src/metrics/state/temporal_metric_storage.cc:71
    #3 0x55c625e5479f in operator() bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/function_ref.h:39
    #4 0x55c625e547d5 in _FUN bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/function_ref.h:38
    #5 0x55c625e56ea2 in opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::common::OrderedAttributeMap const&, opentelemetry::v1::sdk::metrics::Aggregation&)>::operator()(opentelemetry::v1::sdk::common::OrderedAttributeMap const&, opentelemetry::v1::sdk::metrics::Aggregation&) const bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/function_ref.h:87
    #6 0x55c625e55bd6 in opentelemetry::v1::sdk::metrics::AttributesHashMap::GetAllEnteries(opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::common::OrderedAttributeMap const&, opentelemetry::v1::sdk::metrics::Aggregation&)>) const bazel-out/k8-asan-dbg/bin/sdk/_virtual_includes/headers/opentelemetry/sdk/metrics/state/attributes_hashmap.h:89
    #7 0x55c625e537c3 in opentelemetry::v1::sdk::metrics::TemporalMetricStorage::buildMetrics(opentelemetry::v1::sdk::metrics::CollectorHandle*, opentelemetry::v1::nostd::span<std::shared_ptr<opentelemetry::v1::sdk::metrics::CollectorHandle>, 18446744073709551615ul>, opentelemetry::v1::common::SystemTimestamp, opentelemetry::v1::common::SystemTimestamp, std::shared_ptr<opentelemetry::v1::sdk::metrics::AttributesHashMap>, opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::metrics::MetricData)>) sdk/src/metrics/state/temporal_metric_storage.cc:60
    #8 0x55c625e4e1de in opentelemetry::v1::sdk::metrics::SyncMetricStorage::Collect(opentelemetry::v1::sdk::metrics::CollectorHandle*, opentelemetry::v1::nostd::span<std::shared_ptr<opentelemetry::v1::sdk::metrics::CollectorHandle>, 18446744073709551615ul>, opentelemetry::v1::common::SystemTimestamp, opentelemetry::v1::common::SystemTimestamp, opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::metrics::MetricData)>) sdk/src/metrics/state/sync_metric_storage.cc:30
    #9 0x55c625de8800 in opentelemetry::v1::sdk::metrics::Meter::Collect(opentelemetry::v1::sdk::metrics::CollectorHandle*, opentelemetry::v1::common::SystemTimestamp) sdk/src/metrics/meter.cc:307
    #10 0x55c625e7d72e in opentelemetry::v1::sdk::metrics::MetricCollector::Collect(opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::metrics::ResourceMetrics&)>) sdk/src/metrics/state/metric_collector.cc:46
    #11 0x55c625ddce5e in opentelemetry::v1::sdk::metrics::MetricReader::Collect(opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::metrics::ResourceMetrics&)>) sdk/src/metrics/metric_reader.cc:38
    #12 0x55c625dbc5dc in operator() sdk/src/metrics/export/periodic_exporting_metric_reader.cc:57
    #13 0x55c625dbe0ec in __invoke_impl<void, opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > /usr/include/c++/8/bits/invoke.h:60
    #14 0x55c625dbd512 in __invoke<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > /usr/include/c++/8/bits/invoke.h:95
    #15 0x55c625dc4019 in _M_invoke<0> /usr/include/c++/8/thread:244
    #16 0x55c625dc3f05 in operator() /usr/include/c++/8/thread:253
    #17 0x55c625dc372e in operator() /usr/include/c++/8/future:1362
    #18 0x55c625dc2e37 in _M_invoke /usr/include/c++/8/bits/std_function.h:283
    #19 0x55c625dc9e94 in std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>::operator()() const /usr/include/c++/8/bits/std_function.h:687
    #20 0x55c625dc7d7a in std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*) /usr/include/c++/8/future:561
    #21 0x55c625dcf46f in void std::__invoke_impl<void, void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::__invoke_memfun_deref, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) /usr/include/c++/8/bits/invoke.h:73
    #22 0x55c625dcbeb8 in std::__invoke_result<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>::type std::__invoke<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) /usr/include/c++/8/bits/invoke.h:95
    #23 0x55c625dc9757 in std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&)::{lambda()#1}::operator()() const /usr/include/c++/8/mutex:672
    #24 0x55c625dc97a0 in std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&)::{lambda()#2}::operator()() const /usr/include/c++/8/mutex:677
    #25 0x55c625dc97b1 in std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&)::{lambda()#2}::_FUN() /usr/include/c++/8/mutex:677
    #26 0x7f37836d8996 in __pthread_once_slow (/lib/x86_64-linux-gnu/libpthread.so.0+0xf996)
    #27 0x55c625dbaf0c in __gthread_once /usr/include/x86_64-linux-gnu/c++/8/bits/gthr-default.h:699
    #28 0x55c625dc9a33 in void std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) /usr/include/c++/8/mutex:684
    #29 0x55c625dc73de in std::__future_base::_State_baseV2::_M_set_result(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool) /usr/include/c++/8/future:401
    #30 0x55c625dc1b21 in operator() /usr/include/c++/8/future:1667
    #31 0x55c625dc3414 in __invoke_impl<void, std::__future_base::_Async_state_impl<_BoundFn, _Res>::_Async_state_impl(_BoundFn&&) [with _BoundFn = std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >; _Res = void]::<lambda()> > /usr/include/c++/8/bits/invoke.h:60
    #32 0x55c625dc2973 in __invoke<std::__future_base::_Async_state_impl<_BoundFn, _Res>::_Async_state_impl(_BoundFn&&) [with _BoundFn = std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >; _Res = void]::<lambda()> > /usr/include/c++/8/bits/invoke.h:95
    #33 0x55c625dc521d in _M_invoke<0> /usr/include/c++/8/thread:244
    #34 0x55c625dc5071 in operator() /usr/include/c++/8/thread:253
    #35 0x55c625dc43cd in _M_run /usr/include/c++/8/thread:196
    #36 0x7f3784464b2e  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xbbb2e)
    #37 0x7f37836d0fa2 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7fa2)
    #38 0x7f37835ffefe in clone (/lib/x86_64-linux-gnu/libc.so.6+0xf8efe)

0x608000014278 is located 0 bytes to the right of 88-byte region [0x608000014220,0x608000014278)
allocated by thread T6 here:
    #0 0x7f3784617d30 in operator new(unsigned long) (/lib/x86_64-linux-gnu/libasan.so.5+0xead30)
    #1 0x55c625cf28b0 in __gnu_cxx::new_allocator<unsigned long>::allocate(unsigned long, void const*) /usr/include/c++/8/ext/new_allocator.h:111
    #2 0x55c625ce4b44 in std::allocator_traits<std::allocator<unsigned long> >::allocate(std::allocator<unsigned long>&, unsigned long) /usr/include/c++/8/bits/alloc_traits.h:436
    #3 0x55c625cd9979 in std::_Vector_base<unsigned long, std::allocator<unsigned long> >::_M_allocate(unsigned long) /usr/include/c++/8/bits/stl_vector.h:296
    #4 0x55c625ce4bb2 in std::_Vector_base<unsigned long, std::allocator<unsigned long> >::_M_create_storage(unsigned long) /usr/include/c++/8/bits/stl_vector.h:311
    #5 0x55c625cd9b5e in std::_Vector_base<unsigned long, std::allocator<unsigned long> >::_Vector_base(unsigned long, std::allocator<unsigned long> const&) /usr/include/c++/8/bits/stl_vector.h:260
    #6 0x55c625ccc60e in std::vector<unsigned long, std::allocator<unsigned long> >::vector(std::vector<unsigned long, std::allocator<unsigned long> > const&) /usr/include/c++/8/bits/stl_vector.h:460
    #7 0x55c625da0636 in opentelemetry::v1::sdk::metrics::HistogramPointData::HistogramPointData(opentelemetry::v1::sdk::metrics::HistogramPointData const&) bazel-out/k8-asan-dbg/bin/sdk/_virtual_includes/headers/opentelemetry/sdk/metrics/data/point_data.h:55
    #8 0x55c625e76e19 in absl::otel_v1::variant_internal::DestructibleUnionImpl<opentelemetry::v1::sdk::metrics::HistogramPointData, opentelemetry::v1::sdk::metrics::LastValuePointData, opentelemetry::v1::sdk::metrics::DropPointData>::DestructibleUnionImpl<opentelemetry::v1::sdk::metrics::HistogramPointData&>(absl::otel_v1::variant_internal::EmplaceTag<0ul>, opentelemetry::v1::sdk::metrics::HistogramPointData&) bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/./internal/absl/types/../types/internal/variant.h:1016
    #9 0x55c625e76f30 in absl::otel_v1::variant_internal::DestructibleUnionImpl<opentelemetry::v1::sdk::metrics::SumPointData, opentelemetry::v1::sdk::metrics::HistogramPointData, opentelemetry::v1::sdk::metrics::LastValuePointData, opentelemetry::v1::sdk::metrics::DropPointData>::DestructibleUnionImpl<1ul, opentelemetry::v1::sdk::metrics::HistogramPointData&>(absl::otel_v1::variant_internal::EmplaceTag<1ul>, opentelemetry::v1::sdk::metrics::HistogramPointData&) bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/./internal/absl/types/../types/internal/variant.h:1020
    #10 0x55c625e7709d in absl::otel_v1::variant_internal::VariantStateBase<opentelemetry::v1::sdk::metrics::SumPointData, opentelemetry::v1::sdk::metrics::HistogramPointData, opentelemetry::v1::sdk::metrics::LastValuePointData, opentelemetry::v1::sdk::metrics::DropPointData>::VariantStateBase<1ul, opentelemetry::v1::sdk::metrics::HistogramPointData&>(absl::otel_v1::variant_internal::EmplaceTag<1ul>, opentelemetry::v1::sdk::metrics::HistogramPointData&) bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/./internal/absl/types/../types/internal/variant.h:1051
    #11 0x55c625e7315c in absl::otel_v1::variant_internal::VariantStateBaseDestructorNontrivial<opentelemetry::v1::sdk::metrics::SumPointData, opentelemetry::v1::sdk::metrics::HistogramPointData, opentelemetry::v1::sdk::metrics::LastValuePointData, opentelemetry::v1::sdk::metrics::DropPointData>::VariantStateBase<1ul, opentelemetry::v1::sdk::metrics::HistogramPointData&>(absl::otel_v1::variant_internal::EmplaceTag<1ul>, opentelemetry::v1::sdk::metrics::HistogramPointData&) bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/./internal/absl/types/../types/internal/variant.h:1286
    #12 0x55c625e732a8 in absl::otel_v1::variant_internal::VariantMoveBaseNontrivial<opentelemetry::v1::sdk::metrics::SumPointData, opentelemetry::v1::sdk::metrics::HistogramPointData, opentelemetry::v1::sdk::metrics::LastValuePointData, opentelemetry::v1::sdk::metrics::DropPointData>::VariantStateBase<1ul, opentelemetry::v1::sdk::metrics::HistogramPointData&>(absl::otel_v1::variant_internal::EmplaceTag<1ul>, opentelemetry::v1::sdk::metrics::HistogramPointData&) bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/./internal/absl/types/../types/internal/variant.h:1328
    #13 0x55c625e733f4 in absl::otel_v1::variant_internal::VariantCopyBaseNontrivial<opentelemetry::v1::sdk::metrics::SumPointData, opentelemetry::v1::sdk::metrics::HistogramPointData, opentelemetry::v1::sdk::metrics::LastValuePointData, opentelemetry::v1::sdk::metrics::DropPointData>::VariantStateBase<1ul, opentelemetry::v1::sdk::metrics::HistogramPointData&>(absl::otel_v1::variant_internal::EmplaceTag<1ul>, opentelemetry::v1::sdk::metrics::HistogramPointData&) bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/./internal/absl/types/../types/internal/variant.h:1370
    #14 0x55c625e73540 in absl::otel_v1::variant_internal::VariantMoveAssignBaseNontrivial<opentelemetry::v1::sdk::metrics::SumPointData, opentelemetry::v1::sdk::metrics::HistogramPointData, opentelemetry::v1::sdk::metrics::LastValuePointData, opentelemetry::v1::sdk::metrics::DropPointData>::VariantStateBase<1ul, opentelemetry::v1::sdk::metrics::HistogramPointData&>(absl::otel_v1::variant_internal::EmplaceTag<1ul>, opentelemetry::v1::sdk::metrics::HistogramPointData&) bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/./internal/absl/types/../types/internal/variant.h:1413
    #15 0x55c625e736a0 in absl::otel_v1::variant<opentelemetry::v1::sdk::metrics::SumPointData, opentelemetry::v1::sdk::metrics::HistogramPointData, opentelemetry::v1::sdk::metrics::LastValuePointData, opentelemetry::v1::sdk::metrics::DropPointData>::variant<opentelemetry::v1::sdk::metrics::HistogramPointData&, 1ul, opentelemetry::v1::sdk::metrics::HistogramPointData, (void*)0>(opentelemetry::v1::sdk::metrics::HistogramPointData&) bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/./internal/absl/types/variant.h:513
    #16 0x55c625e6f877 in opentelemetry::v1::sdk::metrics::DoubleHistogramAggregation::ToPoint() const sdk/src/metrics/aggregation/histogram_aggregation.cc:181
    #17 0x55c625e6efad in opentelemetry::v1::sdk::metrics::DoubleHistogramAggregation::Merge(opentelemetry::v1::sdk::metrics::Aggregation const&) const sdk/src/metrics/aggregation/histogram_aggregation.cc:161
    #18 0x55c625e520e5 in operator() sdk/src/metrics/state/temporal_metric_storage.cc:71
    #19 0x55c625e5479f in operator() bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/function_ref.h:39
    #20 0x55c625e547d5 in _FUN bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/function_ref.h:38
    #21 0x55c625e56ea2 in opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::common::OrderedAttributeMap const&, opentelemetry::v1::sdk::metrics::Aggregation&)>::operator()(opentelemetry::v1::sdk::common::OrderedAttributeMap const&, opentelemetry::v1::sdk::metrics::Aggregation&) const bazel-out/k8-asan-dbg/bin/api/_virtual_includes/api/opentelemetry/nostd/function_ref.h:87
    #22 0x55c625e55bd6 in opentelemetry::v1::sdk::metrics::AttributesHashMap::GetAllEnteries(opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::common::OrderedAttributeMap const&, opentelemetry::v1::sdk::metrics::Aggregation&)>) const bazel-out/k8-asan-dbg/bin/sdk/_virtual_includes/headers/opentelemetry/sdk/metrics/state/attributes_hashmap.h:89
    #23 0x55c625e537c3 in opentelemetry::v1::sdk::metrics::TemporalMetricStorage::buildMetrics(opentelemetry::v1::sdk::metrics::CollectorHandle*, opentelemetry::v1::nostd::span<std::shared_ptr<opentelemetry::v1::sdk::metrics::CollectorHandle>, 18446744073709551615ul>, opentelemetry::v1::common::SystemTimestamp, opentelemetry::v1::common::SystemTimestamp, std::shared_ptr<opentelemetry::v1::sdk::metrics::AttributesHashMap>, opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::metrics::MetricData)>) sdk/src/metrics/state/temporal_metric_storage.cc:60
    #24 0x55c625e4e1de in opentelemetry::v1::sdk::metrics::SyncMetricStorage::Collect(opentelemetry::v1::sdk::metrics::CollectorHandle*, opentelemetry::v1::nostd::span<std::shared_ptr<opentelemetry::v1::sdk::metrics::CollectorHandle>, 18446744073709551615ul>, opentelemetry::v1::common::SystemTimestamp, opentelemetry::v1::common::SystemTimestamp, opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::metrics::MetricData)>) sdk/src/metrics/state/sync_metric_storage.cc:30
    #25 0x55c625de8800 in opentelemetry::v1::sdk::metrics::Meter::Collect(opentelemetry::v1::sdk::metrics::CollectorHandle*, opentelemetry::v1::common::SystemTimestamp) sdk/src/metrics/meter.cc:307
    #26 0x55c625e7d72e in opentelemetry::v1::sdk::metrics::MetricCollector::Collect(opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::metrics::ResourceMetrics&)>) sdk/src/metrics/state/metric_collector.cc:46
    #27 0x55c625ddce5e in opentelemetry::v1::sdk::metrics::MetricReader::Collect(opentelemetry::v1::nostd::function_ref<bool (opentelemetry::v1::sdk::metrics::ResourceMetrics&)>) sdk/src/metrics/metric_reader.cc:38
    #28 0x55c625dbc5dc in operator() sdk/src/metrics/export/periodic_exporting_metric_reader.cc:57
    #29 0x55c625dbe0ec in __invoke_impl<void, opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > /usr/include/c++/8/bits/invoke.h:60

Thread T6 created by T1 here:
    #0 0x7f378457ddb0 in __interceptor_pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x50db0)
    #1 0x7f3784464db4 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/lib/x86_64-linux-gnu/libstdc++.so.6+0xbbdb4)
    #2 0x55c625dc2145 in _Async_state_impl /usr/include/c++/8/future:1664
    #3 0x55c625dc1670 in construct<std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >, void>, std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > > > /usr/include/c++/8/ext/new_allocator.h:136
    #4 0x55c625dc126d in construct<std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >, void>, std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > > > /usr/include/c++/8/bits/alloc_traits.h:475
    #5 0x55c625dc079d in _Sp_counted_ptr_inplace<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > > > /usr/include/c++/8/bits/shared_ptr_base.h:545
    #6 0x55c625dbfbb3 in __shared_count<std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >, void>, std::allocator<std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >, void> >, std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > > > /usr/include/c++/8/bits/shared_ptr_base.h:677
    #7 0x55c625dbf5fe in __shared_ptr<std::allocator<std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >, void> >, std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > > > /usr/include/c++/8/bits/shared_ptr_base.h:1342
    #8 0x55c625dbf239 in shared_ptr<std::allocator<std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >, void> >, std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > > > /usr/include/c++/8/bits/shared_ptr.h:359
    #9 0x55c625dbed08 in allocate_shared<std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >, void>, std::allocator<std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >, void> >, std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > > > /usr/include/c++/8/bits/shared_ptr.h:706
    #10 0x55c625dbe845 in make_shared<std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > >, void>, std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > > > /usr/include/c++/8/bits/shared_ptr.h:722
    #11 0x55c625dbe2e3 in _S_make_async_state<std::thread::_Invoker<std::tuple<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > > > /usr/include/c++/8/future:1705
    #12 0x55c625dbd79c in async<opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork()::<lambda()> > /usr/include/c++/8/future:1719
    #13 0x55c625dbc99d in opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::DoBackgroundWork() sdk/src/metrics/export/periodic_exporting_metric_reader.cc:68
    #14 0x55c625dcd9ff in void std::__invoke_impl<void, void (opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::*)(), opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader*>(std::__invoke_memfun_deref, void (opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::*&&)(), opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader*&&) /usr/include/c++/8/bits/invoke.h:73
    #15 0x55c625dcac26 in std::__invoke_result<void (opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::*)(), opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader*>::type std::__invoke<void (opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::*)(), opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader*>(void (opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::*&&)(), opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader*&&) /usr/include/c++/8/bits/invoke.h:95
    #16 0x55c625dd2c94 in decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)())) std::thread::_Invoker<std::tuple<void (opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::*)(), opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) /usr/include/c++/8/thread:244
    #17 0x55c625dd2b53 in std::thread::_Invoker<std::tuple<void (opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::*)(), opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader*> >::operator()() /usr/include/c++/8/thread:253
    #18 0x55c625dd2aa7 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::*)(), opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader*> > >::_M_run() /usr/include/c++/8/thread:196
    #19 0x7f3784464b2e  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xbbb2e)

Thread T1 created by T0 here:
    #0 0x7f378457ddb0 in __interceptor_pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x50db0)
    #1 0x7f3784464db4 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/lib/x86_64-linux-gnu/libstdc++.so.6+0xbbdb4)
    #2 0x55c625dbbbcb in opentelemetry::v1::sdk::metrics::PeriodicExportingMetricReader::OnInitialized() sdk/src/metrics/export/periodic_exporting_metric_reader.cc:42
    #3 0x55c625ddc5d4 in opentelemetry::v1::sdk::metrics::MetricReader::SetMetricProducer(opentelemetry::v1::sdk::metrics::MetricProducer*) sdk/src/metrics/metric_reader.cc:21
    #4 0x55c625e7cdfc in opentelemetry::v1::sdk::metrics::MetricCollector::MetricCollector(opentelemetry::v1::sdk::metrics::MeterContext*, std::unique_ptr<opentelemetry::v1::sdk::metrics::MetricReader, std::default_delete<opentelemetry::v1::sdk::metrics::MetricReader> >) sdk/src/metrics/state/metric_collector.cc:23
    #5 0x55c625e43686 in opentelemetry::v1::sdk::metrics::MeterContext::AddMetricReader(std::unique_ptr<opentelemetry::v1::sdk::metrics::MetricReader, std::default_delete<opentelemetry::v1::sdk::metrics::MetricReader> >) sdk/src/metrics/meter_context.cc:49
    #6 0x55c625dd42a1 in opentelemetry::v1::sdk::metrics::MeterProvider::AddMetricReader(std::unique_ptr<opentelemetry::v1::sdk::metrics::MetricReader, std::default_delete<opentelemetry::v1::sdk::metrics::MetricReader> >) sdk/src/metrics/meter_provider.cc:64
    #7 0x55c625c825a8 in initMetrics examples/metrics_simple/metrics_ostream.cc:45
    #8 0x55c625c84898 in main examples/metrics_simple/metrics_ostream.cc:98
    #9 0x7f378352b09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)

SUMMARY: AddressSanitizer: heap-buffer-overflow bazel-out/k8-asan-dbg/bin/sdk/_virtual_includes/headers/opentelemetry/sdk/metrics/aggregation/histogram_aggregation.h:86 in void opentelemetry::v1::sdk::metrics::HistogramMerge<double>(opentelemetry::v1::sdk::metrics::HistogramPointData&, opentelemetry::v1::sdk::metrics::HistogramPointData&, opentelemetry::v1::sdk::metrics::HistogramPointData&)
Shadow bytes around the buggy address:
  0x0c107fffa7f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c107fffa800: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c107fffa810: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c107fffa820: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c107fffa830: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c107fffa840: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00[fa]
  0x0c107fffa850: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 fa
  0x0c107fffa860: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c107fffa870: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c107fffa880: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c107fffa890: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==7421==ABORTING
@supeterposition supeterposition added the bug Something isn't working label Oct 7, 2022
@esigo esigo added this to the Metrics v1.0.0 (GA) milestone Oct 7, 2022
@esigo esigo self-assigned this Oct 7, 2022
@esigo esigo added the priority:p1 Issues that are blocking label Oct 7, 2022
@esigo esigo mentioned this issue Oct 14, 2022
Merged
3 tasks
@lalitb lalitb mentioned this issue Oct 17, 2022
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@esigo esigo

Labels
bug Something isn't working priority:p1 Issues that are blocking
Projects
None yet
Milestone
Metrics v1.0.0 (GA)
Development

Successfully merging a pull request may close this issue.

fix Histogram crash esigo/opentelemetry-cpp
2 participants
@supeterposition @esigo