[release-v2.6] [DOC] TraceQL, metrics-gen, and overrides doc updates …

…for 2.6 (#4054)

docs/sources/tempo/configuration/_index.md

@@ -33,7 +33,7 @@ The Tempo configuration options include:
  - [Filter policy config](#filter-policy-config)
  - [Filter policy](#filter-policy)
  - [Policy match](#policy-match)
- - [Examples](#examples) 
+ - [Examples](#examples)
  - [KVStore config](#kvstore-config)
  - [Search config](#search-config)
  - [WAL config](#wal-config)
@@ -204,6 +204,13 @@ distributor:
  [include_all_attributes: <boolean> | default = false]
  [filter_by_status_error: <boolean> | default = false]
 
+ # Optional.
+ # Enable to log every discarded span to help debug ingestion or calculate span error distributions using the logs.
+ log_discarded_spans:
+ [enabled: <boolean> | default = false]
+ [include_all_attributes: <boolean> | default = false]
+ [filter_by_status_error: <boolean> | default = false]
+
  # Optional.
  # Enable to metric every received span to help debug ingestion
  # This is not recommended for production environments
@@ -1552,6 +1559,10 @@ overrides:
  # Per-user compaction window. If this value is set to 0 (default),
  # then block_retention in the compactor configuration is used.
  [compaction_window: <duration> | default = 0s]
+ # Allow compaction to be deactivated on a per-tenant basis. Default value
+ # is false (compaction active). Useful to perform operations on the backend
+ # that require compaction to be disabled for a period of time.
+ [compaction_disabled: <bool> | default = false]
 
  # Metrics-generator related overrides
  metrics_generator:

docs/sources/tempo/metrics-generator/_index.md

@@ -68,8 +68,11 @@ When multi-tenancy is enabled, the metrics-generator forwards the `X-Scope-OrgID
 
 ## Native histograms
 
-The metrics-generator supports the ability to produce [native histograms](https://grafana.com/docs/grafana-cloud/whats-new/native-histograms/), for
-high-resolution data. Users must [update the receiving endpoint](https://grafana.com/docs/mimir/latest/configure/configure-native-histograms-ingestion/) to ingest native
-histograms, and [update histogram queries](https://grafana.com/docs/mimir/latest/visualize/native-histograms/) in their dashboards.
+[Native histograms](https://grafana.com/docs/grafana-cloud/whats-new/native-histograms/) are a data type in Prometheus that can produce, store, and query high-resolution histograms of observations.
+It usually offers higher resolution and more straightforward instrumentation than classic histograms.
+
+The metrics-generator supports the ability to produce native histograms for
+high-resolution data. Users must [update the receiving endpoint](https://grafana.com/docs/mimir/<MIMIR_VERSION>/configure/configure-native-histograms-ingestion/) to ingest native
+histograms, and [update histogram queries](https://grafana.com/docs/mimir/<MIMIR_VERSION>/visualize/native-histograms/) in their dashboards.
 
 To learn more about the configuration, refer to the [Metrics-generator]({{< relref "../configuration#metrics-generator" >}}) section of the Tempo Configuration documentation.

docs/sources/tempo/setup/upgrade.md

@@ -14,7 +14,7 @@ The upgrade process changes for each version, depending upon the changes made fo
 
 This upgrade guide applies to on-premise installations and not for Grafana Cloud.
 
-For detailed information about any release, refer to the [Release notes](../release-notes/).
+For detailed information about any release, refer to the [Release notes](https://grafana.com/docs/tempo/<TEMPO_VERSION>/setup/upgrade/).
 
 {{< admonition type="tip" >}}
 You can check your configuration options using the [`status` API endpoint]({{< relref "../api_docs#status" >}}) in your Tempo installation.

docs/sources/tempo/traceql/_index.md

@@ -64,38 +64,67 @@ In this example, the search reduces traces to those spans where:
 
 Queries select sets of spans and filter them through a pipeline of aggregators and conditions. If, for a given trace, this pipeline produces a spanset then it is included in the results of the query.
 
-
 ## Selecting spans
 
-TraceQL differentiates between two types of span data: intrinsics, which are fundamental to spans, and attributes, which are customizable key-value pairs. You can use intrinsics and attributes to build filters and select spans.
-
 In TraceQL, curly brackets `{}` always select a set of spans from the current trace. They are commonly paired with a condition to reduce the spans being passed in.
 
-### Intrinsic fields
+TraceQL differentiates between two types of span data: intrinsics, which are fundamental to spans, and attributes, which are customizable key-value pairs.
+You can use intrinsics and attributes to build filters and select spans.
+
+Intrinsic fields are fundamental to scopes.
+Intrinsics are inherently present, as opposed to other key-value pairs (attributes) that are added by a developer.
+
+Intrinsics are always indicated using a `<scope>:`.
+Refer to the Intrinsics table for all current intrinsics.
+
+Custom attributes are prefixed with <scope>. such as `span.`, `resource.` , `link.`, or `event`.
+Resource has no intrinsic values.
+It only has custom attributes.
+The `trace` scope is only an intrinsic and doesn't have any custom attributes at the trace level.
+
+Intrinsics example:
+```
+{ span:name = "foo" }
+{ event:name = "foo" }
+{ trace:id = "1234" }
+{ link:traceID = "1234" }
+```
+
+Attributes example:
+```
+{ span.foo = "bar" }
+{ resource.foo = "bar" }
+{ link.foo = "bar" }
+{ event.foo = "bar" }
+```
 
-Intrinsic fields are fundamental to spans. These fields can be referenced when selecting spans.
-Custom attributes are prefixed with `.`, `span.` or `resource.`, whereas intrinsics are typed directly.
+### Intrinsic fields
 
 The following table shows the current available scoped intrinsic fields:
 
-| **Field** | **Type** | **Definition** | **Example** |
-| ----------------------- | ----------- | --------------------------------------------------------------- | -------------------------------------- |
-| `span:status` | status enum | status: error, ok, or unset | `{ span:status = ok }` |
-| `span:statusMessage` | string | optional text accompanying the span status | `{ span:statusMessage = "Forbidden" }` |
-| `span:duration` | duration | end - start time of the span | `{ span:duration > 100ms }` |
-| `span:name` | string | operation or span name | `{ span:name = "HTTP POST" }` |
-| `span:kind` | kind enum | kind: server, client, producer, consumer, internal, unspecified | `{ span:kind = server }` |
-| `span:id` | string | span id using hex string | `{ span:id = "0000000000000001" }` |
-| `trace:duration` | duration | max(end) - min(start) time of the spans in the trace | `{ trace:duration > 100ms }` |
-| `trace:rootName` | string | if it exists the name of the root span in the trace | `{ trace:rootName = "HTTP GET" }` |
-| `trace:rootService` | string | if it exists the service name of the root span in the trace | `{ trace:rootServiceName = "gateway" }`|
-| `trace:id` | string | trace id using hex string | `{ trace:id = "1234567890abcde" }` |
-| `event:name` | string | name of event | `{ event:name = "exception" }` |
-| `event:timeSinceStart` | duration | time of event in relation to the span start time | `{ event:timeSinceStart > 2ms}` |
-| `link:spanID` | string | link span id using hex string | `{ link:spanID = "0000000000000001" }` |
-| `link:traceID` | string | link trace id using hex string | `{ link:traceID = "1234567890abcde" }` |
-
-`trace:duration`, `trace:rootName`, and `trace:rootService` are trace-level intrinsics and are the same for all spans in the same trace.
+| **Field** | **Type** | **Definition** | **Example** |
+| ------------------------ | ----------- | --------------------------------------------------------------- | -------------------------------------- |
+| `span:status` | status enum | status: error, ok, or unset | `{ span:status = ok }` |
+| `span:statusMessage` | string | optional text accompanying the span status | `{ span:statusMessage = "Forbidden" }` |
+| `span:duration` | duration | end - start time of the span | `{ span:duration > 100ms }` |
+| `span:name` | string | operation or span name | `{ span:name = "HTTP POST" }` |
+| `span:kind` | kind enum | kind: server, client, producer, consumer, internal, unspecified | `{ span:kind = server }` |
+| `span:id` | string | span id using hex string | `{ span:id = "0000000000000001" }` |
+| `trace:duration` | duration | max(end) - min(start) time of the spans in the trace | `{ trace:duration > 100ms }` |
+| `trace:rootName` | string | if it exists the name of the root span in the trace | `{ trace:rootName = "HTTP GET" }` |
+| `trace:rootService` | string | if it exists the service name of the root span in the trace | `{ trace:rootServiceName = "gateway" }`|
+| `trace:id` | string | trace id using hex string | `{ trace:id = "1234567890abcde" }` |
+| `event:name` | string | name of event | `{ event:name = "exception" }` |
+| `event:timeSinceStart` | duration | time of event in relation to the span start time | `{ event:timeSinceStart > 2ms}` |
+| `link:spanID` | string | link span id using hex string | `{ link:spanID = "0000000000000001" }` |
+| `link:traceID` | string | link trace id using hex string | `{ link:traceID = "1234567890abcde" }` |
+
+<!-- instrumentation scope isn't included in the 2.6 documentation
+| `instrumentation:name` | string | instrumentation scope name | `{ instrumentation:name = "grpc" }` |
+| `instrumentation:version`| string | instrumentation scope version | `{ instrumentation:version = "1.0.0" }`|
+-->
+
+The trace-level intrinsics, `trace:duration`, `trace:rootName`, and `trace:rootService`, are the same for all spans in the same trace.
 Additionally, these intrinsics are significantly more performant because they have to inspect much less data then a span-level intrinsic.
 They should be preferred whenever possible to span-level intrinsics.
 
@@ -109,15 +138,24 @@ This example searches all Kubernetes clusters called `service-name` that have a
 { resource.k8s.cluster.name="service-name" && trace:rootName !~ ".*perf.*"}
 ```
 
+
 ### Attribute fields
 
-TraceQL has four different attribute scopes: span attributes, resource attributes, event attributes, and link attributes. By expanding a span in the Grafana UI, you can see both its span attributes (1 in the screenshot) and resource attributes (2 in the screenshot).
+TraceQL has four different attribute scopes: span attributes, resource attributes, event attributes, and link attributes.
+<!-- instrumentation scope isn't in 2.6 >
+instrumentation scope attributes. -->
+
+By expanding a span in the Grafana UI, you can see both its span attributes (1 in the screenshot) and resource attributes (2 in the screenshot).
 
 <p align="center"><img src="assets/span-resource-attributes.png" alt="Example of span and resource attributes." /></p>
 
-Attribute fields are derived from the span and can be customized. Process and span attribute types are [defined by the attribute itself](https:/open-telemetry/opentelemetry-proto/blob/b43e9b18b76abf3ee040164b55b9c355217151f3/opentelemetry/proto/common/v1/common.proto#L30-L38), whereas intrinsic fields have a built-in type. You can refer to dynamic attributes (also known as tags) on the span or the span's resource.
+Attribute fields are derived from the span and can be customized.
+Process and span attribute types are [defined by the attribute itself](https:/open-telemetry/opentelemetry-proto/blob/b43e9b18b76abf3ee040164b55b9c355217151f3/opentelemetry/proto/common/v1/common.proto#L30-L38), whereas intrinsic fields have a built-in type.
+You can refer to dynamic attributes (also known as tags) on the span or the span's resource.
 
-Attributes in a query start with a span scope (for example, `span.http`) or resource scope (for example, `resource.namespace`) depending on what you want to query. This provides significant performance benefits because it allows Tempo to only scan the data you are interested in.
+Attributes in a query start with a span, resource, event, or link scope.
+For example, you could use `span.http` or `resource.namespace`, depending on what you want to query.
+This provides significant performance benefits because it allows Tempo to only scan the data you are interested in.
 
 To find traces with the `GET HTTP` method, your query could look like this:
 
@@ -126,6 +164,7 @@ To find traces with the `GET HTTP` method, your query could look like this:
 ```
 
 For more information about attributes and resources, refer to the [OpenTelemetry Resource SDK](https://opentelemetry.io/docs/reference/specification/resource/sdk/).
+
 #### Examples
 
 Find traces that passed through the `production` environment:
@@ -138,15 +177,29 @@ Find any database connection string that goes to a Postgres or MySQL database:
 { span.db.system =~ "postgresql|mysql" }
 ```
 
+You can use the `event` scope to query events that happen within a span.
+A span event is a unique point in time during the span’s duration.
+While spans help build the structural hierarchy of your services, span events can provide a deeper level of granularity to help debug your application faster and maintain optimal performance.
+To learn more about how you can use span events, read the [What are span events?](https://grafana.com/blog/2024/08/15/all-about-span-events-what-they-are-and-how-to-query-them/) blog post.
+
 You can query for an exception in your span event:
 ```
 { event.exception.message =~ ".*something went wrong.*" }
 ```
 
+If you've instrumented your traces for span links, you can use the `link` scope to query the link data. A span link associates one span with one or more other spans that are a casual relationship.
+For more information on span links, refer to the [Span Links](https://opentelemetry.io/docs/concepts/signals/traces/#span-links) documentation in the Open Telemetry project.
+
 You can search for an attribute in your link:
 ```
 { link.opentracing.ref_type = "child_of" }
 ```
+<!-- instrumentation scope isn't included in the 2.6 release
+Find instrumentation scope programming language:
+```
+{ instrumentation.language = "java" }
+```
+-->
 
 ### Unscoped attribute fields