Add Generic Forwarding documentation (#1872)

* Add generic forwarder documentation

* Update generic_forwarding.md based on comments

docs/tempo/website/configuration/_index.md

@@ -128,6 +128,41 @@ distributor:
  opencensus:
  kafka:
 
+ # Optional.
+ # Configures forwarders that asynchronously replicate ingested traces
+ # to specified endpoints. Forwarders work on per-tenant basis, so to
+ # fully enable this feature, overrides configuration must also be updated.
+ #
+ # Note: Forwarders work asynchronously and can fail or decide not to forward
+ # some traces. This feature works in a "best-effort" manner.
+ forwarders:
+
+ # Forwarder name. Must be unique within the list of forwarders. 
+ # This name can be referenced in the overrides configuration to
+ # enable forwarder for a tenant.
+ - name: <string>
+
+ # The forwarder backend to use
+ # Should be "otlpgrpc".
+ backend: <string>
+
+ # otlpgrpc configuration. Will be used only if value of backend is "otlpgrpc".
+ otlpgrpc:
+
+ # List of otlpgrpc compatible endpoints.
+ endpoints: <list of string>
+ tls:
+
+ # Optional.
+ # Disables TSL if set to true.
+ [insecure: <boolean> | default = false]
+
+ # Optional.
+ # Path to the TLS certificate. This field must be set if insecure = false.
+ [cert_file: <string | default = "">]
+ - (repetition of above...)
+
+
  # Optional.
  # Enable to log every received trace id to help debug ingestion
  # WARNING: Deprecated. Use log_received_spans instead.
@@ -1082,6 +1117,12 @@ overrides:
  # A value of 0 disables the limit.
  [max_bytes_per_tag_values_query: <int> | default = 5000000 (5MB) ]
 
+ # Generic forwarding configuration
+
+ # Per-user configuration of generic forwarder feature. Each forwarder in the list
+ # must refer by name to a forwarder defined in the distributor.forwarders configuration.
+ [forwarders: <list of strings>]
+
  # Metrics-generator configurations
 
  # Per-user configuration of the metrics-generator ring size. If set, the tenant will use a
@@ -1104,7 +1145,7 @@ overrides:
  # overrides settings in the global configuration.
  [metrics_generator_processor_service_graphs_histogram_buckets: <list of float>]
  [metrics_generator_processor_service_graphs_dimensions: <list of string>]
- [metrics_generator_processor_span_metrics_histogram_buckets: <<list of float>]
+ [metrics_generator_processor_span_metrics_histogram_buckets: <list of float>]
  [metrics_generator_processor_span_metrics_dimensions: <list of string>]
 
  # Maximum number of active series in the registry, per instance of the metrics-generator. A

docs/tempo/website/operations/generic_forwarding.md

@@ -0,0 +1,16 @@
+---
+title: Generic forwarding
+weight: 13
+---
+
+# Generic forwarding
+
+Generic forwarding allows asynchronous replication of ingested traces. The distributor writes received spans to both the ingester and defined endpoints, if enabled. This feature works in a "best-effort" manner, meaning that no retries happen if an error occurs during replication. 
+
+>**Warning:** Generic forwarding does not work retroactively. Once enabled, the distributor only replicates freshly ingested spans.
+
+## Configuration
+
+Enabling generic forwarding requires the configuration of two sections. First, define a list of forwarders in the `distributor` section. Each forwarder must specify a unique `name`, supported `backend`, and backend-specific configuration. Second, reference these forwarders in the `overrides` section. This allows for fine-grained control over forwarding and makes it possible to enable this feature globally or on a per-tenant basis.
+
+For a detailed view of all the config options for the generic forwarding feature, please refer to [distributor]({{< relref "../configuration/#distributor" >}}) and [overrides]({{< relref "../configuration/#overrides" >}}) config pages.