diff --git a/README.md b/README.md index e6d0afa4359..5b3dc277511 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ The OpenTelemetry specification describes the cross-language requirements and ex - [Package/Library Layout](specification/library-layout.md) - [Concurrency and Thread-Safety](specification/concurrency.md) - API Specification - - [DistributedContext](specification/api-distributedcontext.md) + - [CorrelationContext](specification/api-correlationcontext.md) - [Propagators](specification/api-propagators.md) - [Tracing](specification/api-tracing.md) - [Metrics](specification/api-metrics.md) diff --git a/specification/api-correlationcontext.md b/specification/api-correlationcontext.md new file mode 100644 index 00000000000..9ed54eea9cf --- /dev/null +++ b/specification/api-correlationcontext.md @@ -0,0 +1,108 @@ +# Correlations API + +
+ +Table of Contents + + +- [Overview](#overview) + - [CorrelationContext](#correlationcontext) + - [Get correlations](#get-correlations) + - [Get correlation](#get-correlation) + - [Set correlation](#set-correlation) + - [Remove correlation](#remove-correlation) + - [Clear correlations](#clear-correlations) +- [CorrelationContext Propagation](#correlationcontext-propagation) +- [Conflict Resolution](#conflict-resolution) + +
+ +## Overview + +The Correlations API consists of: + +- the `CorrelationContext` +- functions to interact with the `CorrelationContext` in a `Context` + +### CorrelationContext + +`CorrelationContext` is used to annotate telemetry, adding context and information to metrics, traces, and logs. +It is an abstract data type represented by a set of name/value pairs describing user-defined properties. +Each name in `CorrelationContext` MUST be associated with exactly one value. +`CorrelationContext` MUST be serialized according to the editor's draft of the [W3C Correlation Context](https://w3c.github.io/correlation-context/) +specification. + +### Get correlations + +Returns the name/value pairs in the `CorrelationContext`. The order of name/value pairs MUST NOT be +significant. Based on the language specification, the returned value can be +either an immutable collection or an immutable iterator to the collection of +name/value pairs in the `CorrelationContext`. + +OPTIONAL parameters: + +`Context` the context containing the `CorrelationContext` from which to get the correlations. + +### Get correlation + +To access the value for a name/value pair by a prior event, the Correlations API +SHALL provide a function that takes a context and a name as input, and returns a +value. Returns the value associated with the given name, or null +if the given name is not present. + +REQUIRED parameters: + +`Name` the name to return the value for. + +OPTIONAL parameters: + +`Context` the context containing the `CorrelationContext` from which to get the correlation. + +### Set correlation + +To record the value for a name/value pair, the Correlations API SHALL provide a function which +takes a context, a name, and a value as input. Returns a new `Context` which +contains a `CorrelationContext` with the new value. + +REQUIRED parameters: + +`Name` the name for which to set the value. + +`Value` the value to set. + +OPTIONAL parameters: + +`Context` the context containing the `CorrelationContext` in which to set the correlation. + +### Remove correlation + +To delete a name/value pair, the Correlations API SHALL provide a function which takes a context +and a name as input. Returns a new `Context` which no longer contains the selected name. + +REQUIRED parameters: + +`Name` the name to remove. + +OPTIONAL parameters: + +`Context` the context containing the `CorrelationContext` from which to remove the correlation. + +### Clear correlations + +To avoid sending any name/value pairs to an untrusted process, the Correlations API SHALL provide +a function to remove all Correlations from a context. Returns a new `Context` +with no correlations. + +OPTIONAL parameters: + +`Context` the context containing the `CorrelationContext` from which to remove all correlations. + +## CorrelationContext Propagation + +`CorrelationContext` MAY be propagated across process boundaries or across any arbitrary boundaries +(process, $OTHER_BOUNDARY1, $OTHER_BOUNDARY2, etc) for various reasons. + +## Conflict Resolution + +If a new name/value pair is added and its name is the same as an existing name, than the new pair MUST take precedence. The value +is replaced with the added value (regardless if it is locally generated or received from a remote peer). diff --git a/specification/api-distributedcontext.md b/specification/api-distributedcontext.md deleted file mode 100644 index 8c38c0af037..00000000000 --- a/specification/api-distributedcontext.md +++ /dev/null @@ -1,226 +0,0 @@ -# DistributedContext API - -An `Entry` is used to label anything that is associated -with a specific operation, such as an HTTP request. - -`DistributedContext` is an abstract data type that represents a collection of entries. -Each key of `DistributedContext` is associated with exactly one value. `DistributedContext` is serializable, -to facilitate propagating it not only inside the process but also across process boundaries. -`DistributedContext` is used to annotate telemetry with the name:value pair `Entry`. -Those values can be used to add dimension to the metric or additional context properties to logs and traces. - -`DistributedContext` is a recommended name but languages can have more language-specific names like `dctx`. - -## Entry - -An `Entry` consists of EntryMetadata, EntryKey, and EntryValue. - -### EntryKey - -`EntryKey` is the name of the Entry. `EntryKey` along with `EntryValue` can be used to -aggregate and group stats, annotate traces and logs, etc. - -#### Restrictions - -- Must contain only printable ASCII (codes between 32 and 126 inclusive) -- Must have length greater than zero and less than 256. -- Must not be empty. - -#### Create - -Creates a new `EntryKey` with the given name in string. This is a static method. - -Required parameter: - -Name of the `EntryKey`. - -#### GetName - -Returns the name of the `EntryKey`. - -### EntryValue - -`EntryValue` is a string. It MUST contain only printable ASCII (codes between -32 and 126) - -#### Create - -Creates a new `EntryValue` with the given value in string. This is a static method. - -Required parameter: - -String value of the `EntryValue`. - -#### AsString - -Returns the string value of the `EntryValue`. - -### EntryMetadata - -`EntryMetadata` contains properties associated with an `Entry`. For now only the property `EntryTTL` -is defined. In future, additional properties may be added to address specific situations. - -The creator of entries determines metadata of an entry it creates. - -#### Create - -Creates a new `EntryMetadata` with the `EntryTTL`. This is a static method. - -Required parameter: - -`EntryTTL` that represents number of hops an entry can propagate. - -#### GetEntryTTL - -Returns the `EntryTTL`. - -#### EntryTTL - -`EntryTTL` is an integer that represents number of hops an entry can propagate. Anytime a sender serializes an entry, -sends it over the wire and receiver deserializes the entry then the entry is considered to have travelled one hop. -There could be one or more proxy(ies) between sender and receiver. Proxies are treated as transparent -entities and they may not create additional hops. Every propagation implementation should support an option -`decrementTTL` (default set to true) that allows proxies to set it to false. - -**For now, ONLY special values (0 and -1) are supported.** - -##### Special Values - -- **NO_PROPAGATION (0)**: An `Entry` with `EntryTTL` value of zero is considered to have local scope and - is used within the process it created. - -- **UNLIMITED_PROPAGATION (-1)**: An `Entry` with `EntryTTL` value of -1 can propagate unlimited hops. - `EntryTTL` value of -1 is typical used to represent a request, processing of which may span multiple entities. - -##### Example for EntryTTL > 0 - -On a server side typically there is no information about the caller besides ip/port, -but in every process there is a notion of "service_name" entry that is added as a "caller" entry before -serialization when a RPC/HTTP call is made. For the "caller" entry, desirable `EntryTTL` value is 1. - -Note that `EntryTTL` value of 1 is not supported at this time. The example is listed here simply to -show a possible use case for `EntryTTL` > 0. - -#### Processing at Receiver and Sender - -For now, limited processing is required on Sender and Receiver. However, for the sake of -completeness, future processing requirement is also listed here. These requirements are marked with -"**(future)**". - -This processing is done as part of entry propagator. - -##### At Receiver - -Upon receiving an entry from a remote entity, an entry extractor: - -- MUST decrement the value of `EntryTTL` by one if it is greater than zero. **(future)** -- MUST treat the value of `EntryTTL` as -1 if it is not present. -- MUST discard the `Entry` for any other value of `EntryTTL`. **(future)** - -##### At Sender - -Upon preparing to send an entry to a remote entity, an entry injector: - -- MUST send the entry AND include `EntryTTL` if its value is greater than 0. **(future)** -- MUST send the entry without `EntryTTL` if its value is -1. Absence of `EntryTTL` on the wire is treated as having `EntryTTL` of -1. - This is to optimize on-the-wire representation of common case. -- MUST not send the entry if the value of `EntryTTL` is 0. - -### Entry Conflict Resolution - -If a new entry conflicts with an existing entry then the new entry takes precedence. Entire `Entry` along -with `EntryValue` and `EntryMetadata` is replaced by the most recent entry (regardless of it is locally -generated or received from a remote peer). Replacement is limited to a scope in which the -conflict arises. When the scope is closed the original value and metadata prior to the conflict is restored. -For example, - -``` -T# - Entry keys -V# - Entry Values -M# - Entry Metadata - -Enter Scope 1 - Current Entries E1=V1/M1, E2=V2/M2 - Enter Scope 2 - Add Entries E3=V3/M3, E2=V4/M4 - Current Entries E1=V1/M1, E2=V4/M4, E3=V3/M3 <== Value/Metadata of E2 is replaced by V4/M4. - Close Scope 2 - Current Entries E1=V1/M1, E2=V2/M2 <== E2 is restored. -Close Scope 1 -``` - -## DistributedContext - -### GetEntries - -Returns the entries in this `DistributedContext`. -The order of entries is not significant. Based on the language specification, -the returned value can be either an immutable collection or an immutable iterator -to the collection of entries in this `DistributedContext`. - -### GetEntryValue - -Returns the `EntryValue` associated with the given `EntryKey`, or null if the given `EntryKey` -is not present. - -Required parameter: - -`EntryKey` entry key to return the value for. - -### Limits - -Combined size of all entries should not exceed 8192 bytes before encoding. -The size restriction applies to the deserialized entries so that the set of decoded - `DistributedContext`s is independent of the encoding format. - -### DistributedContext Propagation - -`DistributedContext` may be propagated across process boundaries or across any arbitrary boundaries -(process, $OTHER_BOUNDARY1, $OTHER_BOUNDARY2, etc) for various reasons. -For example, one may propagate 'project-id' Entry across all micro-services to break down metrics -by 'project-id'. Not all entries in a `DistributedContext` should be propagated and not all entries in a `DistributedContext` -should be accepted from a remote peer. Hence, `DistributedContext` propagator must allow specifying an optional -list of ordered `EntryPropagationFilter`s for receiving entries or for forwarding entries or for both. -An `EntryPropagationFilter` list for receiving MAY be different than that for forwarding. - -If no filter is specified for receiving then all entries are received. -If no filter is specified for forwarding then all entries are forwarded except those that have `EntryTTL` of 0. - -#### EntryPropagationFilter - -Entry Propagation Filter consists of an action (`EntryPropagationFilterAction`) and a condition -(`EntryPropagationFilterMatchOperator` and `EntryPropagationFilterMatchString`). An `EntryKey` -is evaluated against condition of each `EntryPropagationFilter` in order. If the condition is evaluated -to true then action is taken according to `EntryPropagationFilterAction` and filter processing is stopped. -If the condition is evaluated to false then the `EntryKey` is processed against next `EntryPropagationFilter` -in the ordered list. If none of the condition is evaluated to true then the default -action is **Exclude**. - -##### EntryPropagationFilterAction - -This is an interface. Implementation of this interface takes appropriate action on the `Entry` if the -condition (`EntryPropagationFitlerMatchOperator` and `EntryPropagationFilterMatchString`) is evaluated to true. -At a minimum, `Exclude` and `Include` actions MUST be implemented. - -**Exclude** -If the `EntryPropagationFilterAction` is Exclude then any `Entry` whose `EntryKey` evaluates to true -with the condition (`EntryPropagationFitlerMatchOperator` and `EntryPropagationFilterMatchString`) -MUST be excluded. - -**Include** -If the `EntryPropagationFilterAction` is Include then any `Entry` whose `EntryKey` evaluates to true -with the condition (`EntryPropagationFitlerMatchOperator` and `EntryPropagationFilterMatchString`) -MUST be included. - -##### EntryPropagationFilterMatchOperator - -| Operator | Description | -|----------|-------------| -| EQUAL | The condition is evaluated to true if `EntryKey` is exactly same as `EntryPropagationFilterMatchString` | -| NOTEQUAL | The condition is evaluated to true if `EntryKey` is NOT exactly same as `EntryPropagationFilterMatchString` | -| HAS_PREFIX | The condition is evaluated to true if `EntryKey` begins with `EntryPropagationFilterMatchString` | - -##### EntryPropagationFilterMatchString - -It is a string to compare against EntryKey using `EntryPropagationFilterMatchOperator` in order to -include or exclude an `Entry`. diff --git a/specification/library-layout.md b/specification/library-layout.md index 1f609996bdd..a3221ef7960 100644 --- a/specification/library-layout.md +++ b/specification/library-layout.md @@ -17,7 +17,7 @@ api ├── metrics ├── trace │ └── propagation - ├── distributedcontext + ├── correlationcontext │ └── propagation ├── internal └── logs @@ -33,16 +33,11 @@ This directory describes the API that provides in-process context propagation. This directory describes the Metrics API that can be used to record application metrics. -### [/distributedcontext](api-distributedcontext.md) +### [/correlationcontext](api-correlationcontext.md) -This directory describes the DistributedContext API that can be used to manage context propagation +This directory describes the CorrelationContext API that can be used to manage context propagation and metrics-related labeling. -This API consists of a few main classes: - -- `Entry` is used to label anything that is associated with a specific operation, such as an HTTP request. -- An `Entry` consists of `EntryMetadata`, `EntryKey`, and `EntryValue`. - ### [/trace](api-tracing.md) This API consist of a few main classes: @@ -71,7 +66,7 @@ sdk ├── metrics ├── resource ├── trace - ├── distributedcontext + ├── correlationcontext ├── internal └── logs ``` @@ -93,7 +88,7 @@ information about the entity for which stats or traces are recorded. For example by a Kubernetes container can be linked to a resource that specifies the cluster, namespace, pod, and container name. -### `/sdk/distributedcontext` +### `/sdk/correlationcontext` ### [/sdk/trace](sdk-tracing.md) diff --git a/specification/overview.md b/specification/overview.md index 3b45358d931..7d44c016dc7 100644 --- a/specification/overview.md +++ b/specification/overview.md @@ -193,31 +193,28 @@ OpenTelemetry defines the naming convention for metric names as well as a well-known metric names in [Semantic Conventions](data-semantic-conventions.md) document. -## DistributedContext +## CorrelationContext -The **DistributedContext** exists to store labels that describe the context of an operation an application performs. It is intended to enable context that are custom to the application or integrations in contrast to other contexts, such as `SpanContext`. Only one **DistributedContext** should be associated with any particular operation. +In addition to trace propagation, OpenTelemetry provides a simple mechanism for propagating +name/value pairs, called `CorrelationContext`. `CorrelationContext` is intended for +indexing observability events in one service with attributes provided by a prior service in +the same transaction. This helps to establish a causal relationship between these events. -For example, a web service can benefit from including context around what service has sent the request. Or a SaaS provider can include context about the API user or token that is responsible for that request. These values can be consumed from **DistributedContext** and used as an additional dimension for a metric, or additional context for logs and traces. +The `CorrelationContext` implements the editor's draft of the [W3C Correlation-Context specification](https://w3c.github.io/correlation-context/). +While `CorrelationContext` can be used to prototype other cross-cutting concerns, this mechanism is primarily intended +to convey values for the OpenTelemetry observability systems. -**DistributedContext** is a collection of key-value `Entry` pairs, with each key of associated with exactly one value. **DistributedContext** is serializable, -to facilitate propagating it not only inside the process but also across process boundaries. +These values can be consumed from `CorrelationContext` and used as additional dimensions for metrics, +or additional context for logs and traces. Some examples: -**DistributedContext** is a recommended name but languages can have more language-specific names like **dctx**. +- a web service can benefit from including context around what service has sent the request +- a SaaS provider can include context about the API user or token that is responsible for that request +- determining that a particular browser version is associated with a failure in an image processing service -### Entry - -An **Entry** is used to represent the labels that are contained inside the `DistributedContext`, representing values such as the service that originated the request, or vendor-specific data. It consists of an **EntryKey**, an **EntryValue** and an **EntryMetadata**. - -- **EntryKey** is the name of the **Entry**. **EntryKey** along with **EntryValue** - can be used to aggregate and group stats, annotate traces and logs, etc. **EntryKey** is - a string that contains only printable ASCII (codes between 32 and 126 inclusive) and with - a length greater than zero and less than 256. -- **EntryValue** is a string that contains only printable ASCII (codes between 32 and 126). -- **EntryMetadata** contains properties associated with an **Entry**. - For now only the property **EntryTTL** is defined. -- **EntryTTL** is an integer that represents number of hops an entry can propagate. - Anytime a sender serializes an entry, sends it over the wire and a receiver deserializes - the entry then the entry is considered to have travelled one hop. +For backward compatibility with OpenTracing, Baggage is propagated as `CorrelationContext` when +using the OpenTracing bridge. New concerns with different criteria should consider creating a new +cross-cutting concern to cover their use-case; they may benefit from the W3C encoding format but +use a new HTTP header to convey data throughout a distributed trace. ## Resources