Discussion: Should we drop the requirement to percent-encode values of list-members

[basti1302]

We currently have this for "values" of list members:

A string which contains a value identified by the key. Any code points outside of the baggage-octet range MUST be percent-encoded. The percent code point (U+0025) MUST be percent-encoded. Code points which are not required to be percent-encoded MAY be percent-encoded.

It might be better to simply mandate the set of allowed characters, but not prescribe any particular process for encoding or decoding characters outside of the allowed set. For reference, the Cookie-/Set-Cookie-Spec only describes the allowed characters and does not mandate a requirement for any particular encoding, though it mentions that servers that want to store arbitrary data SHOULD encode them in some way.

There is little value in mandating a specific encoding for the following reasons: Parties that want to interact with each other via baggage need to agree on the baggage key any way, hence they can also agree on a encoding outside of baggage-octet, should that be required.

There is however value in having the spec mandating only the things that actually need to be standardized.

The main questions for this change are:

• Is this a breaking change? There are obviously already implementations out there that
  • apply percent-encoding when serializing baggage for downstream propagation
  • percent-decode when deserializing incoming baggage headers
• do we want to apply this change this late in the process (with respect to moving from candidate recommendation to recommendation)

For context, this discussion about came up when discussing #138 (comment) in the working group meeting.

[basti1302]

Copying this comment from @yurishkuro on the other issue here:

Hold on - the spec absolutely must mandate encoding, otherwise the consumer never knows how to interpret the data, whether to decode it or not. We had this ambiguity in jaeger baggage format and it was very painful to handle