docs(textual): Require signing over raw bytes

[amaury1093]

Description

Security concerns around what's signed, pulled out from #12785 (comment).

cc @peterbourgon

---

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

• included the correct type prefix in the PR title
• added ! to the type prefix if API or client breaking change
• targeted the correct branch (see PR Targeting)
• provided a link to the relevant issue or specification
• followed the guidelines for building modules
• included the necessary unit and integration tests
• added a changelog entry to CHANGELOG.md
• included comments for documenting Go code
• updated the relevant documentation or specification
• reviewed "Files changed" and left comments if necessary
• confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

• confirmed the correct type prefix in the PR title
• confirmed ! in the type prefix if API or client breaking change
• confirmed all author checklist items have been addressed
• reviewed state machine logic
• reviewed API design and naming
• reviewed documentation is accurate
• reviewed tests and test coverage
• manually tested (if applicable)

[peterbourgon]

Relevant comments:

• docs(textual): custom Msg renderers #12785 (comment)
• docs(textual): custom Msg renderers #12785 (comment)
• docs(textual): custom Msg renderers #12785 (comment)

tl;dr: signatures verify the integrity of the signed bytes, not a semantic transformation of those bytes

[arirubinstein]

Is there a reason base64 is used as the representation here instead of hex bytes? There could result in ambiguity depending on how the transaction can be tampered with. the digest should be represented in a simpler byte representation, e.g. hex-encoded with expected length validation on read

e.g. here are two base64-encoded payloads which differ in serialized representation but the same raw bytes:

payload1="QzNWwq=="
payload2="QzNWwr=="

% echo -n $payload1 | shasum -a256
b679c1a6f416eb2eb18af4dce3347c3aa74e559aa88a8ef0ef4d6eefb0939c32  -
% echo -n $payload2 | shasum -a256
a7e3cd272e296994116cdf160221a26e71ca6b800abb5c62e90086c245fa115e  -

% echo -n $payload1 | base64 -d | xxd
00000000: 4333 56c2                                C3V.
% echo -n $payload2 | base64 -d | xxd
00000000: 4333 56c2                                C3V.

Additionally, while it doesn't appear to immediately be exploitable in the current form, would it be possible to utilize a scheme involving prepending the length of each payload before each, and/or adopt ASN.1/DER representation of the bytes here in the event " " becomes significant?
e.g. H(len(param1)||param1||separator||len(param2)||param2)

[peterbourgon]

Agreed, but more specifically...

We require that the user signs over these canonical bytes in SIGN_MODE_TEXTUAL, more specifically over the following string:

The string is a rendering of the canonical bytes, and not the canonical bytes themselves. Whatever the on-chain representation is, that's what should be signed, directly. Signatures assert integrity only for the literal msg bytes provided.

[aaronc]

Not sure we need a separator at all, but length prefixing each part makes sense

[amaury1093]

I added length prefix in b15409d.

@arirubinstein Thanks for flagging the non-uniqueness of base64 encoding, I actually wasn't aware. We also use base64 in value renderers. This means that 2 different SignDocTextuals (i.e. 2 different string arrays) can represent the same proto Tx, which breaks the spec on bijectivity. Maybe it's worth indeed considering using hex everywhere.

[alexanderbez]

I would prefer to use HEX for everything personally. Base64 is such a mess.

[plafer]

I agree!

[amaury1093]

Switched to capital hex

[robert-zaremba]

Note1: there is also z-base-32 which is more compact, readable and doesn't have the padding bytes issue.

Note2: we don't need to length prefix the last part. So <len(body_bytes) ++ <body_bytes> ++ <auth_info_bytes> is good enough.

[robert-zaremba]

For z-base-32 support, here is the spec (good read!): https://philzimmermann.com/docs/human-oriented-base-32-encoding.txt

BTW, Algorand is using base32.

[robert-zaremba]

Linking: #13141

[aaronc]

One additional thing we should do is make sure that TxRaw bytes are encoded according to ADR 027 in the tx decoder. This is actually an existing malleability issue for SIGN_MODE_DIRECT that we should address.

[amaury1093]

@aaronc That has already been addressed in #9743.

[aaronc]

Sorry I forgot!

[amaury1093]

This is R4R again. We sign over a hex string now.

Edit: about using Hex instead of base64 in other parts of the spec, I did another PR: #12954

[robert-zaremba]

LGTM. Added a suggestion to consider z-base-32

[webmaster128]

If we use <foo> to mark the insertion of a variable in the document, can we just use a single set of brackets to include the calculation like this?

Suggested change

	*Hash of raw bytes: <HEX(sha256(<len(body_bytes) ++ <body_bytes> ++ len(auth_info_bytes) ++ <auth_info_bytes>))>
	*Hash of raw bytes: <HEX(sha256(len(body_bytes) ++ <body_bytes> ++ len(auth_info_bytes) ++ auth_info_bytes))>

Also we need to specify how len is encoded to bytes. I suggest 4 byte big endian uint32.

[amaury1093]

Good point. I had varint in mind, just to accomodate for arbitrarily long txs.

[webmaster128]

Hmm, varint is complex and not unique. Also we don't need to optimize for output length here. What about 8 byte big endian uint64 instead?

[amaury1093]

OK. uint64 should be plenty.

[arirubinstein]

Just wanted to point out that ASN.1/DER encoding does this natively and has a go lib as well.

[webmaster128]

Yeah, but pulling in a DER dependency for each client comes at a cost, especially when not used consistently thoughout the stack. For Ethereum signatures I had to write DER encoding/decoding by hand in TypeScript, which is not fun. But then again 50 lines of code also don't justify pulling in the whole fully featured ASN.1/DER dependency (if it exists in reasonable quality).

[amaury1093]

Putting automerge on.