Question about multicodec header and object hash

[mildred]

I am wondering how to make sure we won't be changing every hash of every object once we switch from protobufs to ipld. We have the multicodec to handle that transition, but that doesn't work for hashing, becaue hash(protobuf) != hash(multicodec_header + protobuf).

In that case:

• do we want to transfer the full object with the multicodec header, but when computing hash, ignore the header if it is pointing to the old protobuf format?
• do we want to transfer the same object we have now, without the multicodec header (hashing is not changed) and detect when the header is absent and decode it as protobuf?

I'd lean towards the second solution, and this needs to be implemented (I can't see any hint on that being already implemented in the code). Or perhaps that shouldn't be implemented in go-ipld but rather in the caller package, when dealing with an error (header mismatch or something like that).

@jbenet, what do you think ? Do we need to implement a fallback mechanism? In go-ipld or elsewhere?

[jbenet]

@mildred we can go with either,

i think the former case is nicer for the future, in that enforcing there always being a multicodec header is a good thing, instead of having to assume the old one if missing. makes parsers easier. we can leave the hashing part up to the datastructure (ipld) implementation to define, but the data on the wire would be nicely multicodec wrapped.

this may be relevant anyway if we ever want to add native support for reading objects from raw {git, bittorrent, bitcoin, tahoe lafs} objects. (when integrating with these other merkleized datastructures, "wrap them" or "implement native support" is a big debate, and i see the huge benefits and huge drawbacks of either)

---

for people wondering why keep the old protobuf support: it is not a good thing for a project purporting permanence to break these and make them no longer work. yes, it is very annoying to carry the legacy, but fortunately it's not very cumbersome.