Remix clone Hacker News

I was writing a cryptographically-inclined system with serialization in msgpack. At one point, I upgraded the libraries I was using and all my signatures started breaking because the msgpack library started using a different representation under the hood for some of my data structures. That's when I did some research and found ASN.1 DER and haven't really looked back since switching over to it. If you plan on signing your data structures and don't want to implement your own serialization format, give ASN.1 DER a look.

▲

amluto 16 hours ago | parent [-]

If you are planning to sign your data structures, IMO your first choice should be to sign byte strings: be explicit that the thing that is signed is a specific string of bytes (which cryptographic protocol people love to call octets). Anything interpreting the signed data needs to start with those bytes and interpret them — do NOT assume that, just because you have some data structure that you think serializes to those bytes, then that data structure is authentic.

Many, many cryptographic disasters would have been avoided by following the advice above.

▲

zzo38computer 3 hours ago | parent | next [-]

> If you are planning to sign your data structures, IMO your first choice should be to sign byte strings

Yes, that is right; but, the byte sequence can be the canonical form of the data structure, and DER is canonical form.

▲

RainyDayTmrw 14 hours ago | parent | prev [-]

That matches the advice from Latacora[1]. That advice makes a lot of sense from a security correctness and surface area perspective.

There's a potential developer experience and efficiency concern, though. This likely forces two deserialization operations, and therefore two big memory copies, once for deserializing the envelope and once for deserializing the inner message. If we assume that most of the outer message is the inner message, and relatively little of it is the signature or MAC, then our extra memory copy is for almost the full length of the full message.

[1]: https://www.latacora.com/blog/2019/07/24/how-not-to/

▲

amluto 4 hours ago | parent [-]

There are a few serialization/deserialization systems that are close enough to zero-copy that this has no overhead. Cap’n Proto and FlatBuffers were designed around roughly this idea. Even some protobuf implementations allow in-place reads of bytes.

▲

RainyDayTmrw 3 hours ago | parent [-]

Does this work with nesting? I guess, if reading the envelope gets you a pointer to the inner buffer, you can pass that to another read operation. If that can be done safely (with the library ensuring the appropriate checks before it casts/transmutes), that would be very powerful.

	▲	amluto 3 hours ago \| parent [-]
		It should work fine. In C and C++, it's straightforward to YOLO it -- all you need is a pointer and a length. Rust can do more or less the same thing but with compiler-enforced safety. Many GC/ARC languages can efficiently handle slicing buffers, and it mostly comes down to library design. (Even Python can do this, although you generally pay for it in the rather large cost of every other operation...)