> I feel like not understanding why JSON won out is being intentionally obtuse. JSON can easily be hand written, edited, and read for most data.

You are going way out of your way to try to come up with ways to rationalize why JSON was a success. The ugly truth is far simpler than what you're trying to sell: it was valid JavaScript. JavaScript WebApps could parse JSON with a call to eval(). No deserialization madness like XML, no need to import a parser. Just fetch a file, pass it to eval(), and you're done.

> I feel like not understanding why JSON won out is being intentionally obtuse.

I didn’t feel like my comment was the right place to shill for an alternative, but rather to complain about JSON. But since you raise it.

> JSON can easily be hand written, edited, and read for most data.

So can canonical S-expressions!

> Canonical S-expressions are not as easy to read and much harder to write by hand; having to prefix every atom with a length makes is very tedious to write by hand.

Which is why the advanced representation exists. I contend that this:

    (urn:ietf:params:acme:error:malformed
     (detail "Some of the identifiers requested were rejected")
     (subproblems ((urn:ietf:params:acme:error:malformed
                    (detail "Invalid underscore in DNS name \"_example.org\"")
                    (identifier (dns _example.org)))
                   (urn:ietf:params:acme:error:rejectedIdentifier
                    (detail "This CA will not issue for \"example.net\"")
                    (identifier (dns example.net))))))

    {
        "type": "urn:ietf:params:acme:error:malformed",
        "detail": "Some of the identifiers requested were rejected",
        "subproblems": [
            {
                "type": "urn:ietf:params:acme:error:malformed",
                "detail": "Invalid underscore in DNS name \"_example.org\"",
                "identifier": {
                    "type": "dns",
                    "value": "_example.org"
                }
            },
            {
                "type": "urn:ietf:params:acme:error:rejectedIdentifier",
                "detail": "This CA will not issue for \"example.net\"",
                "identifier": {
                    "type": "dns",
                    "value": "example.net"
                }
            }
        ]
    }

As you can see, no you do not.

> Canonical S-expressions are not as easy to read and much harder to write by hand

You don't do that, any more than you read or write machine code in binary. You read and write regular S-expressions (or assembly code) and you translate that into and out of canonical S expressions (or machine code) with a tool (an assembler/disassembler).

The entire reason ACME exists is because you are never writing or reading the CSR by hand.

So of course, ACME is based around a format whose entire reason d'etre is being written and read by hand.

It's weird.

you can use a program to convert between s-expressions and a more readable format. In a world where canonical s-expressions rule, this "more readable format" would probably be an ordinary s-expression

For very big numbers (that could appear in these fields), generating and parsing a base 10 decimal representation is way more cumbersome than using their binary representation.

The DER encoding used in the TLS certificates uses the big endian binary format. OpenSSL API wants the big endian binary too.

The format used by this protocol is a simple one.

It's almost exactly the format that is needed to use these numbers, except JSON can't store binary data directly. Converting binary to base 64 is a simple operation (just bit twiddling, no division), and it's easier than converting arbitrarily large numbers between base 2 and base 10. The 17-bit value happens to be an easy one, but other values may need thousands of bits.

It would be silly for the sender and recipient to need to use a BigNum library when the sender has the bytes and the recipient wants the bytes, and neither has use for a decimal number.

Some parsers, like PHP, may treat 65537 and "65537" the same. Room for vulnerability.

> Go can decode numbers losslessly as strings: https://pkg.go.dev/encoding/json#Number

Yup, and if you’re using JSON in Go you really do need to be using Number exclusively. Anything else will lead to pain.

> I'm half joking, but I'm not sure why S-expressions would be better here. There are LISPs that don't do arbitrary precision math.

Sure, but I’m referring specifically to https://www.ietf.org/archive/id/draft-rivest-sexp-13.html, which only has lists and bytes, and so number are always just strings and it’s up to the program to interpret them.

For actual SERDES, JSON becomes very brittle. It's better to use something like protobuf or cap'n'proto for such cases.

It's more a problem of support and backwards compatibility. JSON and parsers for it are so ubiquitous, and the spec completely lacks any versioning support, that adding a feature would be a breaking change of horrible magnitude, on nearly all levels of the modern software infrastructure stack. I wouldn't be surprised if some CPUs might break from that :D

JSON is a victim of its success: it has become too big to fail, and too big to improve.

There are easy workarounds to getting bigints in JSON: https://github.com/GoogleChromeLabs/jsbi/issues/30#issuecomm...

Because canonical S-expressions don’t have numbers, just atoms (i.e., byte sequences) and lists. It is up to the using code to interpret "34" as the string "34" or the number 34 or the number 13,108 or the number 13,363, which is part of the protocol being used. In most instances, the byte sequence is probably a decimal number.

Now, S-expressions as used for programming languages such as Lisp do have numbers, but again Lisp has bignums. As for parsers of Lisp S-expressions written in other languages: if they want to comply with the standard, they need to support bignums.

I think they mean that Common Lisp has bigints by default

The question is how best to send the modulus, which is a much larger integer. For the reasons below, I'd argue that base64 is better. And if you're sending the modulus in base64, you may as well use the same approach for the exponent sent along with it.

For RSA-4096, the modulus is 4096 bits = 512 bytes in binary, which (for my test key) is 684 characters in base64 or 1233 characters in decimal. So the base64 version is much smaller.

Base64 is also more efficient to deal with. An RSA implementation will typically work with the numbers in binary form, so for the base64 encoding you just need to convert the bytes, which is a simple O(n) transformation. Converting the number between binary and decimal, on the other hand, is O(n^2) if done naively, or O(some complicated expression bigger than n log n) if done optimally.

Besides computational complexity, there's also implementation complexity. Base conversion is an algorithm that you normally don't have to implement as part of an RSA implementation. You might argue that it's not hard to find some library to do base conversion for you. Some programming languages even have built-in bigint types. But you typically want to avoid using general-purpose bigint implementations for cryptography. You want to stick to cryptographic libraries, which typically aim to make all operations constant-time to avoid timing side channels. Indeed, the apparent ease-of-use of decimal would arguably be a bad thing since it would encourage implementors to just use a standard bigint type to carry the values around.

You could argue that the same concern applies to base64, but it should be relatively safe to use a naive implementation of base64, since it's going to be a straightforward linear scan over the bytes with less room for timing side channels (though not none).

Converting large integers to decimal is nontrivial, especially when you don't trust languages to handle large numbers.

Why you wouldn't just use the hexadecimal that everyone else seems to use I don't know. There seems to be a rather arbitrary cutoff where people prefer base64 to hexadecimal.

This sounds like an XY problem to me. There is already an alternative that is at least as secure and only requires a single base-64 string: Ed25519.

PHP (at least old versions I worked with) treats "65537" and 65537 similarly.

Cost.

Too likely that this would not work because silent conversion to number along the way

Just cross your fingers and hope for the best if your data is at any point decoded by a json library that doesn’t support bigints? Python’s ability to handle them is beside the point of they get mangled into ieee754 doubles along the way.

I prefer

  >> import json, decimal
  >> j = "47234762761726473624762746721647624764380000000000000000000000000000000000000000000"
  >> json.loads(j, parse_float=decimal.Decimal, parse_int=decimal.Decimal)
  Decimal('47234762761726473624762746721647624764380000000000000000000000000000000000000000000')

  >> import json
  >> j = "0.47234762761726473624762746721647624764380000000000000000000000000000000000000000000"
  >> json.loads(j)
  0.47234762761726473

  >> import json, decimal
  >> j = "0.47234762761726473624762746721647624764380000000000000000000000000000000000000000000"
  >> json.loads(j, parse_float=decimal.Decimal, parse_int=decimal.Decimal)
  Decimal('0.47234762761726473624762746721647624764380000000000000000000000000000000000000000000')

yes, python falls into the sane language category with arbitrary-precision arithmetic

I doubt that would fix the issue. The real cause is that programmers mostly deal in fixed-size integers, and that’s how they think of integer values, since those are the concepts their languages provide. If you’re going to write a JSON library for your favourite programming language, you’re going to reach for whatever ints are the default, regardless of what the specs or type hints suggest.

Haskell’s Aeson library is one of the few exceptions I’ve seen, since it only parses numbers to ‘Scientific’s (essentially a kind of bigint for rationals.) This makes the API very safe, but also incredibly annoying to use if you want to just munge some integers, since you’re forced to handle the error case of the unbounded values not fitting in your fixed-size integer values.

Most programmers likely simply either don’t consider that case, or don’t want to have to deal with it, so bad JSON libraries are the default.

This is actually a deliberate design choice, which the breathtakingly short JSON standard explains quite well [0]. The designers deliberately didn't introduce any semantics and pushes all that to the implementors. I think this is a defensible design goal. If you introduce semantics, you're sure to annoy someone.

There's an element of "worse is better" here [1]. JSON overtook XML exactly because it's so simple and solves for the social element of communication between disparate projects with wildly different philosophies, like UNIX byte-oriented I/O streams, or like the C calling conventions.

---

[0] https://ecma-international.org/publications-and-standards/st...

[1] https://en.wikipedia.org/wiki/Worse_is_better

Isn't the actual problem that it is supposed to map to JS types, which are badly designed, and thus being infectious for other ecosystems, that don't have these defects?

Yup, imagine if HTML was JSON-like, not XML-like.

Sadly JWT and friends are "standard". In theory the representation and the data are independent and you can marshal and unmarshal correctly.

In practice, "alg:none" is a headache and everyone involved should be ashamed.

Have a read through RFC7159 or 8259 and despair.

> This specification allows implementations to set limits on the range and precision of numbers accepted

JSON is a terrible interoperability standard.

> Aren't JSON parsers technically not following the standard if they don't reliably store a number that is not representable by a IEEE754 double precision float?

That sentence has four negations and I honestly can't figure out what it means.

> hard disagree on that one.

Ok, thanks for your insight.