Why is it replacing true/false with T/F? true/false is already 1 token in all tokenizer I've seen. Even worse is replacing null with ∅. ∅ is a special unicode symbol that takes up 2 tokens compared to the 1 token for null...

▲

wehnsdaefflae 4 minutes ago | parent | next [-]

Maybe they want to be tokenizer agnostic? Then they would need to go by character count, right? Despite these inconsistencies, has anyone actually verified their promise? 350 vs. 10.000 tokens would still be very valuable even if they mess up some edge cases

▲

fallingbananna 39 minutes ago | parent | prev | next [-]

What's even weirder is that the substitution is explained in "How TOON works" section. Yet TOON never describes such behavior anywhere in its spec.

▲

cedws an hour ago | parent | prev | next [-]

Brand new GitHub account, brand new HN account. I stay far away from projects like this these days, they can easily be malicious. GitHub needs some kind of indicator for projects authored by tenured developers with a real identity.

▲

AmazingTurtle 2 hours ago | parent | prev [-]

↲ is also two tokens instead of a simple \n lmao

	▲	hnlmorg an hour ago \| parent [-]
		How is an LF two tokens? Or were you referring to the Unicode symbol? I took their example to mean an actual LF ASCII character but now Ive read your comment, maybe I was being too charitable?