▲ | Dylan16807 4 days ago | |||||||
Okay so you seek by 3 less bytes. Or you accept that if you're randomly losing chunks, you might lose an extra 3 bytes. The real problem is that seeking a few bytes won't work with EMBL. If continuation bytes store 8 payload bits, you can get into a situation where every single byte could be interpreted as a multi-byte start character and there are 2 or 3 possible messages that never converge. | ||||||||
▲ | Sharlin 4 days ago | parent [-] | |||||||
The point is that you don’t have a "seek" operation available. You are given a bytestream and aren’t told if you’re at the start, in a valid position between code points, or in the middle of a code point. UTF-8’s self-synchronizing property means that by reading a single byte you immediately know if you’re in the middle of a code point, and that by reading and discarding at most two additional bytes you’re synchronized and can start/return decoding. That wouldn’t be possible if continuation bytes used all the bits for payload. | ||||||||
|