In the eternal words of the Microsoft Game Development Kit docs[0], "Windows operates natively in UTF-16 [...]. This is a unique burden that Windows places on code that targets multiple platforms."

I wonder when Microsoft will stop being a "UTF-16 house" adding UTF-16 to new projects... They did it again to LSP, and that was in 2016. Maybe by 2045 Microsoft will use UTF-8 for new projects, and then by 2100 we can switch to UTF-8 by default.

[0] https://learn.microsoft.com/en-us/gaming/gdk/docs/gdk-dev/ov...

▲

tredre3 2 days ago | parent | next [-]

> They did it again to LSP, and that was in 2016.

That one is explainable by the fact that it was developed for VS Code, a typescript/javascript project. Javascript uses UTF-16 for strings (for historical reasons unrelated to Microsoft, though they are the same reasons that made MS use UCS-2 during the same time period).

▲

genxy 2 days ago | parent | next [-]

https://en.wikipedia.org/wiki/UTF-16

> UTF-16 is used by the Windows API, and by many programming environments such as Java and Qt. The variable-length character of UTF-16, combined with the fact that most characters are not variable-length (so variable length is rarely tested), has led to many bugs in software, including in Windows itself.

▲

tredre3 2 days ago | parent [-]

Absolutely, UTF-16 is bad and inexcusable in most contexts. Perhaps you misinterpreted my comment or I wasn't expressing myself properly.

I was trying to say: If all the components in your stack are javascript-based, it makes sense that your RPC-API would also use UTF-16. It wasn't Microsoft pushing it for historical Windows reasons as GP implied, it was dictated by a JS stack they chose but didn't create.

▲

genxy 2 days ago | parent [-]

I understood, I was just providing context. Windows, JavaScript, Java are all UTF-16. 65k characters is all you need.

	▲	noosphr a day ago \| parent [-]
		I always thought the js libraries throwing emojies around was stupid. Then I finally sat down to understand enough JS for LLM slop and the having rockets all over the place wasn't just cringe but the quickest way to show you handled utf 16.

▲

Someone a day ago | parent | prev [-]

> Javascript uses UTF-16 for strings

I don’t think the spec requires that. charAt indexes on UTF-16 code points, but I don’t remember ever seeing that it has to do that in O(1) time, so that doesn’t imply the implementation has to use UTF-16 internally.

Also, for modern JavaScript, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... says:

“Note: Avoid re-implementing the solutions above using charAt(). The detection of lone surrogates and their pairing is complex, and built-in APIs may be more performant as they directly use the internal representation of the string. Install a polyfill for the APIs mentioned above if necessary.”

Because of that, I would hope most modern implementations to at least have a variant internal representation that is similar to UTF-8.

▲

kragen 16 hours ago | parent [-]

I wouldn't expect them to. Let us know if you find otherwise!

	▲	Someone 40 minutes ago \| parent [-]
		https://iliazeus.lol/articles/js-string-optimizations-en/ “The ECMAScript standard defines a string as a sequence of 16-bit elements. But in a lot of cases, storing 2 bytes per character is a waste of memory. In fact, quite a lot of strings in most code will be limited to ASCII. That is why V8 has both one-byte and two-byte strings” https://github.com/v8/v8/blob/main/src/objects/string.h confirms that.

▲

GrayShade 21 hours ago | parent | prev [-]

And again in 2026: https://github.com/microsoft/pylance-release/blob/main/docs/...