Remix.run Logo
Big-Endian Testing with QEMU(hanshq.net)
62 points by jandeboevrie 5 hours ago | 42 comments
AKSF_Ackermann 3 hours ago | parent | next [-]

> When programming, it is still important to write code that runs correctly on systems with either byte order

What you should do instead is write all your code so it is little-endian only, as the only relevant big-endian architecture is s390x, and if someone wants to run your code on s390x, they can afford a support contract.

j16sdiz 3 hours ago | parent | next [-]

If you comes to low level network protocol (e.g. writing a TCP stack), the "network byte order" is always big-endian.

edflsafoiewq an hour ago | parent | next [-]

That's a serialization format.

7jjjjjjj an hour ago | parent | prev | next [-]

It goes without saying that all binary network protocols should document their byte order, and that if you're implementing a protocol documented as big endian you should use ntohl and friends to ensure correctness.

However if designing a new network protocol, choosing big endian is insanity. Use little endian, skip the macros, and just add

  #ifndef LITTLE_ENDIAN
    #error
Or the like to a header somewhere.
whizzter 42 minutes ago | parent | prev | next [-]

And honestly at this point it's mostly a historical artifact, if we write that kind of stuff then sure we need to care but to produce modern stuff is a honestly massive waste of time at this point.

FWIW I doing hobby-stuff for Amiga's (68k big-endian) but that's just that, hobby stuff.

skrtskrt 2 hours ago | parent | prev [-]

Prometheus index format is also a big-endian binary file - haven’t found any reference to why it was chosen.

addaon 2 hours ago | parent | prev | next [-]

There's still at least one relevant big-endian-only ARM chip out there, the TI Hercules. While in the past five or ten years we've gone from having very few options for lockstep microcontrollers (with the Hercules being a very compelling option) to being spoiled for choice, the Hercules is still a good fit for some applications, and is a pretty solid chip.

jcalvinowens 2 hours ago | parent | prev | next [-]

Don't ignore endianness. But making little endian the default is the right thing to do, it is so much more ubiquitous in the modern world.

The vast majority of modern network protocols use little endian byte ordering. Most Linux filesystems use little endian for their on-disk binary representations.

There is absolutely no good reason for networking protocols to be defined to use big endian. It's an antiquated arbitrary idea: just do what makes sense.

sllabres 27 minutes ago | parent | prev | next [-]

Not only the System/390. Its also IBM i, AIX, and for many protocols the network byte order. AFAIK the binary data in JPG (1) and Java Class [2] files a re big endian. And if you write down a hexadecimal number as 0x12345678 you are writing big-endian.

(1) for JPG for embedded TIFF metadata which can have both.

[2] https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.ht...

hmry 5 minutes ago | parent [-]

The endianness of file formats and handwriting is irrelevant when it comes to deciding whether your code should support running on big-endian CPUs.

The only question that matters: Do your customers / users want to run it on big-endian hardware? And for 99% of programmers, the answer is no, because their customers have never even been in the same room as a big-endian CPU.

nyrikki 2 hours ago | parent | prev | next [-]

The linked to blog post in the OP explains this better IMHO [0]:

   If the data stream encodes values with byte order B, then the algorithm to decode the value on computer with byte order C should be about B, not about the relationship between B and C.
One cannot just ignore the big/little data interchange problem MacOS[1], Java, TCP/IP, Jpeg etc...

The point (for me) is not that your code runs on a s390, it is that you abstract your personal local implementation details from the data interchange formats. And unfortunately almost all of the processors are little, and many of the popular and unavoidable externalization are big...

[0] https://commandcenter.blogspot.com/2012/04/byte-order-fallac... [1] https://github.com/apple/darwin-xnu/blob/main/EXTERNAL_HEADE...

whizzter 33 minutes ago | parent [-]

MacOS "was" big-endian due to 68k and later PPC cpu's (the PPC Mac's could've been little but Apple picked big for convenience and porting).

Their x86 changeover moved the CPU's to little-endian and Aarch64 continues solidifies that tradition.

Same with Java, there's probably a strong influence from SPARC's and with PPC, 68k and SPARC being relevant back in the 90s it wasn't a bold choice.

But all of this is more or less legacy at this point, I have little reason to believe that the types of code I write will ever end up on a s390 or any other big-endian platform unless something truly revolutionizes the computing landscape since x86, aarch64, risc-v and so on run little now.

GandalfHN 2 hours ago | parent | prev | next [-]

Outsourcing endianness pain to your customers is an easy way to teach them about segfaults and silent data corruption. s390x is niche, endian bugs are not.

Network protocols and file formats still need a defined byte order, and the first time your code talks to hardware or reads old data, little-endian assumptions leak all over the place. Ignoring portability buys you a pile of vendor-specific hacks later, because your team will meet those 'irrelevant' platforms in appliances, embedded boxes, or somebody else's DB import path long before a sales rep waves a support contract at you.

AKSF_Ackermann an hour ago | parent | next [-]

Not sure why you consider that to be an issue, if you need to interact with a format that specifies values to be BE, just always byte-swap. And every appliance/embedded box i had to interact with ran either x86 or some flavour of 32-bit arm (in LE mode, of course).

7jjjjjjj an hour ago | parent | prev [-]

Assuming an 8-bit byte used to be a "vendor specific hack." Assuming twos complement integers used to be a "vendor specific hack." When all the 36-bit machines died, and all the one's complement machines died, we got over it.

That's where big endian is now. All the BE architectures are dying or dead. No big endian system will ever be popular again. It's time for big endian to be consigned to the dustbin of history.

cmrdporcupine 39 minutes ago | parent [-]

> No big endian system will ever be popular again

Cries in 68k nostalgia

bear8642 3 hours ago | parent | prev | next [-]

> the only relevant big-endian architecture is s390x

The adjacent POWER architecture is also still relevant - but as you say, they too can afford a support contract.

AKSF_Ackermann an hour ago | parent [-]

The adjacent POWER architecture seems to be used in ppc64le mode these days.

EPWN3D 3 hours ago | parent | prev [-]

I mostly agree, but network byte ordering is still a thing.

electroly 3 hours ago | parent | prev | next [-]

> When programming, it is still important to write code that runs correctly on systems with either byte order

I contend it's almost never important and almost nobody writing user software should bother with this. Certainly, people who didn't already know they needed big-endian should not start caring now because they read an article online. There are countless rare machines that your code doesn't run on--what's so special about big endian? The world is little endian now. Big endian chips aren't coming back. You are spending your own time on an effort that will never pay off. If big endian is really needed, IBM will pay you to write the s390x port and they will provide the machine.

Retr0id 3 hours ago | parent | next [-]

> There are countless rare machines that your code doesn't run on--what's so special about big endian?

One difference is that when your endian-oblivious code runs on a BE system, it can be subtly wrong in a way that's hard to diagnose, which is a whole lot worse than not working at all.

electroly 3 hours ago | parent | next [-]

That sounds like a problem to deal with as part of your paid IBM s390x porting contract. I guess my point is: why deal with this before IBM is paying you? No other big endian platform matters, and s390x users are 100% large commercial customers. If IBM or one of their customers isn't paying you, there's nobody else who would need it. If IBM is paying you, you can test on a real z/VM that they provide. I see big endian as entirely their burden now; nobody else needs it. If they want it, they can pay for the work.

Retr0id 3 hours ago | parent [-]

I value correct code for purely selfish reasons. The most likely person to try to run my code on a BE system is me.

Retr0id 2 hours ago | parent | next [-]

Also, endian-correct code is usually semantically clearer. For example, if you're reading network-ordered bytes into an int, an unconditional endian swap (which will produce correct results on LE systems but not BE) is less clear than invoking a "network bytes to u32" helper.

eesmith 3 hours ago | parent | prev [-]

There are a lot of odd (by modern standards) machines out there.

You're also the most likely person to try to run your code on an 18 bit machine.

fc417fc802 an hour ago | parent [-]

It might sound outrageous but I guard against this sort of thing. When I write utility code in C++ I generally include various static asserts about basic platform assumptions.

edflsafoiewq an hour ago | parent | prev [-]

Static-assert the machine is little endian.

Retr0id an hour ago | parent [-]

Someone's LLM will comment out that line the moment it causes a build failure

edflsafoiewq an hour ago | parent [-]

Oh brave new world, that has such arguments in it!

CJefferson 3 hours ago | parent | prev [-]

You are correct, honestly, I couldn't disagree more with th article. At this point I can't imagine why it's important.

It's also increasingly hard to test. Particularly when you have large expensive testsuites which run incredibly slowly on this simulated machines.

bluGill 2 hours ago | parent | prev | next [-]

What I really want is memory order emulation. X86 as strong memory order guarantees, ARM has much weaker guarantees. Which means the multi-threaded queue I'm working on works all the time on development x86 machine even if I forget to put in the correct memory-order schematics, but it might or might not work on ARM (which is what my of my users have). (I am in the habit of running all my stress tests 1000 times before I'm willing to send them out, but that doesn't mean the code is correct, it means it works on x86 and passed my review which might miss something)

newpavlov an hour ago | parent [-]

For Rust we have Loom [0], but do not expect for it to work on your whole application.

[0]: https://github.com/tokio-rs/loom

susam an hour ago | parent | prev | next [-]

I wrote a similar post [1] some 16 years ago. My solution back then was to install Debian for PowerPC on QEMU using qemu-system-ppc.

But Hans's post uses user-mode emulation with qemu-mips, which avoids having to set up a whole big-endian system in QEMU. It is a very interesting approach I was unaware of. I'm pretty sure qemu-mips was available back in 2010, but I'm not sure if the gcc-mips-linux-gnu cross-compiler was readily available back then. I suspect my PPC-based solution might have been the only convenient way to solve this problem at the time.

Thanks for sharing it here. It was nice to go down memory lane and also learn a new way to solve the same problem.

[1] https://susam.net/big-endian-on-little-endian.html

mistrial9 an hour ago | parent [-]

"memory lane" !!

zajio1am an hour ago | parent | prev | next [-]

There is one reason not mentioned in the article why it is worth testing code on big-endian systems – some bugs are more visible there than on little-endian systems. For example, accessing integer variable through pointer of wrong type (smaller size) often pass silently on little-endian (just ignoring higher bytes), while read/writ bad values on big-endian.

ncruces an hour ago | parent | prev | next [-]

If you're using Go on GitHub (and doing stuff where this actually matters) adding this to your CI can be as simple as this: https://github.com/ncruces/wasm2go/blob/v0.3.0/.github/workf...

On Linux it's really as simple as installing QEMU binfmt and doing:

   GOARCH=s390x go test
beached_whale 23 minutes ago | parent | prev | next [-]

I've used docker buildx to do this in the past. Easier to work with than qemu directly(it does so under the hood).

bluGill 2 hours ago | parent | prev | next [-]

For most code it doesn't matter. It matters when you are writing files to be read by something else, or when sending data over a network. So make sure the places where those happen are thin shims that are easy to fix if it doesn't work. (that is done write data from everywhere, put a layer in place for this).

eisbaw 3 hours ago | parent | prev | next [-]

I did that many years back, but with MIPS and MIPSel: https://youtu.be/BGzJp1ybpHo?si=eY_Br8BalYzKPJMG&t=1130

presented at Embedded Linux Conf

pragmaticviber 4 hours ago | parent | prev | next [-]

It's all fun and games until you have to figure out if the endianness bug is in your code or in QEMU's s390x emulation.

rurban 4 hours ago | parent [-]

Haven't found any bug in QEMU's s390x, but lots in endian code.

throwaway2027 3 hours ago | parent | prev [-]

Is there any benefit in edge cases to using big-endian these days?