Excuse my ignorance but how does OS development even work? Being a mostly web developer, it just seems to arcane to change an OS - like, pre-application layer.

Not unlike web development, you're working against a contract (in this case, CPU instructions/features, storage formats, bus formats) defined by some megacorporate legacy. You also have support from pieces of embedded firmware running inside storage devices or other peripherals.

The book at http://ostep.org gets into some details.

Another useful illuminator: https://gwern.net/computers

As much as possible is generally done using simulators before testing on any real hardware.

Once you're smart enough, the oscilloscope isn't that different from console.log().