Yeah this stuff isn't even realistic as well.

A number of years ago I was working on something professionally and there was a problem. Only about 1 in 5 boards assembled wouldn't crash the CPU. After much debugging it turned out one of the ICs had an open collector output and it wasn't loaded correctly with a pull up resistor. This caused a cascading failure, held the bus up when initialising the hardware which hit the WDT and reset the CPU over and over again.

If you aren't there designing the thing in the first place, you never read the datasheets, never drew the schematic, never placed the components and thus don't know where to look when something goes wrong. And it does go wrong. And then you're in deep shit.

I worry about people who think they can get a product out of the door with this stuff but can't.

In the pure software domain, this is solved by letting the AI own the entire loop. The AI writes the code, runs the code, tests the code, troubleshoots the code and fixes the code.

Embedded might be resistant to it, because software-hardware interactions are notoriously hard to sim, and AI still struggles with meatspace operations.

Not that it would stop anyone!

You say "people who think they can get a product out of the door with this stuff but can't" and I immediately think: Arduino. That was also seen as a way to introduce people who understand nothing about embedded to embedded. Surely no one would ever go from an Arduino prototype to an actual production run?

Ha ha WRONG. I've seen actual production hardware ship with Arduino firmware, because no one cared enough to fully rewrite that cobbled together Arduino firmware from the first prototype. The FW team just went over it enough to whack-a-mole the most obvious issues, and shipped the result.

So, no. People are absolutely going to ship AI genned embedded hardware, and get away with it too. I bet that by now, someone already did.

I'm with you on the complexity of debugging hardware, as I've been there too. But I will say, automatically adding pullups to floating pins is something that an automated process could succeed at. Not first gen of blindly throwing an LLM at circuits and hoping, but perhaps a later improvement that uses metadata about parts. Perhaps curated metadata about parts, or always starting with a reference design and tweaking from there.

Everything you said is exactly the proper argument against vibe coding.

If you can’t or don’t entirely go over the output, the failure mode is invisible.