  > The most fascinating part of this wave of LLM reverse engineering is seeing how many weird and poor choices were lying in plain sight.

I think what's interesting is that for decades one group of devs has been pushing for higher quality and security while other groups have been pushing for "good enough" or "MVP"[0]. Clearly the group that rushed won.

But in some weird way we're coming to the day of reckoning. I'm sure we'll always have missed bugs but with a lot of what we're seeing it more seems like these were avoidable problems. Code issues I can easily imagine at least one person on the team said "this isn't acceptable, we need to correct it" and them getting overruled. I'm sure each and every one of you has seen, if not experienced, this.

It'll be interesting to see if that continues. I suspect companies will still rush but from my experience even the best models happily produce shoddy code. And there's a feedback loop, the model's performance is influenced by the code's quality. So it's easy to degrade and get into a bad feedback loop. But I think people still value speed over quality, even in this world where Microsoft shit the bed

[0] in quotes because the entire conversation is essentially about what is actually good enough and what actually is the minimum viable product.

▲ mattlondon 15 hours ago | parent | next [-]

I have experienced that quite a few people who are otherwise decent (if not amazing) engineers just have huge blindspots in their knowledge sometimes.

I am sure I am one of them. Known unknowns, and unknown unknowns etc.

You might be shit hot at low-level kernel programming but know nothing about networking for example. Or amazing distributed system design and architecture but missing fundamentals about POSIX process handling etc. And so on and so on. I've seen this even in BigCos where hiring bar is high and they try to recruit "generalists". Not everyone can know everything.

	▲	godelski 4 hours ago \| parent [-]
		I think that's a different problem. Nobody is perfect. No code will ever be perfect (even if formalized). But I'm talking about the category of foreseeable issues. Known knowns. Not even known unknowns. This is a problem not even unique to software. There's many things we do that we know will create problems, or are very likely to, but are in the future so we put them off. Yes, there are trade-offs but that's never how we have those conversations. It's never about weighing the options. We generally have these conversations by dismissing each other's concerns (arguably not a conversation). There's a strong pattern that we dismiss future costs (or future rewards). Why pay $5 now when I could pay $1 is what people hear even if it's "in 10 years you'll make 1000x what you put in" or "in 10 years it'll cost you 1/(10^x) of your net worth". Now matters but we have to remember that so does the future. You can't be penny wise and pound foolish, unless you're a monopoly or extravagantly rich.

▲ _blk 20 hours ago | parent | prev | next [-]

I don't think it's fair to say that those who didn't push for more security pushed against it or just for good enough. They didn't care enough to push back, sure, they were coerced - maybe, they did wrong - probably. But there is a good enough in security too. If you want near-perfect security, keep if off the web. Don't get me wrong, I wish all products had minimal care for good practices, but I won't expect it the random Chinese made product.

▲ dns_snek 13 hours ago | parent | next [-]

> But there is a good enough in security too.

The disagreements usually arise from what exactly is considered "good enough". Your air purifier probably doesn't need quantum-proof cryptography and end-to-end encryption in case the transport layer security is compromised (e.g. emudhra).

The problem is that people will frequently argue that approaches like the one in the article (fetching root certs in an insecure manner) are "good enough" because they don't understand that the whole dance is worthless and connections might as well be unencrypted - at least that's honest.

	▲	godelski 4 hours ago \| parent [-]
		`> people will frequently argue that approaches like the one in the article are "good enough" because they don't understand` The same goes for complexity. I've routinely seen people argue against a simpler system by saying "stop making it complex" simply because they don't understand. Its true that complex means multiple things. But there's a strong push against elegance. I'll give an example. In one job we had to deal with a bunch of different file types. We had a function for each, about 500 lines for each one. Mostly copy pasted. Recognizing that, I wrote one 500 line function that allowed us to write a new file type routine (a common occurrence) in 5 lines. So I removed thousands of lines of code, increased code speed by 10x, decreased developer time by 2x, and by boss (who only ever imported the code, never wrote it) overruled because it was "too complicated" since I was using a C++ abstraction he didn't understand (guy barely knew C++. Like didn't know what a template was). The problem with both of these is that they slowly lower the bar. If the shortcut was "good enough" last time then another shortcut will be "good enough" this time. If X is "too complex" because I struggle to understand it, then Y is too. Which I find extra ironic since our field is quite egotistical and we seem to revel in the fact that we don't understand. But I guess Dijkstra was writing about that half a century ago. Which is plenty of time for things to go to shit

▲ godelski 16 hours ago | parent | prev [-]

  > But there is a good enough in security too.

Of course there is. But you got to at least he good enough to read my footnotes

▲ johndubchak 20 hours ago | parent | prev [-]

> "Clearly the group that rushed won"

This is has been the case since I started developing software in the 90's. As far as I, and likely many others, are concerned it might as well be a Software Aphorism.

I think it will be very interesting as we continue to reverse engineer such things to find out just how poorly "glued together" the Paper Tigers really are....

	▲	godelski 4 hours ago \| parent [-]
		`> This is has been the case since I started developing software in the 90's.` Dijkstra even wrote about it decades before that. But the issue is that this has a feedback loop. It doesn't take long for everything to be made of spaghetti and duct tape. Duct tape is a fine tool, but the real danger is a temporary fix that works is never temporary