I like the analogy. It feels right. But I don't think it actually is, because it leans hard on the _feeling_ of failure, vs. the actual failure.

The failure that actually matters here is product failure. Angry users. Security incidents. Downtime. Broken workflows.

Stressed and uncomfortable developers have, historically, been a leading indicator of product failure. Software is complex and when devs can't handle the breadth of ownership + the depth of expertise at the same time, it means bad work that leads to product failure. It's not clear that agents fail in the same way, or that our collective intuition about how and when to trust agents is sufficiently developed to predict product failure in the same way.

As anecdata: I know a bunch of growth-SaaS/last-batch-of-IPO companies where the devs are all stressed, the feature ship rate is up, and incidents are down. All at the same time.

So yes, quick learners and just-in-time depth is important, but so is intuition about where discomfort is going to produce true failure. The discomfort isn't a feeling to throw away, simply trusting that you'll learn everything in time. Ownership requires holding a model of the system in your head and proactively steering where you dive.

▲

rahulsawhneyonl a few seconds ago | parent | next [-]

You can't teach breadth as a course, but you can deliberately engineer exposure to it. In the teams context, incident work is an example: putting newer engineers on the on-call rotation with a backup, having them write the incident review rather than just read it, and making a habit of reading other teams' postmortems. A good postmortem can reveal where the monsters live, written by someone who just met one.

I also think this matters more with agents, not less. When nobody wrote the code by hand, the incident history and the reasoning in design docs or code end up being the main record of why the system looks the way it does.

▲

augment_me 8 hours ago | parent | prev | next [-]

I think that this was true pre-attention economy. If you look at security incidents and other prod issues, they rarely move the needle today. I work at a hyperscalar and we have been shipping 40% more bugs to prod since 2023 year-to-year, yet user growth and revenue is up about 25% similarly due to more features and more ads.

I think that the whole premise of that failure affects things is based on a older model. You kind of assume that:

1) users are able to easily gain information about alternatives. We have seen that since LLMs replaced search engines, we have much less churn.

2) users are not behaviorally addicted/pushed into patterns. We have patterns that utilize users to keep using the app by exploiting their cognitive weaknesses

3) users have the mental fortitude and stamina to learn new tools instead of accepting flaws in the one they are used to

Some failures and bugs in prod don't matters as much

▲

taurath 7 hours ago | parent | next [-]

Almost the entire structure of software oriented companies, and why devs are expensive, are to prevent these sorts of errors. That job has gotten harder, and in some places impossible as management forces more trust on LLMs and reduces the capacity of devs to look. This is less of a problem in existing products in production and existing cultures - revenue is protected. Eventually those structures break down, as does the user perception of quality. Once the brands have taken enough knocks, it accelerates.

▲

trjordan 8 hours ago | parent | prev | next [-]

I agree that not all failures matter (most don't, in fact). On the other end of the spectrum, I know of a bunch of seed/series A startups that basically operate under a model where they ship everything and fix the stuff people complain about. As long as you do that fast enough, the median user experience is pretty good, because only the first n% actually see the problem before a fix is live.

I'll go out on a limb and guess that at a hyperscaler, shipping a bug to prod means hitting canary or 1% of traffic. Seems fine!

I think my point still stands, even as the shape of failures shift. There's certain ones that still hurt (data corruption, exploited security holes, etc.) because they're hard to notice. It might even be that the primary goal of ownership is to develop intuition about where those failures are likely to happen.

▲

ChrisLTD 8 hours ago | parent | prev | next [-]

There are prod issues, then there are issues that cost me money as a customer.

	▲	Arkada 6 hours ago \| parent [-]
		This very thing. "Shipment of 40% more bugs and fixing only in response to user complaints" could be done in social streams or advertising tech, but not in finance and trading where margin for errors is close to zero. A small bug implemented by an agent in the way R-multiples, stop loss distance, or P&L heatmaps are computed will never be complained about by the user—either he will trade incorrectly and lose his money, or see that single off-by-one difference compared to exchange ledger and churn forever.

▲

matkoniecz 6 hours ago | parent | prev [-]

>We have patterns that utilize users to keep using the app by exploiting their cognitive weaknesses

I hope you are aware that you are outright evil. I also hope that you will manage to stop justifying it as acceptable.

▲

augment_me 2 hours ago | parent | next [-]

You can take the right path and in the end no-one wants to use your product because it's not as engaging.

Instagram tried to take the "good" path in 2014 and lost 90% market share to TikTok who attentionmaxxed things. Then they had to take the exploitative path to claw back users.

As long as it's not regulated for everyone at the same time, you need to do this for survival of your product.

	▲	unsui an hour ago \| parent [-]
		maybe that product doesn't deserve to survive. "Hmmm, but if I cap the Torment Nexus, the competing Torment Core will win out, we can't have that!"

▲

pona-a 4 hours ago | parent | prev | next [-]

HN is one of the few sites where someone can launch into an actual villain monologue, the kind you'd read in leaked corporate emails, and receive approximately no negative reaction.

If this is the face of our industry, that might explain a lot of things...

▲

unsui 3 hours ago | parent | prev [-]

Agreed.

The real alignment issue isn't in the challenges getting AI to align to human values.

It's in humans understanding what is worth aligning to.

"He chose... poorly."

▲

adrq 9 hours ago | parent | prev | next [-]

> Ownership requires holding a model of the system in your head and proactively steering where you dive.

Absolutely, and the feeling of discomfort is a signal that you need to dive, a signal that your mental models might need updating. I think that builds up over time through many dives

▲

adithkb 5 hours ago | parent | prev [-]

[flagged]