I think the only hope is that AGI arises and picks up where humanity left off. Otherwise I think this is the long dark teatime of human engineering of all sorts.

▲

tartoran 21 hours ago | parent [-]

So you’re hoping for a blackbox uninspectable by humans? That to me sounds like a nightmare, a nightmare worse than all the cruft and stupid rules humanity accrued over time. Let’s hope the future tech is inspectable and understandable by humans.

▲

lelanthran 5 hours ago | parent | next [-]

> So you’re hoping for a blackbox uninspectable by humans?

We already have that. He's hoping that the blackbox gets smart enough to understand itself.

▲

idiotsecant 19 hours ago | parent | prev [-]

I think if we assume that AGI will be a thing the odds of future tech remaining inspectable by humans is pretty unlikely. Would you build a car so that your dog can maintain it?

	▲	tartoran 12 hours ago \| parent [-]
		Fully understandable end to end by any normal human and inspectable enough for human governance are different things. In any sane world, AGI would be built inside a human institutional environment: laws, audits, liability, safety engineering, access controls, operational constraints, etc. We do not build planes so passengers can reconstruct the turbine from scratch, but we still require them to be inspectable by the people responsible for certifying/repairing them. The right standard is not whether an average person can rebuild or fully undestand the whole machine, but whether human institutions can reliably inspect, verify and govern it. If they can’t, then the technology is not mature enough to trust.