Around the same time the LLM also faces an existential risk when its advice is wrong.

Not trusting my life to an “intelligence” that has nothing to lose.

Sometimes the things that people say, says more about themselves than the thing they said.

If the only reason you give advice is predicated on the transactional value of that action, I think you've given up on acting ethically.

While to some extent we all are motivated by some abstract transactional value, I think immediately perceiving and reducing everything to a surivivalist, transactional mentality is an empty, cynical existence.

▲

nkrisc 12 hours ago | parent [-]

> If the only reason you give advice is predicated on the transactional value of that action, I think you've given up on acting ethically.

I said nothing of the sort.

Let me give a different example: I would not ride on a plane being flown remotely by a pilot on the ground. But I will ride on a plane with the pilots aboard. I do not necessarily believe the remote pilot would intentionally endanger me, but at the same time they are not facing mortal peril if they screw up. So I would need some other reason to trust them (maybe the remote pilot is a family member).

An AI has nothing to lose for misidentifying a mushroom, and I have my life to lose by trusting it. It can’t even fear for my life on my behalf.

▲

altcognito 11 hours ago | parent [-]

Neither does a radiologist but yet we trust them to read a CT scan. Your relatives can sue if there is malpractice, but in practice, misreading happens, and nobody goes to jail for that.

Everything performs a particular task to the best of its ability and we all accept the level of risk we are willing to tolerate. If machines gain trust from enough sources, it will be accepted.

All I'm saying here is (if you walk back the necessity of having "skin in the game") -- all I'm saying here is, we regularly trust our lives to situations and people we expect to reasonably fulfill a given task. Most of us get in an elevator, or drive in a car, or even just buy food from a store.

	▲	nkrisc 11 hours ago \| parent [-]
		> Neither does a radiologist but yet we trust them to read a CT scan. I also don’t really have any other options in that regard, so there’s no real choice to make, other than getting a second opinion. I may not have a chance for a second opinion if I fully trust an AI on mushroom ID. > Most of us get in an elevator, or drive in a car, or even just buy food from a store. Of course, and I do too. Mostly because I have to and because there are many aspects of those risks that remain under my direct control. At times I have decided to take the stairs instead of the elevator.