Remix.run Logo
captainbland 4 hours ago

This is actually what keeps people using AI: variable reward schedule. It's basically gambling.

dpark 2 hours ago | parent | next [-]

I see this sentiment pretty regularly, and I don’t get it. Variable rewards is not sufficient to establish that it is “ basically gambling”.

Everything in life is variable reward. You invite a friend over, they might accept or they might not. Drive to work, traffic might be good or might be bad. You ask a colleague to finish a task, they might do it or might not or might do a good job or might not.

Everything is variable reward. Is everything gambling?

smugglerFlynn 2 hours ago | parent | next [-]

You invite a friend over, but raccoon appears. Then pigeon appears. Then friend appears but at the last second suddenly becomes a banana. You remember you are out of bananas so you order more and also some cola zero cans on your local grocery delivery app. You are back to the party, but now you have 5 friends in the room, and you run de-duplication query. Now half of your friend is sitting at the sofa, and another half becomes a quarter of banana. Suddenly bananas arrive so you need to open the door. Once you are back there are no friends, pigeons or raccoons but also no bananas and no cola - all the delivery results are gone. This seems to be urgent and important, gotta fix this first before going back to that friend invitation...

pixl97 3 minutes ago | parent | next [-]

Your parties with friends involve a lot more acid than mine. Maybe I'm missing out.

dpark an hour ago | parent | prev [-]

This is not my experience with current AI models at all. But regardless you are not describing anything that sounds like gambling. You are describing a weird hallucinogenic experience.

rnjesus 2 hours ago | parent | prev | next [-]

i’m not sure “variable rewards” is the right term, but i do agree with the op that it is very similar to gambling.

regarding your examples, i think the difference is that with ai, you’re literally sitting in front of a machine, pressing a button, and (almost instantly) getting a result that, if not desired, can immediately be tried for again. you even spend “tokens” to do this, and at least in my native language, “token” brings to mind the coins you’d stick in a slot machine

dpark an hour ago | parent [-]

I don’t see much similarity beyond the most superficial.

If you sit at a slot machine and pump quarters into it, each “turn” is independent. You spin and you win or lose. It’s pure chance and there is no destination. You execute the exact same action over and over and hope random chance brings you more money.

If you sit down in front of a coding harness, the progress is incremental and directed. You ask for a thing, the LLM produces something that is hopefully close to what you wanted. You give it more direction to prod it closer to the end state you want. You are not executing the same action, but incrementally nudging it in the right direction. I’ve literally never restarted from the same initial state with the same prompt and hoped for a different result and I don’t know why anyone would. Rarely I’ve thrown away the progress made and started over but always with a very different prompt that includes learnings from the failed attempt.

rnjesus 28 minutes ago | parent [-]

i agree with you that it’s principally different from a slot machine, and that it’s possible to use it in a way (like you describe) that is much more focused, for lack of a better term, to great effect

most people don’t use ai this way though, and i still feel like the end-psychological reward mechanism is very, very similar to gambling regardless of how well one utilizes it (and this is even more obvious with image generation as you chase that perfect output)

perhaps it’s better to compare it to gacha than slots?

dpark 12 minutes ago | parent [-]

> most people don’t use ai this way though

How do they use it? Surely no one is just repeating the same prompt over and over (except as a Ralph loop perhaps, which is automated). I’m really struggling with the notion that most people just throw the same prompt repeatedly hoping it eventually works. Because that doesn’t sound like gambling. It sounds crazy (and frustrating).

> and i still feel like the end-psychological reward mechanism is very, very similar to gambling regardless of how well one utilizes it

In the sense that you get a dopamine reward when you succeed, sure, but I get the same reward when I code by hand and achieve a successful result.

> and this is even more obvious with image generation as you chase that perfect output

This is fair, because sometimes with image generation the same exact prompt will produce very different output. This is becoming less true as the models get better and it becomes more effective to direct image generation iteratively than to keep starting from scratch with a barely tweaked prompt.

howunfortunate 2 hours ago | parent | prev | next [-]

Same vibe as people saying "addicted to sugar" or "sugar hijacks your reward system"

Sugar is the original point of the reward system!

Barbing 2 hours ago | parent [-]

& you can cheat the reward system.

Do hard work (takes time), get dopamine for successful completion.

Find berries, taste sweet (hopefully safe), eat all, get calories. Doordash Krispy Kreme instead = few too many calories.

(I’m no Luddite in the sense popularly thought of them pre-‘22 [1], though we have to watch skill atrophy)

[1] regressionist? Decelerationist, too loaded perhaps. Someone remembers or knows the word…

utopiah 2 hours ago | parent | prev | next [-]

No, if I use a ruler or a pocket calculator they will reliabley give me the correct result. There is no gambling.

Tadpole9181 25 minutes ago | parent | prev [-]

I would agree that 1-2 years ago models were more "slot machine"-esque - sometimes the output was good, sometimes the output was bad. And as a result, I primarily used them for auto-complete functionality and bouncing ideas around. In those workflows, you can easily ignore it if the spin is wrong.

Not everyone has the desire to work around the system, and many are diametrically opposed to the concept of AI. They get this perception that it's a slot machine because of that inconsistency, and then do the human thing of assuming that other people must just be flawed if they're different from them. They're "addicted to gambling".

Obviously, things have changed. Open models can still be like that, but are often so fast and cheap at iterating it doesn't matter. SOTA models aren't perfect, but are to the point that they're generally much better than the average developer.

But once that perception set in and the meme spreads, it's really hard for some to break out of it. Especially at the pace AI development has been moving. It's just that simple.

gbraad 3 hours ago | parent | prev | next [-]

This is why I also suspect them to waste tokens on purpose.

BikiniPrince an hour ago | parent [-]

Listen Pal, this is load bearing. If you know what is good for you then you will stop asking questions. —Claude

an hour ago | parent | prev | next [-]
[deleted]
smugglerFlynn 2 hours ago | parent | prev | next [-]

Modern LLM services are engineer's pipe dream that was heavily shaped by the shadiest product management dark patterns you can find: applying gambling-style engagement tactics, exploiting cognitive biases, exploiting users' lack of technical understanding to inflate product expectations, using fear mongering in external and investor communications. And that's not even a complete list.

mysterydip 3 hours ago | parent | prev | next [-]

Which also explains why response speed is so important.

raincole an hour ago | parent | prev | next [-]

Gambling but with positive expected value, yes.

thepasswordis an hour ago | parent | prev | next [-]

Literally putting tokens into a machine and hoping profit comes out. Amazing.

jimmaswell an hour ago | parent | prev | next [-]

Programming before AI was always variable reward. It was a gamble against your own time and patience. Maybe I'd waste hours down the wrong rabbit holes trying to find a library that worked for my use case. Maybe I'd waste a day trying to get an API to do something it turned out it couldn't do. Maybe I'd have to redo my entire approach because of some factor I hadn't considered. Something I wrote could have worked on the first try or I could have had to spend the day chasing logic errors (or multiple days chasing memory errors if it was C or C++). Maybe I would just get bored of the project, especially if I realized there were 20 layers of yaks I needed to shave first, and Visual Studio got stuck updating again, and before I could even start actually coding I had to spend the entire evening on an exhausting merge conflict. My entire weekend could be gone with nothing to actually show for it.

I got so sick of all this at some point that I slowly stopped doing anything that wasn't my job. But then AI got better and better and I realized it was the ultimate unblocker. When that dreaded malaise started creeping in signaling it was a project's end because I didn't want to waste any more of my life dealing with bullshit orthogonal to what I was trying to do, I'd give it to the AI. It felt like a miracle the first time this worked, and it still does. If we were previously equipped with shovels to dig through bullshit, we now have a fully automated Bagger 288.

The reward schedule now isn't variable anymore; the chance that I finish something in a good state is 100%. I can focus on the parts I actually enjoy - architecting the broader system, making the parts mesh together in a sensible way that's easy to work with and has some mathematical elegance to it, hand coding the bits I want to be really specific about (but now without the endless frustration of bugfixing or import errors and edgecases being immediately discovered, thanks to the AI).

supern0va 41 minutes ago | parent | next [-]

>Maybe I'd waste a day trying to get an API to do something it turned out it couldn't do.

I was working on a side project recently. I had spent months designing the data model in my spare time, thinking through how to make it as elegant and durable to change as possible in the long term, since (if I launched it) the repercussions for getting it wrong would be significant.

Once I had a working design, it probably would have been several more months to build a working prototype and start testing it.

Instead, Claude knocked out the prototype for me in an afternoon. And it immediately became clear that it didn't work: not because the data model didn't solve all the problems I wanted it to solve, but because it didn't fit the shape of how I quickly learned a normal person would need/want to interact with the product. I was so focused on the long term, that I never thought about what the first five minutes of a user with hands on the thing would need. And the changes needed would be significant.

Maybe there's some variable reward mechanism. But I sure was glad to be able to pull that particular slot machine handle and learn that than waste even more of my time on what was a dead end.

bevr1337 34 minutes ago | parent | prev [-]

> It was a gamble against your own time and patience.

At this point, what do the words even mean? Your own patience and available time are always completely random and fairly distributed across a large enough data set?

> Maybe I'd waste hours down the wrong rabbit holes trying to find a library that worked for my use case. Maybe I'd waste a day trying to get an API to do something it turned out it couldn't do. Maybe I'd have to redo my entire approach because of some factor I hadn't considered.

Our ignorance isn't random chance. As we research and experiment, we reduce the problem area.

tiborsaas an hour ago | parent | prev | next [-]

Is it gambling if I'm beating the house?

varispeed 2 hours ago | parent | prev | next [-]

Something regulators should look at. They don't deliver consistent compute, yet charge consistent money. In my opinion that's fraud.

stavros 3 hours ago | parent | prev [-]

People say this, but I've never seen it. AI has been very consistent in its rewards for me.

wuisce 3 hours ago | parent [-]

You're absolutely right. And it matters.