| ▲ | h05sz487b 4 hours ago |
| > It is very much like playing an instrument. Or it is more like playing a slot machine and you imagine the rest. |
|
| ▲ | cube00 4 hours ago | parent | next [-] |
| This is how I feel whenever I see bold all caps instructions in a system prompt or someone claims they conducted "research" and found the magic prompt template that makes the model pay out. Maybe it works some of the time but it isn't a solution that works everytime. It reminds me of people hovering to play a slot machine when someone gets up and it hasn't paid out as if they've solved slot machines. While I don't mind putting something in a loop until the tests pass, I'm less comfortable doing that when providers are silently rerouting to lower quality models, or in Google's case burning quota faster to ease their own server load without being transparent about what the "standard limits" are to begin with. [1] I'm hopeful I'll be more comfortable with these "slot machines" when frontier models get to the point where they can be run locally on hardware I can actually afford so I know exactly what I'm getting and not jumping at shadows with providers playing tricks behind the scenes to ease their own load without admitting the customer is getting less for their money as they get more popular. [1]: https://support.google.com/gemini/answer/16275805?hl=en&sjid... |
| |
| ▲ | coldtea an hour ago | parent [-] | | >This is how I feel whenever I see bold all caps instructions in a system prompt or someone claims they conducted "research" and found the magic prompt template that makes the model pay out. Maybe it works some of the time but it isn't a solution that works everytime. For such thing to be useful, it's enough that they works substantially more times that not having those instructions in. |
|
|
| ▲ | hodgehog11 3 hours ago | parent | prev | next [-] |
| A poor analogy depending on the setting because you can't adjust the odds with a slot machine, and the ROI is negative by design. If that's your experience, yeah, I wouldn't use an LLM either. |
|
| ▲ | ramon156 4 hours ago | parent | prev | next [-] |
| Instruments are pseudo-random until you know what you're doing. Slot machines are just slot machines |
| |
| ▲ | Forgeties79 2 hours ago | parent | next [-] | | Musical instruments are not random. You’re just doing random inputs. Instruments are consistent, even if the “flavor” and quality varies with different builds. Playing a B on a saxophone always plays a B. | | |
| ▲ | dotancohen 2 hours ago | parent [-] | | Saxophone, being a wind instrument was a bad choice. I can definitely tell which student was blowing when hearing a note. But your analogy remains solid if you substitute e.g. a piano and a reasonably proficient player. A single note would be nearly indistinguishable between players... But a full piece most certainly will sound different. | | |
| ▲ | palata 2 hours ago | parent | next [-] | | While I agree with you, I think it's diverging from the initial point. The original take was "LLMs are very much like playing an instrument". I think they are very much NOT like playing an instrument. While different musicians will produce different results, one musician won't get drastically different results on different days or when trying a different "copy" of the same instrument. If you can play the violin on your violin and I lend you my violin, you will still be able to play very consistently. You may argue that the sound will differ and you will have to adapt slightly, but that's not remotely similar to the randomness coming from LLMs. | |
| ▲ | Forgeties79 2 hours ago | parent | prev [-] | | A poor B is still a B fingering and the sax is supposed to play a B every time. Missing it is human error, not tool error. I can pick up an alto sax, a clarinet, etc. any time, anywhere, and expect the same fingerings to work every time. My individual skill or mistakes or peculiarities of each build are not what is relevant here. LLM’s do not operate consistently and make their own errors while we argue about which incantation makes it less inconsistent, knowing it will never actually perform as expected. I played woodwinds regularly for 15 years so I feel fine with my example. |
|
| |
| ▲ | 3 hours ago | parent | prev [-] | | [deleted] |
|
|
| ▲ | glerk 4 hours ago | parent | prev | next [-] |
| It is a bit of both. A non-deterministic instrument and a predictable slot machine. |
|
| ▲ | psychoslave 4 hours ago | parent | prev [-] |
| I play slot machines as instrument! ;) |
| |
