Remix.run Logo
nerevarthelame 11 hours ago

I agree with most of this, but my one qualm is the notion that LLMs "are particularly good at generating ideas."

It's fair enough that you can discard any bad ideas they generate. But by design, the recommendations will be average, bland, mainstream, and mostly devoid of nuance. I wouldn't encourage anyone to use LLMs to generate ideas if you're trying to create interesting or novel ideas.

dummydummy1234 10 hours ago | parent | next [-]

I have found the one of the better use cases of llms to be a rubber duck.

Explaining a design, problem, etc and trying to find solutions is extremely useful.

I can bring novelty, what I often want from the LLM is a better understanding of the edge cases that I may run into, and possible solutions.

globular-toast 36 minutes ago | parent | next [-]

Do you not know any people who can help? Suddenly realised how lonely this sounds.

apsurd 21 minutes ago | parent [-]

Coordinating with people is hard and only gets harder as you live. And actually, finding someone that is earnestly receptive to hearing you pitch your half-baked startup ideas (just an example) and is in some capacity qualified to be at all helpful, is uhhh, not easy.

globular-toast 10 minutes ago | parent [-]

Really? Sometimes I think I'm not very social, then I read something like this. Don't you have any friends? Colleagues? Maybe that's the problem you need to solve rather than sitting in a room burning energy for endless token streams with LLMs that anyone has access to?

monkaiju 10 hours ago | parent | prev [-]

I always find folks bringing up rubber ducking as a thing LLMs are good at to be misguided. IMO, what defines rubber ducking as a concept is that it is just the developer explaining what their doing to themselves. Not to another person, and not to a thing pretending to be a person. If you have a "two way" or "conversational" debugging/designing experience it isnt rubber ducking, its just normal design/debugging.

The moment I bring in a conversational element, I want a being that actually has problem comprehension and creativity which an LLM by definition does not.

__mharrison__ 31 minutes ago | parent | next [-]

My understanding was that rubber ducking was using a different portion of your brain by speaking the words.

The same discovery often happens when you explain a problem to a coworker and midway through the explanation you say "nvm, I know what I did wrong"

satvikpendem 10 hours ago | parent | prev | next [-]

Sometimes I don't want creativity though, I'm just not familiar enough with the solution space and I use the LLM as a sort of gradient descent simulator to the right solution to my problem (the LLM which itself used gradient descent when trained, meta, I know). I am not looking for wholly new solutions, just one that fits the problem the best, just as one could Google that information but LLMs save even that searching time.

bandrami 5 hours ago | parent [-]

> I'm just not familiar enough with the solution space

Neither is the LLM

jimbokun 3 hours ago | parent | next [-]

No, this is the kind of thing LLMs are very good at. Knowing the specifics and details and minutiae about technologies, programming languages, etc.

satvikpendem 5 hours ago | parent | prev [-]

Oftentimes it is though, good enough for my purposes.

bandrami 4 hours ago | parent [-]

If you're not familiar with the problem space, by definition you don't know whether or not that's the case. The problem spaces I do know well, I know the LLM isn't good at it, so why would I assume it's better at spaces I don't know?

satvikpendem 4 hours ago | parent [-]

I said familiar enough, not familiar. For example, let's say I'm building an app I know needs caching, the LLM is very good at telling me what types of caching to use, what libraries to use for each type, and so on, for which I can do more research if I really want to know specifically what the best library out of all the rest are, but oftentimes its top suggestion is, like I said, good enough for my purpose of e.g. caching.

bandrami 4 hours ago | parent [-]

I still don't get what you're saying. If you possess enough information to accurately judge the LLM's suggestions you possess enough information to decide on your own. There's not really a way around that.

satvikpendem 4 hours ago | parent | next [-]

Of course I'm deciding on my own, I'm not letting the LLM decide for me (although some people do). But the point is whatever the suggestion is is merely an implementation detail that either solves my problem or not, not sure what part of that is confusing. Replace LLM with glorified Google and maybe it's less confusing.

bandrami 4 hours ago | parent [-]

No, Google (at least back when it worked) ranked results based on the feedback of other users, so it was a useful signal.

satvikpendem 4 hours ago | parent [-]

Theoretically the LLM would weight more popular suggestions more too. Regardless you're reading too much into this, either use the LLM or don't, I'm not sure if someone else can convince you. As I said for my purposes of getting shit done it works perfectly fine and works more like a research tool than anything else, especially if it can understand my specific use case unlike general research tools like Google or Stack Overflow.

bandrami 3 hours ago | parent [-]

IDK man this sounds a lot like my junior devs saying "it works fine for me" as they hand in PRs that break prod

satvikpendem 3 hours ago | parent [-]

If you don't review the code it generates then that's still on you. There isn't an excuse for handing in breaking PRs like your juniors. It's a tool at the end of the day and it's the responsibility of the user to utilize it correctly.

jimbokun 3 hours ago | parent | prev [-]

Do you use search engines or do you just memorize all the world’s information?

bandrami 2 hours ago | parent [-]

I don't use search engines for much of anything nowadays (does anybody still?) At work I read documentation if I need to learn something.

WCSTombs 6 hours ago | parent | prev | next [-]

Absolutely, the whole point of the rubber duck is that it's inanimate. The act of talking to the rubber duck makes you first of all describe your problem in words, and secondly hear (or read) it back and reprocess it in a slightly different way. It's a completely free way to use more parts of your brain when you need to.

LLMs are a non-free way for you to make use of less of your brain. It seems to me that these are not the same thing.

Waterluvian 9 hours ago | parent | prev | next [-]

Maybe it’s just a semantic distinction, which, sure. I guess I’d just call it research? It’s basically the “I’m reading blogs, repos, issue trackers, api docs etc. to get a feel for the problem space” step of meaningful engineering.

But I definitely reach for a clear and concise way to describe that my brain and fingers are a firewall between the LLM and my code/workspace. I’m using it to help frame my thinking but I’m the one making the decisions. And I’m intentionally keeping context in my brain, not the LLM, by not exposing my workspace to it.

dexwiz 10 hours ago | parent | prev | next [-]

Sometimes people just need something else to tell them their ideas are valid. Validation is a core principle of therapeutic care. Procrastination is tightly linked to fear of a negative outcome. LLMs can help with both of these. They can validate ideas in the now which can help overcome some of that anxiety.

Unfortunately they can also validate some really bad ideas.

jrowen 9 hours ago | parent | prev | next [-]

I feel I've had the most success with treating it like another developer. One that has specific strengths (reference/checklists/scanning) and weaknesses (big picture/creativity). But definitely bouncing actual questions that I would say to a person off it.

10 hours ago | parent | prev [-]
[deleted]
raincole 10 hours ago | parent | prev | next [-]

Mainstream ideas are often good. That's why they're mainstream. Being different for being different isn't a virtue.

That being said I don't think LLMs are idea generators either. They're common sense spitters, which many people desperately need.

furyofantares 10 hours ago | parent | prev | next [-]

I'm torn.

I sometimes use them when I'm stuck on something, trying to brainstorm. The ideas are always garbage, but sometimes there is a hint of something in one of them that gets me started in a good direction.

Sometimes, though, I feel MORE stuck after seeing a wall of bad ideas. I don't know how to weigh this. I wasn't making progress to begin with, so does "more stuck" even make sense?

I guess I must feel it's slightly useful overall as I still do it.

ozozozd an hour ago | parent | prev | next [-]

Having to fix the LLM’s recommended solution is a good exercise though.

It’s like being a smart-ass for the right reasons, without any social consequences.

jrowen 9 hours ago | parent | prev | next [-]

I think it's just a confusing use of the term "generating." It's thinking of the LLM as a thesaurus. You actually generate the real idea -- and formulate the problem -- it's good at enumerating potential solutions that might inspire you.

jimbokun 3 hours ago | parent [-]

That can be very valuable.

samusiam 7 hours ago | parent | prev | next [-]

"by design, the recommendations will be average"

This couldn't be more wrong. The simplest refutation is just to point out that there are temperature and top-k settings, which by design, generate tokens (and by extension, ideas) that are less probable given the inputs.

dgxyz 10 hours ago | parent | prev | next [-]

All LLM output is always dry as fuck quite frankly. At all levels from ideas and concepts through to the actual copy. And that’s dotted with pure excrement.

I think the only reason it’s seen as good anywhere is there are a lot of tasteless and talentless people who can pretend they created whatever was curled out. This goes for code as well.

If I offend anyone I will not be apologising for it.

gavmor 9 hours ago | parent [-]

Agreed! No LLM is producing Pynchon, Calvino, Borges, Castaneda, Le Guin, Vonnegut.

dgxyz 9 hours ago | parent | next [-]

I think that’s an unfair comparison. It can’t even produce Mills and Boon trash.

fcarraldo 8 hours ago | parent | prev [-]

But can they produce Tom Clancy or James Patterson?

lioeters 5 hours ago | parent [-]

Malcollm Gladwell

NewsaHackO 10 hours ago | parent | prev | next [-]

Yes, I didn't get this portion at all. I feel as though letting an LLM brainstorm ideas for you would be worse in externally framing your thoughts than letting it write/proofread for you. If you pick one idea out of the 10 presented by the LLM, you are still confining yourself to the intersection of what the LLM thinks is important and what you think is important, because then you can never "generate" a thought that the LLM hasn't presented.

paulpauper 10 hours ago | parent | prev | next [-]

LLMs can come sometimes up with novel or non-obvious insights...or just regurgitate google-like results.

j45 10 hours ago | parent | prev [-]

Asking the LLM better will return better than average and bland and mainstream results.

paulryanrogers 10 hours ago | parent | next [-]

How does one ask better? Does better vary per model?

j45 9 hours ago | parent [-]

Yes, its context based.

It's like asking a coworker. Providing too little information, or too much context can give different responses.

Try asking the model to not provide it's most common or average answer.

Been using it this way for 2, almost 3 years.

contagiousflow 10 hours ago | parent | prev [-]

Why would they return "better" results?

j45 9 hours ago | parent [-]

Because AI is not a search engine. It does not return the best search result every time.

What it considers best, is what occurs most often, which can be the most average answers. Unless the service is tuned for search (perplexity, or google itself for example), others will not provide as complete an answer.

How well we ask can make all the difference. It's like asking a coworker. Providing too little information, or too much context can give different responses.

Try asking the model to not provide it's most common or average answer.

Been using it this way for 2, almost 3 years.