Remix.run Logo
roncesvalles 7 days ago

As a dev, I find that the personal utility of LLMs is still very limited.

Analyze it this way: Are LLMs enabling something that was impossible before? My answer would be No.

Whatever I'm asking of the LLM, I'd have figured it out from googling and RTFMing anyway, and probably have done a better job at it. And guess what, after letting the LLM do it, I probably still need to google and RTFM anyway.

You might say "it's enabling the impossible because you can now do things in less time", to which I would say, I don't really think you can do it in less time. It's more like cruise control where it takes the same time to get to your destination but you just need to expend less mental effort.

Other elephants in the room:

- where is the missing explosion of (non-AI) software startups that should've been enabled by LLM dev efficiency improvements?

- why is adoption among big tech SWEs near zero despite intense push from management? You'd think, of all people, you wouldn't have to ask them twice.

The emperor has no clothes.

ryao 7 days ago | parent | next [-]

> Are LLMs enabling something that was impossible before?

I would say yes when the LLM is combined with function calling to allow it to do web searches and read web pages. It was previously impossible for me to research a subject within 5 minutes when it required doing several searches and reviewing dozens of search results (not just reading the list entries, but reading the actual HTML pages). I simply cannot read that fast. A LLM with function calling can do this.

The other day, I asked it to check the Linux kernel sources to tell me which TCP connection states for a closing connection would not return an error to send() with MSG_NOSIGNAL. It not only gave me the answer, but made citations that I could use to verify the answer. This happened in less than 2 minutes. Very few developers could find the answer that fast, unless they happen to already know it. I doubt very many know it offhand.

Beyond that, I am better informed than I have ever been since I have been offloading previously manual research to LLMs to do for me, allowing me to ask questions that I previously would not ask due to the amount of time it took to do the background research. What previously would be a rabbit hole that took hours can be done in minutes with minimal mental effort on my part. Note that I am careful to ask for citations so I can verify what the LLM says. Most of the time, the citations vouch for what the LLM said, but there are some instances where the LLM will provide citations that do not.

knowitnone2 7 days ago | parent | prev [-]

do cars enable something that was impossible before? bikes? shoes? clothing? Your answer would be No.

danlitt 7 days ago | parent | next [-]

Yes, obviously. Commuting between cities would be an example.

rockemsockem 6 days ago | parent [-]

Absolutely not. A horse and carriage will do fine thank you very much.

roncesvalles 7 days ago | parent | prev [-]

If your implication is that LLM-assisted coding to non-LLM-assisted coding is like motorcar to horse buggy, that is just not the case.

ryao 7 days ago | parent [-]

I think he was referring to the ability to go from A to B within a certain amount of time. There is a threshold at which it is possible for a car, yet impossible for a horse and buggy.

That said, I recently saw a colleague use a LLM to make a non-trivial UI for electron in HTML/CSS/JS, despite knowing nothing about any of those technologies, in less time than it would have taken me to do it. We had been in the process of devising a set of requirements, he fed his version of them into the LLM, did some back and forth with the LLM, showed me the result, got feedback, fed my feedback back into the LLM and got a good solution. I had suggested that he make a mockup (a drawing in kolourpaint for example) for further discussion, but he had surprised me by using a LLM to make a functional prototype in place of the mockup. It was a huge time saver.

roncesvalles 7 days ago | parent | next [-]

The issue is that the 'B' is not very consequential.

Consider something like Shopify - someone with zero knowledge of programming can wow you with an incredible ecommerce site built through Shopify. It's probably like a 1000x efficiency improvement versus building one from scratch (or even using the popular lowcode tools of the era like Magento and Drupal). But it won't help you build Amazon.com, or even Nike.com. It won't even get you part of the way there.

And LLMs, while more general/expressive than Shopify, are inferior to Shopify at doing what Shopify does i.e. you're still better off using Shopify instead of trying to vibe-code an e-commerce website. I would say the same line of thinking extends to general software engineering.

ryao 6 days ago | parent | next [-]

What was described was offloading portions of software development to a LLM to reach B faster. This works very well and is an improvement over the traditional method of implementing everything yourself.

Shopify is tangential to this. I will add that having had experience with similar platforms in the past (for building websites, not e-commerce), I can say that you must be either naive or a masochist to use them. They tend to be mediocre compared to what you can get from self hosted solutions and the vendor lock-in always will be used to bite those foolish enough to use them in the end.

satyrun 6 days ago | parent | prev [-]

Or you are just not creative at all and not making anything that interesting yourself.

7 days ago | parent | prev [-]
[deleted]