Remix.run Logo
epolanski 5 days ago

I'll be more detailed in my second reply.

1) Your original post asks a lot if not too much out the LLM, the expectation you have is too big, to the point that to get anywhere near decent results you need a super detailed prompt (if not several spec documents) and your conclusion stands true: it might be faster to just do it manually. That's the state of LLMs as of today. Your post neither hints at such detailed and laborious prompting nor seem to recognize you've asked it too much, displaying that you are not very comfortable with the limitations of the tool. You're still exploring what it can and what it can't do. But that also implies you're yet not an expert.

2) The second takeaway that you're not yet as comfortable with the tools as you think you are is clearly context management. 2/3k locs of code are way too much. It's a massive amount of output to hope for good results (this also ties with the quality of the prompt, with the guidelines and code practices provided, etc, etc).

3) Neither 1 or 2 are criticisms of your conclusions or opinions, if anything, they are confirmations of your point that LLMs are not there. But what I disagree with is the rush into concluding that AI coding provides net 0 benefits out of your experience. That I don't share. Instead of settling on what it could do (help with planning, writing a spec file, writing unit tests, providing the more boilerplate-y part of the code) and use the LLM to reduce the friction (and thus provide a net benefit), you essentially asked it to replace you and found out the obvious: that LLMs cannot take care of non-trivial business logic yet, and even when they can the results are nowhere near being satisfactory. But that doesn't mean that AI-assisted coding is useless and the net benefit is 0, or negative, it only becomes so as the expectations on the tool are too big and the amount of information provided is either too small to return consistent results or too large for the context to be an issue.

Rochus 5 days ago | parent | next [-]

I don't know where your confidence or assumptions come from. Do you work for Anthropic? My prompts for the code generators included an 1.2kLOC code file plus detailed instructions (as described elsewhere), with more details during the session. So I don't think your points apply.

throwaway346434 5 days ago | parent | prev [-]

This is a kind of nuts take; - Senior engineer - Uses tools for non trivial undertaking - Didn't find value in it

Your conclusion from that is "but they are doing it wrong", while also claiming they are saying things they didn't say (0 net benefits, useless, etc).

Do you see how that might undermine your point? That you feel they haven't take the time to understand the tools, but you didn't actually read what what wrote?