Remix.run Logo
m_ke 7 hours ago

It's not an issue of usual vs unusual, I'm saying the models are way better at writing and maintaining django or react code bases than your own hand rolled architecture that you define in some docs that it has to learn and keep in context. All of the models do an amazing job making local edits and working in small greenfield projects but once you get to full production systems with close to a million lines things start to rot. The code still works and the models are able to make progress but all of a sudden you have 3-4 different versions of your concepts sprinkled in random corners of your code base.

My second point is that the models are way better at things like Rust or Lua than Python or JS, because the average person producing code in those languages has way more programming experience, so the code quality of training data online in those languages is higher than the programming 101 medium blogspam type content you see in more popular intro languages.

justincormack 6 hours ago | parent | next [-]

At 600k lines of Rust, not finding those issues yet. You can search for different versions of same concepts, I mean you need to refactor a lot anyway.

Chu4eeno 4 hours ago | parent [-]

Did you actually manually review the 600k lines of code?

And the reason LLMs tend to produce such insanely large codebases is partially what the comment you're replying to explained, they duplicate everything all over the place.

rapind 6 hours ago | parent | prev | next [-]

I just finished an LLM assisted Rust rewrite of a production project after a couple months. It's around 150k LOC, mostly Rust, excluding generated code and tests. Around 200k if I include tests. I'm also now starting to extract / refactor some of it's code into their own reusable crates, so my LOC isn't going to grow and might even shrink a bit. I can't speak to 1M LOC, but I'd imagine that takes a lot of human context to reason about too, and you probably want to break it up into manageable chunks if possible.

LLMs have been a boon. I have a lot of guardrails in place though to keep the LLMs focused and producing correct and ergonomic code. A lot of effort and review has gone into creating these guardrails (choice of language, linting, forced conventions, tests, etc.). A lot of these practices will be useful in future projects. I highly recommend this approach. Don't just copy skills or processes that others publish. Figure it out yourself so the lessons stick. Human in the loop, especially one that has learned through practice and failure, is still immensely valuable IMO. It's a fine line between getting stuff done and yakshaving over your tooling... but that's always been the case, as proven by my vim dotfiles. :)

It's worth noting that the AI industry is filled with tons of snakeoil and BS. You don't need to take someone's course in "21 steps to be productive with AI" or w/e. Just like learning a new programming language, dig right in and try to build something that scratches an itch. Pick a project you have complete autonomy over instead of that "spend all the tokens" crap managers were pushing until recently (this has corrected now right?).

Parent saying "Skill issue" though... What a jackarse.

logicprog 7 hours ago | parent | prev [-]

I haven't hit nearly a million lines, that's true, these projects are "only" 20k and 45k lines of code or so, respectively; however, I do have directed acyclic graphs of worker agents, managed asynchronously by the agent I chat directly with, do huge architectural refactors (the most recent, was to update everything in the codebase that used the sychronous Lua standard library `io` functions to use the asynchronous, multithreaded IO system I created for the editor — at least, everything that could use the new library, which I kept deliberately simple in a few ways, and everything that wasn't already temporary, but you know) and that's turned out quite well, and at least for me, 45k sloc is nothing to sneeze at, especially since with a green field project there's less examples to keep them on track. I do regularly check in to make sure it's using my design to the fullest, as well.

noopprod 6 hours ago | parent [-]

[dead]