Fully agree. I tried to refactor parts of a large code base with Fable+ultracode and it just keeps accidentally merging distinct concepts and making up explanations/reasonings that the code base did not contain.

For example, the code base contains a physical controller. It’s closed loop in that it can react in realtime to changes. But it’s a slightly untypical implementation because this one can even look into the future through simulations. But Fable does not understand that. Instead, I need to remind it every 30 minutes that this is closed loop. It keeps wrongly claiming that the controller was open loop and then based upon that it will make up constraints that don’t actually exist.

The patch-on-patch-on-patch is exactly right, nice way to describe it. It feels like, and I think is, optimized to find the quickest answer not necessarily the right answer.

[dead]

"It is not LLMs fault but you not knowing how to write a prompt". I know I know. But just giving all codebase and saying "rewrite it" is a no go. If e.g. going one class after class LLM will be exceptionally good at keeping the patterns and logics.

I mean it is a tool and you need to understand how the tool works. When there is too little context, where there is so much context so that you are poisoning it, when you are allowing the tool to do patch-on-patch and etc.

That's also true for humans.

I work on a codebase from the early 2000s, a lot of it using webforms, a long abandoned .NET technology. A rewrite preserving all behavior and making no observable changes whatsoever would be amazing. But it’s also tested exactly as well as you’d expect from something like that so I’d rather not let AI go wild.

One obvious target might be rewriting from an unsupported, broken, and/or obsolete target to something that still works. Or moving a project from a platform that no other system in the company uses to the same setup that all the others use.

Of course it won't quite work, but I can definitely see why some people would want that.

Say I rewrite a large codebase from python to C++, preserving all behavior. That's up to a 50x speed up.

Joel is right, but he's also wrong. I've been on the other side of a timid engineering culture that commerical rides roughshod over and its this depressing immeasurable decline. The company stagnates and slowly tailspins around an unmaintainable product until a competitor steals their lunch in a way that that further obscures cause and effect.

Estimates are considerably longer, QA is much harder, integration is full of buckets and rakes, some "senior" devs are afraid to touch stale core code, innovation is stifled, devs are frustrated, hiring is harder, attrition bites. The most frustrating thing is that its very hard to communicate the issues as everyone experiences a fragment of the pain and none of it lines up in a spreadsheet for anyone to appreciate the whole cost. Everything just sucks.

LLMs changing the economy of this sounds great, especially if removes the essential issue with the ground up rewrite, which is the "ground up" part.

That's very true. People put up with the many limitations of off the shelf software because it's cheaper, not because it's better. Developing bespoke software solutions is now a lot cheaper than it used to be. So, there are a lot of cases where that now becomes the better option.

Doing in days what used to take months, is a bit of a game changer. Like with past cost reductions, people will underestimate the work and get it wrong. It helps if you know what you are doing rather than just vibe coding things.

But for rewrites, the sunk cost fallacy becomes a lot cheaper. So, that changes how you deal with stuff that clearly isn't living up to expectations. Unceremoniously replacing what wasn't that expensive to begin with might be the cheaper option relative to fixing it.

Much less if you consider buy vs build+maintain.

I don't think that holds. Internal docs for bespoke frameworks, with examples, are effective at steering AI. The main thing is that both the API and the docs are well written. Easier said than done, but you can ask AI how to write effective documentation for AI.

>if company uses a very well tech stack, development and rewrites will be faster due to AI having way more examples to draw from.

Eh maybe not.

Stuff that has a lot of deprecated features is honestly burdensome on AI. It keeps rediscovering the deprecated features as the understanding that they are deprecated fall outside of the context window.

What you need is something that either never deprecates syntax, or is <10 years old with minimal changes over that time.

I once saw this style be called "broetry"[1], and it's unmistakably LinkedIn-voice. I get that it works because feed algorithms/engagement, but never understood why it seems largely confined to LinkedIn and not other social media sites.

[1] https://fenwick.media/rewild/magazine/dead-broets-society-be...

AI <<can>> do a lot of things, but does it actually do that without an exhaustive test suite (which legacy software generally doesn't have, and it can never be 100%, anyway)?

Between context collapse and hallucinations, how likely is it that the end result isn't slightly polished slop that misses lots of crucial details?

Why, LLMs of course. Isn't that obvious by now?

You are a man of taste and refinement; I could tell it was an LLM, but didn't recognise it was Opus and certainly had no idea which version.

(At least the author sprang for a $20 a month subscription.)

this post is no good. It's a continual rehash of what's going on in the industry. That's how all social media is, it's entirely time sensitive, keep saying the the same things and be the one to say it so the discussion happens on your "content".

OP is playing the game. The post literally says "from LinkedIn" so if you look, he has 500+ connections and 1400 followers. That's not nothing. Good for him, all advice points to this new attention economy we live in.

I'm a bit aged out of all this. And I rode the 2010s wave so I can't give any advice in good conscience. I can only say that I see you and there's a whole world of silent majorities out there with no follow count and no broetry with our name on it. (search for that word in this thread, just learned it, it's great!)

I'm not sure there is any value in knowing shit about AI. I know quite a lot about enterprise organisation level AI, but really, you could just ask an AI and it'd guide you through the processes. Knowledge in general is going to become real cheap in the age of AI. I've been a data archtiect in the past, so I used Opus 4.8 as I would've used a consultant agency on how to do our data architecture for multiple standard systems which can't directly share data with eachother. After a couple of hours with it as a sparring partner, I had some pretty awesome powerpoint decision making slides, one for c-levels and one for it-management.

Since our owners also own an IT consultant agency, I ran the same process through with one of our regular consultants who is an actual awesome data architect. The output was strikingly similar, well except that I/we didn't need to make the slides. I then had him run over the actual slides, and all we changed was adding a { between some arrows to make the source of the arrows more clear.

We're still going to use real human consultants in the loop because they are readily and freely available, and because this is still new. I doubt we'd want to spend 100 consultant hours on something like this in 5 years though. I mean, we'd still do it for decisions where we'd want someone to blame.