From what I read about the freelancers’ perspective, a lot of the newer gigs they get are to clean up work already done by AI instead of the opportunity to start new like they did in the past.

▲

jamestimmins 2 days ago | parent | next [-]

I've done a fair amount of this work.

Pretty challenging once the codebase gets large, but also a lot of fun to take a big mess and bring it under control.

▲

pllbnk 2 days ago | parent [-]

I'm just wondering (without really being an AI fanboy, but using it daily), isn't the whole challenge just more prompting, such as instructing the agents to analyze the codebase, propose refactoring approaches, etc? Maybe it's a bit more sophisticated prompting than a non-technical person would do but prompting nonetheless.

I'm asking because I have read these comments here and there about fixing the mess AI made. I know what mess it usually creates and it doesn't get better with the new models. But I also know that whatever is created by AI kind of works more often than not, and once something breaks, it's just one prompt away to fixing it.

▲

jamestimmins 2 days ago | parent [-]

In my experience, the problem is that once a codebase passes like ~50,000 lines, it's too big for AI to make good architectural choices (this is controversial, but I have yet to see evidence otherwise).

So what I do is mostly use the AI to understand things like architecture, encapsulation, deployment pipelines, types, migration structure, etc.

It's a lot of "Audit the callers of X and give me a report" as well as reading code. I'm looking for stuff that doesn't pass my smell test, especially relating type types/interfaces. I look at the AI recommendations, but they're only useful once I've scoped them down to more granular questions, such as assessing the interfaces between specific layers of a pipeline.

That process is about understanding the shape of the solution that exists and then trying to formulate what shape it should have.

Then it's a lot of identifying a very boring, vanilla architectural target and moving towards it in phases, using the strangler fig pattern pretty aggressively. I review every PR, but mostly at the level of architecture rather than every specific line.

▲

rhdunn 2 days ago | parent [-]

The codebase size will be limited to the LLM context window size. Once the codebase grows larger than the context window, some of the codebase will have to be removed from the context and thus not be known to that LLM instance.

There's also a well documented phenomena [1] where LLMs struggle to recall information in the middle of the context.

[1] https://charlesanthonybrowne.medium.com/how-llms-end-up-igno...

▲

jamestimmins 2 days ago | parent [-]

Good post.

Yeah over the past year I've become convinced that humans' limited working memory is actually a superpower, because it allows us to avoid this type of problem.

	▲	judahmeek a day ago \| parent [-]
		Isn't it less that we avoid the problem of limited working memory & more that we are constantly confronted by it and so we have had to devise processes for quickly organizing data by relevance to immediate & long-term goals, forgetting information that is no longer relevant, and creating new working memory chunks?

▲

tapvt 2 days ago | parent | prev [-]

This feels like more of the same, but the comments in the code aren't as entertaining.

Previously, I'd be contracted to clean up messes from cut-rate dev shops. Now, there's plenty of opportunity for me to clean up messes from AI-driven development.

	▲	moarfaj a day ago \| parent [-]
		[flagged]