The title is a bit exaggerated. The depth of the projects covered in the article is clearly not representative of "all".

In fact, I now prefer to use a purely chat window to plan the overall direction and let LLM provide a few different architectural ideas, rather than asking LLM to write a lot of code whose detail I have no idea about.

▲

OldfieldFund 6 days ago | parent | next [-]

It's a play on the name of the paper that jump-started ChatGPT: "Attention Is All You Need:" https://arxiv.org/abs/1706.03762

▲

skerit 7 days ago | parent | prev | next [-]

I like using Claude-Code, it can be a real timesaver in certain cases.

But it's far from perfect. Really difficult things/big projects are nearly impossible. Even if you break it down into hundred small tasks.

I've tried to make it port an existing, big codebase from one language to another. So it has all of the original codebase in one folder, and a new project in another folder. No matter how much guidance you give it, or how clear you make your todos, it will not work.

▲

crazygringo 7 days ago | parent [-]

What specifically are its modes of failure? I've never tried doing that, do very curious what the roadblocks are.

▲

phist_mcgee 6 days ago | parent [-]

I've done something similar for a refactor.

It simply forgets code exists during a port. It will port part of a function and ignore the rest, it will scan a whole file into context and then forget that a different codepath exists.

I would never rely on it for a 1:1 mapping of large features/code transformations. Small stuff sure, but beyond say a few large files it will miss things and you will be scratching your head for why it's not working.

▲

phito 6 days ago | parent [-]

Sometimes, it also tells you it is done, but if you look at the code, there's a bunch of placeholder comments in the style of "will implement this later if we figure out how to do it".

	▲	skerit 4 days ago \| parent [-]
		Indeed, all of the above. It's impossible to just say "Here is the original file, here is the new file, port everything over" and think that'll be enough. Maybe making a plan for each file and specifically tell it to port each specific function over would work, but at that point you're really just making a literal translation from one language to the other.

▲

dexwiz 6 days ago | parent | prev | next [-]

That's my gist. All of these seem pretty basic apps I would see implemented to demo a new web or REST framework. Comment ranker is cool, but I can't imagine its doing much more than scrape text > call semantic api > modify DOM.

How much of this is buildings versus recalling tutorials in the dataset. For every vibe coded project with 20 lines of requirements, I have a model with 20 different fields all with unique semantic meanings. In focused areas, AI has been okay. But I have yet to see Claude or any model build and scale a code base with the same mindset.

▲

NitpickLawyer 7 days ago | parent | prev [-]

Most harnesses provide this as a "plan" vs. "act" mode now. You first "chat" in plan mode (no access to tools, no instructions to write any code basically), you then can optionally write those plans in a memorybank / plan.md, and then say "now go implement it", and it moves to the "act" mode where it goes through and does it, updating progress in plan.md as it goes.

	▲	pseudosavant 7 days ago \| parent \| next [-]
		I've found it very useful to have items like requirements.md, plans.md, or todo.md, in my LLM focused projects. I'll use AI to help take the ideas I have at that stage and refine them into something more appropriate for ingestion into the next stage. So, when I want it to come up with the plans, it is going to base is mostly on requirements.md, and then I'll have it act on the plans step by step after that.
	▲	chaosprint 6 days ago \| parent \| prev [-]
		the thing is, it's not working as the default mode, which is not ideal imho