> We study a common intermediate setting: a developer pastes one complete file into a chat assistant [5], which sees the file’s internal structure but not its callers or dependencies. We evaluate the first answer, not an agent’s result after iterating on errors.

Very unsurprising results given this methodology. The agentic approach where models are allowed to iterate on changes is a very big part of what has made LLM coding tools actually useful.

▲ yodon 2 hours ago | parent [-]

> We study a common intermediate setting: a developer pastes one complete file into a chat assistant [5], which sees the file’s internal structure but not its callers or dependencies.

Ack! Do people actually try to develop software this way?

▲ refactor_master 2 hours ago | parent | next [-]

I do it from time to time when I’ve run out of tokens at the end of the month. But even then I’ve created a script to pack multiple code files and their dependencies into a single txt, with a tree overview.

And of course you never change the test suite in the process… That would be bad developer practice, LLM or not.

Gemini 3.1 Pro can easily work with this effectively, if not a bit slower than an agent.

	▲	1718627440 an hour ago \| parent [-]
		> But even then I’ve created a script to pack multiple code files and their dependencies into a single txt, with a tree overview. Like: `$ cat $(find project/ -name '*.py' -type f) ?`

▲ plorkyeran 2 hours ago | parent | prev | next [-]

Not any more, but it was common before Claude Code came along (~18 months ago). Based on the model versions used it may have still been the most common thing when the paper was started.

▲ antisthenes an hour ago | parent | prev | next [-]

When I need modification to a small independent Python script (<1000 loc), sure.

Not for anything serious though.

▲ TZubiri 2 hours ago | parent | prev [-]

It's pretty much the standard way to develop in OOP.

You are not supposed to hold all of the code in context, but to have a single object and reason about with what it see.s