> But you can't talk to them about the flow of the code. You can't ask them for their thinking as to why certain things are.

You can absolutely do this. It's even right most of the time.

▲ chmod775 2 hours ago | parent | next [-]

Let's be real. Most of the time you ask an LLM "Why did you do it like this?", it responds with something along the lines of "Oops. My bad. You're right to point this out."

You even have a fair chance of getting a response like that when there isn't anything wrong and the question wasn't rhetorical - which perfectly illustrates the level of the genuine understanding LLMs operate at.

▲ seventhtiger an hour ago | parent | next [-]

When you criticize AI, always remember that the alternative is the average employee. Today's models are pretty good.

	▲	devin an hour ago \| parent [-]
		A lot of people think they're above average. A lot of them are wrong. A lot of average people are producing gigantic messes. At least previous to this they were gated by their mediocrity.

▲ djeastm an hour ago | parent | prev | next [-]

I remember hearing (perhaps last year?) that the model companies have specifically tried to obfuscate the "thinking/reasoning" behind the decisions the models make so as to prevent cheaper models from training on the reasoning logs. So asking one "why did you do it like this" might be not fruitful.

Not sure if that's true or if it might be influencing what you're seeing, but it's a thought.

	▲	NewsaHackO 4 minutes ago \| parent [-]
		I think that has to do more with the thinking "train of thought" that some models show as what the model is processing before making the response. There shouldn't be a distillation risk with actually asking the model to explain why it made a decision and getting the response.

▲ saulpw 2 hours ago | parent | prev | next [-]

This has happened to me, so I put this in my global CLAUDE.md, and it seems to help (I don't remember getting the response you mentioned for awhile now):

    **Lead with the answer when asked how/which/whether.** Name the command/mechanism first; a question seeking understanding isn't a go-ahead to execute. Answer, then offer to act.

▲ dmayle 43 minutes ago | parent | prev | next [-]

That's because of a fundamental misunderstanding of what an LLM is. The only correct answer to "Why did you do it like this?" is that the specific combination of input text and RNG state caused this particular output. There's no reasoning to be had.

* EDIT * What's with the downvoting? That's a correct description of what happened. You can't ask an LLM why it did something and expect a coherent response, because there's no thinking chain, and no stored thinking state... At best, you can get a reconstruction of how the context relates to the output (basically a summarization of the context).

▲ baggy_trough 2 hours ago | parent | prev [-]

Can't remember the last time that happened.

	▲	javier2 37 minutes ago \| parent [-]
		Happened to me at least three times the past 14 days. I point out where it made a design decision that causes data loss. «Oops my mistake»

▲ datsci_est_2015 2 hours ago | parent | prev | next [-]

I believe the “them” the OP was talking about was referring to the people opening the PRs, not the LLMs.

	▲	saulpw 2 hours ago \| parent [-]
		My mistake, that is definitely a different scene.

▲ ssss11 42 minutes ago | parent | prev [-]

And you can certainly tell it the flow you want (and any other constraints) in the prompt.